Icon View Thread

The following is the text of the current message along with any replies.
Messages 1 to 10 of 10 total
Thread DBISAM v3.30 SaveToStream
Thu, Jul 12 2007 7:53 AMPermanent Link

Praveen
Hi,

I'm using dbisam v3.30. I have 2 memory tables Table1 and Table2. I'm using SaveToStream and LoadFromStream to copy the tables i.e. Table1.SaveToStream and
Table2.LoadFromStream.

When I have to load new data to Table1, I perform the ops
Table1.Close;
Table1.EmptyTable;
Table1.Open.
//First append
Table1.Append;
//Fill row with data
Table1.Append;

Control never returns from this second append. The code is in a try..finally..except block. I put breakpoints in these blocks, but it never reaches there.
I'm using Delphi 6.

I had used this code once and faced the same issue. Then I made sure both tables were closed, emptied and opened. It solved the problem. Not happening here.

Does something happen to the table when i use savetostream ? App works fine, if I just copy each row like Table1.Field[].AsString = Table2.Field[].AsString. But this
takes a long time.

-Praveen


Thu, Jul 12 2007 8:39 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Praveen


Not sure what's going on, and not sure if this will work with V3 but try

1. delete table2
2. copy table1 to table 2 eg table1.CopyTable('memory',table1);

Which if it works should avoid the streaming in case that's the problem

The other thing you might try is using insert rather than append.

Roy Lambert
Thu, Jul 12 2007 10:16 AMPermanent Link

Praveen
Hi,

Thanks for the workaround. But it's not Table2 but Table1 that's causing the problem.

I tried to get the stack trace when the problem occurs.
It was something like..

TDBISAMTable.APPEND;
..
..
TDATASET.Post.
..
..
TDataCurSur.AppendRecord
..
..
TDataFile.someproc
TAbstractFile.LockTable or something.. I lost the trace...
...
..
OSEnterCriticalsection.
RtlWait..CriticalSection.

I don't no if this helps.

More info:
-----------
The data is being populated into Table1 in a thread.
The thread calls a callback routine in my class, entering a CriticalSection to work on the Dataset.

Thu, Jul 12 2007 11:01 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Praveen


Does the thread have its own database, session and table components? If so you don't need the criticalsection. If not you're going to get problems.


Roy Lambert
Thu, Jul 12 2007 2:34 PMPermanent Link

Praveen
>Roy Lambert <roy.lambert@skynet.co.uk> wrote:
>Praveen
>Does the thread have its own database, session and table components? If so you don't need the criticalsection. If not you're going to get problems.
>Roy Lambert

Hi Roy,

No, it doesn't have what u mentioned. The Table is connected to a grid. I just pass the table reference to the thread. The app uses SOAP calls to get data from the
server. The data is broken down into packets of 1000 rows. Each packet will be received by separate threads and work on the same table Table1.
They all call the same callback. That's why I use the CriticalSection.

If I don't use streams and just copy rows as mentioned above everythings works out fine.

I'm using streams b'cos it's fast. Say for 1000 records it takes about 10 secs to copy to Table2. Whereas it took around 1 sec to insert the same data to Table1...this is a
totally different problem.

To copy 1000 recs from Table1 to Table2, using Memory streams I got a result of approx 2 secs and with File streams approx 800 msec.

Thu, Jul 12 2007 4:03 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Praveen,

<< I'm using dbisam v3.30. I have 2 memory tables Table1 and Table2. I'm
using SaveToStream and LoadFromStream to copy the tables i.e.
Table1.SaveToStream and Table2.LoadFromStream.

When I have to load new data to Table1, I perform the ops
Table1.Close;
Table1.EmptyTable;
Table1.Open.
//First append
Table1.Append;
//Fill row with data
Table1.Append;

Control never returns from this second append. The code is in a
try..finally..except block. I put breakpoints in these blocks, but it never
reaches there. >>

Where is the LoadFromStream taking place ?

Also do the structures of Table1 and Table2 match *exactly*, including
indexes ?

--
Tim Young
Elevate Software
www.elevatesoft.com

Thu, Jul 12 2007 4:32 PMPermanent Link

Praveen
Tim,

<< Where is the LoadFromStream taking place ?

Also do the structures of Table1 and Table2 match *exactly*, including
indexes ?

--
Tim Young
Elevate Software
www.elevatesoft.com
>>

LoadFromStream takes place in main thread.

fs: TFileStream.create(tempfile, fmcreate)
try
  table1.savetostream(fs);
  fs.Postion := 0;
  table2.loadfromstream(fs);
finally
  fs.free;
  deletefile(tempfile);
end;

Both Table1 and Table2 have same schema.

In short my app works like this. I hit fetch button on screen. Start thread which fetches data from server (in xml) and populates Table1 (datasource of grid). After that,
control returns to main thread. I need copy of Table1 to calculate calculations like (Field1*Field2)/ Field2 Column Total, basically to find Average. So I use Table2.

To refresh screen, I hit fetch button. Table1 and Table2 are both closed, emptied and opened. Table1 reference passed to thread.
First Table1.Append called and data inserted. Calling Table1.Append again -> Table1.Post is called and is stuck in Critical Section in ntdll.dll (as seen in stack trace).

No exceptions thrown.
Fri, Jul 13 2007 2:50 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Praveen


I'm getting confused here. You seem to be saying that you're receiving data into several streams, using this to populate a memory table and then streaming that into a second memory table to merge the data from several threads together. If that's right I don't see how it works because streaming a table into another overwrites it.

Inserting rows into a table linked to a grid will be slow - its the UI update that's the problem.

Try creating a separate database/session/table in each thread point the table to the same memory table and when the thread has finished its insertions post a message back to the main thread telling it to refresh the grid.

Roy Lambert
Fri, Jul 13 2007 4:36 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Praveen,

<< In short my app works like this. I hit fetch button on screen. Start
thread which fetches data from server (in xml) and populates Table1
(datasource of grid). After that, control returns to main thread. I need
copy of Table1 to calculate calculations like (Field1*Field2)/ Field2 Column
Total, basically to find Average. So I use Table2. >>

Where is Table2 then used ?

<< To refresh screen, I hit fetch button. Table1 and Table2 are both closed,
emptied and opened. Table1 reference passed to thread. First Table1.Append
called and data inserted. Calling Table1.Append again -> Table1.Post is
called and is stuck in Critical Section in ntdll.dll (as seen in stack
trace). >>

Okay, it sounds like you've got something messed up with the threading or a
critical section isn't being released.  If you want to send me the code that
you're using, I can take a look and see what is going wrong.

--
Tim Young
Elevate Software
www.elevatesoft.com

Sat, Jul 14 2007 2:14 PMPermanent Link

Praveen
Hi,

Thanks anyway. I just replaced the code by copying row by row. It works fine that way.

Praveen
Image