Icon View Thread

The following is the text of the current message along with any replies.
Messages 1 to 6 of 6 total
Thread Disappearing Records
Tue, Jun 23 2015 2:24 PMPermanent Link

Eduardo

Avatar

I am using DBISAM 4.27 build 5

XE2.4

it is a multi threaded restful server application

most of the time it is attending just one request, but can go up to 10 (very rare)

I have this in several customers, commonly they use this to share data with up to 3 tablets. (take orders)

THe mistery happens when:

- tablet send an order
- my server translate it and save on DBISAM Tables
- i call another functon to print the order. This function creates a FRESH TDBISAM Table, connects to the same table on disk and then PRINT the order
- from a PC accessing the same table we see that the data is not there.
- the tablet cannot retrieve that data anymore.

There is no delete operation available.

There is no way for the user make this happen, it is a fully automated process.

This same server application also  perrmits to use firebird as a datatbase, and this problem never happens.

Each time a request is made to the server it creates all the DBISAM components with its own session name, to isolate each thread.

Problem happens on Windows XP, 7 and 8

I have no more ideas.
Tue, Jun 23 2015 2:47 PMPermanent Link

Raul

Team Elevate Team Elevate

On 6/23/2015 2:24 PM, Eduardo wrote:
> - my server translate it and save on DBISAM Tables
> - i call another functon to print the order. This function creates a FRESH TDBISAM Table, connects to the same table on disk and then PRINT the order
> - from a PC accessing the same table we see that the data is not there.
> There is no delete operation available.

Two things i would check are :

1. Are you using transactions and any chance you print it and then roll
it back ?

2. Do you use any "select .. INTO" clauses ? That is a destructive (i.e.
delete) operation for the destination table.


> I have no more ideas.

if this is reproducible then attach some insert/update/delete triggers
and log the table changes into a dedicated trace log table to see what
operations take place from engine perspective.

Raul
Tue, Jun 23 2015 4:35 PMPermanent Link

Eduardo

Avatar

Thanks Raul

1. Are you using transactions and any chance you print it and then roll
it back ?

I am not using transactions, however I have this code:
   if LocalDatabase.InTransaction then
     LocalDatabase.Commit(true);

as a precaution BEFORE printing.

If eventually for some reason it got in a transaction then i was expecting to commit it before printing.

The printing function is just after this, and it reads everything from the table again.

2. Do you use any "select .. INTO" clauses ? That is a destructive (i.e.
delete) operation for the destination table.

I am not using INTO.

On this particular request to save the customer order there is no selects at all. Just plan DBISAM tables

> I have no more ideas.

if this is reproducible then attach some insert/update/delete triggers
and log the table changes into a dedicated trace log table to see what
operations take place from engine perspective.

I cannot reproduce it. It does happen because I get feedback from customers and they show me the printed cupom and there is not saved records. Even they eventually make mistakes about this, I am pretty sure that some are trully vanished records.

Raul
Wed, Jun 24 2015 3:36 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Eduardo

>- tablet send an order
>- my server translate it and save on DBISAM Tables
>- i call another functon to print the order. This function creates a FRESH TDBISAM Table, connects to the same table on disk and then PRINT the order
>- from a PC accessing the same table we see that the data is not there.
>- the tablet cannot retrieve that data anymore.

I assume that the order is printed correctly, and that the function printing the order is on the same PC as that initially storing it. Further that the PC that can't access the data is a different PC. Finally that the data never "comes back".

If I'm correct, then the whole thing makes me think its some sort of buffering problem. If my final assumption is wrong then simply adding flush / refresh commands should sort it out. If the final assumption is right then a flush may help but I'm very baffled.

DBISAM/Windows do not necessarily flush data down to disk immediately it can be held in the os buffers for a while before hitting the disk, but once DBISAM is aware of its existence it should at some point write it down.

My logic (and Tim will probably destroy it) goes:

1. data received from tablet
2. data written to DBISAM buffer (probably in Windows buffer rather than internal)
3. print routine opens same PC connection to table and because its the same PC has access to Windows buffers and sees the data
4. *** something wipes the Windows buffer **** and DBISAM does not complain, and forgets it was ever there

First suggestion - add a flush in immediately after writing the data to the table.

Second suggestion - write the data out to a simple text file before writing to the table. If that vanishes as well you really have a problem.

Third suggestion - pretty much as Raul but build in full logging and possibly run a windows message logger onto the PC - mind numbingly boring trying to wade through the output but as a desparation measure can help.

Roy Lambert
Wed, Jun 24 2015 4:07 AMPermanent Link

Matthew Jones

Roy Lambert wrote:

>  but build in full logging

This is the simple answer, and only sure way. You need a logging system
that will store everything of importance. For debug use, I use
CodeSite, but while it can be deployed, I went with my own custom
logging system, which is multi-thread safe so that it doesn't cause
inter-thread delays. When someone says there is a problem, I get a file
with a few hundred thousand lines of log, but I can get very forensic
with it, and either identify the issue, or pass on the blame
appropriately. One of my applications is a heavily multi-threaded data
transmission system that hits DBISAM very hard with several writers and
hundreds of readers. I've never lost any data - it is in local file
mode.

What may be of relevance is that some of the logs sometimes show that
some server somewhere did a reboot for a few minutes, and my code then
can't connect during that period. It shows that the reason for the
delay is some other system, but having a log of all DBISAM errors and
other exceptions thrown (I log madExcept errors to the log too) may
reveal some other issue.

Good luck finding it, but you need to start with logging.

--

Matthew Jones
Wed, Jul 8 2015 11:37 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Eduardo,

<< I am using DBISAM 4.27 build 5 >>

4.27 was released *7* years ago.  You should definitely update to the latest
release, especially when using it with the newer XE* versions of Delphi.

Also, is there any Windows network file-sharing taking place, or are the
tables all on the same machine as your REST server ?

<< - from a PC accessing the same table we see that the data is not there.
>>

The same PC, or a different PC ?  So, you can see the table, and the table
is fine (no corruption), but it is completely empty ?  And, when you print
the order, the printed output is fine and shows the records in the table ?

<< - the tablet cannot retrieve that data anymore. >>

What do you mean by "anymore" ?  Was the tablet able to retrieve the data at
some earlier point ?

<< This same server application also  perrmits to use firebird as a
datatbase, and this problem never happens. >>

Apples and oranges, especially if file-sharing is involved.

Thanks,

Tim Young
Elevate Software
www.elevatesoft.com
Image