Hi there..

I am using DBIsam 4.27.5 for ages already. But recently I am having a problem and I am not sure how to solve it.

I have a rest server that uses DBISAM and it creates a complete set of DBISAM components (session database, query) for each thread that is necessary. This rest server let tablets access the same database that is shared on the network and used by another windows software (that works just fine)

This rest server is a datasource provider for a software that I made on android tablets for taking restaurant orders. The windows application is the cashier and adm. This windows app runs on network and is working nice for many years, without any of the problems below.

I am experiencing this:

1- the rest server is session-less, that is, it opens the session, database, tables, retrieve and/or update data and then close everything. This happens everytime that anything happens. However there are records that remains locked sometimes and when the windows program try to access these records it raises an exception (this program is not prepared to deal with that, but it should not have any record locked based on the context that we use it)
2- I am experiencing data loss, eventually some record is missing. I am not 100% sure that is record missing, or if the windows application is not seeing all the data required. The rest server is receiving a json packet that is translated to the DBISAM records and saved. Then I print it AFTER saved, so I know it was saved. However is some circunstances the cashier does not get all the saved data from the rest server.

I know there is not data loss on the tablet-rest server connection. I am using the same software with firebird and works nice.

I am wondering if that is not related to the .lck files. Right now I am creating a folder for each rest server session, that will match one folder for each tablet requesting/updating data.

This scenario works for that tablets, and I retrieve the data just fine when the cashier update the same order.

Eventually I am using wrong the lck files.

Any ideas?

Thanks !

Eduardo wrote:

> Eventually I am using wrong the lck files.
>
> Any ideas?

Hard to help in detail, as obviously we can't see and run your code to
test it. However, the first thing is that DBISAM is fundamentally quite
solid, and works well with multi-threading, so that makes it likely
that there is some sort of issue in your use, rather than the database.
(I have it working in various systems which are heavy threaded (some
run over 200 threads against the database, and some are web services
like yours.)

So, I think you may be on to something with the lock file. I have all
my sessions point at the same location for everything, and I think I'd
worry if any part of a system had its own locations. DBISAM may be
making assumptions that the lockfile is common, and thus breaking
things.

Have a really good look through the flow control of your server too. In
my shopping application, I thought I could cache things, but the way
that the thread pool worked meant that I was sharing the sessions and
queries between multiple requests, and that meant that the first one
that hadn't completed was broken by the second which came in after.
Ensuring isolation is important. If you don't have logging, then be
sure to add some. I recommend Codesite as a tool that handles threads
nicely.

--

Matthew Jones

Den 27-11-2014 kl. 23:00 skrev Eduardo:
> Hi there..
>
> I am using DBIsam 4.27.5 for ages already. But recently I am having a problem and I am not sure how to solve it.
>
> I have a rest server that uses DBISAM and it creates a complete set of DBISAM components (session database, query) for each thread that is necessary. This rest server let tablets access the same database that is shared on the network and used by another windows software (that works just fine)
>
> This rest server is a datasource provider for a software that I made on android tablets for taking restaurant orders. The windows application is the cashier and adm. This windows app runs on network and is working nice for many years, without any of the problems below.
>
> I am experiencing this:
>
> 1- the rest server is session-less, that is, it opens the session, database, tables, retrieve and/or update data and then close everything. This happens everytime that anything happens. However there are records that remains locked sometimes and when the windows program try to access these records it raises an exception (this program is not prepared to deal with that, but it should not have any record locked based on the context that we use it)
> 2- I am experiencing data loss, eventually some record is missing. I am not 100% sure that is record missing, or if the windows application is not seeing all the data required. The rest server is receiving a json packet that is translated to the DBISAM records and saved. Then I print it AFTER saved, so I know it was saved. However is some circunstances the cashier does not get all the saved data from the rest server.
>
> I know there is not data loss on the tablet-rest server connection. I am using the same software with firebird and works nice.
>
> I am wondering if that is not related to the .lck files. Right now I am creating a folder for each rest server session, that will match one folder for each tablet requesting/updating data.
>
> This scenario works for that tablets, and I retrieve the data just fine when the cashier update the same order.
>
> Eventually I am using wrong the lck files.
>
> Any ideas?
>
> Thanks !
>

Hi Eduardo,

Did anything change on the servers, windows updates or something? Can
you check the event logs to see if something suspecius are happening?

Regards,
Hüseyin Aliz

Eduardo

I have two thoughts about this


What you are describing sounds very like lost network connections.

I had something very similar years ago when I first set up a wireless network. I foolishly chose Belkin equipment and it dropped the connection every few seconds.

It also could be a "simple" corruption problem. What does verify show when you think data has been lost?

Roy Lambert

Roy, thank you for the ideas.

However it is not network related. My rest server and database are all locally. Tablets access via wifi but this is not the focus of the problem.

As I said after saving, I send to a printer what I have saved, from the table, not from the memory. So I know it was saved properly.

The threads are isolated, start with its own session/database/tables and are closed after completions. This is not the source of the problem either.

The problem I believe is HOW I am handling the .lck files.

I am now using a folder for EACH thread to store the .lck and the cashier software has its own folder either for .lck.

So there is NO SHARED .lck, probably I am doing this wrong, should it be all using the same .lck including the cashier (that uses the same dbisam version) ????


Roy Lambert wrote:

Eduardo

I have two thoughts about this


What you are describing sounds very like lost network connections.

I had something very similar years ago when I first set up a wireless network. I foolishly chose Belkin equipment and it dropped the connection every few seconds.

It also could be a "simple" corruption problem. What does verify show when you think data has been lost?

Roy Lambert

Huseyin,

"Did anything change on the servers, windows updates or something? Can
you check the event logs to see if something suspecius are happening?"

The problem is not the server itself. Since I am not using server software, those are plain windows XP ou 7 on desktop computers. And there are many instalations using it. It is the way I am using lck files I believe.
Thanks

On 11/27/2014 5:00 PM, Eduardo wrote:
> I am wondering if that is not related to the .lck files. Right now I am creating a folder for each rest server session, that will match one folder for each tablet requesting/updating data.
> This scenario works for that tablets, and I retrieve the data just fine when the cashier update the same order.
> Eventually I am using wrong the lck files.

This would definitely be the source of the problem.

However i'm curious how you end up with multiple lck files (for the same
tables) ?

The lock files are per database in DBISAM (so per directory basically)
since they provide table and transaction locks for db files in that
directory.

Raul

Eduardo

Just had a quick look at the manual for the version of DBISAM I have and I certainly wouldn't set up multiple lock files. The .lck files are there to manage the locking for the application as a whole so I have a suspicion that by having several you're really giving things an opportunity to screw up. I didn't even know you could have multiple ones.

Just to clarify a bit:

1. tablet takes order
2. data is passed across to server
3. data is saved in tables
4. data is printed
5. data vanishes

If that's right you could have a corrupt index so you can't find the data again, or the data could have been overwritten.

So silly questions

a) have you tried verifying & repairing if so what result
b) is there an order number and if so are there gaps appearing

Roy Lambert

Raul

>However i'm curious how you end up with multiple lck files (for the same
>tables) ?

I didn't even know it was possible. Reading your post I now wonder if Eduardo is referring to private directories.

Roy Lambert

On 11/28/2014 8:11 AM, Roy Lambert wrote:
> I didn't even know it was possible. Reading your post I now wonder if Eduardo is referring to private directories.

Same here. The lock file to best of my knowledge is created in the
folder where db files are and private dir should not affect this.

Hence i'm curious about the "using wrong the lck files" part !?

The only other thing i can think of is mixing the LargeFileSupport
setting between different apps access the DB (but again this should not
result in multiple lck files - just screw up locking on existing file).

Raul