"Gordon Turner" <gordon@mycroftcomputing.com> wrote in message
news:67229D4C-9EDB-4A8F-B8F7-F3F5262749A7@news.elevatesoft.com...
>
> In the data module, I use a common event handler for the AfterDelete,
> AfterInsert, and AfterPost that executes the following:
>
> procedure TDataMod.BufferFlush(DataSet: TDataSet);
> begin
>   if not fSkipBufferFlush then
>     (DataSet as TDBISAMTable).FlushBuffers;
> end;
>

Useful, maybe, but that's not the issue. I never use stuff like that. At
most, flush buffers when you commit a transaction.

>
> Files are all opened in Shared mode (it's a multi-user application), but
> in both cases, the user was the only person working in the app at the time
> of the data loss.  I examined the file and it did not show any signs of
> corruption (no change in the number of records when I ran the repair
> utility).  The file in question would not have more than 2000 entries - if
> that many.  The customer did the data entry, and then printed a report
> showing those entries.  The report is generated by a separate query of the
> table.
>

This proves that the data was added correctly to the table. As usual, I
would suspect first my code:

1. Do you have any kind of mass delete that would get rid of all the data
the customer entered on a certain date? Why is the table so small, because
you move data soemplace else and then delete it? Was the data moved in
error? Do you have some sort of "undo" for a day's data entry? If so, is it
possible that it was executed in error?
2. Is it possible that the table was restored from a backup, after the data
had been entered (see next)?
3. Do you have any way of determining if the current table actually at some
point did contain the missing data? For example, gaps in autoinc fields?
4. (unlikely, but ...) any possibility that you did not commit the updates,
and when exiting the application, you do not check for database in
transaction or tables not in dsbrowse?

>
> Is there some sort of caching registry entry that could make a difference?
> I'm really struggling with how to make the customer feel comfortable (and
> me for that matter).
>

If you felt comfortable in this situation, we would doubt your sanity

Robert

Robert wrote:
>
> Useful, maybe, but that's not the issue. I never use stuff like that. At
> most, flush buffers when you commit a transaction.

I also manage record locking with updates (so two users don't try to
update the same employee at the same time), so I want to get changes
into the shared files as quickly as possible.  There is no real
performance hit (except in mass updates - like the data import - where I
skip the flush anyway).

> This proves that the data was added correctly to the table. As usual, I
> would suspect first my code:
>
> 1. Do you have any kind of mass delete that would get rid of all the data
> the customer entered on a certain date? Why is the table so small, because
> you move data soemplace else and then delete it? Was the data moved in
> error? Do you have some sort of "undo" for a day's data entry? If so, is it
> possible that it was executed in error?

No archive facility.  The app tracks employee attendance (or rather
times they were not at work) so the number of entries in the table vary,
depending on the number of employees being tracked and the length of
time the customer has been using the program (one record per day per
employee).  The only way to delete more than one record at a time would
be to delete a parent record - wrapped in a transaction.

> 2. Is it possible that the table was restored from a backup, after the data
> had been entered (see next)?

I thought of that but only some of the day's work was lost, not all of
it.  And in the new instance, the loss occurred on the same day.

> 3. Do you have any way of determining if the current table actually at some
> point did contain the missing data? For example, gaps in autoinc fields?

Unfortunately no.  The key to the affected table is a combination of two
foreign keys plus a date.

> 4. (unlikely, but ...) any possibility that you did not commit the updates,
> and when exiting the application, you do not check for database in
> transaction or tables not in dsbrowse?

Most entries are single record entries, hence no transaction.

> If you felt comfortable in this situation, we would doubt your sanity

I make no claims about my sanity, either before or after the incident


Gordon Turner

Gordon,

<< Another customer reported the same problem yesterday.  This customer is
using a Win XP SP2 machine, but the data files are stored on another users
Win XP SP2 machine rather than a file server.  I "verified" the data files
this morning and there were no problems.

The customer says she made her entries, closed out of the program, and then
went back in later the same day to discover that updates were gone.  No
shutdown or re-boot of her PC.  Since the updates are flushed at the table
level, even if the other PC had been shut down, only one update should have
been lost, not a number of them. >>

Someone is not telling the whole story, or they are missing something that
is occurring with or without their knowledge.  DBISAM will not simply "lose
data" from a table.  Either a machine is getting shut down improperly, or
the records are being deleted.  If you want to have me check for the second
instance, I can do so if you can get me a copy of one of their tables.

<< Could those settings have been the source of the lost updates, preventing
or delaying the buffer flush? >>

No.  The OpLocks are only an issue when you don't call FlushFileBuffers at
the OS level, which in this case you/we are.

--
Tim Young
Elevate Software
www.elevatesoft.com

Gordon


I like the point that if the table was queried to produce a report the data was there.  Which brings the options down to three:

1. DBISAM is wantonly throwing data away
2. Some one (or your program) is randomly deleting data
3. Records are being overwritten

I would personally suspect no. 3. Depending on what the keys are and what you do to make sure they aren't duplicated (I have had problems with Locate and multi-part keys in the past) it could be your code or index corruption. I don't think DBSys reports ALL instances of index repair (I'm sure Tim will tell me if I'm wrong) so it could look like the table is fine but the index is out of whack.

Why not build a simple audit trail in - say just a record of keys used and by who - at least it will help prove your sanity.

Roy Lambert

Tim Young [Elevate Software] wrote:
>
> Someone is not telling the whole story, or they are missing something that
> is occurring with or without their knowledge.  DBISAM will not simply "lose
> data" from a table.  Either a machine is getting shut down improperly, or
> the records are being deleted.  If you want to have me check for the second
> instance, I can do so if you can get me a copy of one of their tables.

Should I just post the files in Binaries or do you want me to email them
to you separately?  I have a copy of the complete set in a ZIP file
(about 35KB).

Gordon Turner