I have a customer who lost entries that they created on a previous day
in my application.  The app itself uses 13 different tables, and the
data is only lost from one.  Here are the specifics (at least for one
instance) - D7, DBISam 3.24 for the app.  Workstation - XP Pro SP2 -
data files located on a W2003 server using MS networking.

In the data module, I use a common event handler for the AfterDelete,
AfterInsert, and AfterPost that executes the following:

procedure TDataMod.BufferFlush(DataSet: TDataSet);
begin
  if not fSkipBufferFlush then
    (DataSet as TDBISAMTable).FlushBuffers;
end;

The fSkipBufferFlush is a flag I use to avoid the flush when I'm
performing a data import function.

Files are all opened in Shared mode (it's a multi-user application), but
in both cases, the user was the only person working in the app at the
time of the data loss.  I examined the file and it did not show any
signs of corruption (no change in the number of records when I ran the
repair utility).  The file in question would not have more than 2000
entries - if that many.  The customer did the data entry, and then
printed a report showing those entries.  The report is generated by a
separate query of the table.

This is the third instance of this happening - three different customers
(see my post from March 13, 2006).  Although that is a small portion of
my 700+ customer base, the fact that data did disappear is disturbing to
say the least.  Some of the updates she made were still present, but
others not.  Running the recovery tools does not recover the missing data.

Is there some sort of caching registry entry that could make a
difference?  I'm really struggling with how to make the customer feel
comfortable (and me for that matter).

Gordon Turner

Gordon,

<< I have a customer who lost entries that they created on a previous day in
my application.  The app itself uses 13 different tables, and the data is
only lost from one.  Here are the specifics (at least for one instance) -
D7, DBISam 3.24 for the app.  Workstation - XP Pro SP2 - data files located
on a W2003 server using MS networking. >>

<< Files are all opened in Shared mode (it's a multi-user application), but
in both cases, the user was the only person working in the app at the time
of the data loss. >>

This information tells me that, most likely, the person did the data entry
and then proceeded to just power-down the machine.  Opportunistic locking on
a Windows 2003 server with XP clients works like this:

1) If only one client workstation has a file(s) open, then XP will cache the
reads and updates to the file until close time.
2) If the workstation is powered down or the connection is broken, then the
data will be lost.

If a repair didn't indicate any corruption, then the above is the only
explanation for the issue apart from the user manually deleting the same
records.    You can check the event log to see when the machine was last
shut down and restarted:

http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/snap_event_viewer.mspx?mfr=true

You can tell when the system is restarted by looking at the Source column
for any instance of "eventlog".  Those are entries where the event logging
was stopped or started, i.e. the machine was stopped or started.

--
Tim Young
Elevate Software
www.elevatesoft.com

Tim Young [Elevate Software] wrote:
>
> This information tells me that, most likely, the person did the data entry
> and then proceeded to just power-down the machine.  Opportunistic locking on
> a Windows 2003 server with XP clients works like this:
>
> 1) If only one client workstation has a file(s) open, then XP will cache the
> reads and updates to the file until close time.
> 2) If the workstation is powered down or the connection is broken, then the
> data will be lost.
>

Is there any way to force the cache to flush?  Obviously, there is no
way to stop a user from shutting down their machine.  (This is a
commercial product, so I don't have direct access to the user's
workstation.)

Gordon,

> Is there some sort of caching registry entry that could make a difference?
> I'm really struggling with how to make the customer feel comfortable (and
> me for that matter).

You might want to try setting the following registry entries upon install of
your package.  We've been doing this for years to deal with BDE problems
before we switched to DBISAM.  They help tremendously with cached file
issues on single user systems with the BDE so they might help in these
situations..

HKey_Local_Machine\System\CurrentControlSet\Services\LanmanServer\Parameters\EnableOpLocks
= 00000000 (Double Word)
HKey_Local_Machine\System\CurrentControlSet\Services\LanmanServer\Parameters\CachedOpenLimit
= 00000000 (Double Word)
HKey_Local_Machine\System\CurrentControlSet\Services\LanmanWorkStation\Parameters\UseOpportunisticLocking
= 00000000 (Double Word)
HKey_Local_Machine\System\CurrentControlSet\Services\LanmanWorkStation\Parameters\UtilizeNtCaching
= 00000000 (Double Word)

--

---------------------------------------
 Terry Swiers
 Millennium Software, LLC
 http://www.1000years.com
 http://www.atrex.com

 Atrex Inventory Control/POS -
    Big business features without spending big business bucks!

Atrex Electronic Support Options:
 Atrex Knowledgebase: http://www.atrex.com/atrexkb.asp
 Email: mailto:support@atrex.com
 Newsgroup: news://news.1000years.com/millennium.atrex
 Fax: 1-925-829-1851
 Phone: 1-925-828-5892 (M-F, 9a-5p Pacific)
 ---------------------------------------

Tim


Shouldn't

procedure TDataMod.BufferFlush(DataSet: TDataSet);
begin
  if not fSkipBufferFlush then
    (DataSet as TDBISAMTable).FlushBuffers;
end;


Have meant that the data was flushed down to disk?

Roy Lambert

Terry Swiers wrote:
>
> You might want to try setting the following registry entries upon install of
> your package.  We've been doing this for years to deal with BDE problems
> before we switched to DBISAM.  They help tremendously with cached file
> issues on single user systems with the BDE so they might help in these
> situations..
>
> HKey_Local_Machine\System\CurrentControlSet\Services\LanmanServer\Parameters\EnableOpLocks
> = 00000000 (Double Word)
> HKey_Local_Machine\System\CurrentControlSet\Services\LanmanServer\Parameters\CachedOpenLimit
> = 00000000 (Double Word)
> HKey_Local_Machine\System\CurrentControlSet\Services\LanmanWorkStation\Parameters\UseOpportunisticLocking
> = 00000000 (Double Word)
> HKey_Local_Machine\System\CurrentControlSet\Services\LanmanWorkStation\Parameters\UtilizeNtCaching
> = 00000000 (Double Word)

I've added that to our setup several years ago, but unfortunately some
customers install the program on a server and then just drag short-cuts
to the user's desktop.  I need to add the checks to the application as
well, but doing those updates requires a reboot before they take effect,
so I need to work out a good way to handle this from a user's
perspective in the app.

Gordon Turner

Gordon,

<< Is there any way to force the cache to flush?  Obviously, there is no way
to stop a user from shutting down their machine.  (This is a
commercial product, so I don't have direct access to the user's
workstation.) >>

Sure, you can use the TDBISAMTable/TDBISAMQuery FlushBuffers method after
any updates, and that should force the OS to flush the data from the local
OpLocks cache back to the file server.

--
Tim Young
Elevate Software
www.elevatesoft.com

Roy,

<< Shouldn't

procedure TDataMod.BufferFlush(DataSet: TDataSet);
begin
if not fSkipBufferFlush then
(DataSet as TDBISAMTable).FlushBuffers;
end;

Have meant that the data was flushed down to disk? >>

Provided that it was executed, yes, it should:

http://support.microsoft.com/default.aspx?scid=kb;en-us;224992&Product=nts40

--
Tim Young
Elevate Software
www.elevatesoft.com

Tim Young [Elevate Software] wrote:
>
> << Shouldn't
>
>  procedure TDataMod.BufferFlush(DataSet: TDataSet);
>  begin
>  if not fSkipBufferFlush then
>  (DataSet as TDBISAMTable).FlushBuffers;
>  end;
>
>  Have meant that the data was flushed down to disk? >>
>
> Provided that it was executed, yes, it should:

The BufferFlush procedure is attached to the AfterDelete, AfterInsert,
and AfterPost methods of all my tables.  The only time the
fSkipBufferFlush flag is False is during a data import process.  Any
update that affects more than one table is wrapped in a transaction.  I
though this should cover any caching/buffer problems.


Gordon Turner

Another customer reported the same problem yesterday.  This customer is
using a Win XP SP2 machine, but the data files are stored on another
users Win XP SP2 machine rather than a file server.  I "verified" the
data files this morning and there were no problems.

The customer says she made her entries, closed out of the program, and
then went back in later the same day to discover that updates were gone.
 No shutdown or re-boot of her PC.  Since the updates are flushed at
the table level, even if the other PC had been shut down, only one
update should have been lost, not a number of them.

It's possible the Registry settings mentioned by Terry had not been set
on the customer's workstation.  She did not know how the IT department
got the application on her workstation (it could have been copied from
the second workstation).

Could those settings have been the source of the lost updates,
preventing or delaying the buffer flush?


Gordon Turner