Icon View Thread

The following is the text of the current message along with any replies.
Messages 1 to 10 of 21 total
Thread Lost Data
Fri, Nov 2 2007 9:36 AMPermanent Link

Gordon Turner
I have a customer who entered data (employee information) into my
application, created a backup of her data (from within the app - create
a zip file of the database files) and closed the application.  Upon
starting the app a couple days later, a number of employee entries were
missing.  Restoring from the backup (from within the app - overlaying
the database files with the files contained in the zip file) did not
restore the missing entries.  The first group of employees she entered
were still there, but none of the later entries.  (She is the only user
of the application.)

This is the second or third customer who has reported this type of
problem and it concerns be greatly.  (The app is D7 using DBISAM 3.24 in
stLocal mode.) I've looked at the backup file and the data in it seems
internally consistent so it's hard to say it's because of a single
corrupted file.  (Running a repair on the tables did not show any
problems being fixed.)  I perform a FlushBuffers in the OnTableClose
event for all tables and close all the tables before creating a backup
file, so it's difficult to say it's a caching problem.  The customer is
running XP Pro SP2 and the data files and application are stored on her PC.

Tim, I've posted the backup file she sent me in Binaries (only a handful
of records in each table).  Is there any way for you to tell if data has
been deleted or if a repair has been performed on any of the tables?

--
Gordon Turner
Mycroft Computing
http://www.mycroftcomputing.com
Fri, Nov 2 2007 11:37 PMPermanent Link

Dave Harrison
Gordon Turner wrote:
> I have a customer who entered data (employee information) into my
> application, created a backup of her data (from within the app - create
> a zip file of the database files) and closed the application.  Upon
> starting the app a couple days later, a number of employee entries were
> missing.  Restoring from the backup (from within the app - overlaying
> the database files with the files contained in the zip file) did not
> restore the missing entries.  The first group of employees she entered
> were still there, but none of the later entries.  (She is the only user
> of the application.)
>
> This is the second or third customer who has reported this type of
> problem and it concerns be greatly.  (The app is D7 using DBISAM 3.24 in
> stLocal mode.) I've looked at the backup file and the data in it seems
> internally consistent so it's hard to say it's because of a single
> corrupted file.  (Running a repair on the tables did not show any
> problems being fixed.)  I perform a FlushBuffers in the OnTableClose
> event for all tables and close all the tables before creating a backup
> file, so it's difficult to say it's a caching problem.  The customer is
> running XP Pro SP2 and the data files and application are stored on her PC.
>
> Tim, I've posted the backup file she sent me in Binaries (only a handful
> of records in each table).  Is there any way for you to tell if data has
> been deleted or if a repair has been performed on any of the tables?
>

Gordon,
     Maybe she entered the data in the wrong directory? Scan the drive
looking for other *.DAT files that may have been created. Is there a
test database she might have been using by mistake?

Dave
Sat, Nov 3 2007 6:07 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Gordon,

<< I have a customer who entered data (employee information) into my
application, created a backup of her data (from within the app - create a
zip file of the database files) and closed the application.  Upon starting
the app a couple days later, a number of employee entries were missing.
Restoring from the backup (from within the app - overlaying
the database files with the files contained in the zip file) did not
restore the missing entries.  The first group of employees she entered were
still there, but none of the later entries.  (She is the only user of the
application.) >>

As Dave indicated - is there any capability to use multiple databases ?
That is usually the most likely culprit besides improper shutdowns or Vista
issues with protected directories (which doesn't apply).

<< This is the second or third customer who has reported this type of
problem and it concerns be greatly. >>

I can assure 100% that DBISAM is not responsible for this issue.  It does
not "lose" data.

<< (The app is D7 using DBISAM 3.24 in stLocal mode.) I've looked at the
backup file and the data in it seems internally consistent so it's hard to
say it's because of a single corrupted file.  (Running a repair on the
tables did not show any problems being fixed.) >>

It's possible with 3.x that an issue with the indexes might escape detection
during the repair.  3.x wasn't as thorough as 4.x and later versions in
terms of validating the indexes during a repair.

<<im, I've posted the backup file she sent me in Binaries (only a handful of
records in each table).  Is there any way for you to tell if data has been
deleted or if a repair has been performed on any of the tables? >>

Which table is the one missing the records ?

--
Tim Young
Elevate Software
www.elevatesoft.com

Sun, Nov 4 2007 1:00 PMPermanent Link

"Robert"

"Tim Young [Elevate Software]" <timyoung@elevatesoft.com> wrote in message
news:76B7BE19-FB32-439A-90D2-A2C378874EC3@news.elevatesoft.com...
> Gordon,
>
> << I have a customer who entered data (employee information) into my
> application, created a backup of her data (from within the app - create a
> zip file of the database files) and closed the application.  Upon starting
> the app a couple days later, a number of employee entries were missing.
> Restoring from the backup (from within the app - overlaying
> the database files with the files contained in the zip file) did not
> restore the missing entries.  The first group of employees she entered
> were still there, but none of the later entries.  (She is the only user of
> the application.) >>
Never say never, but it does not sound at all like a DBISAM issue. Do you
have a filter that was not reset, have you looked at the table with DBSYS,
is it possible that you were doing the updates inside a transaction and you
cancelled instead of comitting?

Robert

>
> As Dave indicated - is there any capability to use multiple databases ?
> That is usually the most likely culprit besides improper shutdowns or
> Vista issues with protected directories (which doesn't apply).
>
> << This is the second or third customer who has reported this type of
> problem and it concerns be greatly. >>
>
> I can assure 100% that DBISAM is not responsible for this issue.  It does
> not "lose" data.
>
> << (The app is D7 using DBISAM 3.24 in stLocal mode.) I've looked at the
> backup file and the data in it seems internally consistent so it's hard to
> say it's because of a single corrupted file.  (Running a repair on the
> tables did not show any problems being fixed.) >>
>
> It's possible with 3.x that an issue with the indexes might escape
> detection during the repair.  3.x wasn't as thorough as 4.x and later
> versions in terms of validating the indexes during a repair.
>
> <<im, I've posted the backup file she sent me in Binaries (only a handful
> of records in each table).  Is there any way for you to tell if data has
> been deleted or if a repair has been performed on any of the tables? >>
>
> Which table is the one missing the records ?
>
> --
> Tim Young
> Elevate Software
> www.elevatesoft.com
>

Sun, Nov 4 2007 11:21 PMPermanent Link

Gordon Turner
Dave Harrison wrote:
>
>      Maybe she entered the data in the wrong directory? Scan the drive
> looking for other *.DAT files that may have been created. Is there a
> test database she might have been using by mistake?

The user can only select a data directory when installing the
application.  That said, the path to the data files is stored in
HKEY_CURRENT_USER.  If the app does not find its data files, it creates
a new set.  So if, for some reason, her local userid changed, she might
access a new set of data files.  But that would only cover loosing all
data, not later entries to the current data files.  To loose later
entries would require changing userIDs twice.  And how would the older
entries appear in the new set of data files?

Very frustrating and confusing as I'm not sure how to reassure the
customer that it won't happen again if I'm not sure how it happened in
the first place.

--
Gordon Turner
Mycroft Computing
http://www.mycroftcomputing.com
Sun, Nov 4 2007 11:26 PMPermanent Link

Gordon Turner
Tim Young [Elevate Software] wrote:
>
> As Dave indicated - is there any capability to use multiple databases ?
> That is usually the most likely culprit besides improper shutdowns or Vista
> issues with protected directories (which doesn't apply).
>
> << This is the second or third customer who has reported this type of
> problem and it concerns be greatly. >>
>
> I can assure 100% that DBISAM is not responsible for this issue.  It does
> not "lose" data.
>
> << (The app is D7 using DBISAM 3.24 in stLocal mode.) I've looked at the
> backup file and the data in it seems internally consistent so it's hard to
> say it's because of a single corrupted file.  (Running a repair on the
> tables did not show any problems being fixed.) >>
>
> It's possible with 3.x that an issue with the indexes might escape detection
> during the repair.  3.x wasn't as thorough as 4.x and later versions in
> terms of validating the indexes during a repair.
>
> <<im, I've posted the backup file she sent me in Binaries (only a handful of
> records in each table).  Is there any way for you to tell if data has been
> deleted or if a repair has been performed on any of the tables? >>
>
> Which table is the one missing the records ?


The immediate problem is the Employee.dat table.  The customer says she
entered about 20 employees and when she started the app again, only the
first 8 were still there.  The process of entering an employee does
involve a transaction as it updates multiple tables behind the scenes,
so rolling back a single transaction could explain one missing employee,
but not 10 - 10 separate transactions.

From her email...

I spent most of the day inputting information.  By the end of the day, I
had about 21 employees with 1 ½ yrs of information, many time off
reasons and their benefit policies. When I finished at the end of the
day, I backed up my information.

When I logged on today, there were only the first 8 employees I posted
and only 5 time off reasons.  I tried to do a restore from my back up
and it just backed it up to the same point of 8 employees.  Is there any
way to tell if my info is still there?   What would cause this?

....

A backup consists of closing all the tables in the database, and
performing a ZIP of the data files.

--
Gordon Turner
Mycroft Computing
http://www.mycroftcomputing.com
Mon, Nov 5 2007 9:01 AMPermanent Link

Gordon Turner
Robert wrote:
> Never say never, but it does not sound at all like a DBISAM issue. Do you
> have a filter that was not reset, have you looked at the table with DBSYS,
> is it possible that you were doing the updates inside a transaction and you
> cancelled instead of comitting?

Thanks for the suggestions, but this is a program I've been selling for
several years - 1000+ customers, and this is only the second customer of
this app complaining about lost data.  And although entering an Employee
into the system involves a transaction, the failure to commit would not
account for some records being saved and others not.  (The user only
clicks on an OK button to save the data.)

--
Gordon Turner
Mycroft Computing
http://www.mycroftcomputing.com
Mon, Nov 5 2007 9:20 AMPermanent Link

Gordon Turner
Sorry for the double response, but I forgot to add some things writing
late last night...

Tim Young [Elevate Software] wrote:
>
> As Dave indicated - is there any capability to use multiple databases ?
> That is usually the most likely culprit besides improper shutdowns or Vista
> issues with protected directories (which doesn't apply).

The app has multi-user capabilities, but the customer says the data
files are located on her PC, which would mean only a single user.  This
is born out by only one record in the Users table.  (The app requires a
record in the Users table for each person accessing the data with the app.)

> I can assure 100% that DBISAM is not responsible for this issue.  It does
> not "lose" data.

I wasn't thinking DBISAM itself lost the data so much as perhaps it was
a caching issue or a corruption problem.  Sorry if I wasn't clear about
this.  The app tests for corruption by examining the TextIndexStopWords
property before opening the tables and if an of a number of corruption
EDBISAMEngine errors occurs, attempts a repair of the data files.

The session component has its LockProtocol set to lpPessimistic and the
setup program sets the following HKEY_LOCAL_MACHINE values...

System\CurrentControlSet\Services\LanmanServer\Parameters
  CachedOpenLimit = 0
  EnableOpLocks = 0
System\CurrentControlSet\Services\LanmanWorkstation\Parameters
  UseOpportunisticLocking = 0
  UtilizeNtCaching = 0

The customer is running Windows XP Pro SP2.

--
Gordon Turner
Mycroft Computing
http://www.mycroftcomputing.com
Mon, Nov 5 2007 9:27 AMPermanent Link

"Robert"

"Gordon Turner" <gordon@mycroftcomputing.com> wrote in message
news:A6057034-B4A0-4588-9AC7-4F969C0EA555@news.elevatesoft.com...
> Robert wrote:
>> Never say never, but it does not sound at all like a DBISAM issue. Do you
>> have a filter that was not reset, have you looked at the table with
>> DBSYS, is it possible that you were doing the updates inside a
>> transaction and you cancelled instead of comitting?
>
> Thanks for the suggestions, but this is a program I've been selling for
> several years - 1000+ customers, and this is only the second customer of
> this app complaining about lost data.  And although entering an Employee
> into the system involves a transaction, the failure to commit would not
> account for some records being saved and others not.  (The user only
> clicks on an OK button to save the data.)
>

Careful, once you assume that it is a DBISAM problem, you stop looking for
the possible real cause. DBISAM is in use in many places, and it does not
lose data. The failure to commit could VERY WELL account for the situation
you described. They entered one group without problems, the second group
failed 100%. It is possible something went wrong with an update, with
opening the tables, who knows, some flag was set incorrectly and, from that
point on, every transaction on the second update  was cancelled intead of
being committed. IOW, debugging 101, "if I wanted to make this happen, how
would I do it?"

Have you followed carefully and without prejudices every path that can lead
to cancelling (or not committing) the transaction?

Robert


Mon, Nov 5 2007 1:11 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Gordon,

<< But that would only cover loosing all data, not later entries to the
current data files.  To loose later entries would require changing userIDs
twice. >>

You are correct.

<< And how would the older entries appear in the new set of data files? >>

They wouldn't, given the scenario that you outline.

--
Tim Young
Elevate Software
www.elevatesoft.com

Page 1 of 3Next Page »
Jump to Page:  1 2 3
Image