Gordon,

<< The immediate problem is the Employee.dat table.  The customer says she
entered about 20 employees and when she started the app again, only the
first 8 were still there.  The process of entering an employee does involve
a transaction as it updates multiple tables behind the scenes, so rolling
back a single transaction could explain one missing employee, but not 10 -
10 separate transactions. >>

There's something screwy with regard to the files in the .zip file that you
posted.  Most specifically, the employee.dat file has a timestamp that is
*later* than the employee.idx file by about 12 minutes.  This is impossible,
since the only way to modify the .idx is to update the contents of the .dat
by updating a record.   Also, I could see if there was a slight difference
due to the commit time required for one or the other, but 12 minutes is way
beyond what would be expected.

So, I guess my next question would be - are the timestamps in the .zip
accurate ?   Did she stop her updates by 12:09 PM  on the 29th of October ?

--
Tim Young
Elevate Software
www.elevatesoft.com

Robert wrote:
>
> Careful, once you assume that it is a DBISAM problem, you stop looking for
> the possible real cause. DBISAM is in use in many places, and it does not
> lose data. The failure to commit could VERY WELL account for the situation
> you described. They entered one group without problems, the second group
> failed 100%. It is possible something went wrong with an update, with
> opening the tables, who knows, some flag was set incorrectly and, from that
> point on, every transaction on the second update  was cancelled intead of
> being committed. IOW, debugging 101, "if I wanted to make this happen, how
> would I do it?"
>
> Have you followed carefully and without prejudices every path that can lead
> to cancelling (or not committing) the transaction?

The user has the option to either click on an OK button to save the data
or a Cancel button to not save the data.  Either way, they are returned
to a screen that lists all of the records in the table.  If the
transaction failed for some reason, an error message would be displayed,
or an exception dialog would be displayed if it was an unhandled error.
(I use EurekaLog to help track these.)  Repeated rollbacks would show up
as entries not being displayed on the screen that lists the data in the
table, not in them being listed and then later being missing.

If the user is reporting their experience accurately (and that IS an
assumption) then data was entered and upon starting the program again,
was no longer there.  Other, highly unlikely causes might be some
Windows process that restored the state of the data files to a previous
state, or some networking process performing the same thing (as, by
default, the data files are stored in CSIDL_COMMON_APPDATA/app/data).

Short of going behind the scenes and deleting the records via DBSYS (and
also updating and deleting records from associated tables) I have no
idea how the user could delete records without knowing they were
deleting them.
--
Gordon Turner
Mycroft Computing
http://www.mycroftcomputing.com

Gordon,

> I spent most of the day inputting information.  By the end of the day, I
> had about 21 employees with 1 ½ yrs of information, many time off reasons
> and their benefit policies. When I finished at the end of the day, I
> backed up my information.
>
> When I logged on today, there were only the first 8 employees I posted and
> only 5 time off reasons.  I tried to do a restore from my back up and it
> just backed it up to the same point of 8 employees.  Is there any way to
> tell if my info is still there?   What would cause this?

Do you have any audit trail behind the scenes that tracks what the user does
and when, more specifically when they make a backup or do a restore?   If
so, check the audit trail and I'll bet that there is a restore in there
rather than a backup.

--

---------------------------------------
 Terry Swiers
 Millennium Software, LLC
 http://www.1000years.com
 http://www.atrex.com

 Atrex Inventory Control/POS -
    Big business features without spending big business bucks!

Atrex Electronic Support Options:
 Atrex Knowledgebase: http://www.atrex.com/atrexkb.asp
 Email: support@atrex.com
 Newsgroup: news://news.1000years.com/millennium.atrex
 Fax: 1-925-829-1851
 Phone: 1-925-828-5892 (M-F, 9a-5p Pacific)
 ---------------------------------------

Gordon,

Terry's suggestion is by far the best explanation so far:
<< check the audit trail and I'll bet that there is a restore in there
rather than a backup. >>

And I can confirm that this scenario does happen, because I have done  
exactly this myself - not once but twice! It happened while "backing up" our
in-house accounting system. In my case it occurred because the two icons
(Backup and Restore) were side by side on the desktop.

After that I put in two "Are you sure?" messages on the Restore process...

Regards,
Dan

Dan Rootham wrote:
> Gordon,
>
> Terry's suggestion is by far the best explanation so far:
> << check the audit trail and I'll bet that there is a restore in there
> rather than a backup. >>
>
> And I can confirm that this scenario does happen, because I have done  
> exactly this myself - not once but twice! It happened while "backing up" our
> in-house accounting system. In my case it occurred because the two icons
> (Backup and Restore) were side by side on the desktop.
>
> After that I put in two "Are you sure?" messages on the Restore process...
>
> Regards,
> Dan
>

Dan,
   I think an even better idea (for people who don't read dialog
windows and click on buttons without understanding what they're doing)
is to put in an edit box that forces the user to type the word "RESTORE"
before it will proceed. Some people are just click happy and this should
slow them down and force them to think.

Dave

Dave,

> Some people are just click happy and this should slow them down and force
> them to think.

"Should" is the operative word there.

We have this type of confirmation in our app along with text that in no
uncertain terms tells them that they will lose any data that was entered
after the backup was created if they continue with the restore.  We actually
had one user call us up and ask for help to get the lost data back and when
asked if he read the warning, he said  "Yes, but I didn't think that it
applied to me.".

Just thought I would share the experience.

--

---------------------------------------
 Terry Swiers
 Millennium Software, LLC
 http://www.1000years.com
 http://www.atrex.com

 Atrex Inventory Control/POS -
    Big business features without spending big business bucks!

Atrex Electronic Support Options:
 Atrex Knowledgebase: http://www.atrex.com/atrexkb.asp
 Email: mailto:support@atrex.com
 Newsgroup: news://news.1000years.com/millennium.atrex
 Fax: 1-925-829-1851
 Phone: 1-925-828-5892 (M-F, 9a-5p Pacific)
 ---------------------------------------

Terry Swiers wrote:
>
> Do you have any audit trail behind the scenes that tracks what the user does
> and when, more specifically when they make a backup or do a restore?   If
> so, check the audit trail and I'll bet that there is a restore in there
> rather than a backup.
>

Unfortunately no.  The restore process does force them to select their
backup file through the standard File Open dialog, so it's not
automatic.  They would have to perform a couple steps before the restore
actually occurred.

Keeping an audit trail of functions is an idea for the future though.
It would also help to track what functions customers use and how often.

--
Gordon Turner
Mycroft Computing
http://www.mycroftcomputing.com

Tim Young [Elevate Software] wrote:
>
> There's something screwy with regard to the files in the .zip file that you
> posted.  Most specifically, the employee.dat file has a timestamp that is
> *later* than the employee.idx file by about 12 minutes.  This is impossible,
> since the only way to modify the .idx is to update the contents of the .dat
> by updating a record.   Also, I could see if there was a slight difference
> due to the commit time required for one or the other, but 12 minutes is way
> beyond what would be expected.
>
> So, I guess my next question would be - are the timestamps in the .zip
> accurate ?   Did she stop her updates by 12:09 PM  on the 29th of October ?
>

Right.  But if the user edited a record without changing the key fields,
I would expect the DAT file to have a later date than the IDX file.  Or
am I missing something here.

I do record locking by physically updating a field in the record, so it
would be possible for the program to update the data in a record
(clearing the lock entry) without the user actually making any changes.
 (The record gets "locked" when it is opened for editing and the lock
gets cleared when the user either Saves or Cancels the record.)

--
Gordon Turner
Mycroft Computing
http://www.mycroftcomputing.com

Hi Gordon,
Just some other things to consider as possible culprits....

Vista virtualization - if the data is stored in the program files directory,
vista plays tricks on you.

System Restore - maybe they went back in time, using system restore?

Other backup/restore programs - are they running any sort of backup at the
system level, and if so, what has it been up to?

Have the user search the whole drive, including "hidden files and
directories" for your data files. ex:  MyTable.DAT

Do they just find the one copy, or are there "others"?

And hey, I really like that idea of having them type "RESTORE" into the
confirmation box...  I'm going to do that. AND leave an audit trail. I'm
thinking of creating an audit trail table, to log backup/restore, repair,
verification results, improper shutdowns, and certain types of deletions. I
suppose this table should be exempt from backup/restore!
--
Chris

Chris,

> I suppose this table should be exempt from backup/restore!

No, actually it shouldn't be exempt from backup and restore or the audit
trail will be out of sync with the data that it is tracking.  What you
should do is to simply log that a restore was done into the restored audit
trail immediately after the restore was completed.  Otherwise your audit
trail will show entries and transactions that are not accounted for in the
data.

One other thing that helps is to add something in the audit trail restore
entry that indicates the date of the original backup that was restored.
That way you can tell them from the audit trail how old of a backup was
restored.

--

---------------------------------------
 Terry Swiers
 Millennium Software, LLC
 http://www.1000years.com
 http://www.atrex.com

 Atrex Inventory Control/POS -
    Big business features without spending big business bucks!

Atrex Electronic Support Options:
 Atrex Knowledgebase: http://www.atrex.com/atrexkb.asp
 Email: mailto:support@atrex.com
 Newsgroup: news://news.1000years.com/millennium.atrex
 Fax: 1-925-829-1851
 Phone: 1-925-828-5892 (M-F, 9a-5p Pacific)
 ---------------------------------------