Sam


<<Vista should not be leading to any corruption, AFAIK. Just slower perf.>>

I am not talking about corruption in Windows Vista, but in Windows XP.
You said FlushBuffers is useless in XP. If that is true, it explains why we are having
so many cases of corruption in our system.

Mauricio Campana Nonino
Nonino Software

Sam,

<< As we currently understand it, the problem is that the win32 call:
FlushFileBuffers was 'broken' on XP and Win2000, and is now 'fixed' in
Vista. >>

That is absolutely false.   See my other reply for why.

--
Tim Young
Elevate Software
www.elevatesoft.com

Sam,

<< The evidence we have:

  See:
http://groups.google.com/groups/search?hl=en&qt_s=1&q=vista+FILE_FLAG_NO_BUFFERING+

 This discussion of the FILE_FLAG_NO_BUFFERING  flag : >>

That is not the same as the FlushFileBuffers API call.  It is a totally
different thing altogether and refers to files that are opened in a way that
bypasses the Windows file cache entirely.  DBISAM does *not* use that call
at all.

Please do not post information like this without being sure of what it is
that you're posting.  It has the potential to start "rumors" that are not
true and take us forever to swat down for good.

--
Tim Young
Elevate Software
www.elevatesoft.com

Mauricio,

<< "Wow". That explains why we are having many corruption cases, even if we
were using FlushBuffers. Could you post the link where did you found it? >>

It is not true, so forget that it was ever posted.

--
Tim Young
Elevate Software
www.elevatesoft.com

Tim Young,

<<It is not true, so forget that it was ever posted.>>

Well, I have read the links and sincerely I have not found
such evidence in nowhere. Anyway, corrupted tables are
consuming almost 50% of our technical support team.

Mauricio Campana Nonino
Nonino Software

"Mauricio Campana Nonino" <nonino@sercomtel.com.br> wrote in message
news:93BD51ED-C2E7-463F-ACA2-7BA7E22DF8B1@news.elevatesoft.com...
> Tim Young,
>
> <<It is not true, so forget that it was ever posted.>>
>
> Well, I have read the links and sincerely I have not found
> such evidence in nowhere. Anyway, corrupted tables are
> consuming almost 50% of our technical support team.
>

I'm sorry I have not followed this thread, but it is odd that you would have
so many problems with corruption in tables. We had a pesky problem with
DBISAM 3.30 (and only with that version) but other than that, DBISAM seems
to be like the energizer bunny, just keeps going and going. I don't have
large deployments in Vista (my customers are smart and refuse so far to
"upgrade") but we have tested Vista rather extensively here, and other than
the known issues with security and where to place the different folders
etc., it seems to work exactly as it does on XP. We do not, however, do
flashbuffers unless absolutely necessary. Maybe that is the difference.

Could you elaborate a bit on your environment, your programming, the types
of errors?

Robert

Sam,
<<From our research, pre-Vista, the FlushFileBuffers did not necessarily
flush everything to
disk, but on Vista it really really does. So the perf hit is really real.
....>>
<<Many moons ago, when we first implemented FlushBuffers in DBISAM, we did a
perf analysis,
and found that FlushBuffers had almost zero perf hit.>>

I find this very hard to accept.

I remember that before we selected DBISam a few years ago, we too did very
thorough testing on DBISam and one of the things we tested was the affects
of ForceFlushBuffer = true.  Back then we didn't have an XP machine so our
tests were done on Win2000 and Win ME. I remember that depending on the type
of operation, you will suffer huge performance hit if you leave
ForceFlushBuffer = true even on Win2000 and Win ME.

I just had to verify your post so I did another quick test bench using
DBISam V3.30  :

Time in milliseconds to Insert 1000 records programmatically. Database on
local hard disk.

I. XP 32bit : Celeron 2.2GHZ, 736RAM
No Flush Buffers : 312
Table.FlushBuffer at end of operation : 469 (done after posting last record)
Session.ForceFlushBuffer = true : 99703

II. Vista 64bit : Intel Core Duo 1.8GHZ, 2GBRam
No Flush Buffers : 165
Table.FlushBuffer at end of operation : 262 (done after posting last record)
Session.ForceFlushBuffer = true : 68025

As you can see, this just confirms my previous post. You will take a huge
performance hit whether its XP or Vista when you leave ForceFlushBuffers to
true when doing intensive database insert or edit operations. In fact there
is nothing in the results to suggest that it is much slower in Vista than
when compared to XP.  Between no FlushBuffers and Table.FlushBuffers its
hardly noticeable to the end user. It won't take you more than an hour to
verify this yourself and create your own test bench.

Regards
Gregory Sebastian

Some responses:

a) Sorry about the "FILE_FLAG_NO_BUFFERING" mention. I was completely wrong to mention
that. I made an assumption about how DBISAM works internally and did not actually check it.

b) The ForceFlushBuffers session property: We found, when testing on XP,
  that our TRANSACTIONAL performance was almost unaffected. Our transaction
  time is ~1 second per, of which a small small fraction is actual
  DBISAM overhead. So we were good with this.

c) The reason we used ForceFlushBuffers at the session level is that
  some users were hitting our software so hard, that our software
  would hang, and when it was terminated, the DBISAM data would be
  damaged. By using ForceFlushBuffers, this corruption problem was
  solved.

d) The problem is that when ForceFlushBuffers is on at the session level,
  Operations like CREATE TABLE, ALTER TABLE, CREATE INDEX, and mass
  INSERT operations are all HORRIBLY slow. They are 10x slower on
  Vista than on XP.

e) We are now evaluating removing the ForceFlushBuffers session level flag
  and moving to a table.FlushBuffers only on transactional operations.


f) "If that is true, it explains why we are having so many cases
   of corruption in our system."

   Proper use of table.flushbuffers should help you fix this.


I apologize for any confusion I have caused. From the start, my only
goal has been:

1) For the DBISAM guys to explain to us all at a low level "Property X
  in DBISAM is behaving differently (slower) on Vista, and here is why"

  We don't have this yet... oh well.


2) To find a workaround for the problem. It seems we have one now.



Thanks to all!!!

Robert,

First, I want to be clear I understand "table corruptions" are
windows problems, not DBIsam. If I were using any other databases,
I will be having the the same problems, maybe worse ones. Second,
I am not experiencing any problems with Vista yet. Almost no one of
our customers are using it.

>>I don't have large deployments in Vista
>>(my customers are smart and refuse so far to "upgrade")

The corruption problems we are experiencing are not related with Vista,
but with XP and even worse with 98 machines. Just a few customers are
using Vista here too.

>>We do not, however, do flashbuffers unless absolutely necessary.

We are making intensive use of FlushBuffers.

>>Could you elaborate a bit on your environment, your programming,
>>the types of errors?

Sure. We have more than 400 customers. The enviroments are of all
kind: Windows 98, 2000, XP, etc. But  XP is the big one. Currently we
are only using File Share access. We plan to use just CS in the future.
We have discovered many things about the issue:

1) FAT32 is much more problematic than NTFS;
2) Many of the times, the problem only ocurrs when using our system in a LAN;
3) If we disable "write caching on the disk", the problem is minimized;

Thanks,

Mauricio Campana Nonino
Nonino Software

"Mauricio Campana Nonino" <nonino@sercomtel.com.br> wrote in message
news:678778E0-36C3-44AC-B6FC-9600B8E6622A@news.elevatesoft.com...
>
> The corruption problems we are experiencing are not related with Vista,
> but with XP and even worse with 98 machines. Just a few customers are
> using Vista here too.
>

What are the errors? What is the DBISAM version? How do you fix them?

Robert