Tim

<<Are you monitoring the memory via the task manager ?  If so, which statistic
column are you monitoring ?

This is important, because if you're still seeing the same amount of memory
consumption, then most likely it isn't related to the memory stream.>>

I'm monitoring with the task manager, i'm sendng you an image... of which column I'm using it

Tim

I did an small application to do some tests, and also you can replicate the situation.

Please look at it and give me your opinion.

Saul


Small attachments (eg your previous one of 78Kb) I don't think anyone will moan about to much, but 17Mb ones belong either in the binaries ng or direct to Tim. In fact, unless you're wanting other people to look at it, or think they'll benefit from it I would recommend direct to Tim, especially since he's changed the support model.

Remember a good few people are still on dial up and this is very nasty for them.

Roy Lambert [Team Elevate]

Saul,

<< I did an small application to do some tests, and also you can replicate
the situation. >>

Two things:

1) Please don't post attachments of that size in the normal newsgroup.  We
have Binaries newsgroup for that purpose.

2) I cannot replicate what you're seeing with the project.  My page file
usage stays steady or goes up by about 1 meg and then returns to the same
value.  I've run the document load continuously with no difference in the
page file usage.

3) Using the page file usage as a monitor of memory consumption is not
particularly accurate unless something is consuming memory at a runaway rate
that would be reflected in a drastic increase in the page file usage.

--
Tim Young
Elevate Software
www.elevatesoft.com

Time

<<1) Please don't post attachments of that size in the normal newsgroup.  We
have Binaries newsgroup for that purpose.>> Sorry about that..

<<2) I cannot replicate what you're seeing with the project.  My page file
usage stays steady or goes up by about 1 meg and then returns to the same
value.  I've run the document load continuously with no difference in the
page file usage.>> Please run the "open tables" option first and see how the memory goes up about 1 meg, then without close the app please run
the "Load Documents" option and after that run again the "open tables" option again and see the memory please. The problem shows up after you
run the "Load Documents" option and try to open one or more tables doesn't matter the size of them.

Thanks

Saul,

<< Please run the "open tables" option first and see how the memory goes up
about 1 meg, then without close the app please run the "Load Documents"
option and after that run again the "open tables" option again and see the
memory please. The problem shows up after you run the "Load Documents"
option and try to open one or more tables doesn't matter the size of them.
>>

That doesn't make any difference at all, nor should it.  Running the Open
methods against tables that are already open won't do anything at all - it's
a NOP.

--
Tim Young
Elevate Software
www.elevatesoft.com

Tim, we were discussing speed issues in DBISAM over in the ElevateDB
newsgroup. I thought I would move over to this group and create a new
thread.

I have no real, concrete problem at the moment, which is perhaps why you
find my comments a bit confusing. But here are the details.

I have a DBISAM 4/Delphi 7 product that stores files and boxes. It was
written to replace an existing application that was written in MS Access
97. We still use this MS Access 97 application. The reason we wrote the
Delphi app is that we can distribute it to others, whereas the MSAccess
app we cannot. So the first thing the Delphi app has to do is match the
MS Access app for speed. Since we still use this app every day, this is
easy to do.

I have some trouble in matching the speed of MS Access, which is one of
the reasons for this thread. Please note that I have not done exact
benchmarks in each system - it is more a 'user-feeling' thing.

- The Problem -
Most of our customers (as in our in-house system) runs with tables of
around 100,000 items with 2 or 3 online users at any one time max. It is
usually one 1 user. In fact, this is where my problem comes in - when we
have 2 or more users, speed drops right off. I have raised this problem
before - it is nothing new. It has been explained as a Windows locking
issue, and I accept that. The thing is, I still have to fix it, or find
some way around it.

To be more specific, my Delphi app is installed on each local
workstation. Each user "points to" a directory on a Windows 2003 server
that contains the DBISAM database files. So most users use a "local"
DBISAM database connection. This is where I am having a problem with speed.

I also have a DBISAM server (the exact one that is included with DBISAM)
that I can also run on the Windows 2003 server, pointing to the same
DBISAM database folder mentioned above. Users can then choose to connect
via a 'local' connection, or this C/S connection.

We see very little difference in speed using either method.

- What to do? -
I am starting to think about the speed issue again, and it struck me
that since the multi-user locking problem seems file based, perhaps a
single file database system might be the answer to my problem? I asked
this in another thread, and I am now satisfied that this is NOT a solution.

My next thought was to beef up the C/S server. At the moment I just use
the one provided by ElevateSoft. I implement no server based procedures,
nor do I use transactions, etc. This seems a valid way to go, and I
would appreciate any general comments you have in this area.

BTW - every table has been designed to third normal form. All queries
are optimized using the DBSys program. Indexes are used for every search
field. I do not think I can do much in the database design area. But
certainly if anyone has any suggestions in this area, I would love to
see them.

In any case, with one user on the system, speed is quite acceptable. It
is only when we add one more user that we get problems.


Any thoughts?

Kerry

Switch from local (file-server) to remote (client-server) is much more than
give to the user the oportunit to change it. If we are talking about speed
issues then you must develop the application thinking in a way to fit your
needs in terms of performance.

It is really hard to describe but in some situations transactions will
increase a lot the performance.

When you say that MS-Access is faster than DBISAM, what are you looking ?
Could you be more specific ? DBISAM has some troubles with speed, but if you
know where is the roles then you can avoid them.

Are you using queries or tables ? If queries then you are using RequestLive
"True" ? What about the indexes ? All the queries are optimized ? Are you
using Optimistic locking model ?

Eduardo

"Kerry Neighbour" <kerry@dojitraders.com> wrote in message
news:E68AE120-E907-42E6-8763-2B75C385F2A8@news.elevatesoft.com...
>
> - The Problem -
> Most of our customers (as in our in-house system) runs with tables of
> around 100,000 items with 2 or 3 online users at any one time max. It is
> usually one 1 user. In fact, this is where my problem comes in - when we
> have 2 or more users, speed drops right off. I have raised this problem
> before - it is nothing new. It has been explained as a Windows locking
> issue, and I accept that. The thing is, I still have to fix it, or find
> some way around it.
>
> To be more specific, my Delphi app is installed on each local workstation.
> Each user "points to" a directory on a Windows 2003 server that contains
> the DBISAM database files. So most users use a "local" DBISAM database
> connection. This is where I am having a problem with speed.
>
> I also have a DBISAM server (the exact one that is included with DBISAM)
> that I can also run on the Windows 2003 server, pointing to the same
> DBISAM database folder mentioned above. Users can then choose to connect
> via a 'local' connection, or this C/S connection.
>
> We see very little difference in speed using either method.
>

This would seem to contradict the fact that there is a significant decrease
in speed with more than one user due to Windows locking, since AFAIK locking
has no impact with C/S. Or maybe there is more than one issue.

Do you see the same speed hit when you go to more than one user in C/S?

What type of locking do you use in your database (optimistic or
pessimistic)? Are the speed issues evident in queries, opening tables,
posting changes?

For what it's worth, I have customers running the same exact software on
what look like pretty equivalent systems, and one is twice as fast as the
other. One of these days I'll try to figure out why.

Robert

Kerry,

<< Tim, we were discussing speed issues in DBISAM over in the ElevateDB
newsgroup. I thought I would move over to this group and create a new
thread. >>

No problem.

<< Most of our customers (as in our in-house system) runs with tables of
around 100,000 items with 2 or 3 online users at any one time max. It is
usually one 1 user. In fact, this is where my problem comes in - when we
have 2 or more users, speed drops right off. I have raised this problem
before - it is nothing new. It has been explained as a Windows locking
issue, and I accept that. The thing is, I still have to fix it, or find some
way around it. >>

Yes, but you can't "fix" it without possibly increasing the DBISAM buffering
to very large amounts.  The issue is that the 2nd+ user performance is the
*real* performance, and the 1st user performance is simulated by Windows,
whereby all of the updates and reads are occurring locally as much as
possible.  The only reason that Access performs reasonably well in such a
scenario is because it uses a *lot* more memory for buffering in each
instance of Access, just like the BDE.  As I said, you can try bumping up
these engine settings:

http://www.elevatesoft.com/manual?action=mancompprop&id=dbisam4&product=d&version=7&comp=TDBISAMEngine&prop=MaxTableDataBufferSize

http://www.elevatesoft.com/manual?action=mancompprop&id=dbisam4&product=d&version=7&comp=TDBISAMEngine&prop=MaxTableIndexBufferSize

and see if that will help.  Try 32 megs, or 33554432 bytes for each setting
on your most widely-used tables.

However, usually this type of issue indicates a problem with the way the
application is written, such as un-optimized filters and queries that cause
table scans, locates that aren't using indexes and causing table scans, or
something similar.  DBISAM's default buffering settings are designed to be
optimal in terms of memory consumption, but only when things are optimized
properly.  If they aren't, then you'll be pulling a lot more data across
then what the default buffering settings will be able to cache, so you'll
end up pulling the same data across repeatedly.

<< We see very little difference in speed using either method. >>

This is impossible.  As Robert indicated, the C/S product is not subject to
opportunistic locking as long as you have the database tables located on the
same machine as the DBISAM Database Server, and therefore there's no way you
could see the 2nd+ user slowdown that you're seeing with a file-sharing
arrangement.  Is that the case here, or are you trying to access the
database tables from another machine other than the DBISAM Database Server
machine ?

<< My next thought was to beef up the C/S server. At the moment I just use
the one provided by ElevateSoft. I implement no server based procedures, nor
do I use transactions, etc. This seems a valid way to go, and I would
appreciate any general comments you have in this area. >>

You're going to want to use server-side procedures if you have batch
processes that perform a lot of while not eof...next type of processing.
Barring that, you're going to want to at least use the RemoteReadSize
property to optimize how the records are pulled across the wire:

http://www.elevatesoft.com/manual?action=mancompprop&id=dbisam4&product=d&version=7&comp=TDBISAMDataSet&prop=RemoteReadSize

Finally, if you want to send me the application, I can tell you what you
need to change to have it run optimally, and I guarantee that it will run
faster than Access when I'm done.

--
Tim Young
Elevate Software
www.elevatesoft.com