Login ProductsSalesSupportDownloadsAbout |
Home » Technical Support » DBISAM Technical Support » Support Forums » DBISAM General » View Thread |
Messages 11 to 20 of 23 total |
Problems with memory |
Fri, May 23 2008 9:26 AM | Permanent Link |
Saul_fg | Tim
<<Are you monitoring the memory via the task manager ? If so, which statistic column are you monitoring ? This is important, because if you're still seeing the same amount of memory consumption, then most likely it isn't related to the memory stream.>> I'm monitoring with the task manager, i'm sendng you an image... of which column I'm using it Attachments: taskmanager.JPG |
Fri, May 23 2008 11:20 AM | Permanent Link |
Saul_fg | Tim
I did an small application to do some tests, and also you can replicate the situation. Please look at it and give me your opinion. |
Fri, May 23 2008 12:01 PM | Permanent Link |
Roy Lambert NLH Associates Team Elevate | Saul
Small attachments (eg your previous one of 78Kb) I don't think anyone will moan about to much, but 17Mb ones belong either in the binaries ng or direct to Tim. In fact, unless you're wanting other people to look at it, or think they'll benefit from it I would recommend direct to Tim, especially since he's changed the support model. Remember a good few people are still on dial up and this is very nasty for them. Roy Lambert [Team Elevate] |
Fri, May 23 2008 2:07 PM | Permanent Link |
Tim Young [Elevate Software] Elevate Software, Inc. timyoung@elevatesoft.com | Saul,
<< I did an small application to do some tests, and also you can replicate the situation. >> Two things: 1) Please don't post attachments of that size in the normal newsgroup. We have Binaries newsgroup for that purpose. 2) I cannot replicate what you're seeing with the project. My page file usage stays steady or goes up by about 1 meg and then returns to the same value. I've run the document load continuously with no difference in the page file usage. 3) Using the page file usage as a monitor of memory consumption is not particularly accurate unless something is consuming memory at a runaway rate that would be reflected in a drastic increase in the page file usage. -- Tim Young Elevate Software www.elevatesoft.com |
Fri, May 23 2008 2:57 PM | Permanent Link |
Saul_fg | Time
<<1) Please don't post attachments of that size in the normal newsgroup. We have Binaries newsgroup for that purpose.>> Sorry about that.. <<2) I cannot replicate what you're seeing with the project. My page file usage stays steady or goes up by about 1 meg and then returns to the same value. I've run the document load continuously with no difference in the page file usage.>> Please run the "open tables" option first and see how the memory goes up about 1 meg, then without close the app please run the "Load Documents" option and after that run again the "open tables" option again and see the memory please. The problem shows up after you run the "Load Documents" option and try to open one or more tables doesn't matter the size of them. Thanks |
Sat, May 24 2008 3:22 PM | Permanent Link |
Tim Young [Elevate Software] Elevate Software, Inc. timyoung@elevatesoft.com | Saul,
<< Please run the "open tables" option first and see how the memory goes up about 1 meg, then without close the app please run the "Load Documents" option and after that run again the "open tables" option again and see the memory please. The problem shows up after you run the "Load Documents" option and try to open one or more tables doesn't matter the size of them. >> That doesn't make any difference at all, nor should it. Running the Open methods against tables that are already open won't do anything at all - it's a NOP. -- Tim Young Elevate Software www.elevatesoft.com |
Mon, May 26 2008 2:48 AM | Permanent Link |
Kerry Neighbour | Tim, we were discussing speed issues in DBISAM over in the ElevateDB
newsgroup. I thought I would move over to this group and create a new thread. I have no real, concrete problem at the moment, which is perhaps why you find my comments a bit confusing. But here are the details. I have a DBISAM 4/Delphi 7 product that stores files and boxes. It was written to replace an existing application that was written in MS Access 97. We still use this MS Access 97 application. The reason we wrote the Delphi app is that we can distribute it to others, whereas the MSAccess app we cannot. So the first thing the Delphi app has to do is match the MS Access app for speed. Since we still use this app every day, this is easy to do. I have some trouble in matching the speed of MS Access, which is one of the reasons for this thread. Please note that I have not done exact benchmarks in each system - it is more a 'user-feeling' thing. - The Problem - Most of our customers (as in our in-house system) runs with tables of around 100,000 items with 2 or 3 online users at any one time max. It is usually one 1 user. In fact, this is where my problem comes in - when we have 2 or more users, speed drops right off. I have raised this problem before - it is nothing new. It has been explained as a Windows locking issue, and I accept that. The thing is, I still have to fix it, or find some way around it. To be more specific, my Delphi app is installed on each local workstation. Each user "points to" a directory on a Windows 2003 server that contains the DBISAM database files. So most users use a "local" DBISAM database connection. This is where I am having a problem with speed. I also have a DBISAM server (the exact one that is included with DBISAM) that I can also run on the Windows 2003 server, pointing to the same DBISAM database folder mentioned above. Users can then choose to connect via a 'local' connection, or this C/S connection. We see very little difference in speed using either method. - What to do? - I am starting to think about the speed issue again, and it struck me that since the multi-user locking problem seems file based, perhaps a single file database system might be the answer to my problem? I asked this in another thread, and I am now satisfied that this is NOT a solution. My next thought was to beef up the C/S server. At the moment I just use the one provided by ElevateSoft. I implement no server based procedures, nor do I use transactions, etc. This seems a valid way to go, and I would appreciate any general comments you have in this area. BTW - every table has been designed to third normal form. All queries are optimized using the DBSys program. Indexes are used for every search field. I do not think I can do much in the database design area. But certainly if anyone has any suggestions in this area, I would love to see them. In any case, with one user on the system, speed is quite acceptable. It is only when we add one more user that we get problems. Any thoughts? |
Mon, May 26 2008 6:28 AM | Permanent Link |
"Eduardo \(HPro\)" | Kerry
Switch from local (file-server) to remote (client-server) is much more than give to the user the oportunit to change it. If we are talking about speed issues then you must develop the application thinking in a way to fit your needs in terms of performance. It is really hard to describe but in some situations transactions will increase a lot the performance. When you say that MS-Access is faster than DBISAM, what are you looking ? Could you be more specific ? DBISAM has some troubles with speed, but if you know where is the roles then you can avoid them. Are you using queries or tables ? If queries then you are using RequestLive "True" ? What about the indexes ? All the queries are optimized ? Are you using Optimistic locking model ? Eduardo |
Mon, May 26 2008 8:36 AM | Permanent Link |
"Robert" | "Kerry Neighbour" <kerry@dojitraders.com> wrote in message news:E68AE120-E907-42E6-8763-2B75C385F2A8@news.elevatesoft.com... > > - The Problem - > Most of our customers (as in our in-house system) runs with tables of > around 100,000 items with 2 or 3 online users at any one time max. It is > usually one 1 user. In fact, this is where my problem comes in - when we > have 2 or more users, speed drops right off. I have raised this problem > before - it is nothing new. It has been explained as a Windows locking > issue, and I accept that. The thing is, I still have to fix it, or find > some way around it. > > To be more specific, my Delphi app is installed on each local workstation. > Each user "points to" a directory on a Windows 2003 server that contains > the DBISAM database files. So most users use a "local" DBISAM database > connection. This is where I am having a problem with speed. > > I also have a DBISAM server (the exact one that is included with DBISAM) > that I can also run on the Windows 2003 server, pointing to the same > DBISAM database folder mentioned above. Users can then choose to connect > via a 'local' connection, or this C/S connection. > > We see very little difference in speed using either method. > This would seem to contradict the fact that there is a significant decrease in speed with more than one user due to Windows locking, since AFAIK locking has no impact with C/S. Or maybe there is more than one issue. Do you see the same speed hit when you go to more than one user in C/S? What type of locking do you use in your database (optimistic or pessimistic)? Are the speed issues evident in queries, opening tables, posting changes? For what it's worth, I have customers running the same exact software on what look like pretty equivalent systems, and one is twice as fast as the other. One of these days I'll try to figure out why. Robert |
Mon, May 26 2008 11:40 AM | Permanent Link |
Tim Young [Elevate Software] Elevate Software, Inc. timyoung@elevatesoft.com | Kerry,
<< Tim, we were discussing speed issues in DBISAM over in the ElevateDB newsgroup. I thought I would move over to this group and create a new thread. >> No problem. << Most of our customers (as in our in-house system) runs with tables of around 100,000 items with 2 or 3 online users at any one time max. It is usually one 1 user. In fact, this is where my problem comes in - when we have 2 or more users, speed drops right off. I have raised this problem before - it is nothing new. It has been explained as a Windows locking issue, and I accept that. The thing is, I still have to fix it, or find some way around it. >> Yes, but you can't "fix" it without possibly increasing the DBISAM buffering to very large amounts. The issue is that the 2nd+ user performance is the *real* performance, and the 1st user performance is simulated by Windows, whereby all of the updates and reads are occurring locally as much as possible. The only reason that Access performs reasonably well in such a scenario is because it uses a *lot* more memory for buffering in each instance of Access, just like the BDE. As I said, you can try bumping up these engine settings: http://www.elevatesoft.com/manual?action=mancompprop&id=dbisam4&product=d&version=7&comp=TDBISAMEngine&prop=MaxTableDataBufferSize http://www.elevatesoft.com/manual?action=mancompprop&id=dbisam4&product=d&version=7&comp=TDBISAMEngine&prop=MaxTableIndexBufferSize and see if that will help. Try 32 megs, or 33554432 bytes for each setting on your most widely-used tables. However, usually this type of issue indicates a problem with the way the application is written, such as un-optimized filters and queries that cause table scans, locates that aren't using indexes and causing table scans, or something similar. DBISAM's default buffering settings are designed to be optimal in terms of memory consumption, but only when things are optimized properly. If they aren't, then you'll be pulling a lot more data across then what the default buffering settings will be able to cache, so you'll end up pulling the same data across repeatedly. << We see very little difference in speed using either method. >> This is impossible. As Robert indicated, the C/S product is not subject to opportunistic locking as long as you have the database tables located on the same machine as the DBISAM Database Server, and therefore there's no way you could see the 2nd+ user slowdown that you're seeing with a file-sharing arrangement. Is that the case here, or are you trying to access the database tables from another machine other than the DBISAM Database Server machine ? << My next thought was to beef up the C/S server. At the moment I just use the one provided by ElevateSoft. I implement no server based procedures, nor do I use transactions, etc. This seems a valid way to go, and I would appreciate any general comments you have in this area. >> You're going to want to use server-side procedures if you have batch processes that perform a lot of while not eof...next type of processing. Barring that, you're going to want to at least use the RemoteReadSize property to optimize how the records are pulled across the wire: http://www.elevatesoft.com/manual?action=mancompprop&id=dbisam4&product=d&version=7&comp=TDBISAMDataSet&prop=RemoteReadSize Finally, if you want to send me the application, I can tell you what you need to change to have it run optimally, and I guarantee that it will run faster than Access when I'm done. -- Tim Young Elevate Software www.elevatesoft.com |
« Previous Page | Page 2 of 3 | Next Page » |
Jump to Page: 1 2 3 |
This web page was last updated on Saturday, May 4, 2024 at 12:54 AM | Privacy PolicySite Map © 2024 Elevate Software, Inc. All Rights Reserved Questions or comments ? E-mail us at info@elevatesoft.com |