Icon View Thread

The following is the text of the current message along with any replies.
Messages 1 to 10 of 17 total
Thread DBISAM rollout issues
Mon, Oct 16 2006 6:39 AMPermanent Link

Paul T. Gardner
Hello,

We are rolling out DBISAM accross some 200 sites, replacing Paradox as back-end to an on site application providing stock management and
ordering etc.

The following are the two key reasons given by the *implementation team* as to why the rollout for DBISAM was stopped: -

1.  DBISAM error message #11276 - the DBISAM server dies on the machine and connection is lost to the database. As a result the application
crashes. This resolved by restarting the machine and in some cases running a data repair (35-40mins).  
2.  DBISAM sites slower than Paradox sites - Some sites have reported that the new database has slowed down the dispensing process, in extreme
cases the PC has frozen. This has occurred on sites with the older PCs, and has been resolved by swapping the machines over with newer ones.

Regarding these two points - the details are reported as...

1. DBISAM error message #11276

1.1  We have investigated all the failures up to last week and have found that the error has occurred more on the older machines then the newer
ones.
1.2  We are currently investigating whether the error is H/W or S/W related, by testing on the two types of PCs we have in the estate.    We have now
found that the problem is software related because the application communications (modem based ordering) and application itself are holding onto
the 100% CPU on the older machines, and are not on the newer ones.
1.3  Swap out the old machines with new ones for the sites with the most issues.  Completed. We found that by moving the site to the new PC we
have not seen DBISAM server die.
1.4  Check what the impact of switching on CPU monitoring on the machines and test the impact.  This slowed the machine down, however it did
prove that the application process were using 100% CPU.
1.5  Check free processes and RAM to establish a baseline on both types of machines in ops test.
1.8  Carry out the above steps on three other sites with high level of failures.

2. DBISAM sites slower than paradox sites

2.1  Conduct a test dispensing on Paradox/DBISAM sites using Testman at a specified time and on a weekday when other processes are running.
Conducted this test on 10 sites and have found that Paradox is faster. Will be doing the same on the other 15 sites soon

Whereas I think the older machines may be suffering more as a result of these issues, as I understand the sites with newer machines are still using
up to 50% CPU, which is still high.  A colleague found the following thread on your website, which started our investigation into memory
management:

'We've investigated some of these cases together with the customers and in all these cases the actual
bottleneck of their apps was not the memory manager, but an extreme number
of thread switches due to un optimized inter-thread communication (race conditions,
dead-locks, ...) which forced the system to switch thread environment far
more often than needed. A very good indication for this is when the CPU usage
in the taskmanager shows 100% and lots of this time is spent in system calls
(the optional red graph). Once we had removed these issues the application
was already performing far superior than before even with the Borland MM
and then also the impact of replacing witht he Nexus MM was clearly visible.'

The application using DBISAM at our sites is written in Delphi, though I personally have not had an in-depth look at the code.  I wanted to present
the information above to you for your thoughts as currently one of the options forwarded to us is to replace all the older machines at the sites.  An
expensive and knee-jerk solution to say the least.

Any help/thoughts appreciated.

Rgds

PTG.
Mon, Oct 16 2006 8:11 AMPermanent Link

adam
Dear Paul,

I cannot make a detailed technical analysis of the problems you quote. I used Paradox
until about 1999 & on swaping to DBISAM found that bringing legacy projects over from
Paradox was a bit complicated, and in the end I developed different ways of handling data
in the DBISAM projects. Overall I saw massive increases in performance & stability with
the switch & would never switch back ... so I am amazed by what you are saying.

However, the systems I was using (back then) were fairly simple, I had never developed
really complex DB Structures with Paradox, so perhaps part of the issue with your
'upgrade' to DBISAM is that you have a more complex DB model & a lot of what I would call
"Paradox specific Code".

You may well have to really dig down in how the code is handling transactions & posting of
data & review / change these for DBISAM.

Also, I am guessing that you are using Client Server in "internet" mode? In a Modem-based
data transfer situation I would say that that is essential.

--

Note that I now work a lot in Africa (I am writing from there right now), often with quite
old & slow computers. DBISAM is always happier & faster on new machines of course, but
generally works OK on older machines. A good sized DB app created with Delphi will consume
at *most* half of the resources as (for instance) Microsoft Word ... even allowing for
Reporting etc.

I hope this is useful.

Adam
Mon, Oct 16 2006 8:19 AMPermanent Link

"R. Tipton"
Paul #11276 = Remote Communication Lost
Are both ports open on the server ?
I take it the 200 connections have been setup.
Have the clients checked their firewalls both
the built in ones on their router + the XP firewall.

I'am guessing here but I think as the server dies
the 200 Delphi apps are eating its memory and
you could try looking in and around  session
settings.
Someone will come along soon and let you know
but please let us know on this one.
Rita


"Paul T. Gardner" <paul@gardner.net> wrote in message
news:365B278F-051D-44F5-B228-0C64A121FD00@news.elevatesoft.com...
> Hello,
>
> We are rolling out DBISAM accross some 200 sites, replacing Paradox as
> back-end to an on site application providing stock management and
> ordering etc.
>
> The following are the two key reasons given by the *implementation team*
> as to why the rollout for DBISAM was stopped: -
>
> 1.  DBISAM error message #11276 - the DBISAM server dies on the machine
> and connection is lost to the database. As a result the application
> crashes. This resolved by restarting the machine and in some cases running
> a data repair (35-40mins).
> 2.  DBISAM sites slower than Paradox sites - Some sites have reported that
> the new database has slowed down the dispensing process, in extreme
> cases the PC has frozen. This has occurred on sites with the older PCs,
> and has been resolved by swapping the machines over with newer ones.
>
> Regarding these two points - the details are reported as...
>
> 1. DBISAM error message #11276
>
> 1.1  We have investigated all the failures up to last week and have found
> that the error has occurred more on the older machines then the newer
> ones.
> 1.2  We are currently investigating whether the error is H/W or S/W
> related, by testing on the two types of PCs we have in the estate. We have
> now
> found that the problem is software related because the application
> communications (modem based ordering) and application itself are holding
> onto
> the 100% CPU on the older machines, and are not on the newer ones.
> 1.3  Swap out the old machines with new ones for the sites with the most
> issues.  Completed. We found that by moving the site to the new PC we
> have not seen DBISAM server die.
> 1.4  Check what the impact of switching on CPU monitoring on the machines
> and test the impact.  This slowed the machine down, however it did
> prove that the application process were using 100% CPU.
> 1.5  Check free processes and RAM to establish a baseline on both types of
> machines in ops test.
> 1.8  Carry out the above steps on three other sites with high level of
> failures.
>
> 2. DBISAM sites slower than paradox sites
>
> 2.1  Conduct a test dispensing on Paradox/DBISAM sites using Testman at a
> specified time and on a weekday when other processes are running.
> Conducted this test on 10 sites and have found that Paradox is faster.
> Will be doing the same on the other 15 sites soon
>
> Whereas I think the older machines may be suffering more as a result of
> these issues, as I understand the sites with newer machines are still
> using
> up to 50% CPU, which is still high.  A colleague found the following
> thread on your website, which started our investigation into memory
> management:
>
> 'We've investigated some of these cases together with the customers and in
> all these cases the actual
> bottleneck of their apps was not the memory manager, but an extreme number
> of thread switches due to un optimized inter-thread communication (race
> conditions,
> dead-locks, ...) which forced the system to switch thread environment far
> more often than needed. A very good indication for this is when the CPU
> usage
> in the taskmanager shows 100% and lots of this time is spent in system
> calls
> (the optional red graph). Once we had removed these issues the application
> was already performing far superior than before even with the Borland MM
> and then also the impact of replacing witht he Nexus MM was clearly
> visible.'
>
> The application using DBISAM at our sites is written in Delphi, though I
> personally have not had an in-depth look at the code.  I wanted to present
> the information above to you for your thoughts as currently one of the
> options forwarded to us is to replace all the older machines at the sites.
> An
> expensive and knee-jerk solution to say the least.
>
> Any help/thoughts appreciated.
>
> Rgds
>
> PTG.
>

Mon, Oct 16 2006 9:30 AMPermanent Link

"David Farrell-Garcia"
If your Dbisam 4 C/S applicatoin is slower then a desktop database like
Paradox then I would think that you have a serious design flaw in your
application.  Have you optimized your app for C/S?


Have you considered:

1. building optimized queries to reduce the amount of data returned?

2. setting dbisam properties to bring data over in chunks?

3. not using lookups (very slow on any C/S system) or re-designd your
apps to cache lookup tables locally?

4 Handled disconnect errors?

5. setting time-outs for clients?

I am sure others can add  to this list.  Generally speaking you should
see a huge performance increase when moving from  Paradox to Dbisam C/S
LAN or WAN applications.  You do have to think in different terms when
it comes to designing your application.

--
David Farrell-Garcia
Whidbey Island Software, LLC
Mon, Oct 16 2006 12:02 PMPermanent Link

Jason Lee
Please list the specs on a typical "older" machine.
Mon, Oct 16 2006 1:29 PMPermanent Link

"Frans van Daalen"

"Paul T. Gardner" <paul@gardner.net> wrote in message
news:365B278F-051D-44F5-B228-0C64A121FD00@news.elevatesoft.com...
> Hello,
>
> We are rolling out DBISAM accross some 200 sites, replacing Paradox as
> back-end to an on site application providing stock management and
> ordering etc.
>
> The following are the two key reasons given by the *implementation team*
> as to why the rollout for DBISAM was stopped: -
>
> 1.  DBISAM error message #11276 - the DBISAM server dies on the machine
> and connection is lost to the database.

Standard dbsrv.exe or custom one? Standard or custom settings?


Mon, Oct 16 2006 3:13 PMPermanent Link

"Lucian"
> Standard or custom settings?

How is that important, which settings? I use only custom settings, am I
in trouble?

Lucian
Mon, Oct 16 2006 3:32 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Paul,

You've presented a lot of information here, some of it helpful, some of it
not.  However, let me see if I can at least address what you've posted:

<< 1. DBISAM error message #11276

1.1  We have investigated all the failures up to last week and have found
that the error has occurred more on the older machines then the newer
ones. >>

This is essentially the surfacing of a direct problem with the TCP/IP
(Winsock) layer.  Any time DBISAM cannot send or receive data as required,
it will issue this error.  More than likely this is a hardware or connection
reliability issue.  However, the 11276 error is *not* a fatal error.  If you
retry the operation, then DBISAM will try and reconnect to the existing
session on the database server and pick up where it left off.

What I need from you to investigate this aspect further is:

1) The database server configuration settings - Connection Timeout, Dead
Session Expiration Time, Max Dead Sessions, etc.  You can find this
information on the second tab page of the Server Administration Utility once
you've logged into the database server as an administrator.

2) The client-side remote TDBISAMSession property settings for the
RemoteTimeout, RemotePing, and RemotePingInterval properties.  Also, whether
you have the OnRemoteTimeout and/or OnRemoteReconnect event handlers defined
for the remote session.


<< 2. DBISAM sites slower than paradox sites >>

Are you comparing DBISAM C/S access with Paradox local/file-server access ?

<< Whereas I think the older machines may be suffering more as a result of
these issues, as I understand the sites with newer machines are still using
up to 50% CPU, which is still high.  A colleague found the following thread
on your website, which started our investigation into memory
management: >>

Actually, you've got things a bit confused:

a) That message was Hannes talking about NexusDB, not DBISAM.
b) Memory manager issues with multi-processor machines show up as
lower-than-expected CPU consumption, not excessive CPU consumption.  This is
caused by the threads constantly being put into wait states by the
processors because they are all blocking on the same resource, be it a
critical section, semaphore, etc.

<< The application using DBISAM at our sites is written in Delphi, though I
personally have not had an in-depth look at the code.  I wanted to present
the information above to you for your thoughts as currently one of the
options forwarded to us is to replace all the older machines at the sites.
An expensive and knee-jerk solution to say the least. >>

You shouldn't have to replace any machines.  However, it would be great if
you could give some more information on what your setup is - C/S or local,
how many users at each installation, what kind of machines and
configuration, what OS, etc.

--
Tim Young
Elevate Software
www.elevatesoft.com

Tue, Oct 17 2006 6:52 AMPermanent Link

"Frans van Daalen"

"Lucian" <thanks@no.spam> wrote in message
news:A200E751-0A7B-4F0A-94EB-22B99FC09531@news.elevatesoft.com...
>> Standard or custom settings?
>
> How is that important, which settings? I use only custom settings, am I
> in trouble?
>
> Lucian

Only if your settings are not good enough for your setup  Smile But Tim has
already stepped in :>>

Tue, Oct 17 2006 9:52 AMPermanent Link

Paul T. Gardner
Guys, firstly - thanks for your swift responses.  I haven't been as quick to get back due to a desktop rollout!  Nothing to do with this issue, however, as it's a head office thing...

Some detail is definitely lacking.  My post was fairly quickly executed as it was mostly a copy/paste of the implementation team report with my own details added in.  In all of this it must be
remembered that the application using DBISAM is a third party product, in which I have not been involved in the development.  So thanks for your patience so far!

A crucial detail I omitted was that the setup is *local* NOT C/S.  Each machine, running W2K has a copy of DBISAM on board.

Adam, it may be that I will have to dig down into the code, but at the moment I have to request this from the developer.  Unfortunately, decisions made higher up the tree have meant we
have chosen the supplier we have and they are not very forthcoming with their code.

David Farrell-Garcia - we are using 3.30.1 as specified by the supplier and it is used in a local setup.  Your points are very interesting and I will forward them to the developers.  
Unfortunately, as you will all become aware, I am a bit 'piggy-in-the-middle' here.

Jason Lee, the older machines are Fujitsu Siemenss Scenic Ds and Ns:

Scenic D Key Features
Processor  Intel Celeron 1 GHz  
Installed Memory  128 MB (SDRAM)  
Operating System  Microsoft Windows ME  
Processor Type  Intel Celeron  
Processor Speed  1 GHz  
Processor Manufacturer  Intel  
Max Processors Qty.  1  
Motherboard
Bus Speed  100 MHz  
Video Output Interface  AGP  
Memory
RAM Technology  SDRAM  
Installed RAM  128 MB  
Installed Video Memory  4 MB  
Installed Cache Memory  128 KB  
Hard Drive
Hard Drive Capacity  20 GB  
Hard Drive Interface  DMA/ATA-66 (Ultra)  
CD / DVD
Optical Drive Type  CD-ROM  
Optical Drive Read Speed  48x (CD)  
Other Drives
Floppy Drive  3.5" 1.44 MB floppy  
Audio / Video
Audio Output Type  Sound Card  
MPN  D810E-E8F  
Product ID  21095711  

Scneic N Key Features
Processor  Intel Pentium 4 2.6 GHz  
Chipset  Intel 845GE  
Operating System  Microsoft Windows 2000  
Recommended Use  Corporate Business, Small Business  
Processor
Processor Type  Intel Pentium 4  
Processor Speed  2.6 GHz  
Processor Manufacturer  Intel  
Socket Type  Socket 478  
Motherboard
Bus Speed  400 MHz  
Memory
RAM Technology  DDR SDRAM  
Max Supported RAM  1 GB  
Number of Memory Slots  2 x DIMMs  
Supported RAM Speeds  266 MHz  
Technical Features
Integrated Input/Output Ports  Serial Port x 1 • RJ45 Lan Port x 1 • PS/2 Mouse x 1 • PS/2 Keyboard x 1 • Parallel Port (ECP/EPP/SPP) x 1  
Expansion Bays  2 x 5.25" (External Access) • 2 x 3.5" (Internal Access) • 1 x 3.5" (External Access)  
Hard Drive
Hard Drive Capacity  40 GB  
Hard Drive Interface  DMA/ATA-100 (Ultra)  
Hard Drive Rotation Speed  5,400 RPM  
Controller Type  DMA/ATA-100 (Ultra)  
CD / DVD
Optical Drive Type  CD-ROM  
Optical Drive Read Speed  48x (CD)  
Other Drives
Floppy Drive  3.5" 1.44 MB floppy  
Audio / Video
Video Out Ports  15 Pin D-Sub VGA port x 1  
Audio Input  Microphone Jack • 1 x Line In  
Audio Output Type  Sound Card • Headphones • Line out  
Integrated Audio  AC97 Audio Codec  
Networking
Networking Type  Integrated 10/100 Network Card  
Data Link Protocol  Ethernet • Fast Ethernet  

Tim, I will need to get more info to you as soon as I can.  At the moment, on the test machine all I can see is that it's v.3.30 running on ports 12005 and 12006.  Also, thanks for pointing out
my misunderstanding of the situation re Nexus/DBISAM - a colleague fowarded the text, so I didn't see it in context.  It would seem it has pointed us in the right direction, however... hence
I'm on this site.  I think the last bit of info that I can give at the moment that you've requested is that there are one to three users at each site, usually one pc inthe area we are talking
about - sometimes two.  All running a local instance of DBISAM linked to this Delphi app.  The machines are Win2K.

As stated, I will get some more details soon, it's just been a while since I've worked on this part of the project and there's alot else going on.  Sometime difficult to see the wood from the
trees!

Thanks again guys.  Much appreciated.
Page 1 of 2Next Page »
Jump to Page:  1 2
Image