Hi All-
We have a series of applications running with an EDBServer for about a week.  There are a number of issues remaining to be cleared up.  But I'm out of my depth regarding the biggest.  The EDBServer crashes regularly.

Access errors, #9999
I don't know if the addresses help at all.  It's not always the same
Address: 0000000000647150 in edbsrvr.exe; address 0000000000000008
I can provide other access errors with different addresses
The function is the LogEvents are:
 Free Cursor
 Free Statement
 Start Transaction
 Rollback Transaction
 Close Database
Also, Error 9999, Invalid pointer operation
 Rollback Transaction
 Close Database

Runtime error 231 at 0…..40E852
Too many nested exceptions
Could be the result of a query crashing repeatedly.  The user kept trying.
Or repeated access errors.

List index out of bounds (0)

Needless to say, it's a serious situation.

EDBSrvr is 2.28 v1, 64 bit.  Intending to update when things are stable, but can update before if that's advisable.
Question: Can I update 2.28 to the current version, or do I need to go in steps?
Running on a VMWare Windows 2012 VM.
Remote, using the default port
Ansi

Workstations are Windows 10 and 7.  There's also one XP that's attached to production equipment.  It can't reasonably be replaced for the user, but it can be stopped from using the applications and edbserver if it's necessary

Development environment is Delphi 5 running in a Windows XP Virtualbox VM.

There are some things I've requested from our IT providers, but I have no confirmation that all has been done:
 Stop drive cacheing on the 2012 VM
 Stop virus checking for now
 Stop Datto backup or time it for off hours
 And, what I haven't yet requested is to stop defragging until we're stable, or limit it to off hours.

Considering that the code is from Delphi 5, I've considered changing to EDBSrvr 32 bit.
My understanding is it doesn't matter

The timing of the server crashes has varied.  Once a day for a couple of days in a row.  Once every two days after that.  Yesterday a couple of times and again now.  

I can include unfixed errors from the applications if that is of any help

I've seen some references in the newsgroup about closing sessions when exiting an application.  Is this something that is recommended?

Thank you,
Judd
Judd

jkr


This advice is a bit controversial - turn off / comment out all of your exception handling so that you can get a clear picture of the root cause.

This "Too many nested exceptions" suggests to me that you're not handling things correctly - maybe trying to restart something when it crashes from within the except part of the try..except block so that the exception od never cleared.

Roy Lambert

Will do.  That's exactly the area I had been working with before rolling out the system.  I've been trying to catch some particular errors, route messages to myself, and let the program carry on.  I can get rid of those efforts and see where we're at.

Today will be busy.  If I have time.  Otherwise, tomorrow..
Judd

On 10/30/2019 5:54 PM, jkr wrote:
> We have a series of applications running with an EDBServer for about a week.  There are a number of issues remaining to be cleared up.  But I'm out of my depth regarding the biggest.  The EDBServer crashes regularly.
>
> Access errors, #9999

The 9999 error is supposed to be rare and as i recall indicates the
actual error originated outside EDB itself.

How exactly is the EDB Server configured - specifically is everything
local - meaning exe and config and databases etc on local drive and
specified using local file path (I.e. c:\mydata etc) and not using UNC
or other network path or such.

Do all clients connect remotely or do you have mixed mode ?

Do you use global buffered file I/O ?


> EDBSrvr is 2.28 v1, 64 bit.  Intending to update when things are stable, but can update before if that's advisable.
> Question: Can I update 2.28 to the current version, or do I need to go in steps?

Yes you can but there have been few small changes you need to be aware -
see release notes. Good thing is that from 2.28 to latest should be
pretty easy as that introduced number of changes .

I'd start with incident reports :
https://www.elevatesoft.com/incident?category=edb

There have been number of fixes in 2.29-2.31 but see if any of them
apply to you - for example there is a AV fix for transaction rollback in
2.29 and number of fixes for Global File I/O Buffering as we ll as
others in later ones.
Also check the rest to see if there might be any impact on existing code.

> Considering that the code is from Delphi 5, I've considered changing to EDBSrvr 32 bit.
> My understanding is it doesn't matter

No - for remote connections the server being 32/64 does not matter. I
assume you are running the same versions on both side (i.e. 2.28
components in app and 2.28 ebd server)

Raul

On 10/31/2019 10:04 AM, Raul wrote:
>> EDBSrvr is 2.28 v1, 64 bit.  Intending to update when things are
>> stable, but can update before if that's advisable.
>> Question: Can I update 2.28 to the current version, or do I need to go
>> in steps?
>
> Yes you can but there have been few small changes you need to be aware -
> see release notes. Good thing is that from 2.28 to latest should be
> pretty easy as that introduced number of changes .
>
> I'd start with incident reports :
> https://www.elevatesoft.com/incident?category=edb

Forgot to mention that even for 2.28 there are number of newer builds
since build 1 you';re using - 2.28 Build 7 is latest

Raul

Hi Raul-

Server
  Remote, not mixed
  ANSI
  Defaults for session timing parameters.
  Standard NULL: yes
  Caching modules: no
  Buffered I/O: no
  Port: 12010
  Not using UNC paths; mapped drive
 (applications are now stored locally; Fixed issue with “external exception  C0000006)

Sesssion
  Remote, for all clients
  ANSI
  Defaults on Remote tab
  (Except IP address of course)
  Pessimistic file locking, default times
  Row change detection checked
  Force Buffer flushes to disk checked
  Keep tables open checked

Same EDB version everywhere

I’m not sure whether to go direct to 2.31, or perhaps go to the final 2.28 and then 2.31.
I’d rather go to 2.31
I’m not sure how much updating will help, but sooner or later it must be done.
I do think applications are the foremost problem, and I have a good idea what to do with that.
I also see where I’ll need to get a network environment going for testing.  That will be a bit complicated.

As a side note, Force Buffer flushes to disk and Row change detection are not working the way I thought they would.  In the issues with the various versions I do see one or more related to “Force Buffer….”

All help is much appreciated
Judd

On 11/1/2019 9:57 AM, jkr wrote:

> Server
>     Standard NULL: yes

Good - this option has been removed in 2.31 but since you use standard
you're OK  (make sure you open all forms and date modules so dfm's are
updated).

>     Caching modules: no
>     Buffered I/O: no
>     Port: 12010
>     Not using UNC paths; mapped drive

This would be one of the 1st culprits i'd look into ?

Mapped drives can be notoriously finicky - both in terms of Windows
timing them out and any brief network interruptions preventing access to
files and causing weird app level errors.

I would try temporarily moving everything locally just to eliminate this
from being a source of problems.

You can still do things like do backups to mapped drive or such if that
is the reason you use them.


> I’m not sure whether to go direct to 2.31, or perhaps go to the final 2.28 and then 2.31.
> I’d rather go to 2.31

If you're thinking of updating then i would suggest go to 2.31 - number
of useful fixes in 2.31 and anything new would be released as 2.31 anyways.

There are just a few breaking changes and one with most possible impact
(StandardNullBehavior removal) should have no impact for you.

> As a side note, Force Buffer flushes to disk and Row change detection are not working the way I thought they would.  In the issues with the various versions I do see one or more related to “Force Buffer….â€�

Not sure which changes you are referring to but i think most of those
are related to Global File I/O Bufferring which you do not have enabled
(i.e.
https://www.elevatesoft.com/manual?action=viewtopic&id=edb2sql&topic=Buffering_Caching)
- since you're running all remote this might be worthwhile to check out
though.

In regards to "ForceBufferFlush" I'm not sure this is even used with
remote sessions since edbserver is one that does all actual disk i/o and
not clients - i might be totally wrong though on this.

Can you elaborate on what does not work in regards to change detection ?

Raul

<< We have a series of applications running with an EDBServer for about a week.  There are a number of issues remaining to be cleared up.  But I'm out of my depth regarding the biggest.  The EDBServer crashes regularly.

Access errors, #9999
I don't know if the addresses help at all.  It's not always the same
Address: 0000000000647150 in edbsrvr.exe; address 0000000000000008
I can provide other access errors with different addresses
The function is the LogEvents are:
 Free Cursor
 Free Statement
 Start Transaction
 Rollback Transaction
 Close Database
Also, Error 9999, Invalid pointer operation
 Rollback Transaction
 Close Database >>

You might be seeing this issue:

https://www.elevatesoft.com/incident?action=viewaddr&category=edb&release=2.29&incident=4695

Also, you can run the debug version of the EDB Server, which will generate .edbdmp files in the temporary files folder for the user account under which the EDB Server is running whenever there is an unhandled exception like an AV.  Those .edbdmp files have important information in them regarding the call stack, etc. that will pinpoint where the issue is occurring.

However, considering that you're using 2.28, my first suggestion would be to update to 2.31 first and make sure that the problem still exists in 2.31.  You shouldn't have any issues updating to 2.31 as long as you're using the standard NULL behavior, which I believe you are.

Finally, I'm not checking in on the support forums as much right now, so you can follow up with me on this issue via email (support@elevatesoft.com).

Thanks,

Tim Young
Elevate Software
www.elevatesoft.com

Raul wrote:

On 11/1/2019 9:57 AM, jkr wrote:

> Server
>     Standard NULL: yes

>     Not using UNC paths; mapped drive
I would try temporarily moving everything locally just to eliminate this
from being a source of problems.

Our setup may conform to what you're saying, at least in part.
The Database is setup with a UNC path in the EDB Manager
The EDB server is setup with the Config and Temp folders referencing off Drive C of the Windows server where they are located.  I didn't have success trying any other way.
Applications are all stored locally on the workstations.

There are a series of paths to auxiliary folders, all on the same mapped drive.  In fact, all the paths are MappedDrive:\foldername.  
  Backups (still manually triggered), PDF output, common storage for app input like logos, diagrams, etc.
  Also updates to the apps to be copied to the workstations
I think these are in line with what you're suggesting: "You can still do things like do backups to mapped drive or such if that is the reason you use them."

> Can you elaborate on what does not work in regards to change detection ?
The general program flow is Browse Window > Edit Windows
The Browse windows are, with a very few exceptions, read only.
The edit windows work with memory tables containing copies of the data
Saving moves the edited data back to the database

Error 1008, "The row has been modified since last cached...."
Some browse windows are dealing with data that can move from one workstation to another, and then back again to the first. So the change is not surprising.  That the change is not recognized is what is surprising. I think a properly placed Refresh will take care of this.  But I thought and hoped that "change detection" would.

Error 1007  "The row has been deleted since last cached for the table...."
Same with this.  The situation is normal.  But it's not being caught.

I don't think these will be major issues using Refresh.  
I think I also need to review my transactions and have all reads happen within them.

I hope this is clear enough.  I won't see posts on Sat. Which I see has already started.
Thanks
Judd

On 11/2/2019 1:03 AM, jkr wrote:
> On 11/1/2019 9:57 AM, jkr wrote:
>
> Our setup may conform to what you're saying, at least in part.
> The Database is setup with a UNC path in the EDB Manager

Is UNC pointing to remote or local location ?

If you can (meaning data is local) I would suggest switching to local
paths and not UNC. This would eliminate this as a possible culprit.

> There are a series of paths to auxiliary folders, all on the same mapped drive.  In fact, all the paths are MappedDrive:\foldername.
>     Backups (still manually triggered), PDF output, common storage for app input like logos, diagrams, etc.
>     Also updates to the apps to be copied to the workstations
> I think these are in line with what you're suggesting: "You can still do things like do backups to mapped drive or such if that is the reason you use them."

Are these just accessed by the applications or also by edbserver ?

if just apps then it should have no impact on edbserver.


> Error 1008, "The row has been modified since last cached...."
> Some browse windows are dealing with data that can move from one workstation to another, and then back again to the first. So the change is not surprising.  That the change is not recognized is what is surprising. I think a properly placed Refresh will take care of this.  But I thought and hoped that "change detection" would.

Yes refresh should fix this. The 1008 (and 1007) can both be normal in a
multi session setup - EDB will cache some data in each local session and
as long as local cache contains what you need it will not go back to
source data (i.e. server table in your case) for simple "view". it will
do  so in case updates though (including delete).

This in general is desirable for performance purposes and using either
refresh or even handling the errors in code (something like 1007 might
be candidate to handle silently might be an option - not sure of your
use case).

One can do per row locking but that is lot more work in app  (you'd need
to consistently in the app query ServerSessionLocks to see if current
record is locked and if not use LockCurrentRecord to lock it for your
current session and make sure you release it when done etc).

Raul