Hi all,

My DBISAM version is V4.31 build3 (I use it C/S). To be able to see the (remote) DBSRVR memory consumption I have recompiled the DBSRVR to show the actual memory status, see the source code below I found somewhere on the internet.

changes apply to sourcefile dbisamsv.pas

function TDataServer.GetMemoryUsage: double;
var
  AMemoryManagerState: TMemoryManagerState;
  SBTState: TSmallBlockTypeState;
begin
  Result:=0;
  GetMemoryManagerState(AMemoryManagerState);
  with AMemoryManagerState do
     begin
     for SBTState in SmallBlockTypeStates do
        Result:=Result+SBTState.AllocatedBlockCount*SBTState.UseableBlockSize;
     Result:=Result+TotalAllocatedMediumBlockSize;
     Result:=Result+TotalAllocatedLargeBlockSize;
     end;
  Result:=Result/1024;  //Result in kB
end;

The function result works fine and I can see the DBSRVR memory status going up and down as expected (using RemoteSession.GetRemoteMemoryUsage).

But once I execute a query statement with "INTO MEMORY\TEMPTABLE" then after closing the application the final GetMemoryUsage result is always 96,6 kB higher than it was before. How many times I run the query in the client-application does not matter. I do execute a DROP MEMORY\TEMPTABLE after use. Calling RemoveRemoteMemoryTables has no effect on this.

If I execute the client application again and again memory consumption on the server keeps on growing. Even if I use DBSYS to test this situation it has the exact same result.

the simple test SQL I use is

select MyField
into memory\temptable
from MyTable
;
drop table memory\temptable

Only after restarting the DBSRVR the memory consumption it shows is back to 'normal' again. My clients using my application, working all day starting and ending the application experience the same thing and now and then the only 'workaround' is to restart the DBSRVR.

Is there anything I can do to prevent this memory growth, or is the implementation of GetMemoryUsage not correct? Has anyone ever experienced something like this? I don't think it has to do with fragmentation because in my test situation I run the server on my own pc, with only one user.

One question i have is whether the session was actually closed on server when you close your app? When you check dbsrvr session list is it empty once you app disconnects?

i would also suggest checking an unmodified version of the dbsrvr to see if its memory consumption - using something like vmmap (http://technet.microsoft.com/en-us/sysinternals/dd535533).

We've been using dbsrvr for a long time and have not seen this problem in general - dbsrvr does not keep consuming memory.

Raul


<<
J.H. Vis wrote:

Only after restarting the DBSRVR the memory consumption it shows is back to 'normal' again. My clients using my application, working all day starting and ending the application experience the same thing and now and then the only 'workaround' is to restart the DBSRVR.

Is there anything I can do to prevent this memory growth, or is the implementation of GetMemoryUsage not correct? Has anyone ever experienced something like this? I don't think it has to do with fragmentation because in my test situation I run the server on my own pc, with only one user.
>>

Raul, thanks for your response.

I've been testing with the default (out of the box) DBSRVR (V4.30 build4) as you suggested and have checked memory with vmmap.
At first I did not specifically close the RemoteSession in my TestApp, but I've now tested both situations with and without closing and as far as I can see this makes no difference. Actually I don't understand all the figures that vmmap shows, but what I do see is that the Private Data keeps on growing. When I change my TestApp not to create a memory\ result then all figures remain the same. I've added a ScreenPrint of the vmmap results after testing for quite some hours running my TestApp (with creating a memory\ result) over and over again.

Is this PrivateData figure from vmmap something to worry about or do I forget something when closing my app?

Johan,

Private Data AFAIK is the real memory usage (vs Working Set which is
virtual memory allocation and can fluctuate) so if it keeps increasing
then yes i'd say the memory is not being freed.
This does not necessarily indicate a leak though just that something is
not closed until dbsrvr exits.

The 4.30 is an older version (as is 4.31) as we're currently at 4.34 b4.
There are number of fixes in newer version which might help though i'm
not seeing anything obvious in incident reports.

One thing i would suggest is calling RemoveAllRemoteMemoryTables
function (of your remote session) when your app closes/disconnects and
see if it helps :

http://www.elevatesoft.com/manual?action=viewmethod&id=dbisam4&product=d&version=7&comp=tdbisamsession&method=removeallremotememorytables

Raul


On 12/10/2012 3:37 AM, Johan Dorrestijn wrote:
> Raul, thanks for your response.
>
> I've been testing with the default (out of the box) DBSRVR (V4.30 build4) as you suggested and have checked memory with vmmap.
> At first I did not specifically close the RemoteSession in my TestApp, but I've now tested both situations with and without closing and as far as I can see this makes no difference. Actually I don't understand all the figures that vmmap shows, but what I do see is that the Private Data keeps on growing. When I change my TestApp not to create a memory\ result then all figures remain the same. I've added a ScreenPrint of the vmmap results after testing for quite some hours running my TestApp (with creating a memory\ result) over and over again.
>
> Is this PrivateData figure from vmmap something to worry about or do I forget something when closing my app?
>

J.H.,

<< Only after restarting the DBSRVR the memory consumption it shows is back
to 'normal' again. My clients using my application, working all day starting
and ending the application experience the same thing and now and then the
only 'workaround' is to restart the DBSRVR.

Is there anything I can do to prevent this memory growth, or is the
implementation of GetMemoryUsage not correct? Has anyone ever experienced
something like this? I don't think it has to do with fragmentation because
in my test situation I run the server on my own pc, with only one user. >>

More than likely you're just seeing the vagaries of the FastMM4
sub-allocator, specifically situations where larger allocations are not
released back to the VMM because some of the smaller blocks allocated from
the larger VM allocation are not freed yet.  It shouldn’t apply to the
memory file blocks, which are 64KB blocks and should be pass-through
allocated directly from the VMM in Windows, but it may apply to other
aspects of the operations being executed.

However, just to be sure, I checked the memory file deletion for in-memory
tables and everything is definitely getting freed properly.  You can also
test this with this simple test:

Query:

SELECT * INTO "memory\temp" FROM customer;
DROP TABLE "memory\temp";

Code:

procedure TForm1.Button1Click(Sender: TObject);
var
  I: Integer;
begin
  for I:=1 to 1000 do
     begin
     DBISAMQuery1.ExecSQL;
     Label1.Caption:=IntToStr(I);
     Application.ProcessMessages;
     end;
end;

When I run the above with the latest 4.34 B4, I don't see any movement in
the memory allocation using the task manager.

Tim Young
Elevate Software
www.elevatesoft.com

Raul,

I have tested the same situation with the latest version V4.34 B4, but the result remain the same.

Tim,

My test-situation is not quite the same as yours: running the same query within a program for 1000 times indeed does not influence memory allocation. But if you run the query just once, close your app en start the same app again then I see the memory allocation growing. To test this I run the app in a batch-file (my test app runs the query once and closes automatically) like:

:again
MemCheckProject.exe
goto again

Maybe not quite a real-life situation but it shows what I experience. And you could imagine that apps start and close over time, so eventually the effect will be the same I guess.
As for allocated memory not being freed yet: I have waited for some time (more than 1 hr) after the test to see if memory was released, but I didn't see it happen.
All above testing with the latest build V4.34 B4 using Windows7 as Client and using Windows7 or WindowsServer2008 as Server.

Johan Dorrestijn

Johan,

<< Maybe not quite a real-life situation but it shows what I experience. And
you could imagine that apps start and close over time, so eventually the
effect will be the same I guess. As for allocated memory not being freed
yet: I have waited for some time (more than 1 hr) after the test to see if
memory was released, but I didn't see it happen. All above testing with the
latest build V4.34 B4 using Windows7 as Client and using Windows7 or
WindowsServer2008 as Server. >>

So, you're saying that you're *not* seeing the issue when you use the code
that I described in a single process executing the code 1000 times, but if
you execute the same process 1000 times you *do* see the issue, even though
it's using the exact same code ?

You can imagine that I'm a bit skeptical of what you're describing.

Tim Young
Elevate Software
www.elevatesoft.com

No, I see the issue once when executing the code 1000 times in a single process. I see the issue n times when executing the single process n times. It appears process dependant.
Johan Dorrestijn
Dolfijn Software Solutions b.v.

Johan,

<< No, I see the issue once when executing the code 1000 times in a single
process. I see the issue n times when executing the single process n times.
It appears process dependant. >>

Windows will not allow memory to remain allocated for a process past the
process lifetime, so I can't see how there is an issue with the memory after
the process is terminated.

Tim Young
Elevate Software
www.elevatesoft.com

"Tim Young [Elevate Software]" wrote:

<<Windows will not allow memory to remain allocated for a process past the
process lifetime, so I can't see how there is an issue with the memory after
the process is terminated.>>

Yes I know, but I work C/S and the process I meant is the process on the Client workstation. The extra memory that seems to be allocated is what I see on the server machine where DBSRVR resides. If I execute the described process once on the client workstation then I measure a little bit of memory remaining allocated on the server. When I execute this process on the client workstation n times the result of allocated memory on the server is n times as large. During execution of the one single process on the client workstation it does not matter how many MEMORY\ tables I CREATE or DROP - the remaining allocated memory on the server is always the same. Only after restarting the DBSRVR process on the server the allocated memory is freed. At least that is what FastMM or VMMap tells me.
Johan Dorrestijn
Dolfijn Software Solutions b.v.