Icon View Thread

The following is the text of the current message along with any replies.
Messages 11 to 14 of 14 total
Thread Bulk data copy
Fri, Nov 23 2007 12:59 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Paul,

<< This works beautifully with largeish datasets, and I get very similar
times in local and remote modes, but on my largest dataset it always throws
up a EPriviledge exception on the LoadFromStream call (89660 rows x 44
Columns). I know that the docs say not to use this method on large datasets
because you will run out of memory, but I'm checking my memory when this
code is running and It's not running out at all. Any insights into this
issue. I'm, guessing that I'm going to have to try and break up large tables
into smaller bite size chunks, but I'm unsure as to how to proceed on that
front. >>

Are you using the default Delphi memory manager ?   If so, then try using
the FastMM memory manager instead on the client.  It may help alleviate this
issue, which is that the application's address space is becoming severely
fragmented due to the number of re-allocations occurring during the
SaveToStream/LoadFromStream.   This is one of the reasons why the manual
says not to use these methods with large datasets.

--
Tim Young
Elevate Software
www.elevatesoft.com

Mon, Nov 26 2007 11:41 AMPermanent Link

Paul Endersby
Hi Tim,

I had high hopes for FastMM4, but alas it was not to be. I still get the same EPriviledged exception at the same position. Frown

Any other suggestions greatly received.

"Tim Young [Elevate Software]" <timyoung@elevatesoft.com> wrote:

Paul,

<< This works beautifully with largeish datasets, and I get very similar
times in local and remote modes, but on my largest dataset it always throws
up a EPriviledge exception on the LoadFromStream call (89660 rows x 44
Columns). I know that the docs say not to use this method on large datasets
because you will run out of memory, but I'm checking my memory when this
code is running and It's not running out at all. Any insights into this
issue. I'm, guessing that I'm going to have to try and break up large tables
into smaller bite size chunks, but I'm unsure as to how to proceed on that
front. >>

Are you using the default Delphi memory manager ?   If so, then try using
the FastMM memory manager instead on the client.  It may help alleviate this
issue, which is that the application's address space is becoming severely
fragmented due to the number of re-allocations occurring during the
SaveToStream/LoadFromStream.   This is one of the reasons why the manual
says not to use these methods with large datasets.

--
Tim Young
Elevate Software
www.elevatesoft.com

Mon, Nov 26 2007 5:10 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Paul,

<< I had high hopes for FastMM4, but alas it was not to be. I still get the
same EPriviledged exception at the same position. Frown>>

Sorry, but the only way you're probably going to avoid this issue is to cut
down on the size of the stream.

--
Tim Young
Elevate Software
www.elevatesoft.com

Tue, Nov 27 2007 5:44 AMPermanent Link

Mauricio Campana Nonino
Paul Endersby

<<Any other suggestions greatly received.>>

What is the '.Dat'  file size?

Are you using transactions?

I am transfering a 70 Mb file with success here.

Mauricio Campana Nonino
Nonino Software
« Previous PagePage 2 of 2
Jump to Page:  1 2
Image