Icon View Thread

The following is the text of the current message along with any replies.
Messages 11 to 16 of 16 total
Thread IMPORT TABLE performance
Thu, Apr 23 2009 7:39 AMPermanent Link

Dan
<<ALTER TABLE MyTable
MAX ROW BUFFER SIZE 1048576

That will permit ElevateDB to use more memory for buffering rows during the
import.>>

In fact, less memory gives more performance in my case. The best setting was 16k.
Thu, Apr 23 2009 5:05 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Dan,

<< In fact, less memory gives more performance in my case. The best setting
was 16k. >>

That's possible, especially with such a small row size.

--
Tim Young
Elevate Software
www.elevatesoft.com

Fri, Apr 24 2009 4:37 AMPermanent Link

Dan
I agree that skipping both CSV format and IMPORT, and write the data directly
to the file in ElevateDB table format will cause the code to be exposed to any changes in
ElevateDB.

Reindexing is fast, no problem here.

This bottleneck will dramatically affect the queries/second indicator on the server. The
import is done in a temporary table open by the user. I understand that is not a widely
requested feature and is not a priority for the mainstream ElevateDB developement. But it
is important for my project and if you feel that there is a way to clearly improve
performance in my special case, I prefer to let that be done in the ElevateDB core and not
let my code be exposed to even minor changes in table structure. So if you are interested
and this can be done, we can discuss a special consultancy fee to support the improvement
I"m interested in.
Fri, Apr 24 2009 2:34 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Dan,

<< This bottleneck will dramatically affect the queries/second indicator on
the server. The import is done in a temporary table open by the user. I
understand that is not a widely requested feature and is not a priority for
the mainstream ElevateDB developement. But it is important for my project
and if you feel that there is a way to clearly improve performance in my
special case, I prefer to let that be done in the ElevateDB core and not let
my code be exposed to even minor changes in table structure. So if you are
interested and this can be done, we can discuss a special consultancy fee to
support the improvement I"m interested in. >>

To be honest, I'm not sure how much more I can do with it without it
straying significantly outside of the ElevateDB engine, which is not
something that is desired.

How about this:  can you can send me a project that does a sample import and
that replicates the performance characteristics that you're seeing ?  That
will at least give me something to work with.

--
Tim Young
Elevate Software
www.elevatesoft.com

Mon, Apr 27 2009 6:08 AMPermanent Link

Dan
<<How about this:  can you can send me a project that does a sample import and
that replicates the performance characteristics that you're seeing ?  That
will at least give me something to work with.>>

I'll do that ASAP, thank you very much.
Tue, Apr 28 2009 2:32 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Dan,

<< I'll do that ASAP >>

Thanks,

--
Tim Young
Elevate Software
www.elevatesoft.com

« Previous PagePage 2 of 2
Jump to Page:  1 2
Image