Icon View Thread

The following is the text of the current message along with any replies.
Messages 1 to 5 of 5 total
Thread Insert Thousands to millions records in one table
Mon, Nov 11 2019 11:05 AMPermanent Link

Javierus

CYBER Informatica Avanzada, S.L.

Hello,
Is there any way of improving the insert speed of such amount of records?

When I need to do that, I create a table and fill it. Exclusive use is not a problem; in fact is desired. I don't need any safety net for that table: if the process fails before completion, the table will be dropped, so no need of any kind of protective measures
Mon, Nov 11 2019 12:14 PMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Javierus


1. Remove any and all indices / constraints
2. Run it in a loop committing every 1000 records or so
3. Play with the various buffering options

Roy Lambert
Mon, Nov 11 2019 1:50 PMPermanent Link

Javierus

CYBER Informatica Avanzada, S.L.

Thanks for your answer, will try that.
Dou you mean opening a transaction for each 1000 records, will be faster than no transaction?
Tue, Nov 12 2019 3:09 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Javierus


My understanding is that if you don't explicitly start and control a transaction yourself each INPUT statement will effectively create its own transaction.


Roy Lambert
Mon, Dec 9 2019 12:55 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Javierus wrote:

<< Is there any way of improving the insert speed of such amount of records?

When I need to do that, I create a table and fill it. Exclusive use is not a problem; in fact is desired. I don't need any safety net for that table: if the process fails before completion, the table will be dropped, so no need of any kind of protective measures >>

Have you tried opening the table exclusively and just appending records to it ?  When you open a table exclusively, ElevateDB will buffer it in a maximal fashion so that it writes out records in batches automatically.

Also, do you want to use a table directly, or SQL ?

Tim Young
Elevate Software
www.elevatesoft.com
Image