Hi,

I am evaluating your product. I am trying to import a large file into a table. But I get the above error message all the time. Its around the same record every time. But when I make a seperate file with just the records that are around the problem zone, they import fine. So it seems it has something to do with the size. I checked the "Support large files" in de server. I do not have any threads, just 1 connection to the server. The file I am trying to import is 1.8Gb, and contains a field that I import in a CLOB. When I skip this field in the import. It works fine.

I disabled Virusscanner and possible interfering software. I use Win7 64bit, tried it on different drives (a SSD and a Striped Raid array).

Any suggestions?

<< I am evaluating your product. I am trying to import a large file into a
table. But I get the above error message all the time. Its around the same
record every time. But when I make a seperate file with just the records
that are around the problem zone, they import fine. So it seems it has
something to do with the size. I checked the "Support large files" in de
server. I do not have any threads, just 1 connection to the server. The file
I am trying to import is 1.8Gb, and contains a field that I import in a
CLOB. When I skip this field in the import. It works fine. >>

Would it be possible for me to download the file from you using Remote
Desktop/Skype/etc. ?  I will need to test it here with the file to see what
is going on.

--
Tim Young
Elevate Software
www.elevatesoft.com

"Tim Young [Elevate Software]" wrote:

<<Would it be possible for me to download the file from you using Remote
Desktop/Skype/etc. ?  I will need to test it here with the file to see what
is going on.>>

I could put the file on our website so you can download it.....I will include the Table structure of the table I import the data into, and the import statement....
I will let you know where to download when I uploaded the file

Kick wrote:

<<I could put the file on our website so you can download it.....I will include the Table structure of the table I import the data into, and the import statement....
I will let you know where to download when I uploaded the file >>

The import stops at record 1772800. But when the CLOB field is smaller it will stop later. So it seems to have samething to do with the CLOB field, as I mentioned earlier when I do not import this field it works fine.

You can download the file from www.enk.nl\tmp\Artlev-1.zip

The table structure:

CREATE TABLE "artikelen" (
 "mutatiecode" varchar(1) DEFAULT NULL,
 "artikelcodeleverancier" varchar(20) NOT NULL,
 "glnleverancier" varchar(13) NOT NULL ,
 "gtin" varchar(14) DEFAULT NULL,
 "ingangsdatumartikel" date DEFAULT NULL,
 "codebestelbaarheid" varchar(3) DEFAULT NULL,
 "codeverwerkbaar" varchar(3) DEFAULT NULL,
 "statuscode" varchar(3) DEFAULT NULL,
 "gtinopvolger" varchar(14) DEFAULT NULL,
 "artikelcodeopvolger" varchar(20) DEFAULT NULL,
 "gtinvoorganger" varchar(14) DEFAULT NULL,
 "artikelcodevoorganger" varchar(20) DEFAULT NULL,
 "aantalgebruikseenheden" FLOAT DEFAULT NULL,
 "gebruikseenheid" varchar(3) DEFAULT NULL,
 "glnfabrikant" varchar(13) DEFAULT NULL,
 "productcodefabrikant" varchar(20) DEFAULT NULL,
 "gtinproduct" varchar(14) DEFAULT NULL,
 "artikelcodefabrikantartikel" varchar(20) DEFAULT NULL,
 "gtinfabrikantartikel" varchar(14) DEFAULT NULL,
 "afnamegroep" varchar(17) DEFAULT NULL,
 "cbsnummer" varchar(17) DEFAULT NULL,
 "artikelomschrijving" varchar(70) DEFAULT NULL,
 "codeverpakkingsvorm" varchar(3) DEFAULT NULL,
 "brutogewicht" FLOAT DEFAULT NULL,
 "gewichtseenheid" varchar(3) DEFAULT NULL,
 "hoogteverpakking" FLOAT DEFAULT NULL,
 "lengteverpakking" FLOAT DEFAULT NULL,
 "breedteverpakking" FLOAT DEFAULT NULL,
 "eenheidafmeting" varchar(3) DEFAULT NULL,
 "eenheidafname" varchar(3) DEFAULT NULL,
 "minimumafname" FLOAT DEFAULT NULL,
 "stapgrootteafname" FLOAT DEFAULT NULL,
 "levertijd" INTEGER DEFAULT NULL,
 "eenheidlevertijd" varchar(6) DEFAULT NULL,
 "ingangsdatumprijs" date DEFAULT NULL,
 "codebtwtarief" varchar(1) DEFAULT NULL,
 "btwpercentage" FLOAT DEFAULT NULL,
 "fabrikantprijsvolgen" varchar(3) DEFAULT NULL,
 "brprbewerkingstoeslag" decimal(19,4) DEFAULT NULL,
 "kortingsgroep" varchar(35) DEFAULT NULL,
 "brutoinkoopprijs" decimal(19,4) DEFAULT NULL,
 "prijsper" FLOAT DEFAULT NULL,
 "prijseenheid" varchar(3) DEFAULT NULL,
 "nettoprijs" decimal(19,4) DEFAULT NULL,
 "omrekenfactorprijsbesteleenh" FLOAT DEFAULT NULL,
 "valutacode" varchar(3) DEFAULT NULL,
 "mutatiedatum" date DEFAULT NULL,
 "Keywords" CLOB,
 PRIMARY KEY ("artikelcodeleverancier","glnleverancier")
);

The import statement:
IMPORT TABLE Artikelen FROM "Artlev.Tmp" IN STORE importfiles (Mutatiecode,ArtikelcodeLeverancier,GLNLeverancier,GTIN,IngangsdatumArtikel,CodeBestelbaarheid,CodeVerwerkbaar,Statuscode,GTINOpvolger,ArtikelcodeOpvolger,GTINVoorganger,ArtikelcodeVoorganger,AantalGebruikseenheden,Gebruikseenheid,GLNFabrikant,ProductcodeFabrikant,GTINproduct,ArtikelcodeFabrikantArtikel,GTINfabrikantArtikel,AfnameGroep,CBSNummer,Artikelomschrijving,Codeverpakkingsvorm,BrutoGewicht,Gewichtseenheid,HoogteVerpakking,LengteVerpakking,BreedteVerpakking,EenheidAfmeting,EenheidAfname,MinimumAfname,StapgrootteAfname,Levertijd,EenheidLevertijd,IngangsdatumPrijs,CodeBTWTarief,BTWPercentage,FabrikantprijsVolgen,BrPrbewerkingstoeslag,kortingsgroep,BrutoInkoopprijs,PrijsPer,Prijseenheid,Nettoprijs,OmrekenfactorPrijsBesteleenh,Valutacode,keywords)
DELIMITER CHAR #9
QUOTE CHAR #14
DATE FORMAT 'yyyy-mm-dd'

Kick,

<< You can download the file from www.enk.nl\tmp\Artlev-1.zip >>

Thanks, I'm downloading right now and should be able to tell you what is
going on shortly.

--
Tim Young
Elevate Software
www.elevatesoft.com

Kick,

<< The import stops at record 1772800. But when the CLOB field is smaller it
will stop later. So it seems to have samething to do with the CLOB field, as
I mentioned earlier when I do not import this field it works fine. >>

How big is the largest CLOB column value that you're importing, and what is
the average size of the CLOB column values (approximately) ?  Basically what
is happening is that the virtual address space is being exhausted, most
likely due to very large CLOB columns being imported.

I can't look at this information myself because I can't find a hex/text
editor that can load a 1.7GB text file.  My existing hex editor keeps
choking on it.

--
Tim Young
Elevate Software
www.elevatesoft.com

Tim,

I now checked the whole thing again with your fix. It works like a charm! It now processes the file without problems, after loading the file, it contains 6,500,000 records. Total data generated is app. 8,7Gb. It loads in 489secs!!!! That must be some kind of record! Haven't seen that kind of speed on any DB.

Thanks!

Kick,

<< I now checked the whole thing again with your fix. It works like a charm!
It now processes the file without problems, after loading the file, it
contains 6,500,000 records. Total data generated is app. 8,7Gb. It loads in
489secs!!!! That must be some kind of record! Haven't seen that kind of
speed on any DB. >>

Cool, I'm glad that it's working well for you.

--
Tim Young
Elevate Software
www.elevatesoft.com