Icon View Thread

The following is the text of the current message along with any replies.
Messages 1 to 5 of 5 total
Thread File size issue
Tue, Jun 12 2007 11:47 PMPermanent Link

Dondi
Greetings :

I have an ASCII CSV file with quotes as the field delimiters. The file
is 2,968,426 KB. I am trying to split the file but I cannot figure out
how big to make each part so that it does not split the last record.

I am using Windows XP Pro SP2 - NTFS.
The record has 44 alpha fields totaling 809 characters.

I am using the Database System Utility to import the records.

I would like to make each part about 1 gig.

I cannot get this right. I know that I am missing something simple here.

Thanks ...
Dom
Wed, Jun 13 2007 4:00 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Dom,

<< I have an ASCII CSV file with quotes as the field delimiters. The file is
2,968,426 KB. I am trying to split the file but I cannot figure out how big
to make each part so that it does not split the last record.

I am using Windows XP Pro SP2 - NTFS. The record has 44 alpha fields
totaling 809 characters.

I am using the Database System Utility to import the records.

I would like to make each part about 1 gig. >>

Why do you need to split the file ?  DBISAM can access and use a file up to
4Gb without large file support turned on.

--
Tim Young
Elevate Software
www.elevatesoft.com

Wed, Jun 13 2007 4:43 PMPermanent Link

Dondi
Tim,

<< Why do you need to split the file ?  DBISAM can access and use a file
up to
4Gb without large file support turned on. >>

That is what I thought. But after many attempts to import the entire
file in the Database System Utility, I kept getting an error message.

"Error 11309 invalid number of fields in import data." I took a screen
shot of the message if you want I can attach it.

Yet when I split the file, the smaller file imports and exports with no
problems. I thought that I needed to recompile the Utility to enable
large file support.

I searched the NG's and found one reference to this issue but you
indicated that it had been resolved.

I am using Version 4.25 (4.25 Build 5) XP Pro SP2 on an NTFS Drive.

Thanks again for your quick response.

Later ...
Dom
Thu, Jun 14 2007 2:55 PMPermanent Link

Dondi
Tim :

I tracked down the problem to an error in the text file. It had an entry
for "O'Hare" as "O"Hare" and the Database System Utility appropriately
threw up the error message I cited in the previous post. Once I
corrected the error I was able to import the file.

In order to load the file in a text editor I had to split it into 39
parts. Each text file ended up as 76,800KB. Yet when I imported the file
the resulting DAT file ended up as 271,435KB. At this rate I do not
believe that I will be able to import the original 2,968,426KB text file
without exceeding the DBISAM file size limitation. Am I missing
something here?

Similarly, when I export the DBISAM file the resulting text file ends as
66,387KB. A big difference from the 271,435KB DAT file.

When I add up the number of characters in the 44 fields of the text file
I get 809, yet the Database System Utility reports the record size as
904. Again, am I doing something wrong?

Thanks for all your help.

Later ...
Dom
Thu, Jun 14 2007 4:26 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Dom,

<< I tracked down the problem to an error in the text file. It had an entry
for "O'Hare" as "O"Hare" and the Database System Utility appropriately threw
up the error message I cited in the previous post. Once I corrected the
error I was able to import the file. >>

Good news, I'm glad it's now working.

<< In order to load the file in a text editor I had to split it into 39
parts. Each text file ended up as 76,800KB. Yet when I imported the file the
resulting DAT file ended up as 271,435KB. At this rate I do not believe that
I will be able to import the original 2,968,426KB text file without
exceeding the DBISAM file size limitation. Am I missing something here? >>

The size of a DBISAM table is completely dependent upon how you've defined
the table structure and how large you've made each column in the table.

<< Similarly, when I export the DBISAM file the resulting text file ends as
66,387KB. A big difference from the 271,435KB DAT file. >>

DBISAM uses fixed-length records, hence there will be rows with a lot of
empty space in them if you have defined them as particularly large string
fields.

<< When I add up the number of characters in the 44 fields of the text file
I get 809, yet the Database System Utility reports the record size as 904.
>>

You can't compare the two - comma-delimited text files use variable-length,
textual rows and DBISAM uses fixed-length, binary records.

--
Tim Young
Elevate Software
www.elevatesoft.com

Image