Dave

>     So if you have 100 million records and a max of 1 million rows per
> table, then you have 100 tables. If someone wanted to store 1 billion
> rows you're looking at 1000 tables and 10 billion rows you're looking at
> 10,000 tables.
>

Quite right. We are talking about loading, evaluation and breaking down
into multiple databases/folders. DBISAM can handle quite some. Each
database can contain 4096 tables (and more if you modify DBISAM source).
"In server" access can be to "as many folders" as you want.  This way
data acess to large amounts of data for further processing can be
extremely fast.

>     You brought up an interesting point about the 6 threads. When
> loading in a lot of data the process is disk bound.

We evaluate original phone call data and prepare this data for near real
time reporting, alerting, invoicing etc. Reading the data from file is
about 5% of the process.

 The only advantage I
> see of using multiple threads with multiple tables is to put the tables
> on a separate hard drive and load the data using separate machines. Of
> course you have to have data that is easily separated into distinct
> tables in order to make searches work properly. It's not going to work
> very well if you need to do a full text search on 1000 tables.
>

I have never worked with text search so I can not comment on it. However
breaking data down in logical chunks that can be searched should always
be possible.

Arnd

"Dave Harrison" <daveh_18824@spammore.com> wrote in message
news:E54F97D9-848A-4933-AC1E-529188A95BA4@news.elevatesoft.com...
>
> Rita,
>     >now Google wants EDB.<
>     Huh? Are you saying Google buying ElevateSoft? Is Google switching its
> propietary database over to EDB? Enquiring minds want to know.

No I said DBisam can be faster than Oracle
now Google wants EDB the last part was meant as a joke I missed the
smiley wink like so
Rita

ps Tim would probaly welcome a buyout from Google tho if he could
leak the approach his sales would skyrocket pre takeover it would be
a win win situation.

>     What was the operation that took 70 minutes on SQL Server and only
> 16 seconds on DBISAM? Did you have to denormalize the tables to
> eliminate table joins, was that why it was so slow?
>

Dave,

it was a phone call evaluation process. It had nothing to do with
normalization. We just recently visited a potential customer using an
extremely expensive system (hardware 100.000 $ only) with one of the
famous databases underlying. They were proud of evaluating 10 million
phone calls within 12 hours.

Having said all that. I simply want to make the point that DBISAM and
ElevateDB are extremely powerfull, both do not need to fear any
competition and I think they are some of the best databases available
for developers.

Arnd

"Dave Harrison" <daveh_18824@spammore.com> wrote in message
news:5108C439-C52A-43DC-9536-D25077B1636A@news.elevatesoft.com...

Dave are you not a great fan of DBisam ?

Working examples like Arnd posted are fact.
MSSQL 70 minutes moved to DBisam down
to 10 minutes then optimise the tables and its
down to16 seconds is what others have found
as well.

I'am sorry but I have tried a lot of databases
and I was blown away by the speed DBisam
had over all of them.

Rita (Oracles a toy for rich boys)

Dave,

<< It will take a long long time to add tens of millions of rows to an EDB
table, or DBISAM or Nexusdb, or any other Delphi database for that matter.
They are not really enterprise class databases and Oracle's reign is quite
safe. >>

I hate to say this Dave, but you're really going to be eating your words by
the end of this year when we release the ElevateDB Enterprise Server.

--
Tim Young
Elevate Software
www.elevatesoft.com

Rita wrote:
> "Dave Harrison" <daveh_18824@spammore.com> wrote in message
> news:E54F97D9-848A-4933-AC1E-529188A95BA4@news.elevatesoft.com...
>
>>Rita,
>>    >now Google wants EDB.<
>>    Huh? Are you saying Google buying ElevateSoft? Is Google switching its
>>propietary database over to EDB? Enquiring minds want to know.
>
>
> No I said DBisam can be faster than Oracle
> now Google wants EDB the last part was meant as a joke I missed the
> smiley wink like so
> Rita
>
> ps Tim would probaly welcome a buyout from Google tho if he could
> leak the approach his sales would skyrocket pre takeover it would be
> a win win situation.
>

Tim wouldn't do that. He took a vow of poverty.

A lot of database products are being snatched up lately. Oracle bought
Innodb 2 years ago, then they bought MaxDb a few months back, then Sun
bought MySQL late last year for about a billion. What would Tim do with
a spare billion dollars? He'd probably want to take the weekend off or
something crazy like that!

Dave

Rita wrote:

> "Dave Harrison" <daveh_18824@spammore.com> wrote in message
> news:5108C439-C52A-43DC-9536-D25077B1636A@news.elevatesoft.com...
>
> Dave are you not a great fan of DBisam ?

Actually I think DBISAM is great for tables with a few hundred thousand
rows and would be my choice for a vertical market application.

>
> Working examples like Arnd posted are fact.
> MSSQL 70 minutes moved to DBisam down
> to 10 minutes then optimise the tables and its
> down to16 seconds is what others have found
> as well.

I have spent a lot of time last year evaluating DBISAM, NexusDb, EDB,
and a few others. I even went to the extent of writing my own table
manager to organize the data into 35 tables (each with less than 1
million rows) like Arnd has done. It's speed increase when loading data
was negligible compared with 1 large table. This was due to the index
overhead. EDB didn't prove to be faster at loading data than DBISAM.
Even dropping the indexes and rebuilding them later was too slow.

> I'am sorry but I have tried a lot of databases
> and I was blown away by the speed DBisam
> had over all of them.

You're a woman so you know what they say, Size does matter!
I need a database that can comfortably load 35 to 100 million rows into
a table. I've done that with MySQL and it was quite fast. I wish I could
do it with DBISAM or EDB or NexusDb but their indexing overhead does not
permit it. MySQL is 100x faster at loading data and building indexes
compared to any other database I've tried. A lot of that has to do with
MySQL's memory management (AWE is built in) and you can control the
amount of memory you can assign to indiviudal tasks like indexing etc..
If you can allocate 1gb of RAM to build indexes, it makes a big
difference over databases that are using only 2mb. It's no longer disk
bound and is quite fast by 2 orders of magnitude.

So for now it is MySQL for me. (Unless Tim releases his enterprise
version of EDB with improved memory management).

Dave

Tim Young [Elevate Software] wrote:

> Dave,
>
> << It will take a long long time to add tens of millions of rows to an EDB
> table, or DBISAM or Nexusdb, or any other Delphi database for that matter.
> They are not really enterprise class databases and Oracle's reign is quite
> safe. >>
>
> I hate to say this Dave, but you're really going to be eating your words by
> the end of this year when we release the ElevateDB Enterprise Server.
>

Hey Tim,
    I really hope you can do it. If you're successful I'd even go as
far as erecting a statue of you in front of city hall. It won't be 10
miles high like that of Arthur Dent, but it would be the best snow
sculpture in town.

Dave

"Dave Harrison" <daveh_18824@spammore.com> wrote in message
news:C8BB336B-3801-4559-92CB-49A9AE39AD7F@news.elevatesoft.com...
> A lot of database products are being snatched up lately. Oracle bought
> Innodb 2 years ago, then they bought MaxDb a few months back, then Sun
> bought MySQL late last year for about a billion. What would Tim do with a
> spare billion dollars? He'd probably want to take the weekend off or
> something crazy like that!
>

Dave he could dump Sam and move in with me
Rita

"Dave Harrison" <daveh_18824@spammore.com> wrote in message
news:5CD661FF-5A99-44D9-B29F-64A09FD30738@news.elevatesoft.com...
>     I really hope you can do it. If you're successful I'd even go as far
> as erecting a statue of you in front of city hall. It won't be 10 miles
> high like that of Arthur Dent, but it would be the best snow sculpture in
> town.
>

Leave a little piece sticking out for the piegons to perch on
Rita (naughty mode)