>> For file for each table.

My last sentence should read - One file for each table.

Peter


>I am not sure what this means. Currently a table has a number of files.
>I can see those in a disk drives folder.
>
>Will all those files be hidden within a single file?
>
>Tim Young replied "Yes".
>
>So it has not been said that there will be one file for the whole database.
>
>Quite clearly there will still be many files in a database. For file for each table.

That's one way of interpreting it. I'd go for the one file to rule them all ie it would encompass the configuration, catalog, edbtbl, edbblb, edbidx files that currently constitute a database. I would expect those files to still exist in so shape or form within the one master file so your final statement is liable to be correct BUT and its a very big but they will all be directly managed by ElevateDB not via OS calls.


Roy

On 5/13/2018 7:36 PM, Peter Evans wrote:
> I asked the following :-
>
> I am not sure what this means. Currently a table has a number of files.
> I can see those in a disk drives folder.
>
> Will all those files be hidden within a single file?
>
> Tim Young replied "Yes".
>
> So it has not been said that there will be one file for the whole database.

OK.

However Tim has also said that design plan is for "single file database"
so all interpretations are valid at this point.

(see
https://www.elevatesoft.com/forums?action=view&category=edb&id=edb_general&page=1&msg=14009#14009)

My own interpretation is for a "single file per database" when this
feature comes out eventually

Raul

On 5/14/2018 2:51 AM, Roy Lambert wrote:
> That's one way of interpreting it. I'd go for the one file to rule them all ie it would encompass the configuration, catalog, edbtbl, edbblb, edbidx files that currently constitute a database. I would expect those files to still exist in so shape or form within the one master file so your final statement is liable to be correct BUT and its a very big but they will all be directly managed by ElevateDB not via OS calls.

Definitely interesting how that turns out - one could go with "oinly 1
file" or 1 file per database (and catalog/config/etc is just another
database).

Pure speculation now but once you do that then you do need to manage the
files internally anymore - one can deal with manage blocks/pages of the
database engine and that opens up all kinds in interesting options for
things like locking and concurrency and backup.

Raul

On 5/14/2018 8:56 AM, Raul wrote:
> Pure speculation now but once you do that then you do need to manage the
> files internally anymore - one can deal with manage blocks/pages of the
> database engine and that opens up all kinds in interesting options for
> things like locking and concurrency and backup.

Not enough coffee - this should say

"...do NOT need to manage the files internally anymore - one can deal
with blocks/pages at the database engine level and that opens up all
kinds of interesting options for locking and concurrency and backup."

Raul

Peter,

<< Tim stated, inter alia, that it is only the table files that become one file for that table. So a database will still consist of files corresponding to one each for each table. >>

Sorry for the confusion: no, this is not correct.  By "single file", I mean that (typically) the entire database will be contained in a single file.

<< Why? >>

You can do all sorts of neat stuff with a single-file format that you can't necessarily do with separate files per table:

1) It's easier for the engine to capture database-wide activity for write-ahead logs and other fail-safe measures.

2) The internal structure can be optimized by moving bits around without requiring the developer to do anything.  For example, the database engine can optimize index and BLOB storage in ways that aren't possible with the current format without a proliferation of more physical files.

3) It's much easier for customers to keep track of one file than multiple files when copying/moving databases around.

Tim Young
Elevate Software
www.elevatesoft.com

Lance,

<< Do you forsee any major performance hits as a single file vs as is? >>

No, in fact you should see better performance since the I/O buffering that was just added in 2.28 will become the default primary buffering mechanism (implemented a little differently, of course).

There will still be additional buffer management at the session/table level so that each session isn't needing to go directly to the primary buffer manager for everything, which is in line with Oracle and others that use multiple buffer pools to achieve better buffer locality (and is how EDB has always worked, with 2.28 making the primary buffer management more explicit than just using the OS file system cache). The biggest problem you can run into with a "great big buffer manager to rule them all" in a database engine is that individual sessions/connections tend to "fight" each other by causing each others most recently-accessed data to get ejected from the buffer manager.  Using multiple layers of buffering solves this problem pretty well.

Tim Young
Elevate Software
www.elevatesoft.com

Tim


>3) It's much easier for customers to keep track of one file than multiple files when copying/moving databases around.

You forgot the obvious corollary - its much easier for customers to delete one file than multiple files <vbg>

Roy

Roy,

<< You forgot the obvious corollary - its much easier for customers to delete one file than multiple files <vbg> >>

Absolutely, but that hasn't seemed to be much of a concern to almost every other database format out there, so I'm going to chalk it up to "going along with the crowd"...

Tim Young
Elevate Software
www.elevatesoft.com

Hi Tim,

<< You can do all sorts of neat stuff with a single-file format that you can't necessarily do with separate files per table:

1) It's easier for the engine to capture database-wide activity for write-ahead logs and other fail-safe measures.

2) The internal structure can be optimized by moving bits around without requiring the developer to do anything.  For example, the database engine can optimize index and BLOB storage in ways that aren't possible with the current format without a proliferation of more physical files.

3) It's much easier for customers to keep track of one file than multiple files when copying/moving databases around. >>

Which I first heard about the idea of a single-file database for EDB way back when I didn't really like it.  Now I think it's an amazing idea and am really looking forward to it.

= Steve