Icon View Thread

The following is the text of the current message along with any replies.
Messages 1 to 10 of 43 total
Thread Questions on Catalogs
Wed, Oct 22 2008 11:02 AMPermanent Link

"Stefano Monterisi"
Hi Tim,
I have started porting of my applications from DBISAM 4 to EDB; I have some
questions......on catalogs;
1) I have applications that, by design, works with data splitted in several
folders, one foreach company (for ex. COMP0001, COMP0002, etc.);
every folder contain files with the same name (customers, orders, etc), and
one folder for the common tables (for ex. countries, vat, cities, accounts,
etc);
2) I have also applications that, by design,  have folders that contains all
data mixed (for ex. customers0001. dat, orders0001.dat, customers0002.dat
orders0002.dat, etc);
How works catalogs in those situations?  There is one catalog foreach folder
in situation 1) and only one in situation  2) ?
Is possible to copy the catalog across folders in situation 1, for ex., or
every catalog is locked with its data files?
There is a 3) situation, very important for me: A big association  that
collects data from peripheral centers at end of year for big data
processing; Peripherical centers have the same data names in different coded
folder forech (PF0101, RM0201, MI0604);
What problem on catalogs may be rise when the association receive many data
folders with catalogs inside?...and if they mix catalog's files? Smile

My old request on ONE file database is related on the opportunity to name it
different foreach situation; it contain catalog and data transparently for
developer and user, so database0001, database0002 etc. is more simple to
manage.

Thanks in advance,

Stefano Monterisi



Wed, Oct 22 2008 12:29 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Stefano,

<< How works catalogs in those situations? >>

Database directories in EDB work just like they do with DBISAM.  The only
difference is that there is one additional file per database directory, and
that is the database catalog.

<< Is possible to copy the catalog across folders in situation 1, for ex.,
or every catalog is locked with its data files? >>

No.

<< There is a 3) situation, very important for me: A big association  that
collects data from peripheral centers at end of year for big data
processing; Peripherical centers have the same data names in different coded
folder forech (PF0101, RM0201, MI0604); What problem on catalogs may be rise
when the association receive many data folders with catalogs inside?...and
if they mix catalog's files? Smile>>

You cannot mix catalog files, nor can you have more than one catalog file in
the same database directory.  Trying to do so will simply corrupt the
database installation.

<< My old request on ONE file database is related on the opportunity to name
it different foreach situation; it contain catalog and data transparently
for developer and user, so database0001, database0002 etc. is more simple to
manage. >>

Sure, but there's a lot of downsides to a single-file database that far
outweigh the convenience of having a single file that acts like a document.

--
Tim Young
Elevate Software
www.elevatesoft.com

Wed, Oct 22 2008 1:40 PMPermanent Link

"Stefano Monterisi"
Hi Tim,

> << Is possible to copy the catalog across folders in situation 1, for ex.,
> or every catalog is locked with its data files? >>
>
> No.

You intend that is'nt possible to copy catalog across folder?




> << My old request on ONE file database is related on the opportunity to
> name it different foreach situation; it contain catalog and data
> transparently for developer and user, so database0001, database0002 etc.
> is more simple to manage. >>
>
> Sure, but there's a lot of downsides to a single-file database that far
> outweigh the convenience of having a single file that acts like a
> document.
>
You can make it as an optional features, so I can use one file database when
application design required it (so customers cannot manipulate or mix files)
or use traditional multi-files; What's the reason because SQL server,
Firebird and others have one files?
In Dbisam I can add tables to database simply copying them into directory,
that make multi-file approach very useful, but now with catalogs I cannot; I
must use the same logic of a single-file database.
Sorry but now, with EDB catalogs, we have all disadvantage of single-file
database without its advantageSmile
There are margins for improve it?

Thank you, Tim.
Good job;

Stefano Monterisi



> --
> Tim Young
> Elevate Software
> www.elevatesoft.com
>
>

Wed, Oct 22 2008 2:13 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Stefano,

<< You intend that is'nt possible to copy catalog across folder? >>

You can, but you need to make sure that you copy any of the table files
also, or else you will have a situation where the catalog does not match the
table files.

<< You can make it as an optional features, so I can use one file database
when application design required it (so
customers cannot manipulate or mix files) or use traditional multi-files; >>

It's not that simple.  The entire design of a database engine requires that
it use one way or the other.

<< What's the reason because SQL server, Firebird and others have one files?
>>

They don't do mult-user file-sharing access.

<< In Dbisam I can add tables to database simply copying them into
directory, that make multi-file approach very useful, but now with catalogs
I cannot; I must use the same logic of a single-file database. Sorry but
now, with EDB catalogs, we have all disadvantage of single-file database
without its advantageSmile>>

That's simply not true.  How many people that use single-file database
servers/engines do so because they want to copy database files around ?
Probably not that many.  It's probably one of the least important aspects of
a database engine.

The advantages of having database catalogs are numerous - RI, user security,
catalog queries, catalog-level procedures and functions, etc.  If you do
some searching on these newsgroups, I've gone through these advantages many
times, so you can read about them here.

--
Tim Young
Elevate Software
www.elevatesoft.com

Thu, Oct 23 2008 5:02 AMPermanent Link

"Stefano Monterisi"
Tim,

> << You intend that is'nt possible to copy catalog across folder? >>
>
> You can, but you need to make sure that you copy any of the table files
> also, or else you will have a situation where the catalog does not match
> the table files.

I want known if catalog contain only data structure/relations information or
if it is involved directly on data content;
If it manage only data structure, then the same catalog can be used (copied)
into several folder that contain databases with the same exact structure but
with different contents; Otherwise the catalog is related directly with data
contents.....

> << What's the reason because SQL server, Firebird and others have one
> files?
> >>
>
> They don't do mult-user file-sharing access.
>
OK.

> << In Dbisam I can add tables to database simply copying them into
> directory, that make multi-file approach very useful, but now with
> catalogs I cannot; I must use the same logic of a single-file database.
> Sorry but now, with EDB catalogs, we have all disadvantage of single-file
> database without its advantageSmile>>
>
> That's simply not true.  How many people that use single-file database
> servers/engines do so because they want to copy database files around ?
> Probably not that many.  It's probably one of the least important aspects
> of a database engine.

Tim, can I add a table (already filled) to a database in one shot into EDB?
Or I must first create the empty table and then fill records within a loop?
Ok, this now isn't the most important aspect....Smile


>
> The advantages of having database catalogs are numerous - RI, user
> security, catalog queries, catalog-level procedures and functions, etc.
> If you do some searching on these newsgroups, I've gone through these
> advantages many times, so you can read about them here.

I think that catalog is a great advantage, but I intend it as a objecte
that, yes,contains the data structure/relational information, but that is
used by several database that have (MUST HAVE)  the same structure; So I can
have one catalog for 100 folders (for ex.);  I can be very happy with one
catalog file for different databases, because if  I update the catalog, then
all related databases MUST change (automatically?) the structure on
connect....
(Honestly I don't known if Edb already works in this way, I am discovering
it now)

Good job,

Stefano Monterisi

> --
> Tim Young
> Elevate Software
> www.elevatesoft.com
>
>

Thu, Oct 23 2008 5:36 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Stefano

>I think that catalog is a great advantage, but I intend it as a objecte
>that, yes,contains the data structure/relational information, but that is
>used by several database that have (MUST HAVE) the same structure; So I can
>have one catalog for 100 folders (for ex.); I can be very happy with one
>catalog file for different databases, because if I update the catalog, then
>all related databases MUST change (automatically?) the structure on
>connect....
> (Honestly I don't known if Edb already works in this way, I am discovering
>it now)

I don't know if this is even possible (almost certainly not) but I really like the idea. One reason I won't be making as much use as possible of the catalog is that I also have multiple databases all with the same structure so for me its far better to keep the sql that could be stored in the catalog in the .exe. Simply so that I only have to worry about changing in one place rather than 3 or 4. If one catalog could control several databases that would mean I would happily stuff things in there.

Roy Lambert
Thu, Oct 23 2008 1:46 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Stefano,

<< I want known if catalog contain only data structure/relations information
or if it is involved directly on data content;
If it manage only data structure, then the same catalog can be used
(copied) into several folder that contain databases with the same exact
structure but with different contents; Otherwise the catalog is related
directly with data contents..... >>

It only contains metadata (structure) information.  However, the table files
themselves are versioned so that they match the catalog, so you can't just
mix and match table files with catalogs.  IOW, don't start copying them
around and mixing them up.  You'll corrupt your installation and have a mess
on your hands.

<< Tim, can I add a table (already filled) to a database in one shot into
EDB? Or I must first create the empty table and then fill records within a
loop? Ok, this now isn't the most important aspect....Smile>>

A table cannot exist outside of a database in EDB, therefore there's no way
to actually have the instance that you're referring to.

<< I think that catalog is a great advantage, but I intend it as a objecte
that, yes,contains the data structure/relational information, but that is
used by several database that have (MUST HAVE)  the same structure; So I can
have one catalog for 100 folders (for ex.);  I can be very happy with one
catalog file for different databases, because if  I update the catalog, then
all related databases MUST change (automatically?) the structure on
connect.... >>

EDB does not work that way.  The catalog is not just a data dictionary that
stores the structure of the tables, and then the table files *also* have
their own structure that can be compared to the data dictionary.  In EDB,
*only* the catalog has the metadata about a table.  The table files *only*
contain the data, nothing else.  Therefore, it is imperative that EDB make
sure that the table file being accessed matches the catalog via its version
number.  If it doesn't, then EDB must prevent access because otherwise it
would cause all sorts of AVs and errors.

--
Tim Young
Elevate Software
www.elevatesoft.com

Thu, Oct 23 2008 1:50 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Roy,

<< I don't know if this is even possible (almost certainly not) but I really
like the idea. One reason I won't be making as much use as possible of the
catalog is that I also have multiple databases all with the same structure
so for me its far better to keep the sql that could be stored in the catalog
in the .exe. Simply so that I only have to worry about changing in one place
rather than 3 or 4. If one catalog could control several databases that
would mean I would happily stuff things in there. >>

Stefano and you really aren't thinking this through.  There's a lot more to
the database catalog then simply "information about the database".  What you
want is a data dictionary, which is different from an actual database
catalog.  There's nothing preventing you from using a data dictionary in
your application to ensure that the actual database catalog and tables match
your desired structure.

--
Tim Young
Elevate Software
www.elevatesoft.com

Fri, Oct 24 2008 4:59 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Tim

><< I don't know if this is even possible (almost certainly not) but I really
>like the idea. One reason I won't be making as much use as possible of the
>catalog is that I also have multiple databases all with the same structure
>so for me its far better to keep the sql that could be stored in the catalog
>in the .exe. Simply so that I only have to worry about changing in one place
>rather than 3 or 4. If one catalog could control several databases that
>would mean I would happily stuff things in there. >>
>
>Stefano and you really aren't thinking this through. There's a lot more to
>the database catalog then simply "information about the database". What you
>want is a data dictionary, which is different from an actual database
>catalog. There's nothing preventing you from using a data dictionary in
>your application to ensure that the actual database catalog and tables match
>your desired structure.

I believe I am thinking it through. We may be thinking along different paths but that doesn't mean I haven't thought it through. What it does mean, because I don't have the source (probably couldn't understand it if I had), and the published documentation doesn't tell me, is what prevents such an approach. As an example the catalog holding the current autoinc value to be used on the next insert would be a reason for having to maintain a 1:1 relationship. Whilst I accept that version numbers must also match unlike in your post to Stefano I don't see the version number being an obstacle. It might be difficult (but not impossible) to ensure the correct versioning.

Unless I misunderstand how a data dictionary could be used I don't think that would be a solution to what I'd love to see. I'm using more sql for the non-interactive parts of my app and it would be nice to offload the code to the catalog to shrink the size of the .exe. As and when table structures get changed I have no concerns about having to alter those in multiple databases. What I don't want to have to do is maintain program code, even sql, in multiple locations. The code gets changed much more frequently than structure and whilst its equally achievable its something I'd prefer not to have to do. I know that with your scripting capabilities I'll (most of the time) just be able to send out a gargantuan script and get the users to run it but I'll still hate replicated code.

Roy Lambert
Fri, Oct 24 2008 3:36 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Roy,

<< I believe I am thinking it through. We may be thinking along different
paths but that doesn't mean I haven't thought it through. What it does mean,
because I don't have the source (probably couldn't understand it if I had),
and the published documentation doesn't tell me, is what prevents such an
approach. As an example the catalog holding the current autoinc value to be
used on the next insert would be a reason for having to maintain a 1:1
relationship. Whilst I accept that version numbers must also match unlike in
your post to Stefano I don't see the version number being an obstacle. It
might be difficult (but not impossible) to ensure the correct versioning. >>

No, the catalog does not store any "data" such as autoinc values.  Those are
all stored in the table.

What you're missing is that you're talking about an exhaustive metadata
resolution process *every time a database or table is opened*.  It would
have to be that way if we went with a design like what Stefano is
recommending.  Furthermore, it doesn't even begin to deal with how to
reconcile missing RI structural information, and how easy it would be to
break such an RI design.  That's why I'm saying that you're not thinking
this through.

<< Unless I misunderstand how a data dictionary could be used I don't think
that would be a solution to what I'd love to see. I'm using more sql for the
non-interactive parts of my app and it would be nice to offload the code to
the catalog to shrink the size of the .exe. As and when table structures get
changed I have no concerns about having to alter those in multiple
databases. What I don't want to have to do is maintain program code, even
sql, in multiple locations. The code gets changed much more frequently than
structure and whilst its equally achievable its something I'd prefer not to
have to do. I know that with your scripting capabilities I'll (most of the
time) just be able to send out a gargantuan script and get the users to run
it but I'll still hate replicated code. >>

You hate replicated code, but you don't have a problem with replicated
metadata and structural information that could lead to all sorts of issues ?
I'll take the replicated code. Smiley

--
Tim Young
Elevate Software
www.elevatesoft.com

Page 1 of 5Next Page »
Jump to Page:  1 2 3 4 5
Image