Icon View Thread

The following is the text of the current message along with any replies.
Messages 11 to 20 of 31 total
Thread Inserting Record Efficiently
Thu, May 17 2007 4:01 PMPermanent Link

"Robert"

"Norman L. Kleinberg" <nlk11021@yahoo.com> wrote in message
news:741935E3-0FC5-4BC6-9448-F1D0D8ABCA40@news.elevatesoft.com...
>
> I don't need it to be empty, you're correct. What I want, however, is to
> minimize network
> traffic
> so I need this Query, which will be Live, to have to fetch as few records
> as possible.
>

The query will fetch as few records as are needed by the application. It
seems to me that closing and reopening a query will be a lot more expensive
in time and resources.  Not to forget that changing the WHERE clause
probably forces an unprepare and you have to start all over again preparing
the query before you can open.

A live query is just like a tTable, and all the same criteria apply. If you
want to limit the number of records visible in a live query (I don't quite
see why that would be necessary in your case), it is much faster and
efficient to use a filter. DBISAM is amazingly fast in resolving filters,
and in most cases the results even on farly slow networks are instantaneous.

Robert


Fri, May 18 2007 6:15 PMPermanent Link

Norman L. Kleinberg
Roy Lambert <roy.lambert@skynet.co.uk> wrote:

Norman


OK lesson 1

SELECT * INTO "Memory\Fred" FROM BandA WHERE _BoxNo = -100000

Creates a new database in memory with the exact structure as the source. The problem is
the AutoInc. With an existing record it will pick up the existing Autoinc and that will be
preserved when you post. A new record will create its own, but you should be able to set
it to null after the initial post.


Lesson 2 (returning the data to the table) will follow if I ever figure it out Smiley

Roy Lambert


Roy:

Re: Lesson 1:  what happens when you insert a new record? Will _BoxNo = -100000? If not
then, at least from my experiments, when you Post that new record it will disappear.
Unless I am completely missing the point, which I may be. Smile


Norman
Fri, May 18 2007 7:30 PMPermanent Link

Norman L. Kleinberg
>"Robert" <ngsemail2005withoutthis@yahoo.com.ar> wrote:


>"Norman L. Kleinberg" <nlk11021@yahoo.com> wrote in message
>news:741935E3-0FC5-4BC6-9448-F1D0D8ABCA40@news.elevatesoft.com...
>>
>> I don't need it to be empty, you're correct. What I want, however, is to
>> minimize network
>> traffic
>> so I need this Query, which will be Live, to have to fetch as few records
>> as possible.
>>

>The query will fetch as few records as are needed by the application. It
>seems to me that closing and reopening a query will be a lot more expensive
>in time and resources.  Not to forget that changing the WHERE clause
>probably forces an unprepare and you have to start all over again preparing
>the query before you can open.

>A live query is just like a tTable, and all the same criteria apply. If you
>want to limit the number of records visible in a live query (I don't quite
>see why that would be necessary in your case), it is much faster and
>efficient to use a filter. DBISAM is amazingly fast in resolving filters,
>and in most cases the results even on farly slow networks are instantaneous.


>Robert

Robert:

IF you are saying that a Live query with a WHERE clause that limits the records to a small
subset of the entire database is just like a TTable or Live Query with no WHERE clause
then I'm doing a lot of work for nothing and my inexperience is showing. My understanding
was that a Live Query with a small result set (where the record selection is done on the
Server) is quicker than a Live Query with no WHERE clause (if only because you don't need
to pass the entire database across the wire). I want my user to scroll records so I've
decided to use a "canned" query hooked to a grid; when they decide to Insert a record I
then need to obtain a "live" dataset (forgetting for the moment about UpdateObjects). My
choices would be a Live query with no WHERE clause JUST to add one record (or a TTable) or
an empty (or nearly empty query) which can be generated in less time.

Again, however, if you are saying that there's no difference between a SELECT * FROM
LARGE_DATABASE and SELECT * FROM LARGE_DATABASE WHERE (only a few records satisfy), when
dealing with a LIVE query, then I'm ashamed and embarrassed, but wiser. Smile

=NLK=

Sat, May 19 2007 3:56 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Norman

>Re: Lesson 1: what happens when you insert a new record? Will _BoxNo = -100000? If not
>then, at least from my experiments, when you Post that new record it will disappear.
>Unless I am completely missing the point, which I may be. Smile

What my example was doing was creating an in memory table. As such its totally separate from the source table. Essentially you can think of it as creating a new table with the structure of the original table and copying the data to it. As a new table it has no where condition or filter applied to it.

In the case where you are creating a new entry then the table created will have no data so you would have to issue an insert. In the other case you'd issue an edit.

The problem is returning the data to the source table. Its not difficult, just requires a small loop eg, totally untested


If wasinserting then begin
Offset := 1;
sourcetable.insert;
end else begin
Offset := 0;
sourcetable.edit;
end;
for Cntr := Offset to memorytable.fieldCount - 1 do sourcetable.fields[cntr].assign(memorytable.fields[cntr]);
sourcetable.post;

Roy Lambert
Sat, May 19 2007 9:36 AMPermanent Link

Norman L. Kleinberg
>Roy Lambert <roy.lambert@skynet.co.uk> wrote:

>Norman

>>Re: Lesson 1: what happens when you insert a new record? Will _BoxNo = -100000? If not
>>then, at least from my experiments, when you Post that new record it will disappear.
>>Unless I am completely missing the point, which I may be. Smile

>What my example was doing was creating an in memory table. As such its totally separate
from the source table. Essentially you can think of it as creating a new table with the
structure of the original table and copying the data to it. As a new table it has no where
condition or filter applied to >it.

>In the case where you are creating a new entry then the table created will have no data
so you would have to issue an insert. In the other case you'd issue an edit.

>The problem is returning the data to the source table. Its not difficult, just requires a
small loop eg, totally untested


>If wasinserting then begin
>Offset := 1;
>sourcetable.insert;
>end else begin
>Offset := 0;
>sourcetable.edit;
>end;
>for Cntr := Offset to memorytable.fieldCount - 1 do
sourcetable.fields[cntr].assign(memorytable.fields[cntr]);
>sourcetable.post;

>Roy Lambert

Ahh, yes, the INTO clause; see that now and realize that what you were offering was a
"COPY STRUCTURE" analog. Thanks, sorry for missing that.

However, you also alluded to my "problem" (not really a tremendous issue, just a
curiosity). If I create a separate table and then copy the info to the main table (I have
a copy record routine in place already that works if I'm careful) I need to basically
instantiate/open a TDBISAMTable component representing that DataSet. I'm assuming that is
a bandwidth waster and something I shouldn't do, if I'm running remotely and there are a
significant number of records in the Table. So I'm back to SQL Insert and a 105 field
Insert Query (which of course is not the end of the world).

The real possibility remains that having a T(DBISAM)Table representing this remote dataset
is not a big deal and I'm actually wasting more time writing these posts.

In any case, thanks for your patience.

=NLK=
Sat, May 19 2007 9:52 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Norman


With C/S the memory table will be at the server end so I don't know about traffic. You might want to add some sort of monitor just to see what the traffic levels are.

Don't worry about wasting my time, I'm just going round the bend trying to figure out why a bit of my code involving a TMS grid works for one case and not another.

Roy Lambert
Tue, May 22 2007 8:22 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Norman,

<< IF you are saying that a Live query with a WHERE clause that limits the
records to a small subset of the entire database is just like a TTable or
Live Query with no WHERE clause then I'm doing a lot of work for nothing and
my inexperience is showing. >>

That's what he's saying.

<< My understanding was that a Live Query with a small result set (where the
record selection is done on the Server) is quicker than a Live Query with no
WHERE clause (if only because you don't need to pass the entire database
across the wire). >>

That is incorrect.  Only the number of records required for display are
transferred to the client, irregardless of whether you're using a table
straight-up, or a query (of any kind).  You can also control the number of
records transferred in one chunk by using the TDBISAMTable or TDBISAMQuery
RemoteReadSize property:

http://www.elevatesoft.com/dbisam4d7_tdbisamdataset_remotereadsize.htm

--
Tim Young
Elevate Software
www.elevatesoft.com

Tue, May 22 2007 8:23 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Norman,

<< The real possibility remains that having a T(DBISAM)Table representing
this remote dataset is not a big deal and I'm actually wasting more time
writing these posts. >>

You're not wasting your time with these posts since you're going to save
yourself a lot of coding now that you know that you can simply use a
TDBISAMTable and be done with it. Smiley

--
Tim Young
Elevate Software
www.elevatesoft.com

Thu, May 24 2007 11:45 AMPermanent Link

Norman L. Kleinberg
>"Tim Young [Elevate Software]" <timyoung@elevatesoft.com> wrote:

>Norman,

><< IF you are saying that a Live query with a WHERE clause that limits the
>records to a small subset of the entire database is just like a TTable or
>Live Query with no WHERE clause then I'm doing a lot of work for nothing and
>my inexperience is showing. >>

>That's what he's saying.

><< My understanding was that a Live Query with a small result set (where the
>record selection is done on the Server) is quicker than a Live Query with no
>WHERE clause (if only because you don't need to pass the entire database
>across the wire). >>

>That is incorrect.  Only the number of records required for display are
>transferred to the client, irregardless of whether you're using a table
>straight-up, or a query (of any kind).  You can also control the number of
>records transferred in one chunk by using the TDBISAMTable or TDBISAMQuery
>RemoteReadSize property:

>http://www.elevatesoft.com/dbisam4d7_tdbisamdataset_remotereadsize.htm

>--
>Tim Young
>Elevate Software
>www.elevatesoft.com

Tim:

Thanks for chiming in. When you say "that's incorrect" I'm inferring that you mean a
limited number of records are sent over the wire with EITHER Query or Table components (as
opposed to my thinking that it was only the Query).

But that begs a question (sorry if I'm being dense here): when I Open a T(DBISAM)Query
with a WHERE clause requesting, say, ONE record I can see how only ONE record will be
sent. But when I open a T(DBISAM)Table, how are the records limited? Is it that the act of
OPENING a TTable doesn't send ANYTHING over the wire? What happens then when I, say, try
to FIND a record in a TTable? Is the searching done remotely and only the resulting record
sent?

It's clear I'm missing something here but don't know what it is.

Thanks again.


Norman
Thu, May 24 2007 5:17 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Norman,

<< Thanks for chiming in. When you say "that's incorrect" I'm inferring that
you mean a limited number of records are sent over the wire with EITHER
Query or Table components (as opposed to my thinking that it was only the
Query). >>

Correct.  All TDataSet-descendant components in DBISAM work the same way
with respect to how records are fetched from the database server.

<< But that begs a question (sorry if I'm being dense here): when I Open a
T(DBISAM)Query with a WHERE clause requesting, say, ONE record I can see how
only ONE record will be sent. But when I open a T(DBISAM)Table, how are the
records limited? >>

They are limited by two things:

1) How many records are required to populate a data-aware control (i.e. grid
may need 10 records, but an edit will only need 1 at a time)

and

2) The setting of the RemoteReadSize property

whichever of the two is bigger is the number that DBISAM uses to determine
how many records to fetch from the database server.

<< Is it that the act of OPENING a TTable doesn't send ANYTHING over the
wire? >>

By itself, no.  The actual fetching of the records is another
request/response.

<< What happens then when I, say, try to FIND a record in a TTable? Is the
searching done remotely and only the resulting record sent? >>

Correct.

--
Tim Young
Elevate Software
www.elevatesoft.com

« Previous PagePage 2 of 4Next Page »
Jump to Page:  1 2 3 4
Image