Hi,

I'm having problems using the Database.LoadRows method. I have reduced the application down to two statements:

procedure TDatabaseDescendant .LoadData;
begin
 LoadColumns(MyDataset);
 LoadRows(MyDataset);
end;

We are using a custom web server that supplies the JSON needed and if I omit the LoadColumns it all works fine (adding the columns in design time).

When we use the LoadColumns there is no GET request on debugging our web server. Hence the dataset won't have any columns defined and we get a

Error: Persistent load error(At least one column must be defined for the "MyDataset" dataset)

when the LoadRows executes.

Does anyone have an idea?

Regards

Thomas Werner
Å-DATA

On 1/18/2017 4:45 PM, tompaw wrote:
> I'm having problems using the Database.LoadRows method. I have reduced the application down to two statements:
> procedure TDatabaseDescendant .LoadData;
> begin
>   LoadColumns(MyDataset);
>   LoadRows(MyDataset);
> end;


These are both async operations - when "LoadColumns" returns the web
request has not completed yet so when you're calling "LoadRows" your
MyDataset actually has no columns.

Easy way to to test is to place 2 buttons on your form - one triggers
"LoadColumns(MyDataset);" and other "LoadRows(MyDataset);".

This should work fine since usually async operation is matter of 10's or
100's of ms - however this would quickly show you that requests is going
out for both.


> We are using a custom web server that supplies the JSON needed and if I omit the LoadColumns it all works fine (adding the columns in design time).
> When we use the LoadColumns there is no GET request on debugging our web server. Hence the dataset won't have any columns defined and we get a
>  Error: Persistent load error(At least one column must be defined for the "MyDataset" dataset)

I can confirm that LoadColumns does generate a proper server requests
here - using browser debugger and a sample ewb app i i have i see this
request going out in my case:

http://localhost:8080/databases?method=columns&database=MyData&dataset=MyData


Error you're seeing is because "LoadRows" will actually not execute a
call if it sees no columns (and since LoadColumns has not finished yet
there are no columns at that time).

You could insert a dummy column design time so you get past this error
and LoadRows actually executes. however design of tdatabase/tdataset
currently does not really allow this AFAIK.

They both internally set the ColumnLoad property and due to the async
nature of things overwrite each other:
- LoadColumns sets ColumnLoad:=True; and requests is queued
- LoadRows sets ColumnLoad:=True; and requests is queued
- at some point in the future (after web request call completes)
- RequestComplete is called and it checks the "ColumnLoad" property and
then you should see error "excpected "rows" but found "columns"".


Proper way would be to do this as follows:
- call LoadColumns
- wait for this to complete
- call LoadRows


Unfortunately at this time AfterLoad does not fire for LoadColumns so
you might have to find another way to figure out when this is done (like
timer and check ColumnCount property for dataset).

A kludge but i believe Tim is planning to add some additional events for
this - not sure if part of 2.06 though.


Raul

On 1/18/2017 6:28 PM, Raul wrote:
> They both internally set the ColumnLoad property and due to the async
> nature of things overwrite each other:
> - LoadColumns sets ColumnLoad:=True; and requests is queued
> - LoadRows sets ColumnLoad:=True; and requests is queued

This should be :
LoadRows sets ColumnLoad:=False; and requests is queued

Raul

Raul wrote:

On 1/18/2017 6:28 PM, Raul wrote:
> They both internally set the ColumnLoad property and due to the async
> nature of things overwrite each other:
> - LoadColumns sets ColumnLoad:=True; and requests is queued
> - LoadRows sets ColumnLoad:=True; and requests is queued

Thanks Raul!

Actually I discovered that our server did the Ansi to UTF-8 conversion twice and garbled the data returned...

During my debugging I noticed the async behaviour av LoadColumns, I'll try your suggestions. And if Tim fixes EWB so the AfterLoad fires av LoadColumns it's perfect.

Regards

Thomas

Another comment that might help somebody...

During my debugging I discovered that the EWB IDE caches data from database requests differently than Firefox/ IE 11. In some cases when you run a program that generates an error, edit the source an rerun from the IDE it won't request new data from your web server. If you shut down the IDE and start it again it works again.

This is why I initially wondered if the LoadColumns actually works, while debugging our server I never got any GET request.

When I finally tried to start my application from Firefox everything was OK.

This might be due to some problems with CACHE headers, we use the Indy 10 TIdHTTPServer as base with default settings for caching. If anyone has information about how EWB IDE handles caching of database requests it would be helpful....
Regards

Thomas Werner
A-DATA Infosystem AB

I wrote my own server with RealThinClient http controls.  When I send to the browser I include the following in the header responses.

           Cache-Control: must-revalidate, post-check=0, pre-check=0
           Cache-Control: no-cache, no-store

You need to tell the browser not to cache the pages (at least for the ones you don;t want cached).

If you search these newgroups for 'Cache-Control' or 'cache' there are posts.

Do you do that in your Indy Server?

Walter

On 1/20/2017 10:04 AM, tompaw wrote:
> During my debugging I discovered that the EWB IDE caches data from database requests differently than Firefox/ IE 11. In some cases when you run a program that generates an error, edit the source an rerun from the IDE it won't request new data from your web server. If you shut down the IDE and start it again it works again.

it should not - that likely is the browser.

> This might be due to some problems with CACHE headers, we use the Indy 10 TIdHTTPServer as base with default settings for caching. If anyone has information about how EWB IDE handles caching of database requests it would be helpful....

if you're using Indy 10  then minimally you'd want to do something like
this for anything to do with these types of requests (where you don't
want caching) :

AResponseInfo.CacheControl := 'no-cache';
AResponseInfo.Expires := 0;


Raul

<<Raul wrote:
if you're using Indy 10  then minimally you'd want to do something like
this for anything to do with these types of requests (where you don't
want caching) :

AResponseInfo.CacheControl := 'no-cache';
AResponseInfo.Expires := 0;>>


Thanks, this fixed my problem.
Regards

Thomas Werner
A-DATA Infosystem AB

Tom,

<< I'm having problems using the Database.LoadRows method. I have reduced the application down to two statements:

procedure TDatabaseDescendant .LoadData;
begin
 LoadColumns(MyDataset);
 LoadRows(MyDataset);
end;

We are using a custom web server that supplies the JSON needed and if I omit the LoadColumns it all works fine (adding the columns in design time).  >>

This is a known issue that will be fixed shortly in 2.06.  The problem is that there isn't an OnLoadColumns event handler so that you can chain the event handlers together in terms of intercepting the columns load and then executing the LoadRows method call.

Tim Young
Elevate Software
www.elevatesoft.com