Icon View Thread

The following is the text of the current message along with any replies.
Messages 31 to 40 of 53 total
Thread How To Perform A Multi-Field Dataset Search?
Mon, Oct 5 2015 11:41 PMPermanent Link

Raul

Globestar Systems

Team Elevate Team Elevate

On 10/5/2015 8:38 PM, Jim Gallagher wrote:
> Thanks, Raul and Matthew.  OK, I think I understand.  Your concerns are with dataset access to server tables.  In my ignorance, I avoid that access method, since I don't have grid-oriented applications, and having a dataset access server data sounds a little scary to me.  Assuming that DBISAM tables are password protected, and are accessed via https: from a module with the user supplying the username and password once only up front, then even local DBISAM access on a server could be made secure.

Absolutely, if you're doing your own server modules then security is as
good as you can make it (assuming you use SSL of course). I'm pretty
sure DBISAM is fine when using DBSRVR in the back as well since it can
do auth.

Dataset access is a really nice feature to have and lets you write EWB
apps very rapidly if needed - and easy to secure if you approach it
right (authenticate all requests with username/pass or session id - i
personally prefer the latter).

Raul
Tue, Oct 6 2015 12:23 AMPermanent Link

Raul

Globestar Systems

Team Elevate Team Elevate

On 10/5/2015 11:04 PM, Frederick Chin wrote:
> DBISAM table is MNPSMAIN.DAT with an index on the NAME column.

Whatever index you may or may not have in DBISAM is not directly
relevant to the EWB - it is not visible nor reflected in any way in EWB
dataset. DBISAM data is turned into a JSON string (representing rows and
columns and sent to EWB - it's no longer DBISAM data when it reaches
EWB, just EWB dataset data).


> Example 1:-
> lOK returns True if name and password combination are in my DBISAM table and False if not. This is a correct result.

> Example 2:-
> With the Sort methods taken out, lOK still returns True if name and password combination are in my DBISAM table and False if not. This is a correct result.

> Example 3:-
> With the Sort methods AND the password column's SortDirection taken out, lOK still returns True if name and password combination are in my DBISAM table and False if not! This is a correct result but why? Is there some sort of cache in place?

Yes - you are doing a regular matching search so sorting does not really
come into play.

Sort does come into effect if you need to do a nearestmatch search (i.e.
Find(TRUE,... ) since the data would need to be sorted in order for EWB
to guess at the nearest match.

If you look at the definition of the sort
(http://www.elevatesoft.com/manual?action=viewmethod&id=ewb2&comp=TDataSet&method=Find)
then it's specified in there ( "The NearestMatch parameter can only be
set to True if there is an active sort on the dataset, ...")

Raul
Tue, Oct 6 2015 12:25 AMPermanent Link

Raul

Globestar Systems

Team Elevate Team Elevate

On 10/6/2015 12:23 AM, Raul wrote:
> Yes - you are doing a regular matching search so sorting does not really
> come into play.

I meant to say Yes ("this is expected result") and not that there is a
cache of any sort - there is not one.

Raul
Tue, Oct 6 2015 12:44 AMPermanent Link

Frederick Chin

Raul,

/*
Yes - you are doing a regular matching search so sorting does not really
come into play.

Sort does come into effect if you need to do a nearestmatch search (i.e.
Find(TRUE,... ) since the data would need to be sorted in order for EWB
to guess at the nearest match.

If you look at the definition of the sort
(http://www.elevatesoft.com/manual?action=viewmethod&id=ewb2&comp=TDataSet&method=Find)
then it's specified in there ( "The NearestMatch parameter can only be
set to True if there is an active sort on the dataset, ...")
*/

Thanks for the explanations.

I did a modification of the code and commented out the sorting calls and the result is still true for the combination found,

        with Mnpsmain do begin
             {
             Columns['name'].SortDirection:=sdAscending;
             Columns['password'].SortDirection:=sdAscending;
             SortCaseInsensitive:=True;
             Sort;
             }
             InitFind;
             Columns['name'].AsString:=edtName.Text;
             Columns['password'].AsString:=cValue;
             lOK:=Find(False,True);
        end;

From what I am seeing here, EWB is performing a brute force record by record search if I am not performing a NearestSearch, which I don't often do, and I'd better not be loading a 100,000 record table.

Frederick
Tue, Oct 6 2015 8:34 AMPermanent Link

Raul

Globestar Systems

Team Elevate Team Elevate

On 10/6/2015 12:44 AM, Frederick Chin wrote:
>  From what I am seeing here, EWB is performing a brute force record by record search if I am not performing a NearestSearch, which I don't often do, and I'd better not be loading a 100,000 record table.

EWB Find will take advantage of the sort but does not require it.

If you look at the framework source code then Find that can use the sort
essentially performs a binary search (InternalSortFind) and otherwise
has to potentially look at all rows (InternalFind).

So if you're planning to run lot of Find operations on the same dataset
then cost of sorting might be worthwhile to speed up the finds.

In terms of amount of data yes you should try to limit the data you load
down to EWB by doing more restrictive queries. This way the back end
database can use all its indexes etc and you reduce amount of data that
needs to be transmitted over network and most importantly parsed by the
EWB/Javascript once it arrives.

Raul
Wed, Oct 7 2015 12:18 AMPermanent Link

Frederick Chin

Raul,

/*
EWB Find will take advantage of the sort but does not require it.

If you look at the framework source code then Find that can use the sort
essentially performs a binary search (InternalSortFind) and otherwise
has to potentially look at all rows (InternalFind).

So if you're planning to run lot of Find operations on the same dataset
then cost of sorting might be worthwhile to speed up the finds.

In terms of amount of data yes you should try to limit the data you load
down to EWB by doing more restrictive queries. This way the back end
database can use all its indexes etc and you reduce amount of data that
needs to be transmitted over network and most importantly parsed by the
EWB/Javascript once it arrives.
*/

Thanks. This information is useful.

I will be testing the performance of EWB with the 100,000 record table next week with a WIFI connection and see what kind of performance I can get.

Frederick
Wed, Oct 7 2015 5:50 AMPermanent Link

Walter Matte

Tactical Business Corporation

Frederick Chin wrote:

I will be testing the performance of EWB with the 100,000 record table next week with a WIFI connection and see what kind of performance I can get.


<Scratching Head> Why in the world would you want 100,000 records download into a browser!!    Do the heavy lifting on the server.  Look at the browser as the UI, low overhead display of data and collection of data with validation.  </Scratching Head>

If you have a long list, send a 100 at a time to the browser and manage that.  Look at existing web sites like stores - if you search for something they may give you 10 or 20 at a time, and you click "more" or "page 2" or something to get the next page.  No one downloads all the data all at once into the browser.

These records better by 1 byte in length!!

Walter
Wed, Oct 7 2015 10:12 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Frederick,

<< With the Sort methods AND the password column's SortDirection taken out, lOK still returns True if name and password combination are in my DBISAM table and False if not! This is a correct result but why? Is there some sort of cache in place? >>

The only purpose of sorting is sorting, find optimization, and nearest-match finds.  IOW, a Find will *still* return the correct result, it will just do so a bit slower if you don't have a matching sort.

Find works like a combination of filtering, Locate, and FindKey, all rolled into one.

Tim Young
Elevate Software
www.elevatesoft.com
Wed, Oct 7 2015 10:20 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Frederick,

<< I think I am going to use a dynamic DNS together with stunnel for the computer hosting the DBISAM tables. Will this work? >>

Yes, that will work, but you will *still* need to provide some form of authentication, either in the form of using the DBISAM Database Server and making *it* perform the authentication:

EWB Client -> EWB Web Server -> DataSet Definition with Authentication Info -> Connects to DBISAM Database Server

Or you can use an EWB Web Server module along with TEWBDatabaseAdapter/TEWBDataSetAdapter components to surface your various tables/queries.  The web server modules and adapters allow you to intercept authentication requests from EWB client applications and perform your own authentication.  See the datasetmodule example project included with EWB and the EWB Web Server Modules installations for information on how to do this:

http://www.elevatesoft.com/download?action=info&category=ewb&type=ewbmodule&majorversion=2&version=2.01

This is information on how you get started with web server modules after installing:

http://www.elevatesoft.com/manual?action=topics&id=ewb2mod&product=rsdelphi&version=XE&section=getting_started

Essentially, web server modules allow you to set up an API for your application.  They are essentially the middleware that allow you to move your native Delphi code on to the server-side for doing all of the heavy lifting like authentication, report generation, etc.

Tim Young
Elevate Software
www.elevatesoft.com
Wed, Oct 7 2015 10:27 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Jim,

<<  Assuming that DBISAM tables are password protected, and are accessed via https: from a module with the user supplying the username and password once only up front, then even local DBISAM access on a server could be made secure.  >>

You don't really need the DBISAM tables to be password protected.  Just make sure that:

1) The tables are *not* located in any sub-directory of the content folder setting for the EWB Web Server.  Doing so will allow anyone to download them just like any other content.

2) You implement authentication via the TEWBModule:

http://www.elevatesoft.com/manual?action=viewmethod&id=ewb2mod&product=rsdelphi&version=XE&comp=TEWBModule&method=AuthenticateUser

or, if using the TEWBDatabaseAdapter and its HandleRequest method for dataset requests:

http://www.elevatesoft.com/manual?action=viewevent&id=ewb2mod&product=rsdelphi&version=XE&comp=TEWBDatabaseAdapter&event=OnAuthenticateUser

Also, see here:

http://www.elevatesoft.com/manual?action=viewtopic&id=ewb2mod&product=rsdelphi&version=XE&topic=Handling_Requests

under "Authenticating Requests" for more information on how the authentication information comes into a web server module.

OR....

You use pre-defined datasets and let the underlying database engine perform the authentication.  With EDB, this works for both local and remote sessions, but with DBISAM you'll need to use remote sessions with the DBISAM Database Server in order to enable database-level authentication.  And, this is very important, this *will not* provide *general* authentication, only authentication for the dataset requests.  If you're doing more than just dataset requests, then you'll need to look into use the TEWBModule AuthenticateUser method along with the OnAuthenticateUser event handler to implement more general authentication.

Tim Young
Elevate Software
www.elevatesoft.com
Previous PagePage 4 of 6Next Page
Jump to Page:  1 2 3 4 5 6
Image