Icon View Thread

The following is the text of the current message along with any replies.
Messages 1 to 10 of 18 total
Thread Post notification to DataAware Controls
Tue, Mar 16 2010 5:27 PMPermanent Link

Robert Rowlands

Hello.

I'm still struggling with speed issues on a VPN but things have greatly improved with the much welcomed support on these forums.

Grovel over, I now come cap in hand again.

I'm using CodeSite to help me profile the application and it highlights all (lots of them) bottlenecks in my code.

The application has a main query with a load of detail queries linked to various fields in the main query.  There's an autoinc for Client data, Architect data, Contractor data etc. All of the detail queries are displayed in DevEx Grids.

I have long (unnecessary?) delays when changing the data in a field in the main query that has no detail queries linked to it.  It could be a date or something.

The problem appears to be that when I post after editing the field all of the detail queries want to reload their data from the server even though their controlling field in the main query is unchanged.

I noted a question on the DevEx support forum and the reply was that the 'dataset sends the "DataChanged" notification to its data links' ... and.. 'it is impossible to force the DataController not to react to this notification'.  The DataController referred to here is the DevEx one associated with their data-aware components.

Is there any way I can stop this message going out when I post a change to an innocuous field in my main query?

I DisableControls and then edit the field in code.  I them Post and then EnableControls.  CodeSite shows it takes 10 seconds to go from the EnableControls instruction to the next line of code and I can see loads of network traffic.  As an aside I'm also using Begin/EndUpdate on all the linked DevEx DataControllers.

Is perhaps the only way to not have Master/Detail relationships and manually reload the detail queries in the AfterScroll event of the main query?  Bit of a pain but if I can get the extra 10 seconds...

Thanks.  I'm now going to hit that cheap Tesco Cuvee Prestige...
Wed, Mar 17 2010 5:11 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Robert

I don't use DevEx so can't help directly. Do you have the source? If so you may be able to find the code where this notification is sent and subvert it. If not you may be able to unlink the detail queries and manage them yourself. A lot of my stuff like this I do in a dataset's AfterScroll or DBEdit's OnChange event.

Roy Lambert [Team Elevate]
Wed, Mar 17 2010 5:38 AMPermanent Link

Robert Rowlands

Hi Roy.

Yes, I suspect I should unlink the detail queries and handle in the main query events.

I think I now understand why some people seem to see dataware as a curse and do everything manually.

Rob.
Wed, Mar 17 2010 11:04 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Robert,

<< I have long (unnecessary?) delays when changing the data in a field in
the main query that has no detail queries linked to it.  It could be a date
or something.

The problem appears to be that when I post after editing the field all of
the detail queries want to reload their data from the server even though
their controlling field in the main query is unchanged. >>

Yes, but that, by itself, shouldn't be that big of a deal if the loading is
done in an optimized manner.  How long of a delay are you seeing, and did
you profile this using the remote tracing in DBISAM (RemoteTrace=True) ?
More than likely the issue is just a simple case of records being loaded
like this:

Get Record
Get Record
Get Record
Get Record

instead of

Get 4 Records

Although the issue could be with lookup fields, or something similar.

--
Tim Young
Elevate Software
www.elevatesoft.com

Sat, Mar 20 2010 5:22 PMPermanent Link

Robert Rowlands

Tim.

I've decided to try the unlinked option but am having difficulties with another table.

This has to be streamed to the Client for performance purposes and a lot of queries use it.

It contains around 3000 records with 33 fields in each.  It takes 50 seconds to load to the client machine.

The odd thing is that after a few seconds there is virtually no network activity yet something associated with the streaming is taking place.

The query on the remote server is loaded with the following code.

 frmRemoteTrace.cxMemo1.Lines.Add('Query Opened   - '+DateTimeToStr(Now));
 DMMain.qryContacts.Open;
 frmRemoteTrace.cxMemo1.Lines.Add('Data streamed   - '+DateTimeToStr(Now));

In the OnOpen event of qryContacts is the following code (courtesy of Bruno).

 CreateTableContacts; //Reverse Engineer code from DBSys
 lMemoryStream:=nil;
 try
   SessionData.RemoteTrace := True;
   lMemoryStream:=TMemoryStream.Create;
   qryContacts.SaveToStream(lMemoryStream);
   SessionData.RemoteTrace := False;
   qryContacts.Close; // Not needed anymore.
   tblContactsMem.LoadFromStream(lMemoryStream); // Stuff it in local memory table
 finally
   lMemoryStream.Free;
 end;

The RemoteTrace file is attached.

At the start there is a lot of traffic on the Local Area Connection then a long period of absolutely nothing happening.  I think the log demonstrates this.  Id did however check CPU usage and when the network traffic had stopped it is running a 100%.

Have I messed up again here or is this 'normal'?

Thanks.



Attachments: tblcontacts.txt
Mon, Mar 22 2010 12:15 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Robert,

<< It contains around 3000 records with 33 fields in each.  It takes 50
seconds to load to the client machine.

The odd thing is that after a few seconds there is virtually no network
activity yet something associated with the streaming is taking place.

The query on the remote server is loaded with the following code.

frmRemoteTrace.cxMemo1.Lines.Add('Query Opened   - '+DateTimeToStr(Now));
DMMain.qryContacts.Open;
frmRemoteTrace.cxMemo1.Lines.Add('Data streamed   - '+DateTimeToStr(Now));

In the OnOpen event of qryContacts is the following code (courtesy of
Bruno). >>

There isn't any OnOpen event - there's a BeforeOpen event and an AfterOpen
event.  In general, however, you shouldn't Close the query that you're
opening in either event - doing so could potentially cause weird things to
happen.

Did you time the LoadFromStream method call ?  It should be fairly easy to
see which methods are causing the bulk of the time, and the SaveToStream
indicates that it is taking ~4secs to execute, based upon the remote trace.

--
Tim Young
Elevate Software
www.elevatesoft.com

Mon, Mar 22 2010 3:58 PMPermanent Link

Robert Rowlands

Tim.

Sorry, I mean in the AfterOpen event.

I have moved the code out of this now following your comment.

I suspect I have to accept the time it takes to do the loading.  I've attached a more detailed report.  The codes has been changed as follows.

 if not assigned(frmRemoteTrace) then
    frmRemoteTrace := TfrmRemoteTrace.Create(Application);
 frmRemoteTrace.Show;
        frmRemoteTrace.cxMemo1.Lines.Add('Query Opened   - '+DateTimeToStr(Now));
 DMMain.qryContacts.Open;

 if DMMain.tblContactsMem.Exists then
 begin
   DMMain.tblContactsMem.Close;
   DMMain.tblContactsMem.DeleteTable;
 end;

 // Create the memory table for Events
 CreateTableContacts; //Reverse Engineer code from DBSys
 lMemoryStream:=nil;
 try
   DMMain.SessionData.RemoteTrace := True;
   lMemoryStream:=TMemoryStream.Create;
          frmRemoteTrace.cxMemo1.Lines.Add('SaveToStream   - '+DateTimeToStr(Now));
   DMMain.qryContacts.SaveToStream(lMemoryStream);
   // close the query so I can access the MemoryStream
          frmRemoteTrace.cxMemo1.Lines.Add('Closing qryContacts   - '+DateTimeToStr(Now));
   DMMain.qryContacts.Close;
          frmRemoteTrace.cxMemo1.Lines.Add('Loading from Stream  - '+DateTimeToStr(Now));
   DMMain.tblContactsMem.LoadFromStream(lMemoryStream);
          frmRemoteTrace.cxMemo1.Lines.Add('Loading complete  - '+DateTimeToStr(Now));
 finally
   lMemoryStream.Free;
 end;
 frmMain.tvContacts.DataController.BeginUpdate;
 frmMain.dsContacts.DataSet := DMMain.tblContactsMem;
        frmRemoteTrace.cxMemo1.Lines.Add('Opening Mem Table  - '+DateTimeToStr(Now));
 DMMain.tblContactsMem.Open;
        frmRemoteTrace.cxMemo1.Lines.Add('Updating Mem Table Fields  - '+DateTimeToStr(Now));
 DMMain.tblContactsMem.FieldDefs.Update;
 DMMain.tblContactsMem.Close;
 DMMain.tblContactsMem.FieldDefs.Add('FullName',ftString,50,False,'','','','',fcNoChange);
        frmRemoteTrace.cxMemo1.Lines.Add('Altering Mem Table  - '+DateTimeToStr(Now));
 DMMain.tblContactsMem.AlterTable;
 DMMain.tblContactsMem.Open; // TDBISAMTable (in memory) ready for use
        frmMain.tvContacts.DataController.EndUpdate;
 DMMain.SessionData.RemoteTrace := False;
        frmRemoteTrace.cxMemo1.Lines.Add('All Finished  - '+DateTimeToStr(Now));


I have another little issue.  God knows why, but the OnCalcFields event of tblContactsMem does not fire.  The AutoCalcFields property is set to True.

Is there any way I can force it to run?

Finally, I seem to be taking a disproportionate amount of your time.  Please let me know if I should be paying for a Support Plan.

Thanks.

Rob.



Attachments: tblcontacts.txt
Tue, Mar 23 2010 7:57 AMPermanent Link

John Hay

Robert

With the total data size at only 1.5mb the 42 seconds for loading the stream
is almost unbeliveably excessive.

Can you test the streaming in a "noddy" app which references only the memory
table and remote query?

John

Tue, Mar 23 2010 10:07 AMPermanent Link

Robert Rowlands

John.

Your surprise had made me look at the creation of the memtable t make sure it appears correct.

I was about to print it out and noted it needs full text search on the memo fields.  Perhaps this takes the time?

The table creation code is as follows.

with TableToCreate do
        begin
        DatabaseName:= 'Memory';
        TableName:='MemContacts';
        Exclusive:=True;
        if (not Exists) then
           begin
           with FieldDefs do
           begin
              Clear;
              Add('GroupAutoInc',ftInteger);
              Add('GroupRef',ftInteger);
              Add('FileAs',ftString,75);
              Add('JobTitle',ftString,50);
              Add('CompanyName',ftString,50);
              Add('Address',ftMemo);
              Add('PhoneWork',ftString,20);
              Add('Fax',ftString,20);
              Add('PhoneHome',ftString,20);
              Add('eMail',ftString,100);
              Add('Categories',ftString,100);
              Add('Notes',ftMemo);
              Add('CompanyMainTelephoneNumber',ftString,20);
              Add('FirstName',ftString,20);
              Add('Name',ftString,50);
              Add('HomeAddress',ftMemo);
              Add('LastName',ftString,30);
              Add('MiddleName',ftString,10);
              Add('PhoneMobile',ftString,20);
              Add('ReferredBy',ftString,50);
              Add('Suffix',ftString,10);
              Add('Title',ftString,10);
              Add('WebSite',ftString,50);
              Add('Flag',ftBoolean,0,False,'False');
              Add('JobsReferals',ftBoolean,0,False,'False');
              Add('SalutationDear',ftString,30);
              Add('AccountRef',ftString,10);
              Add('AccountsFlag',ftBoolean);
              Add('ReferelRefNo',ftInteger);
              Add('Selected',ftBoolean);
              Add('MailMerge',ftMemo);
              Add('bInvAddress',ftBoolean);
              Add('InvAddress',ftMemo);
           end;
           with IndexDefs do
              begin
              Clear;
              Add('','GroupAutoInc',[ixPrimary,ixUnique]);
              Add('CompanyName','CompanyName');
              Add('FileAs','FileAs');
              Add('Flag','Flag');
              Add('FullName','Name');
              Add('GroupRef','GroupRef');
              Add('JobsReferals','JobsReferals');
              Add('GroupAutoInc','GroupAutoInc');
              Add('Selected','Selected');               
              end;
           StopWords:=TStringList.Create;
           try
              with StopWords do
                 begin
                 Add('A');
                 Add('AN');
                 Add('AND');
                 Add('BE');
                 Add('FOR');
                 Add('HOW');
                 Add('IN');
                 Add('IS');
                 Add('IT');
                 Add('OF');
                 Add('ON');
                 Add('OR');
                 Add('THAT');
                 Add('THE');
                 Add('THIS');
                 Add('TO');
                 Add('WAS');
                 Add('WHAT');
                 Add('WHEN');
                 Add('WHICH');
                 Add('WHY');
                 Add('WILL');
                 end;
              CreateTable(0,1,3,True,'SolutionsIT','',4096,512,0,'CompanyName;Name;Categories',StopWords,
                    #1+#2+#3+#4+#5+#6+#7+#8+#9+#10+#11+#12+#13
                    +#14+#15+#16+#17+#18+#19+#20+#21+#22+#23
                    +#24+#25+#26+#27+#28+#29+#30+#31+#32+'*+'+
                    ',-./:;<=>\`',
                    '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_ab'+
                    'cdefghijklmnopqrstuvwxyz'+
                    ' '+
                    ''+
                    '');
           finally
              StopWords.Free;
           end;
           end;
        end;
  finally
     TableToCreate.Free;

Do you think the stuff at the end would cause a problem?

The query opened on the remote server could be a table component if I deleted some redundant fields in the server table.  Perhaps that would be quicker as I now suspect the indexes are being re-built on the client machine which could be taking up the time.  Or perhaps it makes no difference.
Tue, Mar 23 2010 11:17 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Robert

>I was about to print it out and noted it needs full text search on the memo fields. Perhaps this takes the time?

Try deleting the full text index and see if it speeds it up. My guess is that it will. I remember full text indexing being fairly slow on DBISAM. It was one of the main drivers for me to move to ElevateDB.

Roy Lambert

Page 1 of 2Next Page
Jump to Page:  1 2
Image