On 5/18/2015 3:56 AM, Matthew Jones wrote:
> Your analysis is good, but I want to pick up on this because while
> generally true, it may not be. I think that if you pin the site

Agreed - now that many of the devices are mobile the connectivity is
back in the forefront.

> These things are not important, because you should assume that you may
> lose a connection at any time, and that is what is important. Assume
> that any part of your connection may fail, and handle it appropriately.

Pending commits is actually designed just for this if using datasets.

Server request does give you fine control already so i'm sure most of us
will write another connection layer on top for more purposes - at app
level - that allows queuing, retry ,etc.

I wonder if this might be a useful code to share. It could even be a
custom component (non UI).


Raul

Raul,

Good post.

<< Cancel option seems to be not directly accessible at this time (unless i
missed something). What i think needs to be done is basically to cancel the
current requests from internal TServerRequestQueue and then clean up
internal tdatabase commit state  (basically what "EndCommit" does today - in
fact might be ok to just call it). >>

If you do that, you'll wipe out all pending commits, and therefore, the
entire database state and all datasets will immediately become invalid
because they represent a state that no longer exists.  As long as there are
pending commits, then everything is okay.  It is up to the application and
the user to determine the appropriate course of action in the face of
commits that cannot succeed.  If the application doesn't want to tolerate
it, then it will shut off *all* ability to update datasets after any commit
error.  *I* wouldn't do this, because it involves a lot of event handler
assignments and code for very little pay-off, but it's possible to do.

What *I* would do is simply inform the user that a commit has failed, that
this represents a connection problem, and that they should resolve the
connection problem *first* before proceeding with any other updates to the
database.  Then, I would give them a very prominent option to retry the
commits manually.  However, remember that any *subsequent* commit call (with
transaction level 0) will result in a retry of *all* pending commits
automatically, so you don't necessarily event need to give them an option to
manually commit again.

If someone has designed their EWB application so that commit pivot points
are decided in the *client* application, then they need to go back to the
drawing board, because that is a very bad design.  And, by commit pivot
points, I mean a case where some other operation or interaction with an
external system happens in the EWB application in response to a commit
(think ATMs, but don't, because nobody would actually design something like
that).  Those types of operations should be performed by the back-end web
server when it is guaranteed that a commit has reached the back-end and has
been successfully applied.

<< This also comes into play if the back-end issues an error (say DB key
violation). >>

I would never design an EWB application in a way that would allow it to
experience key violations.  Such a design is simply toxic when it comes to
an optimistic update architecture, which is what *any* web application has
because server requests are *always* async.  Having said that, at some point
I will probably try to introduce an ability to reconcile rows after a commit
failure due to a database constraint, but there are a lot of issues with
doing so, so it will be something that takes a lot of extra code and still
won't be able to solve all such possible constraint issues.  Especially
those for which the front-end EWB applications lacks the context to know
what to do with the error.

<< - OnAllCommitsCompleted or similar callback that fires when EndCommit is
called with FPendingCommits = 0.. It might not be a bad idea to have
callback option in RequestComplete instead also for additional flexibility.
>>

This would only be useful if one were interested in blocking all database
operations until a single commit completed.  This is insanity in a browser
environment where *everything* is asynchronous.  It's like trying to block
the Pacific Ocean with a couple of sandbags.  Plus, if you could do
this, then EWB would have no need for pending commits in the first place.

<< - public function CancelCurrentCommit that you can call from
OnCommitError and that does as described above (cancel serverrequest and
call endcommit). >>

See above.  I'm not sure what purpose such an event would have....

<< i personally like the OnAllCommitsCompleted callback concept though and
once its called you proceed  (i.e. inform user or remove progress etc). >>

Again, you're fighting a losing battle trying to make something synchronous
that is inherently asynchronous.  This is why I introduced the concept of
pending commits - EWB 1 tried to do it your way and lost (commit events
became pseudo-race conditions with other dataset events).  It simply doesn't
work because the browser won't *let* it work.

Tim Young
Elevate Software
www.elevatesoft.com

On 5/18/2015 10:45 AM, Tim Young [Elevate Software] wrote:
> If you do that, you'll wipe out all pending commits, and therefore, the
> entire database state and all datasets will immediately become invalid
> because they represent a state that no longer exists.  As long as there

Good point - i forgot about the local state in EWB datasets.

> I would never design an EWB application in a way that would allow it to
> experience key violations.  Such a design is simply toxic when it comes
> to an optimistic update architecture, which is what *any* web
> application has because server requests are *always* async.  Having said

Agreed but i also would want to enforce business rules on the back-end
and not in UI so there will be a possibility of commit failing by the
back-end.

Need to give this more thought and maybe it's the kind of thing i just
handle myself with server request without using any of the dataset
functionality. it would be just few key areas.


> reconcile rows after a commit failure due to a database constraint, but
> there are a lot of issues with doing so, so it will be something that
> takes a lot of extra code and still won't be able to solve all such
> possible constraint issues.  Especially those for which the front-end
> EWB applications lacks the context to know what to do with the error.

Dump data and reload option ?


> Again, you're fighting a losing battle trying to make something
> synchronous that is inherently asynchronous.  This is why I introduced
> the concept of pending commits - EWB 1 tried to do it your way and lost
> (commit events became pseudo-race conditions with other dataset
> events).  It simply doesn't work because the browser won't *let* it work.

I actually do want the async behavior - callback when all pending
operations have completed rather than me constantly polling with timer.

Again might be just my way of thinking at this time but our app (that we
want to write web version with EWB) has operations that we want to know
completed before other areas of the app get enabled (rest of app is
usable still during this time).

Might be something we should redesign but for revision 1 we want to keep
look and usage similar to our desktop app since users already know how
to use it and almost all of them like the way it works (they just want
to do same thing from web and tablets etc).

Raul

Raul,

I just noticed that I'm using too many asterisks for emphasis.  I'm
emphasizing *everything*.

<< Agreed but i also would want to enforce business rules on the back-end
and not in UI so there will be a possibility of commit failing by the
back-end. >>

Normally, I would agree.  But, because the browser forces asynch behaviors
on the database commits, it becomes somewhat necessary to validate data on
the front-end also.

<< Dump data and reload option ? >>

The only problem with this is that the user might be right in the middle of
another update.  The key issue here is that the user be allowed to reconcile
any errors, without necessarily interrupting what they're currently doing.
That's why, as Run D.M.C. would say, it's tricky.

<< Again might be just my way of thinking at this time but our app (that we
want to write web version with EWB) has operations that we want to know
completed before other areas of the app get enabled (rest of app is usable
still during this time). >>

If you're talking about authentication or something similar, then you would
probably want to use a generic TServerRequest for such an operation.  If you
have relevant details but can't say publicly, just email them to me and I'll
give them a look to see if they're something that you should use datasets
for.

Tim Young
Elevate Software
www.elevatesoft.com

The callback - event driven programming makes more sense to me than having a timer constantly checking, since the framework is being told that the commits are not pending anymore.

I
Walter

Tim,

<< I just noticed that I'm using too many asterisks for emphasis.  I'm  emphasizing *everything*. >>

They do show area of focus well and bring the point across.


<<  Normally, I would agree.  But, because the browser forces asynch behaviors
on the database commits, it becomes somewhat necessary to validate data on
the front-end also.  >>

This one i need to think about - one can definitely validate on front-end but our current web services enforces checking on back-end also (original idea was for our customers to write small web apps so we check all input


<<
The only problem with this is that the user might be right in the middle of
another update.  The key issue here is that the user be allowed to reconcile
any errors, without necessarily interrupting what they're currently doing.
That's why, as Run D.M.C. would say, it's tricky.
>>

I see what you mean.

I'm still bit unclear on the whole "how do i cancel" option (i.e. user cannot restore the connection).


<< If you're talking about authentication or something similar, then you would
probably want to use a generic TServerRequest for such an operation.  >>

We do use TServerRequest for this as there is lot of extra (unrelated) data i like to return as part of the login.

<< If you  have relevant details but can't say publicly, just email them to me and I'll
give them a look to see if they're something that you should use datasets for. >>

The original proof of concept was done using ewb1 and datasets but the more i think about this using custom classes (i.e. combined with the save to/load from JSON capability in EWB2) would likely be a better way and give more flexibility.

Raul

I  do appreciate the asynchronous way of thinking in EWB2. Not double about it.... In most cases that will be the way to go.

However, in certain cases I would also be absolutely dependent on knowing when all commits has completed successfully.

In my case, one of my EWB apps is a personnel performance appraisal system that runs on oil and gas platforms and vessel with very slow and lossy satellite internet. The user would typically get a link on his email and click on that link to open the EWB app. There the user click a bunch of radio buttons and write some comments. At the end, the user will submit the data to the server. Due to the nature of slow satellite internet, the commit operation can take up to 30 seconds to complete. It's critical that the user stay put and and wait for a confirmation, or wait for an error to pop up so he can retry. After the user presses the commit button, I will show a spinner and a please wait message. All users that work on a oil and gas platform are used to this and accept the nature of this internet connection. I simply cannot just tell the user that the data is saved immediately as he will then close the browser and move on. I need to show him a progress and have him wait for a confirmation.

I would also appreciate a call back function when all commits are successful instead of having a TTimer poll the PendingCommits  and close the spinner when it reaches 0.

Well, this i just my thoughts and humble wishes to solve some special cases

Eivind

Random thoughts - so I Commit the current client dataset transactions.  At the Server when the transactions are applied to the database a number of stored procedures fire that generate new data and new records in other tables that I now want to show in the browser.  Could be a bunch of statistical data.....

When do I know to Database.Load(NewStatsData)?

Walter

On 5/19/2015 5:01 AM, Walter Matte wrote:
> When do I know to Database.Load(NewStatsData)?

In general case just load data right after commit without waiting. The
server requests are serialized and queued by Database so the load will
not happen until commit completes (since it was in the queue first).

Any retry logic applies to both (actually anything in the queue so you
can build up other loads and/or commits as well).

The only thing you really need to worry about is dealing with the errors
- i.e. retry in OnCommitError  and/or show manual retry button or such.

If you do know (like Elvind) that connections can be super slow (i
consider 30 sec super slow for single http query) then progress dialog
might make sense at some point since he cannot use shorter http timeout.
For rest of us enforcing a short http timeout and dealing with
OnCommitError likely is enough.

Raul

Raul,

<< This one i need to think about - one can definitely validate on front-end
but our current web services enforces checking on back-end also (original
idea was for our customers to write small web apps so we check all input
>>

The real issue here is not that we can't do the dataset reconciliation, but
rather that it makes porting the dataset handling to other environments
*extremely* difficult because they'll now *also* have to handle generating
reconciliation information for the datasets that experienced commit errors
(this isn't new, and was also an issue with EWB1).  Then there's the whole
issue of how to handle this information in the UI....

<< I'm still bit unclear on the whole "how do i cancel" option (i.e. user
cannot restore the connection). >>

Well, the general idea is: if the connection is completely gone, then it
doesn't matter since the application is now useless.  If the connection
can be restored at any point, then the commits can be retried.

Tim Young
Elevate Software
www.elevatesoft.com