Incremental request delay

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Incremental request delay

Ralf Jung
Hi everyone,

I am using an incremental input to filter a list of items, i.e. the xforms-
value-changed is connected to the submission querying the database for the
filtered dataset. Since filtering for short strings is rather expensive, I want
the form to wait a bit before sending the request, since the user will most
likely enter more than one character. So I tried increasing oxf.xforms.delay-
before-incremental-request to 500ms, but looking at the database log, even if
I type quickly, several requests are sent. Even if I set the delay to 5
seconds, the filter is applied almost immediately (less than a second). Is that
a bug somewhere, or am I using the property the wrong way? Unfortunately I
found no documentation for it and only learned that it existed by looking at
the source JavaScript file.

I am using the latest stable Orbeon release (3.8).

Kind regards,
Ralf Jung


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Incremental request delay

Alessandro  Vernet
Administrator
Ralf,

Wow - this is a nice catch. I couldn't believe my eyes. This is indeed
a bug (not too much of a surprise so far), but this one has been
introduced in a refactoring done in November 2008. Here is the
relevant section of that 2008 commit:

http://github.com/orbeon/orbeon-forms/commit/547d4776bc7077ba434d9978eea8c62688bd42ac#L0R2316

I checked all the other changes done at that time (you never know!),
and this one was the only faulty one. So I fixed this, and the fix
will be in the next nightly build. For reference, the bug is:

http://forge.ow2.org/tracker/index.php?func=detail&aid=315584&group_id=168&atid=350207

You'll let us know how the new build works for you.

Alex

On Sun, Oct 24, 2010 at 4:31 AM, Ralf Jung <[hidden email]> wrote:

> Hi everyone,
>
> I am using an incremental input to filter a list of items, i.e. the xforms-
> value-changed is connected to the submission querying the database for the
> filtered dataset. Since filtering for short strings is rather expensive, I want
> the form to wait a bit before sending the request, since the user will most
> likely enter more than one character. So I tried increasing oxf.xforms.delay-
> before-incremental-request to 500ms, but looking at the database log, even if
> I type quickly, several requests are sent. Even if I set the delay to 5
> seconds, the filter is applied almost immediately (less than a second). Is that
> a bug somewhere, or am I using the property the wrong way? Unfortunately I
> found no documentation for it and only learned that it existed by looking at
> the source JavaScript file.
>
> I am using the latest stable Orbeon release (3.8).
>
> Kind regards,
> Ralf Jung
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws
>
>


--
Orbeon Forms - Web forms, open-source, for the Enterprise -
http://www.orbeon.com/
My Twitter: http://twitter.com/avernet


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: Re: Incremental request delay

Ralf Jung
Hi,

> Wow - this is a nice catch. I couldn't believe my eyes. This is indeed
> a bug (not too much of a surprise so far), but this one has been
> introduced in a refactoring done in November 2008. Here is the
> relevant section of that 2008 commit:
>
> http://github.com/orbeon/orbeon-forms/commit/547d4776bc7077ba434d9978eea8c6
> 2688bd42ac#L0R2316
Wow, glad you found that out :)

> I checked all the other changes done at that time (you never know!),
> and this one was the only faulty one. So I fixed this, and the fix
> will be in the next nightly build. For reference, the bug is:
>
> http://forge.ow2.org/tracker/index.php?func=detail&aid=315584&group_id=168&
> atid=350207
>
> You'll let us know how the new build works for you.
I solved the problem in the mean time by using xxforms:script. This also
allowed to be get the behaviour that I actually wanted - to update the form
not some fixed time span after the first key press, but some time span after the
last. So, pressing a key every 100ms, 10 times in a row, should only result in
one update after the last keystroke. Is this possible with the built-in
settings? Besides, I am hesitant to use a nightly build in production.

I'll try to do a quick test of the nightly later this week.

Kind regards,
Ralf Jung


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Incremental request delay

Alessandro  Vernet
Administrator
Ralf,

I am not sure what the "this" refers to when you say "Is this possible
with the built-in settings?", and I don't really see how using
<xxforms:script> can be used to work around this. But the fix I did is
very simple (it can't be simpler!). See:

http://github.com/orbeon/orbeon-forms/commit/00478eb2be71d6409b068beee33d2202123a1366

So you could just re-apply this change in your build. You can extract
xforms.js from WEB-INF/lib/orbeon-resources-public.jar, do this
change, regenerate the xforms-min.js from xforms.js using the YUI
compressor (http://developer.yahoo.com/yui/compressor/), and update
xforms.js and xforms-min.js in WEB-INF/lib/orbeon-resources-public.jar
with your patched versions.

Alex

On Tue, Oct 26, 2010 at 1:14 PM, Ralf Jung <[hidden email]> wrote:

> Hi,
>
>> Wow - this is a nice catch. I couldn't believe my eyes. This is indeed
>> a bug (not too much of a surprise so far), but this one has been
>> introduced in a refactoring done in November 2008. Here is the
>> relevant section of that 2008 commit:
>>
>> http://github.com/orbeon/orbeon-forms/commit/547d4776bc7077ba434d9978eea8c6
>> 2688bd42ac#L0R2316
> Wow, glad you found that out :)
>
>> I checked all the other changes done at that time (you never know!),
>> and this one was the only faulty one. So I fixed this, and the fix
>> will be in the next nightly build. For reference, the bug is:
>>
>> http://forge.ow2.org/tracker/index.php?func=detail&aid=315584&group_id=168&
>> atid=350207
>>
>> You'll let us know how the new build works for you.
> I solved the problem in the mean time by using xxforms:script. This also
> allowed to be get the behaviour that I actually wanted - to update the form
> not some fixed time span after the first key press, but some time span after the
> last. So, pressing a key every 100ms, 10 times in a row, should only result in
> one update after the last keystroke. Is this possible with the built-in
> settings? Besides, I am hesitant to use a nightly build in production.
>
> I'll try to do a quick test of the nightly later this week.
>
> Kind regards,
> Ralf Jung
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws
>
>


--
Orbeon Forms - Web forms, open-source, for the Enterprise -
http://www.orbeon.com/
My Twitter: http://twitter.com/avernet


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Re: Incremental request delay

Ralf Jung
Hi,

> I am not sure what the "this" refers to when you say "Is this possible
> with the built-in settings?", and I don't really see how using
> <xxforms:script> can be used to work around this.
"this" = "to update the form not some fixed time span after the first key press,
but some time span after the last. So, pressing a key every 100ms, 10 times in
a row, should only result in one update after the last keystroke"

Kind regards,
Ralf Jung


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Re: Incremental request delay

Ralf Jung
In reply to this post by Alessandro Vernet
Hi Alex,

> So you could just re-apply this change in your build. You can extract
> xforms.js from WEB-INF/lib/orbeon-resources-public.jar, do this
> change, regenerate the xforms-min.js from xforms.js using the YUI
> compressor (http://developer.yahoo.com/yui/compressor/), and update
> xforms.js and xforms-min.js in WEB-INF/lib/orbeon-resources-public.jar
> with your patched versions.
I did that, and verified that the changed files are in the jar, and restarted
tomcat. But it's still using the old version, as I can see from opening
/xforms-server/xforms-dialog-yuicalendar-min.js
Maybe I should better try a nightly... hm. Does it suffice to replace all the
jar files with the new ones? I did some configuration to disable the
/orbeon/exist and so on and do not want to re-do all that.

Kind regards,
Ralf Jung


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Re: Re: Incremental request delay

Alessandro  Vernet
Administrator
In reply to this post by Ralf Jung
Hi Ralf,

OK, I get it. I don't think you can implement this behavior of waiting
x ms after the last rather than the first keystroke to send an Ajax
request with your own <xxforms:script>, without resorting to huge
hacks :). The reason we implemented it this way is that if users type
continually, if we wait for x ms after the last keystroke, we might
not sent a request for a long time, which might not give the
impression that the field is "incremental enough". I.e. imagine those
two use cases:

1. Auto-translate content of a text area, or auto-rendering of
Markdown: users will have to stop typing to see the result.
2. Auto-save: here again, data won't be saved until after users stop typing.

This is not to say that there are no use cases for waiting until users
are "done" typing. I think the best way would be to implement this
alternate behavior in xforms.js, an trigger it based on a property.

Alex

On Wed, Oct 27, 2010 at 5:04 AM, Ralf Jung <[hidden email]> wrote:

> Hi,
>
>> I am not sure what the "this" refers to when you say "Is this possible
>> with the built-in settings?", and I don't really see how using
>> <xxforms:script> can be used to work around this.
> "this" = "to update the form not some fixed time span after the first key press,
> but some time span after the last. So, pressing a key every 100ms, 10 times in
> a row, should only result in one update after the last keystroke"
>
> Kind regards,
> Ralf Jung
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws
>
>


--
Orbeon Forms - Web forms, open-source, for the Enterprise -
http://www.orbeon.com/
My Twitter: http://twitter.com/avernet


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Re: Re: Incremental request delay

Alessandro  Vernet
Administrator
In reply to this post by Ralf Jung
Ralf,

Have you regenerated the xforms-min.js? The
xforms-dialog-yuicalendar-min.js is not a real file on disk: it is
produced by the Orbeon Forms by concatenating some JavaScript files it
finds in the resources. This is done to minimize the number of HTTP
requests the clients needs to make. While you are still testing your
changes, you can set the following two properties, so the actual
xforms.js is loaded:

    <property as="xs:boolean" name="oxf.xforms.minimal-resources"
                     value="false"/>
    <property as="xs:boolean" name="oxf.xforms.combine-resources"
                     value="false"/>

Also, instead of repackaging the xforms.js into the jar, you can put
it in your WEB-INF/resources/ops/javascript. Any file you put in
WEB-INF/resources by default overrides files in the jars/classpath.
And if you are curious why, you'll find the configuration for this in
your web.xml. Also, you might find some information about editing the
Orbeon Forms JavaScript code at the page below:

http://wiki.orbeon.com/forms/doc/contributor-guide/development-environment/javascript-development

I hope this helps,

Alex

On Wed, Oct 27, 2010 at 5:43 AM, Ralf Jung <[hidden email]> wrote:

> Hi Alex,
>
>> So you could just re-apply this change in your build. You can extract
>> xforms.js from WEB-INF/lib/orbeon-resources-public.jar, do this
>> change, regenerate the xforms-min.js from xforms.js using the YUI
>> compressor (http://developer.yahoo.com/yui/compressor/), and update
>> xforms.js and xforms-min.js in WEB-INF/lib/orbeon-resources-public.jar
>> with your patched versions.
> I did that, and verified that the changed files are in the jar, and restarted
> tomcat. But it's still using the old version, as I can see from opening
> /xforms-server/xforms-dialog-yuicalendar-min.js
> Maybe I should better try a nightly... hm. Does it suffice to replace all the
> jar files with the new ones? I did some configuration to disable the
> /orbeon/exist and so on and do not want to re-do all that.
>
> Kind regards,
> Ralf Jung
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws
>
>


--
Orbeon Forms - Web forms, open-source, for the Enterprise -
http://www.orbeon.com/
My Twitter: http://twitter.com/avernet


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: Re: Incremental request delay

Ralf Jung
Hi,

> Have you regenerated the xforms-min.js?
Yes, I re-generated and re-packed both of the js files.

> Also, instead of repackaging the xforms.js into the jar, you can put
> it in your WEB-INF/resources/ops/javascript. Any file you put in
> WEB-INF/resources by default overrides files in the jars/classpath.
> And if you are curious why, you'll find the configuration for this in
> your web.xml. Also, you might find some information about editing the
> Orbeon Forms JavaScript code at the page below:
>
> http://wiki.orbeon.com/forms/doc/contributor-guide/development-environment/
> javascript-development
Thanks for this, good to know the "overloading" of Orbeon's virtual file system
(as I call it for myself) works even for the shipped core resources.

> The reason we implemented it this way is that if users type
> continually, if we wait for x ms after the last keystroke, we might
> not sent a request for a long time, which might not give the
> impression that the field is "incremental enough". I.e. imagine those
> two use cases:
>
> 1. Auto-translate content of a text area, or auto-rendering of
> Markdown: users will have to stop typing to see the result.
> 2. Auto-save: here again, data won't be saved until after users stop
> typing.
I agree that there are usecases for both types of behaviour. In this case, it
is about a filter, and filtering for short strings is relatively costy. Usually,
the user has some name in his mind and will enter at least 4 or 5 characters,
in which case I do not want the shorter queries to be executed.

Also, with these additional options, it seems to work now, thanks a lot! This
is much cleaner than my Javascript solution. It was not *that* hacky though:
<xxforms:script ev:event="DOMActivate">
if (filterTimer != undefined) window.clearTimeout(filterTimer);
var time = ORBEON.xforms.Document.getValue("filter").length == 1 ? 500 : 200;
// single-character queries are quite costy, so let's see if we can avoid them
filterTimer = window.setTimeout("filterTimer = undefined;
ORBEON.xforms.Document.dispatchEvent('summary-model', 'filter-update');",
time);
</xxforms:script>
But it went through the server, so if the fixed incremental update does not
suffice, I will try to add some local event listener and disable incremental and
trigger the update myself. Somehow...

Btw, a timeout of 5000ms is still not honoured, it gets update after a second
or two - but I guess that is caused by some of the other limits like delay-
before-force-incremental-request.

Kind regards,
Ralf Jung


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Incremental request delay

Alessandro  Vernet
Administrator
Ralf,

I see: you dispatch an custom event when appropriate from JavaScript.
Indeed, this is less hacky than what I had in mind :). So I gather you
went the route of dispatching your own event rather than using the
fixed incremental mode. Do you still have any open issue with this, or
are you all good?

Alex

On Thu, Oct 28, 2010 at 11:26 AM, Ralf Jung <[hidden email]> wrote:

> Hi,
>
>> Have you regenerated the xforms-min.js?
> Yes, I re-generated and re-packed both of the js files.
>
>> Also, instead of repackaging the xforms.js into the jar, you can put
>> it in your WEB-INF/resources/ops/javascript. Any file you put in
>> WEB-INF/resources by default overrides files in the jars/classpath.
>> And if you are curious why, you'll find the configuration for this in
>> your web.xml. Also, you might find some information about editing the
>> Orbeon Forms JavaScript code at the page below:
>>
>> http://wiki.orbeon.com/forms/doc/contributor-guide/development-environment/
>> javascript-development
> Thanks for this, good to know the "overloading" of Orbeon's virtual file system
> (as I call it for myself) works even for the shipped core resources.
>
>> The reason we implemented it this way is that if users type
>> continually, if we wait for x ms after the last keystroke, we might
>> not sent a request for a long time, which might not give the
>> impression that the field is "incremental enough". I.e. imagine those
>> two use cases:
>>
>> 1. Auto-translate content of a text area, or auto-rendering of
>> Markdown: users will have to stop typing to see the result.
>> 2. Auto-save: here again, data won't be saved until after users stop
>> typing.
> I agree that there are usecases for both types of behaviour. In this case, it
> is about a filter, and filtering for short strings is relatively costy. Usually,
> the user has some name in his mind and will enter at least 4 or 5 characters,
> in which case I do not want the shorter queries to be executed.
>
> Also, with these additional options, it seems to work now, thanks a lot! This
> is much cleaner than my Javascript solution. It was not *that* hacky though:
> <xxforms:script ev:event="DOMActivate">
> if (filterTimer != undefined) window.clearTimeout(filterTimer);
> var time = ORBEON.xforms.Document.getValue("filter").length == 1 ? 500 : 200;
> // single-character queries are quite costy, so let's see if we can avoid them
> filterTimer = window.setTimeout("filterTimer = undefined;
> ORBEON.xforms.Document.dispatchEvent('summary-model', 'filter-update');",
> time);
> </xxforms:script>
> But it went through the server, so if the fixed incremental update does not
> suffice, I will try to add some local event listener and disable incremental and
> trigger the update myself. Somehow...
>
> Btw, a timeout of 5000ms is still not honoured, it gets update after a second
> or two - but I guess that is caused by some of the other limits like delay-
> before-force-incremental-request.
>
> Kind regards,
> Ralf Jung
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws
>
>


--
Orbeon Forms - Web forms, open-source, for the Enterprise -
http://www.orbeon.com/
My Twitter: http://twitter.com/avernet


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Re: Incremental request delay

Ralf Jung
Hi Alex,

> I see: you dispatch an custom event when appropriate from JavaScript.
> Indeed, this is less hacky than what I had in mind :).
I better don't ask what that was then :D

> So I gather you
> went the route of dispatching your own event rather than using the
> fixed incremental mode.
Actually, these options (together with the xforms.js fix you sent me) turned
out to work great - without any custom JavaScript, or needless roundtrips
through the server:
<property as="xs:string"  name="oxf.xforms.delay-before-incremental-request"
value="250"/>
<property as="xs:string"  name="oxf.xforms.delay-before-force-incremental-
request" value="1000"/>

> Do you still have any open issue with this, or
> are you all good?
It's fine now, thanks a lot!

Kind regards,
Ralf Jung


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws