Efficient POST/PUT queries

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Efficient POST/PUT queries

Ralf Jung
Hi list,

when loading data from a HTTP service with GET, using the url-generator is (in
my experience) much more efficient than using xforms-submission. However, the
url-generator lacks a way to use different HTTP methods - if I want to issue a
large POST or PUT request, I seem to still have to use the xforms-submission
processor. That will not only start a full-flown XForms session, it will also
parse the whole request and response in a DOM tree, which is horribly
inefficient.
I guess for PUT and DELETE, which usually do not have a response (besides
"failed or not"), I could also use the url-serializer, but I could not find any
documentation about which HTTP method it uses or how to configure that
(http://wiki.orbeon.com/forms/doc/developer-guide/processors-other-
serializers#TOC-URL-serializer). Not that serializing to a "DELETE" would make
much sense.
However, this is obviously no solution for POST as the response is of interest
there. I attempted to extend the url-generator to support post, put and
delete. Unfortunately, Apache HttpClient sends weird "chunked POST" requests
if it is handed an InputStream, so the full POST body has to be buffered in a
byte[]. This is however still more memory efficient than parsing it into a DOM
tree. I tested it briefly with my usecase, and it worked. However I am sure I
did not get the caching part right - I could not figure out what all that
validity stuff does, so I just had it return null for non-GET queries. Seems to
work ;-)

I'd welcome any feedback whether something like this is desired, and whether I
chose the right direction to implement it.

The attached patch is based on current master.

Kind regards,
Ralf


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws

0001-implemenet-POST-and-PUT-for-URLGenerator.patch (14K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Efficient POST/PUT queries

Ralf Jung
Is this the totally wrong approach, or is such a feature just not useful
enough?
Any feedback would be welcome.

Kind regards,
Ralf

On Monday 25 July 2011 22:45:00 Ralf Jung wrote:

> Hi list,
>
> when loading data from a HTTP service with GET, using the url-generator is
> (in my experience) much more efficient than using xforms-submission.
> However, the url-generator lacks a way to use different HTTP methods - if
> I want to issue a large POST or PUT request, I seem to still have to use
> the xforms-submission processor. That will not only start a full-flown
> XForms session, it will also parse the whole request and response in a DOM
> tree, which is horribly inefficient.
> I guess for PUT and DELETE, which usually do not have a response (besides
> "failed or not"), I could also use the url-serializer, but I could not find
> any documentation about which HTTP method it uses or how to configure that
> (http://wiki.orbeon.com/forms/doc/developer-guide/processors-other-
> serializers#TOC-URL-serializer). Not that serializing to a "DELETE" would
> make much sense.
> However, this is obviously no solution for POST as the response is of
> interest there. I attempted to extend the url-generator to support post,
> put and delete. Unfortunately, Apache HttpClient sends weird "chunked
> POST" requests if it is handed an InputStream, so the full POST body has
> to be buffered in a byte[]. This is however still more memory efficient
> than parsing it into a DOM tree. I tested it briefly with my usecase, and
> it worked. However I am sure I did not get the caching part right - I
> could not figure out what all that validity stuff does, so I just had it
> return null for non-GET queries. Seems to work ;-)
>
> I'd welcome any feedback whether something like this is desired, and
> whether I chose the right direction to implement it.
>
> The attached patch is based on current master.
>
> Kind regards,
> Ralf


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Efficient POST/PUT queries

Erik Bruchez
Administrator
Ralf,

Sorry, I hadn't yet looked at the patch. I think the approach makes sense.

I haven't looked at all the details of the patch. Two concerns:

* currently, this only allows posting XML, right? that is, you can't
post binary/text content?
* we have to look at the implication for caching, if any (i.e. the URL
generator shouldn't cache at all in this case)

-Erik

On Tue, Aug 9, 2011 at 3:06 AM, Ralf Jung <[hidden email]> wrote:

> Is this the totally wrong approach, or is such a feature just not useful
> enough?
> Any feedback would be welcome.
>
> Kind regards,
> Ralf
>
> On Monday 25 July 2011 22:45:00 Ralf Jung wrote:
>> Hi list,
>>
>> when loading data from a HTTP service with GET, using the url-generator is
>> (in my experience) much more efficient than using xforms-submission.
>> However, the url-generator lacks a way to use different HTTP methods - if
>> I want to issue a large POST or PUT request, I seem to still have to use
>> the xforms-submission processor. That will not only start a full-flown
>> XForms session, it will also parse the whole request and response in a DOM
>> tree, which is horribly inefficient.
>> I guess for PUT and DELETE, which usually do not have a response (besides
>> "failed or not"), I could also use the url-serializer, but I could not find
>> any documentation about which HTTP method it uses or how to configure that
>> (http://wiki.orbeon.com/forms/doc/developer-guide/processors-other-
>> serializers#TOC-URL-serializer). Not that serializing to a "DELETE" would
>> make much sense.
>> However, this is obviously no solution for POST as the response is of
>> interest there. I attempted to extend the url-generator to support post,
>> put and delete. Unfortunately, Apache HttpClient sends weird "chunked
>> POST" requests if it is handed an InputStream, so the full POST body has
>> to be buffered in a byte[]. This is however still more memory efficient
>> than parsing it into a DOM tree. I tested it briefly with my usecase, and
>> it worked. However I am sure I did not get the caching part right - I
>> could not figure out what all that validity stuff does, so I just had it
>> return null for non-GET queries. Seems to work ;-)
>>
>> I'd welcome any feedback whether something like this is desired, and
>> whether I chose the right direction to implement it.
>>
>> The attached patch is based on current master.
>>
>> Kind regards,
>> Ralf
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws
>
>


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Efficient POST/PUT queries

Ralf Jung
Hi Erik,

> * currently, this only allows posting XML, right? that is, you can't
> post binary/text content?
That's true. I assume it should use code similar to the serializers to support
all types of content? I will have a look at that.

> * we have to look at the implication for caching, if any (i.e. the URL
> generator shouldn't cache at all in this case)
Since I have no idea how caching works at all, I do not know if this is the
case, or not.

Kind regards,
Ralf


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Efficient POST/PUT queries

Ralf Jung
In reply to this post by Erik Bruchez
Hi Erik,


> * currently, this only allows posting XML, right? that is, you can't
> post binary/text content?
> * we have to look at the implication for caching, if any (i.e. the URL
> generator shouldn't cache at all in this case)
Attached version of the patch should now support all kinds of contents in the
POST body. As usual, XML can be passed like this

<p:input name="data" href="#xml-post-body" transform="oxf:xml-converter">
<config/></p:input>

However, currently the body is never cached, and there is no way to change the
serialization settings - I did not know the best way to do that.

As for the caching of the result, I tried to make it so that with a non-GET
method, it's never cached. But I do not know if that's what I actually did,
and I might have broken caching completely along the way.

Kind regards,
Ralf


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws

0001-implemenet-POST-and-PUT-for-URLGenerator.patch (17K) Download Attachment