HTTP Service and mediatype

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

HTTP Service and mediatype

bwallis42
I'm trying to define a call to an http service but orbeon doesn't understand the returned media type.

Our REST service returns a content type of

Content-Type=application/vnd.infomedix+json;v="33"

unless we supply an Accept header value to ask for something else, so we could say

Accept=application/vnd.infomedix+xml;v="33"
or even
Accept=text/xml

in the request headers to get xml. This call will return a content type header with the same media string as was passed in.  This is all done so we can have control over our formats and versions of our REST interface.

Both of the above are specifying that the underlying structure is "json" or "xml" respectively.

When the type is the json variant I would expect that orbeon would recognise this as json and convert it into the appropriate xml model in the instance but this is not happening, instead I get an error

2017-01-27 16:32:50,981 72373259 ERROR [ajp-0.0.0.0-8209-2] processor.XFormsServer - xforms-submit-error - Mediatype is neither text nor XML for replace="text": application/vnd.infomedix+json
2017-01-27 16:32:50,987 72373265 ERROR [ajp-0.0.0.0-8209-2] processor.XFormsServer - xforms-submit-done|error - setting body document {body: "
{
  "patientNs" : "IHN",
  "dob" : "19801012",
...
  "firstName" : "Emily",
  "lastName" : "YOUNG",
}"}

I could use the xml variant if I could supply an Accept header to the HTTP service definition but there doesn't seem to be a way to do that.

So,
1)  Is there a way to populate the Accept header? If not is it possible to get that added to the service definition?
2)  I think the error (above) is a bug, the content type of application/vnd.infomedix+json is a valid json type and should be accepted and converted into the xml model instance.

thanks.
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

Alessandro  Vernet
Administrator
Hi Brian,

1. You should be able to set the Accept header with an <xf:header>, and this is even the example that was used in the XForms spec [1].
2. Are you using Orbeon Forms 2016.1, or newer? I am asking as JSON support [2], and the automatic conversion of JSON to XML with XForms submissions is new in that version.

[1] https://www.w3.org/community/xformsusers/wiki/XForms_2.0#The_header_Element
[2] https://doc.orbeon.com/xforms/submission-json.html

Alex
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

bwallis42
Thanks Alex,
  I'll give the xf:header entry a go. I think it would be nice to be able to do this via the UI instead of having to edit the form source, not a great option for our customers.

I am using 2016.3 (trying to keep up to date!) so the json support should work but I don't think it recognises the form of media type we are returning by default, i.e. application/vnd.infomedix+json;v=33

I think this should work as that type is indicating that the base structure is json but is a vendor specific schema (and is version 33 of our API).

The Accept header should get me going for the moment as our API will return the requested media type, i.e. if I ask for text/xml then the media type returned will be text/xml.

thanks.
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

bwallis42
Hi Alex,
  I've tried the header in my submission and it doesn't work. The header is not inserted into the request (I even used wireshark to make sure it wasn't there).

I just added the xf:header element to the submission as shown below. Is this the correct way to do it?

thanks
brian...

              <xf:instance id="PatientInfo-instance" class="fr-service" xxf:exclude-result-prefixes="#all">
                <body xmlns:secure="java:org.orbeon.oxf.util.SecureUtils"
                      xmlns:frf="java:org.orbeon.oxf.fr.FormRunner"
                      xmlns:p="http://www.orbeon.com/oxf/pipeline"
                      xmlns:fbf="java:org.orbeon.oxf.fb.FormBuilder">&lt;params/&gt;</body>
            </xf:instance>
            <xf:submission id="PatientInfo-submission" class="fr-service"
                           ref="instance('fr-service-request-instance')"
                           resource="{concat(xxf:property('im.webservice.uri.patient'),'/patient/IHN:919191')}"
                           method="get"
                           serialization="none"
                           mediatype="none"
                           replace="instance"
                           instance="fr-service-response-instance">
                               <xf:header combine="replace" name="Accept" value="'text/xml'"/>
            </xf:submission>

Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

Alessandro  Vernet
Administrator
Hi Brian,

It could be a bug. Could you create a minimal standalone example that shows this, for instance doing a submission replace="all" to https://httpbin.org/headers and setting a header, and that shows that the header wasn't received?

Alex
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

bwallis42
OK, I think I have it. This doesn't work

<xf:header combine="replace" name="Accept" value="text/xml"/>

but this does

<xf:header combine="replace">
       <xf:name>Accept</xf:name>
       <xf:value>text/xml</xf:value>
</xf:header>

I think the first one is XForms 2.0 (https://www.w3.org/community/xformsusers/wiki/XForms_2.0#The_header_Element)

and the second one is XForms 1.1 (https://www.w3.org/TR/xforms/#submit-header)

So I can get it working for our application by editing the source and including the XForms 1.1 header element in the submission.

I have a few suggestions/bugs:

1) Support for adding headers from the dialog would be better than editing the source.

2) If I add the header to the source to set Accept to application/json it doesn't seem to be used when I click the test button in the HTTP Service editor. It does work when the form is tested. By not work I mean it shows a red "Test Results" tab for my service that is returning (by default) application/vnd.infomedix+json as I assume it is not recognising the result as json.

3) Sometimes the HTTP Service editor dialog pops up with neither tab selected. I think this occurs if you had the test tab open when you closed it last time. It is remembering the last tab that was open and trying to display that one.

4) I think the Content-Type headers returned that are of the form application/vnd.xxx+json should work and convert the json to xml as it does for application/json.

5) documenting the correct header format in the HTTP Services page would be useful for future users.

thanks,
brian...




Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

Alessandro  Vernet
Administrator
Hi Brian,

1. We have an RFE for this (see link below); I added a +1 from you.
2. You're saying that the test button on the HTTP Service editor doesn't work when the service is returning JSON, but the service works when running the form?
3. What do you mean by "with neither tab selected"? Could you share a screenshot?
4+5. Got it, we will look into it.

https://github.com/orbeon/orbeon-forms/issues/2530

Alex
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

bwallis42
Alessandro Vernet wrote
1. We have an RFE for this (see link below); I added a +1 from you.
Thanks.

2. You're saying that the test button on the HTTP Service editor doesn't work when the service is returning JSON, but the service works when running the form?
The header definition I add to the source apparently does not get used when the test button is clicked but it is used when the service is run from the form. Without the accept header the service is returning the content type of application/vnd.infomedix+json which orbeon is not recognising as json hence the error.

3. What do you mean by "with neither tab selected"? Could you share a screenshot?
There are two problems I've seen with this dialog

I.) appearing like this if the 'test' tab was selected the last time it was saved or cancelled



II.) appearing with no data the first time the http service dialog is used for an existing http service when editing an existing form template



If you cancel the http service dialog and open it again you get the correct settings in the dialog



An example where these problems occur can be found in the form cpf/form1 on demo.orbeon.com, there is an HTTP service defined in this form called 'headers'

4+5. Got it, we will look into it.
Thanks again Alex, your help is appreciated as always.
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

Alessandro  Vernet
Administrator
Hi Brian,

2. Got it: I added a noted about this to the description of issue 2530, so when we "officially" support Form Builder sending headers, we also make sure that this works with the text functionality inside the dialog.

3. Got it: we have issue 3092 which covers the dialog coming up empty. Hopefully this will also solved the content of the tabs completely missing. We'll have to check that once 3092 is fixed, which will hopefully happen soon. We'll fix it for sure, but I still added a +1 from you on the issue.

https://github.com/orbeon/orbeon-forms/issues/3092

Alex
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

Alessandro  Vernet
Administrator
In reply to this post by Alessandro Vernet
Hi Brian,

For the content type, JSON has two competing standards, RFC 7159 and ECMA-404. I don't see anything about the MIME type in  ECMA-404, but RFC 7159 registers the MIME type `application/json`. I am not sure if there is a spec for that, but it seems to be a custom practice to register MIME types of the form `application/xxx+json` for specific JSON formats, so we might want to support that pattern in Orbeon Forms. Would you agree with that assessment?

https://tools.ietf.org/html/rfc7159
http://www.iana.org/assignments/media-types/media-types.xhtml

Alex
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

bwallis42
Sound OK to me.

There is also some detail about vnd. and +suffix in rfc6838, in particular that the +suffix should be a registered +suffix for that structured media type, i.e. +xml means it should be xml syntax and +json means it should be json syntax. rfc7303 talks about the +xml suffix but I didn't find anything specifically about +json.

Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

Alessandro  Vernet
Administrator
Hi Brian,

Got it, so we created issue 3101 to support `application/xxx+json`, and are shooting to have this in 2017.1.

https://github.com/orbeon/orbeon-forms/issues/3101

Alex
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

Alessandro  Vernet
Administrator
And FYI, Erik just implemented the support for `application/xxx+json`.

Alex
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Service and mediatype

Alessandro  Vernet
Administrator
Also, I created the following issue for the support of the `name` and `value` attributes on `xf:header`:

https://github.com/orbeon/orbeon-forms/issues/3102

Alex
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet