Autosave Broken in 4.9?

Update/Summary: The main issue was the addition of the FormRunnerAuthFilter in 4.9 and the fact that we have and maintain our own web.xml for the orbeon war. That filter was not included in our web.xml and so did not add the orbeon-username and orbeon-group headers that are required for autosave to work correctly.

I've been having trouble getting autosave to work since I upgraded to 4.9. I just rolled back to 4.8.1 and it is working again. In 4.9 when I access the edit url for a form orbeon doesn't call the search function on my persistence API to see if there are any draft forms, in 4.8.1 it does.

The query is  a post to /cpf/search/cpf/fredd with the following content

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<search>
    <drafts>only</drafts>
    <page-size>10</page-size>
    <page-number>1</page-number>
    <lang>en</lang>
</search>

which returns the draft if there is one, orbeon then prompts the user to select the saved or draft version for edit.

I noticed also that the autosaves are not happening at all with 4.9 as well.

Is this a regression or has something changed in the way it is configured. The configuration that works for 4.8.1 is

    <property as="xs:boolean" name="oxf.fr.persistence.cpf.autosave" value="true" />
<!-- Autosave every 5 seconds (production installs should be longer) -->
    <property as="xs:integer" name="oxf.fr.detail.autosave-delay" value="5000" />

Hi, Any comment on this? Is autosave working for anyone on 4.9?

thanks
brian...

Brian,

It was tested as part if the 4.9 release and shouldn't have changed much since 4.8.x. Just to be sure, do you satisfy the conditions listed here under "The following conditions must be met for autosave to happen"?

    https://github.com/orbeon/orbeon-forms/wiki/Form-Runner-~-Autosave

-Erik

Erik Bruchez wrote

It was tested as part if the 4.9 release and shouldn't have changed much since 4.8.x. Just to be sure, do you satisfy the conditions listed here under "The following conditions must be met for autosave to happen"?

    https://github.com/orbeon/orbeon-forms/wiki/Form-Runner-~-Autosave

Hi Eric,
  I've updated to 4.10 now and I still cannot get drafts to work correctly. I have re-read the conditions and I think I am meeting them since, as described below, the autosave does work, sometimes.

Autosave works if I view the form (http://localhost/orbeon/fr/cpf/freddd/view/201509011)  and click on the "Edit" button but it does not work if I go directly to the form runner edit URL (http://localhost/orbeon/fr/cpf/freddd/edit/201509011) as used from the summary page.

If I use the "Edit" button and then abandon the edit after seeing an autosave and then reload the view URL and click edit again I am not prompted to reuse the draft that was saved. In my persistence logging it is apparent that the persistence layer was not even asked if there are any drafts (I used to see a GET for the draft version when running version 4.8).

The configuration options for this in my app are

    <property as="xs:boolean" name="oxf.fr.persistence.cpf.autosave" value="true" />
    <property as="xs:integer" name="oxf.fr.detail.autosave-delay.cpf.*" value="1000" />
    <property as="xs:boolean" name="oxf.fr.persistence.cpf.permissions" value="true" />
    <property as="xs:boolean" name="oxf.fr.persistence.cpf.versioning" value="false" />

I've also configured the mysql persistence and see the same behaviour. Go to the view URL and click Edit and autosave occurs, go direct to the edit url and it doesn't and in either case I never get prompted to recover a draft version when I go to edit and there is a draft.

The mysql configuration is

    <property as="xs:boolean" name="oxf.fr.persistence.mysql.active" value="true" />
    <property as="xs:string" name="oxf.fr.persistence.provider.mysql.*.*" value="mysql" />
    <property as="xs:boolean" name="oxf.fr.persistence.mysql.versioning" value="false" />
    <property as="xs:anyURI" name="oxf.fr.persistence.mysql.uri" value="/fr/service/mysql" />
    <property as="xs:string" name="oxf.fr.persistence.mysql.datasource" value="orbeon" />
    <property as="xs:boolean" name="oxf.fr.persistence.mysql.autosave" value="true" />  
    <property as="xs:integer" name="oxf.fr.detail.autosave-delay.mysql.*" value="5000" />

Version string is "Orbeon Forms 4.10.0.201508261723 PE"

CONTENTS DELETED

CONTENTS DELETED

Thanks for checking Bruno. We also tested autosave with 4.10 as part of our test plan:

    https://github.com/orbeon/orbeon-forms/wiki/Contributors-~-Test-Plan-~-Autosave-and-Permissions

So Brian, I am a bit puzzled. I think I have only ever tried this going directly to /new or /edit.

This said, again, I can't rule out a bug. I will try to test autosave with MySQL again, going to /edit directly.

-Erik

I suspect there is something in my configuration that is causing a problem and I think it is in the area of authentication. I am using standard web based authentication but I think the setup of the groups may be affecting how this works. If I deploy the standard orbeon 4.10 war it works OK.

I'm going to be travelling interstate over the few weeks so will chase this up next month.

thanks

Brian,

That sound good. It'd be great if you didn't give up on this one because if there is a bug in Orbeon Forms we want to fix it!

-Erik

Back onto this. I'm going through how our authentication works and comparing it to the default authentication configured in the standard orbeon war (when uncommented in the web.xml).

I cannot see any differences but there must be something in the session that we are not setting up that is used by orbeon to decide if autosave is used or not. Is the a description of what is required in an authenticated session for autosave to work? Can you point me to the code used for this decision?

thanks

When I view a form and then click edit, what is the sequence of operations that I should see at the persistence layer?

Should I see some sort of query that tries to determine if there is a draft version of the form before the query that gets the current version of the form? All I am seeing is a query to get the form template followed by a query to get the form. There isn't any (visible) attempt to find out if there is a draft version first.

I seem to recall that in version 4.8 that there was a query to get the draft version that would return a 404 if there wasn't one prior to the query to fetch the current version.

I have versioning and permissions disabled for our persistence layer.

The sequence of calls to our persistence layer is as follows

GET /cpf/crud/cpf/freddd/form/form.xhtml Request Content-Type: null
Path Parameters:
  appname: [cpf]
  formname: [freddd]
Query Parameters:
  document: [201509041]
Request Headers:
  orbeon-versioning: false
  if-modified-since: Tue, 29 Sep 2015 07:18:25 GMT
  cookie: JSESSIONIDSSO=0CEF79D25DE003CA4E1A6C71E3B4F4E4, JSESSIONID=BFAA70DD362F51A69C3D79F08449E351
  orbeon-for-document-id: 201509041
  orbeon-username: system
  orbeon-token: 804c8e4c73008ae6da50a6240351a94e9475fb1f
  host: localhost:8080
  connection: Keep-Alive
  cookie2: $Version=1
  accept-encoding: gzip,deflate
Cookies:
JSESSIONIDSSO: path="null" value="0CEF79D25DE003CA4E1A6C71E3B4F4E4" version=0 domain="null" value=-1 secure=false comment="null"
JSESSIONID: path="null" value="BFAA70DD362F51A69C3D79F08449E351" version=0 domain="null" value=-1 secure=false comment="null"
orbeon-for-document-id = 201509041
Returning form template cpf/freddd/5
200 OK /cpf/crud/cpf/freddd/form/form.xhtml Response Content-Type: null
Response Headers:
  X-Powered-By: Servlet/3.0; JBossAS-6
  Pragma: No-cache
  Cache-Control: no-cache
  Expires: Thu, 01 Jan 1970 10:00:00 EST
  orbeon-form-definition-version: 5
Beans found for class au.com.infomedix.orbeon.persistence.Persistence : [Managed Bean [class au.com.infomedix.orbeon.persistence.Persistence] with qualifiers [@Any @Default]]
GET /cpf/crud/cpf/freddd/data/201509041/data.xml Request Content-Type: null
Path Parameters:
  id: [201509041]
  appname: [cpf]
  formname: [freddd]
Query Parameters:
Request Headers:
  orbeon-versioning: false
  orbeon-token: 804c8e4c73008ae6da50a6240351a94e9475fb1f
  orbeon-username: system
  cookie: JSESSIONIDSSO=0CEF79D25DE003CA4E1A6C71E3B4F4E4, JSESSIONID=BFAA70DD362F51A69C3D79F08449E351
  host: localhost:8080
  connection: Keep-Alive
  cookie2: $Version=1
  accept-encoding: gzip,deflate
Cookies:
JSESSIONIDSSO: path="null" value="0CEF79D25DE003CA4E1A6C71E3B4F4E4" version=0 domain="null" value=-1 secure=false comment="null"
JSESSIONID: path="null" value="BFAA70DD362F51A69C3D79F08449E351" version=0 domain="null" value=-1 secure=false comment="null"
Returning xform document for cpf/freddd/data/201509041, file "/Users/bwallis/InfoMedix/JBoss/UDR/udr_data/repository/././partition-1/documentstore/2015/09/29/201509291.xform"
200 OK /cpf/crud/cpf/freddd/data/201509041/data.xml Response Content-Type: null
Response Headers:
  X-Powered-By: Servlet/3.0; JBossAS-6
  Pragma: No-cache
  Cache-Control: no-cache
  Expires: Thu, 01 Jan 1970 10:00:00 EST

Brian,

You can see the code which checks whether to enable autosave here:

    https://github.com/orbeon/orbeon-forms/blob/8b285b4657fc1e4cf07b3602a773048876f810f9/src/resources/apps/fr/includes/persistence/persistence-model.xml#L43

autosaveSupported() is here:

    https://github.com/orbeon/orbeon-forms/blob/8b285b4657fc1e4cf07b3602a773048876f810f9/src/main/scala/org/orbeon/oxf/fr/FormRunnerPersistence.scala#L86

and xpathAllAuthorizedOperations(), which is tricky because it ends up calling a lot of authorization code:

    https://github.com/orbeon/orbeon-forms/blob/8b285b4657fc1e4cf07b3602a773048876f810f9/src/main/scala/org/orbeon/oxf/fr/FormRunnerPermissions.scala#L62

You should see a call to the search API to check the draft.

What is `cpf`? Is that a custom persistence layer?

-Erik

Thanks Eric,
  Yes, cpf is our custom persistence layer, it is an interface into our (custom) medical records database.

I've been examining how that code works and trying to use the debugger to see what is going on but my eclipse with scala support is not working very well, I can set breakpoints but examining values returned from functions is difficult so it is hard to work out what is going on. What do you use for development/debug of the scala code? Also, is there anyway to see what the code in the persistence-model.xml is doing?

thanks,
brian...

Brian,

And your persistence layer does support autosave, and it is enabled?

    <property
      as="xs:boolean"
      name="oxf.fr.persistence.cpf.autosave"
      value="true"/>

For debugging, we use IntelliJ and have no experience with Eclipse, sorry. Scala debugging in IntelliJ is far from perfect but usually serviceable.

For persistence-model.xml that's a bit trickier. You can enable XForms logging:

   http://wiki.orbeon.com/forms/doc/developer-guide/xforms-logging

To trace what's happening you can also log with xf:message as is done here:

    https://github.com/orbeon/orbeon-forms/blob/b7629170534a9dfd9bf9e89d37269f2b32767fce/src/resources/apps/fr/includes/persistence/persistence-model.xml#L89

-Erik

> And your persistence layer does support autosave, and it is enabled?

And if I had checked above in the thread, I would have seen that yes, that property is set.

-Erik

Correct, the property is set and we support autosave in our persistence layer implementation.

Autosave is working as noted earlier as long as I click on the "Edit" button when viewing a form. It doesn't work if I use the edit URL. The reason for this is the following code in persistence-model.xml:

    <xf:var name="autosave-enabled"     value="frf:autosaveSupported($app, $form)
                                           and $autosave-delay gt 0
                                           and normalize-space(xxf:get-request-header('orbeon-username')) != ''
                                           and frf:xpathAllAuthorizedOperations(
                                                    xxf:instance('fr-form-metadata')/permissions,
                                                    xxf:get-request-header('orbeon-username'),
                                                    xxf:get-request-header('orbeon-group')
                                               ) = ('*', 'update')
                                           and $mode = ('new', 'edit')
                                           and not($is-fb)"/>

When I go directly to the edit page (url http://localhost/orbeon/fr/cpf/freddd/edit/201509041) orbeon-username is not set as a request header and so autosave-enabled is always false. If click edit in the view page (url http://localhost/orbeon/fr/cpf/freddd/view/201509041) then the request header orbeon-username is set and autosave can evaluate as true.

If I remove the authorization checks from the above expression and use

    <xf:var name="autosave-enabled"     value="frf:autosaveSupported($app, $form)
                                           and $autosave-delay gt 0 
                                           and $mode = ('new', 'edit')
                                           and not($is-fb)"/>

then autosave works when I go directly to the edit url and I also get prompted to select the draft or saved document if there is a draft (the search request is sent to my persistence layer).

The only thing that isn't working now is when I view the document that has a draft version and click the edit button. In this case I am not prompted to select the draft or saved document and it just opens the saved document (no query to the persistence layer). autosave in this case is set to true so there is something else going on here.

brian...

I just tried adding 'view' the the mode check and I get prompted to select the saved or draft version when viewing and then the edit button edits the correct one. This is not quite right of course as you want to be prompted when you edit, not when you view but this is another clue as to what might be going on. (this also broke the view and the form didn't display correctly).

brian...

Brian,

It seems to me then that the key is to understand why you don't have the `orbeon-username` header set in edit mode. The username must be available at that point.

BTW that header is set internally based on the authentication method (`header` or `container` method).

-Erik

Erik Bruchez wrote

Brian,

It seems to me then that the key is to understand why you don't have the `orbeon-username` header set in edit mode. The username must be available at that point.

BTW that header is set internally based on the authentication method (`header` or `container` method).

-Erik

So, if I understand you, when I do a request to edit a form, ie: go to the url http://localhost/orbeon/fr/cpf/freddd/edit/201509041, the orbeon-username value is injected into that request while it is being processed by the orbeon server. The value for this would come from the current session since I am using container based authentication and have oxf.fr.authentication.method set to "container".

Where does this occur? I had a look through the code and haven't found it.

> So, if I understand you, when I do a request to edit a form, ie: go to the url http://localhost/orbeon/fr/cpf/freddd/edit/201509041, the orbeon-username value is injected into that request while it is being processed by the orbeon server.

That's correct. This is done by a filter (FormRunnerAuthFilter) since 4.9, see:

    http://blog.orbeon.com/2015/05/orbeon-forms-49.html

and:

    https://github.com/orbeon/orbeon-forms/blob/b7629170534a9dfd9bf9e89d37269f2b32767fce/descriptors/orbeon-war-web.xsl#L204

But I assume that filter is in place in your app, or nothing would work.

And the value would come not from the session, but from the container. The container itself keeps a session with the user with the JSESSIONID cookie or similar.

-Erik