Data lost on "Save Document"

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Data lost on "Save Document"

lazx
Just ran into a large problem.  Here is the situaton:

Version: Orbeon 3.7 Form Runner/Builder Nightly Build

First 4 completed forms were saved successfully. For the 5th+ form the following happens:

1) A new form is created.
2) Information is entered into the form
3) Save Document is pressed -> Form Runner reports that the document has been saved successfully
4) Document is NOT saved to the database (eXist).

Ouch!

Any suggestions on what is going on here??
Reply | Threaded
Open this post in threaded view
|

Re: Data lost on "Save Document"

lazx
Let me describe the situation a little more.

There were multiple users filling out forms simultaneously.  There was also some (x)queries in eXist occurring when the users were completing these forms.

I looked into the eXist database, and it appears the the "resource" collection for each form data document (data.xml) was not created - and nor was the data.xml document in the eXist db - so, the data is really lost.

As of 5 minutes ago, it seems that I am able to save form data again.  At this point, I am the only one using Form Runner.

Let me know if there is any other information needed.

Thanks!

lazx wrote
Just ran into a large problem.  Here is the situaton:

Version: Orbeon 3.7 Form Runner/Builder Nightly Build

First 4 completed forms were saved successfully. For the 5th+ form the following happens:

1) A new form is created.
2) Information is entered into the form
3) Save Document is pressed -> Form Runner reports that the document has been saved successfully
4) Document is NOT saved to the database (eXist).

Ouch!

Any suggestions on what is going on here??
Reply | Threaded
Open this post in threaded view
|

Re: Re: Data lost on "Save Document"

Erik Bruchez
Administrator
I am not sure why this is happening, but it sounds lie there is an  
eXist error, and the error should be propagated to the client so you  
know the data wasn't saved. But it clearly isn't.

Do you see errors in the logs?

-Erik

On Jul 10, 2008, at 12:28 PM, lazx wrote:

>
> Let me describe the situation a little more.
>
> There were multiple users filling out forms simultaneously.  There  
> was also
> some (x)queries in eXist occurring when the users were completing  
> these
> forms.
>
> I looked into the eXist database, and it appears the the "resource"
> collection for each form data document (data.xml) was not created -  
> and nor
> was the data.xml document in the eXist db - so, the data is really  
> lost.
>
> As of 5 minutes ago, it seems that I am able to save form data  
> again.  At
> this point, I am the only one using Form Runner.
>
> Let me know if there is any other information needed.
>
> Thanks!
>
>
> lazx wrote:
>>
>> Just ran into a large problem.  Here is the situaton:
>>
>> Version: Orbeon 3.7 Form Runner/Builder Nightly Build
>>
>> First 4 completed forms were saved successfully. For the 5th+ form  
>> the
>> following happens:
>>
>> 1) A new form is created.
>> 2) Information is entered into the form
>> 3) Save Document is pressed -> Form Runner reports that the  
>> document has
>> been saved successfully
>> 4) Document is NOT saved to the database (eXist).
>>
>> Ouch!
>>
>> Any suggestions on what is going on here??
>>
>
> --
> View this message in context: http://www.nabble.com/Data-lost-on-%22Save-Document%22-tp18389797p18390030.html
> Sent from the ObjectWeb OPS - Users mailing list archive at  
> Nabble.com.
>
>
> --
> You receive this message as a subscriber of the [hidden email]  
> mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws
--
Orbeon Forms - Web Forms for the Enterprise Done the Right Way
http://www.orbeon.com/



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Data lost on "Save Document"

lazx
Hi Erik,

Thanks for your help.

I have checked the log in the /var/tomcat/orbeon/exist directory - did not see anything out of the norm.

Do you suggest I check anywhere else?

Also, while I have your attention - can you point me to some guidelines/instructions/config_files on how to get orbeon use a non-embedded exist/etc database?  For updating the nightly builds, the exist database not persisting is not going to work :)  And I'd also like to try the 1.3 dev release of exist.  

Would it be possible for orbeon to save to 2 locations during the submission process?  ie, eXist 1.2 db and 1.3beta...

Thanks again Erik for all your work - great app :)


Erik Bruchez wrote
I am not sure why this is happening, but it sounds lie there is an  
eXist error, and the error should be propagated to the client so you  
know the data wasn't saved. But it clearly isn't.

Do you see errors in the logs?

-Erik

On Jul 10, 2008, at 12:28 PM, lazx wrote:

>
> Let me describe the situation a little more.
>
> There were multiple users filling out forms simultaneously.  There  
> was also
> some (x)queries in eXist occurring when the users were completing  
> these
> forms.
>
> I looked into the eXist database, and it appears the the "resource"
> collection for each form data document (data.xml) was not created -  
> and nor
> was the data.xml document in the eXist db - so, the data is really  
> lost.
>
> As of 5 minutes ago, it seems that I am able to save form data  
> again.  At
> this point, I am the only one using Form Runner.
>
> Let me know if there is any other information needed.
>
> Thanks!
>
>
> lazx wrote:
>>
>> Just ran into a large problem.  Here is the situaton:
>>
>> Version: Orbeon 3.7 Form Runner/Builder Nightly Build
>>
>> First 4 completed forms were saved successfully. For the 5th+ form  
>> the
>> following happens:
>>
>> 1) A new form is created.
>> 2) Information is entered into the form
>> 3) Save Document is pressed -> Form Runner reports that the  
>> document has
>> been saved successfully
>> 4) Document is NOT saved to the database (eXist).
>>
>> Ouch!
>>
>> Any suggestions on what is going on here??
>>
>
> --
> View this message in context: http://www.nabble.com/Data-lost-on-%22Save-Document%22-tp18389797p18390030.html
> Sent from the ObjectWeb OPS - Users mailing list archive at  
> Nabble.com.
>
>
> --
> You receive this message as a subscriber of the ops-users@ow2.org  
> mailing list.
> To unsubscribe: mailto:ops-users-unsubscribe@ow2.org
> For general help: mailto:sympa@ow2.org?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws

--
Orbeon Forms - Web Forms for the Enterprise Done the Right Way
http://www.orbeon.com/



--
You receive this message as a subscriber of the ops-users@ow2.org mailing list.
To unsubscribe: mailto:ops-users-unsubscribe@ow2.org
For general help: mailto:sympa@ow2.org?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Data lost on "Save Document"

Alessandro Vernet
Administrator
On Thu, Jul 10, 2008 at 7:24 PM, lazx <[hidden email]> wrote:
> I have checked the log in the /var/tomcat/orbeon/exist directory - did not
> see anything out of the norm.

You should find errors with submissions in the orbeon.log. If you are
using Tomcat, it should be in Tomcat's log directory. You can change
the location of this file in config/log4j.xml.

> Also, while I have your attention - can you point me to some
> guidelines/instructions/config_files on how to get orbeon use a non-embedded
> exist/etc database?  For updating the nightly builds, the exist database not
> persisting is not going to work :)  And I'd also like to try the 1.3 dev
> release of exist.

You can change the location of eXist in properties.xml. You will see
two references there:

1) oxf.fr.persistence.service.exist.uri points to the eXist used by Form Runner.
2) oxf.xforms.store.application.uri points to the eXist used by the
XForms Store.

Most likely you just want to change #1 and not #2.

Alex
--
Orbeon Forms - Web 2.0 Forms, open-source, for the Enterprise
Orbeon's Blog: http://www.orbeon.com/blog/
Personal Blog: http://avernet.blogspot.com/
Twitter - http://twitter.com/avernet


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Data lost on "Save Document"

lazx
org.orbeon.oxf.xforms.process.XFormsServer + org.orbeon.oxf.xforms.process.XFormsModelSubmission were both commented out in the log4j.xml...  assuming orbeon was therefore not logging that information.

I have uncommented those "categories" and am wondering how do I log "categories" of differing priorities (ie, warn, info, debug) to the same logging file.

Does this "root" entity act as a catch all for all debug/warn/info log messages???

<root>
  <priority value="debug"/>
  <appender-ref ref="RollingFileAdapter"/>
</root>


Thanks Alessandro + Erik

Alessandro Vernet wrote
On Thu, Jul 10, 2008 at 7:24 PM, lazx <lazx888@gmail.com> wrote:
> I have checked the log in the /var/tomcat/orbeon/exist directory - did not
> see anything out of the norm.

You should find errors with submissions in the orbeon.log. If you are
using Tomcat, it should be in Tomcat's log directory. You can change
the location of this file in config/log4j.xml.

> Also, while I have your attention - can you point me to some
> guidelines/instructions/config_files on how to get orbeon use a non-embedded
> exist/etc database?  For updating the nightly builds, the exist database not
> persisting is not going to work :)  And I'd also like to try the 1.3 dev
> release of exist.

You can change the location of eXist in properties.xml. You will see
two references there:

1) oxf.fr.persistence.service.exist.uri points to the eXist used by Form Runner.
2) oxf.xforms.store.application.uri points to the eXist used by the
XForms Store.

Most likely you just want to change #1 and not #2.

Alex
--
Orbeon Forms - Web 2.0 Forms, open-source, for the Enterprise
Orbeon's Blog: http://www.orbeon.com/blog/
Personal Blog: http://avernet.blogspot.com/
Twitter - http://twitter.com/avernet


--
You receive this message as a subscriber of the ops-users@ow2.org mailing list.
To unsubscribe: mailto:ops-users-unsubscribe@ow2.org
For general help: mailto:sympa@ow2.org?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws


-----
Orbeon Forms - Web 2.0 Forms, open-source, for the Enterprise
http://www.orbeon.com/
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Data lost on "Save Document"

lazx
In reply to this post by Alessandro Vernet
This problem is seeming to be related to lack/misconfiguration of memory.

catalina.out:
Exception in thread "exist_QuartzScheduler_QuartzSchedulerThread" Exception in thread "exist_QuartzScheduler_QuartzSchedulerThread" java.lang.OutOfMemoryError: PermGen space
Exception in thread "exist_QuartzScheduler_QuartzSchedulerThread" java.lang.OutOfMemoryError: PermGen space
Exception in thread "exist_QuartzScheduler_QuartzSchedulerThread" java.lang.OutOfMemoryError: PermGen space
Jul 10, 2008 4:41:05 PM org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
SEVERE: Caught exception (java.lang.OutOfMemoryError: PermGen space) executing org.apache.tomcat.util.net.LeaderFollowerWorkerThread@10580fa, terminating thread
Exception in thread "http-8080-Processor20" java.lang.OutOfMemoryError: PermGen space
Exception in thread "http-8080-Processor14" java.lang.OutOfMemoryError: PermGen space
java.lang.reflect.InvocationTargetException
Exception in thread "main" java.lang.OutOfMemoryError: PermGen space

Tomcat memory configuration:
CATALINA_OPTS="-Xmx256M"

Exist memory configuration:
<db-connection cacheSize="48M" collectionCache="24M" database="native"
        files="exist-data" pageSize="4096">

Orbeon memory configuration (am I missing some options?):
<property as="xs:integer" name="oxf.cache.size"                                  value="500"/>



Did I need to have the tomcat -Xms memory option set to 256M??  Also, is 256M enough memory for eXist + orbeon to coexist on tomcat?


Alessandro Vernet wrote
On Thu, Jul 10, 2008 at 7:24 PM, lazx <lazx888@gmail.com> wrote:
> I have checked the log in the /var/tomcat/orbeon/exist directory - did not
> see anything out of the norm.

You should find errors with submissions in the orbeon.log. If you are
using Tomcat, it should be in Tomcat's log directory. You can change
the location of this file in config/log4j.xml.

> Also, while I have your attention - can you point me to some
> guidelines/instructions/config_files on how to get orbeon use a non-embedded
> exist/etc database?  For updating the nightly builds, the exist database not
> persisting is not going to work :)  And I'd also like to try the 1.3 dev
> release of exist.

You can change the location of eXist in properties.xml. You will see
two references there:

1) oxf.fr.persistence.service.exist.uri points to the eXist used by Form Runner.
2) oxf.xforms.store.application.uri points to the eXist used by the
XForms Store.

Most likely you just want to change #1 and not #2.

Alex
--
Orbeon Forms - Web 2.0 Forms, open-source, for the Enterprise
Orbeon's Blog: http://www.orbeon.com/blog/
Personal Blog: http://avernet.blogspot.com/
Twitter - http://twitter.com/avernet


--
You receive this message as a subscriber of the ops-users@ow2.org mailing list.
To unsubscribe: mailto:ops-users-unsubscribe@ow2.org
For general help: mailto:sympa@ow2.org?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws


-----
Orbeon Forms - Web 2.0 Forms, open-source, for the Enterprise
http://www.orbeon.com/
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Re: Data lost on "Save Document"

Erik Bruchez
Administrator
Many applications seem to hit this issue (with Tomcat, Eclipse, etc.).  
It is not entirely clear why this space runs out.

Do you hot-redeploy your app into Tomcat, or do you shutdown Tomcat  
instead?

This said the permgen space is a separate VM setting. It can be  
increased with a VM option, e.g.:

   -XX:MaxPermSize=128m

This may or may not solve your problem. Please let us now.

-Erik

On Jul 14, 2008, at 7:26 AM, lazx wrote:

>
> This problem is seeming to be related to lack/misconfiguration of  
> memory.
>
> catalina.out:
> Exception in thread "exist_QuartzScheduler_QuartzSchedulerThread"  
> Exception
> in thread "exist_QuartzScheduler_QuartzSchedulerThread"
> java.lang.OutOfMemoryError: PermGen space
> Exception in thread "exist_QuartzScheduler_QuartzSchedulerThread"
> java.lang.OutOfMemoryError: PermGen space
> Exception in thread "exist_QuartzScheduler_QuartzSchedulerThread"
> java.lang.OutOfMemoryError: PermGen space
> Jul 10, 2008 4:41:05 PM
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
> SEVERE: Caught exception (java.lang.OutOfMemoryError: PermGen space)
> executing  
> org.apache.tomcat.util.net.LeaderFollowerWorkerThread@10580fa,
> terminating thread
> Exception in thread "http-8080-Processor20"  
> java.lang.OutOfMemoryError:
> PermGen space
> Exception in thread "http-8080-Processor14"  
> java.lang.OutOfMemoryError:
> PermGen space
> java.lang.reflect.InvocationTargetException
> Exception in thread "main" java.lang.OutOfMemoryError: PermGen space
>
> Tomcat memory configuration:
> CATALINA_OPTS="-Xmx256M"
>
> Exist memory configuration:
> <db-connection cacheSize="48M" collectionCache="24M" database="native"
>        files="exist-data" pageSize="4096">
>
> Orbeon memory configuration (am I missing some options?):
> <property as="xs:integer" name="oxf.cache.size"
> value="500"/>
>
>
>
> Did I need to have the tomcat -Xms memory option set to 256M??  
> Also, is
> 256M enough memory for eXist + orbeon to coexist on tomcat?
>
>
>
> Alessandro Vernet wrote:
>>
>> On Thu, Jul 10, 2008 at 7:24 PM, lazx <[hidden email]> wrote:
>>> I have checked the log in the /var/tomcat/orbeon/exist directory -  
>>> did
>>> not
>>> see anything out of the norm.
>>
>> You should find errors with submissions in the orbeon.log. If you are
>> using Tomcat, it should be in Tomcat's log directory. You can change
>> the location of this file in config/log4j.xml.
>>
>>> Also, while I have your attention - can you point me to some
>>> guidelines/instructions/config_files on how to get orbeon use a
>>> non-embedded
>>> exist/etc database?  For updating the nightly builds, the exist  
>>> database
>>> not
>>> persisting is not going to work :)  And I'd also like to try the  
>>> 1.3 dev
>>> release of exist.
>>
>> You can change the location of eXist in properties.xml. You will see
>> two references there:
>>
>> 1) oxf.fr.persistence.service.exist.uri points to the eXist used by  
>> Form
>> Runner.
>> 2) oxf.xforms.store.application.uri points to the eXist used by the
>> XForms Store.
>>
>> Most likely you just want to change #1 and not #2.
>>
>> Alex
>> --
>> Orbeon Forms - Web 2.0 Forms, open-source, for the Enterprise
>> Orbeon's Blog: http://www.orbeon.com/blog/
>> Personal Blog: http://avernet.blogspot.com/
>> Twitter - http://twitter.com/avernet
>>
>>
>> --
>> You receive this message as a subscriber of the [hidden email]  
>> mailing
>> list.
>> To unsubscribe: mailto:[hidden email]
>> For general help: mailto:[hidden email]?subject=help
>> OW2 mailing lists service home page: http://www.ow2.org/wws
>>
>>
>> -----
>> Orbeon Forms - Web 2.0 Forms, open-source, for the Enterprise
>> http://www.orbeon.com/
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Data-lost-on-%22Save-Document%22-tp18389797p18445079.html
> Sent from the ObjectWeb OPS - Users mailing list archive at  
> Nabble.com.
>
>
> --
> You receive this message as a subscriber of the [hidden email]  
> mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws
--
Orbeon Forms - Web Forms for the Enterprise Done the Right Way
http://www.orbeon.com/



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Re: Data lost on "Save Document"

lazx
This post was updated on .
Thanks Erik...  will give that a try.  So many memory options with tomcat! :(

I restart tomcat to deploy the app.

And some maybe related logs...   doubt its the source of the error, but may as well report.

Warnings that eXist is producing:

2008-07-15 11:29:49,422 INFO  EXistServlet  - registering XMLDB driver                                                  
2008-07-15 11:29:49,454 INFO  EXistServlet  - Using default user guest for all unauthorized requests.                    
2008-07-15 11:29:49,480 WARN  EXistServlet  - Looking for a valid Parser...                                              
Checking for Xerces, found version Xerces-J 2.9.0 needed version Xerces-J 2.9.1                                          
Warning: Failed find a valid Parser!                                                                                    

Please add an appropriate Parser to the class-path, e.g. in the 'endorsed' folder of the servlet container or in the 'endorsed' folder of the JRE.

2008-07-15 11:29:49,484 WARN  EXistServlet  - Looking for a valid Transformer...
Checking for Saxon, found version 8.8 needed version 9.0.0.4
Checking for Saxon, found version 8.8 needed version 9.0.0.3
Checking for Xalan, found version Xalan Java 2.5.1 needed version Xalan Java 2.7.0
Checking for Saxon, found version 8.8 needed version 8.9.0.3
Warning: Failed find a valid Transformer!

Please add an appropriate Transformer to the class-path, e.g. in the 'endorsed' folder of the servlet container or in the 'endorsed' folder of the JRE.


Erik Bruchez wrote
Many applications seem to hit this issue (with Tomcat, Eclipse, etc.).  
It is not entirely clear why this space runs out.

Do you hot-redeploy your app into Tomcat, or do you shutdown Tomcat  
instead?

This said the permgen space is a separate VM setting. It can be  
increased with a VM option, e.g.:

   -XX:MaxPermSize=128m

This may or may not solve your problem. Please let us now.

-Erik
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Re: Re: Data lost on "Save Document"

Erik Bruchez
Administrator
Good. Please let us know.

The eXist warnings should not be a problem.

-Erik

On Jul 15, 2008, at 8:41 AM, lazx wrote:

>
> Thanks Erik...  will give that a try.  So many memory options with  
> tomcat! :(
>
> And some maybe related logs...   doubt its the source of the error,  
> but may
> as well report.
>
> Warnings that eXist is producing:
>
> 2008-07-15 11:29:49,422 INFO  EXistServlet  - registering XMLDB driver
> 2008-07-15 11:29:49,454 INFO  EXistServlet  - Using default user  
> guest for
> all unauthorized requests.
> 2008-07-15 11:29:49,480 WARN  EXistServlet  - Looking for a valid  
> Parser...
> Checking for Xerces, found version Xerces-J 2.9.0 needed version  
> Xerces-J
> 2.9.1
> Warning: Failed find a valid Parser!
>
> Please add an appropriate Parser to the class-path, e.g. in the  
> 'endorsed'
> folder of the servlet container or in the 'endorsed' folder of the  
> JRE.
>
> 2008-07-15 11:29:49,484 WARN  EXistServlet  - Looking for a valid
> Transformer...
> Checking for Saxon, found version 8.8 needed version 9.0.0.4
> Checking for Saxon, found version 8.8 needed version 9.0.0.3
> Checking for Xalan, found version Xalan Java 2.5.1 needed version  
> Xalan Java
> 2.7.0
> Checking for Saxon, found version 8.8 needed version 8.9.0.3
> Warning: Failed find a valid Transformer!
>
> Please add an appropriate Transformer to the class-path, e.g. in the
> 'endorsed' folder of the servlet container or in the 'endorsed'  
> folder of
> the JRE.
>
>
>
> Erik Bruchez wrote:
>>
>> Many applications seem to hit this issue (with Tomcat, Eclipse,  
>> etc.).
>> It is not entirely clear why this space runs out.
>>
>> Do you hot-redeploy your app into Tomcat, or do you shutdown Tomcat
>> instead?
>>
>> This said the permgen space is a separate VM setting. It can be
>> increased with a VM option, e.g.:
>>
>>   -XX:MaxPermSize=128m
>>
>> This may or may not solve your problem. Please let us now.
>>
>> -Erik
>>
> --
> View this message in context: http://www.nabble.com/Data-lost-on-%22Save-Document%22-tp18389797p18468382.html
> Sent from the ObjectWeb OPS - Users mailing list archive at  
> Nabble.com.
>
>
> --
> You receive this message as a subscriber of the [hidden email]  
> mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws
--
Orbeon Forms - Web Forms for the Enterprise Done the Right Way
http://www.orbeon.com/



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws