Exceptions and response times under load
Locked
 
|
Hi all,
We see some weird behavior (under 'normal' load we see no problems) when Orbeon is under 'heavy' load (approx 6 req./sec). This is version 3.7.1. First of all the response times go skyhigh (sometimes above 10 sec), then we also see a number of exceptions (including NullPointerException). The exceptions often happen in both ajax requests and page load/submission. 1: 2009-07-24 00:37:22,873 ERROR [org.orbeon.oxf.webapp.ProcessorService] Exception at line 24 of oxf:/config/xforms-widgets.xsl (dispatching XForms event: eve nt='xforms-refresh', target id='xf-2') java.lang.NullPointerException at org.orbeon.oxf.xforms.XFormsControls$UpdateBindingsListener.startRepeatIteration(XFormsControls.java:690) at org.orbeon.oxf.xforms.XFormsControls.visitControlElementsHandleRepeat(XFormsControls.java:442) at org.orbeon.oxf.xforms.XFormsControls.visitControlElementsHandleRepeat(XFormsControls.java:483) at org.orbeon.oxf.xforms.XFormsControls.visitControlElementsHandleRepeat(XFormsControls.java:483) 2: 2009-07-24 q,856 ERROR [org.orbeon.oxf.webapp.ProcessorService] Exception at line 24 of oxf:/config/xforms-widgets.xsl (dispatching XForms event: eve nt='xforms-refresh', target id='xf-2') java.util.EmptyStackException at java.util.Stack.peek(Stack.java:79) at org.orbeon.oxf.xforms.XFormsContextStack.getCurrentBindingContext(XFormsContextStack.java:488) at org.orbeon.oxf.xforms.XFormsContextStack.pushBinding(XFormsContextStack.java:275) 3: 2009-07-24 00:16:11,061 WARN [org.exist.xmldb.LocalCollection] Resource 763F95D3-27C4-4CA8-2984-CD96D316791F not found 2009-07-24 00:16:11,082 ERROR [org.orbeon.oxf.webapp.ProcessorService] Exception at line 140, column 60 of oxf:/ops/pfc/xforms-epilogue.xpl (reading process or output: name='document', id='xhtml-data') org.orbeon.oxf.common.OXFException: [Lorg.orbeon.oxf.xforms.XFormsContainingDocument;@7a297 is not an XFormsContainingDocument at org.orbeon.oxf.xforms.state.XFormsDocumentCache$XFormsContainingDocumentPoolableObjetFactory.destroyObject(XFormsDocumentCache.java:160) at org.orbeon.oxf.util.SoftReferenceObjectPool.invalidateObject(SoftReferenceObjectPool.java:122) at org.orbeon.oxf.xforms.processor.XFormsToXHTML.doIt(XFormsToXHTML.java:234) 4: 2009-07-24 00:18:58,081 ERROR [org.orbeon.oxf.webapp.ProcessorService] Exception at line 140, column 60 of oxf:/ops/pfc/xforms-epilogue.xpl (reading process or output: name='document', id='xhtml-data') org.orbeon.oxf.common.OXFException: [Lorg.orbeon.oxf.xforms.XFormsContainingDocument;@18d1276 is not an XFormsContainingDocument at org.orbeon.oxf.xforms.state.XFormsDocumentCache$XFormsContainingDocumentPoolableObjetFactory.destroyObject(XFormsDocumentCache.java:160) at org.orbeon.oxf.util.SoftReferenceObjectPool.invalidateObject(SoftReferenceObjectPool.java:122) at org.orbeon.oxf.xforms.processor.XFormsToXHTML.doIt(XFormsToXHTML.java:234) 5: 2009-07-24 00:30:20,471 ERROR [org.orbeon.oxf.webapp.ProcessorService] Exception at line 140, column 60 of oxf:/ops/pfc/xforms-epilogue.xpl (reading process or output: name='document', id='xhtml-data') org.orbeon.oxf.common.OXFException: Attempt to clear context stack. at org.orbeon.oxf.xforms.XFormsContextStack.popBinding(XFormsContextStack.java:493) at org.orbeon.oxf.xforms.control.XFormsControl.getLabelHelpHintAlertValue(XFormsControl.java:291) 6: 2009-07-24 00:32:24,836 ERROR [org.orbeon.oxf.webapp.ProcessorService] Exception at line 24 of oxf:/config/xforms-widgets.xsl (dispatching XForms event: eve nt='xforms-refresh', target id='xf-2') org.orbeon.oxf.common.OXFException: Attempt to clear context stack. at org.orbeon.oxf.xforms.XFormsContextStack.popBinding(XFormsContextStack.java:493) at org.orbeon.oxf.xforms.XFormsControls.visitControlElementsHandleRepeat(XFormsControls.java:489) at org.orbeon.oxf.xforms.XFormsControls.visitControlElementsHandleRepeat(XFormsControls.java:368) 7: 2009-07-24 02:02:24,399 ERROR [org.orbeon.oxf.webapp.ProcessorService] Exception at line 55, column 65 of oxf:/ops/xforms/xforms-server.xpl (reading process or output: name='response', id='xforms-response') java.util.EmptyStackException at java.util.Stack.peek(Stack.java:79) at org.orbeon.oxf.xforms.XFormsContextStack.getCurrentBindingContext(XFormsContextStack.java:488) at org.orbeon.oxf.xforms.XFormsContextStack.pushBinding(XFormsContextStack.java:275) Any idea how to prevent this? Any help is appreciated! |
|
|
Btw this looks awfully similar to http://forge.ow2.org/tracker/index.php?func=detail&aid=311644&group_id=168&atid=350207 and http://www.nabble.com/eXist-%27Resource--xyz--not-found%27-warning-messages-td20190651.html because we also see the warnings and similar behavior.
Siarhei |
Re: Re: Exceptions and response times under load
|
|
Siarhei,
That looks like something that could happen when XForms documents get serialized/deserialized, if there is a bug in serialization/deserialization. If there is a bug, I am not sure if it is still in nightly build, but you could give one of those a try. At any rate, did you configure this property to approximately match the number of true concurrent users: <property as="xs:integer" name="oxf.xforms.cache.documents.size" value="50"/> -Erik On Mon, Jul 27, 2009 at 5:49 AM, Siarhei<[hidden email]> wrote: > > Btw this looks awfully similar to > http://forge.ow2.org/tracker/index.php?func=detail&aid=311644&group_id=168&atid=350207 > and > http://www.nabble.com/eXist-%27Resource--xyz--not-found%27-warning-messages-td20190651.html > because we also see the warnings and similar behavior. > > Siarhei > > ----- > IPROFS > > Claus Sluterweg 125 > > 2012 WS Haarlem > > http://www.iprofs.nl www.iprofs.nl > -- > View this message in context: http://www.nabble.com/Exceptions-and-response-times-under-load-tp24647127p24680129.html > Sent from the ObjectWeb OPS - Users mailing list archive at Nabble.com. > > > > -- > You receive this message as a subscriber of the [hidden email] mailing list. > To unsubscribe: mailto:[hidden email] > For general help: mailto:[hidden email]?subject=help > OW2 mailing lists service home page: http://www.ow2.org/wws > > -- You receive this message as a subscriber of the [hidden email] mailing list. To unsubscribe: mailto:[hidden email] For general help: mailto:[hidden email]?subject=help OW2 mailing lists service home page: http://www.ow2.org/wws |
|
|
Hi Erik,
Yes this is the default setting and it does approximately match a max number of users we use in load test. We also noticed that the performance doesn't degrade immediately but after a significant number of submissions, also the errors start showing up as response times increase. Regards, Siarhei
|
|
|
We have reduced the number of errors and response times by solution described in http://www.nabble.com/SessionListener-disabled-by-default--td24697170.html
But all these errors still show up under load. Regards, Siarhei
|
Re: Re: Re: Exceptions and response times under load
|
Administrator
|
This is not good, and it would need a fairly serious investigation. I
still suspect a bug in the deserialization process. -Erik On Thu, Jul 30, 2009 at 6:06 AM, Siarhei<[hidden email]> wrote: > > We have reduced the number of errors and response times by solution described > in > http://www.nabble.com/SessionListener-disabled-by-default--td24697170.html > > But all these errors still show up under load. > > Regards, > Siarhei > > > Siarhei wrote: >> >> Hi Erik, >> >> Yes this is the default setting and it does approximately match a max >> number of users we use in load test. >> >> We also noticed that the performance doesn't degrade immediately but after >> a significant number of submissions, also the errors start showing up as >> response times increase. >> >> Regards, >> Siarhei >> >> >> Erik Bruchez-3 wrote: >>> >>> Siarhei, >>> >>> That looks like something that could happen when XForms documents get >>> serialized/deserialized, if there is a bug in >>> serialization/deserialization. If there is a bug, I am not sure if it >>> is still in nightly build, but you could give one of those a try. >>> >>> At any rate, did you configure this property to approximately match >>> the number of true concurrent users: >>> >>> <property as="xs:integer" name="oxf.xforms.cache.documents.size" >>> value="50"/> >>> >>> -Erik >>> >>> On Mon, Jul 27, 2009 at 5:49 AM, Siarhei<[hidden email]> wrote: >>>> >>>> Btw this looks awfully similar to >>>> http://forge.ow2.org/tracker/index.php?func=detail&aid=311644&group_id=168&atid=350207 >>>> and >>>> http://www.nabble.com/eXist-%27Resource--xyz--not-found%27-warning-messages-td20190651.html >>>> because we also see the warnings and similar behavior. >>>> >>>> Siarhei >>>> >>>> ----- >>>> IPROFS >>>> >>>> Claus Sluterweg 125 >>>> >>>> 2012 WS Haarlem >>>> >>>> http://www.iprofs.nl www.iprofs.nl >>>> -- >>>> View this message in context: >>>> http://www.nabble.com/Exceptions-and-response-times-under-load-tp24647127p24680129.html >>>> Sent from the ObjectWeb OPS - Users mailing list archive at Nabble.com. >>>> >>>> >>>> >>>> -- >>>> You receive this message as a subscriber of the [hidden email] >>>> mailing list. >>>> To unsubscribe: mailto:[hidden email] >>>> For general help: mailto:[hidden email]?subject=help >>>> OW2 mailing lists service home page: http://www.ow2.org/wws >>>> >>>> >>> >>> >>> -- >>> You receive this message as a subscriber of the [hidden email] mailing >>> list. >>> To unsubscribe: mailto:[hidden email] >>> For general help: mailto:[hidden email]?subject=help >>> OW2 mailing lists service home page: http://www.ow2.org/wws >>> >>> >> >> > > > ----- > IPROFS > > Claus Sluterweg 125 > > 2012 WS Haarlem > > http://www.iprofs.nl www.iprofs.nl > -- > View this message in context: http://www.nabble.com/Exceptions-and-response-times-under-load-tp24647127p24738119.html > Sent from the ObjectWeb OPS - Users mailing list archive at Nabble.com. > > > > -- > You receive this message as a subscriber of the [hidden email] mailing list. > To unsubscribe: mailto:[hidden email] > For general help: mailto:[hidden email]?subject=help > OW2 mailing lists service home page: http://www.ow2.org/wws > > -- You receive this message as a subscriber of the [hidden email] mailing list. To unsubscribe: mailto:[hidden email] For general help: mailto:[hidden email]?subject=help OW2 mailing lists service home page: http://www.ow2.org/wws |
Re: Re: Exceptions and response times under load
|
Administrator
|
In reply to this post by Siarhei-2
Siarhei,
Like Erik said, this looks like a serious issue, and we would like to investigate it. But really to make progress on this, what we need first is a reproducible test case. I know it isn't as simple as creating a test for the XForms sandbox; maybe a JMeter test would do it, just hitting the standard examples? Alex |
Re: Re: Exceptions and response times under load
|
|
Hi Alex, Eric,
We're encountering the same problems as mentioned in this thread when we stress our application (which depends on Orbeon). Did you figure out a solution to this problem? TIA, Jasper Linthorst |
Re: Re: Re: Exceptions and response times under load
|
Administrator
|
Jasper,
The key to progressing on this kind of issues is always reproducibility, and in truth we have not reproduced the issue mentioned in this thread. Since lots of changes and improvements have occurred since then and 3.8, the answer is "maybe". Which version are you using at this point? Did you try 3.8 or a nightly build? -Erik On Tue, Jun 1, 2010 at 3:38 AM, Jasper Linthorst <[hidden email]> wrote: > > Hi Alex, Eric, > We're encountering the same problems as mentioned in this thread when we > stress our application (which depends on Orbeon). Did you figure out a > solution to this problem? > > TIA, > Jasper Linthorst > -- > View this message in context: http://orbeon-forms-ops-users.24843.n4.nabble.com/Exceptions-and-response-times-under-load-tp44828p2238324.html > Sent from the Orbeon Forms (ops-users) mailing list archive at Nabble.com. > > > -- > You receive this message as a subscriber of the [hidden email] mailing list. > To unsubscribe: mailto:[hidden email] > For general help: mailto:[hidden email]?subject=help > OW2 mailing lists service home page: http://www.ow2.org/wws > > -- You receive this message as a subscriber of the [hidden email] mailing list. To unsubscribe: mailto:[hidden email] For general help: mailto:[hidden email]?subject=help OW2 mailing lists service home page: http://www.ow2.org/wws |
«
Return to Orbeon Forms community mailing list
|
1 view|%1 views
| Free forum by Nabble | Edit this page |