exception when xs:base64Binary exceeds oxf.xforms.cache.session.size

Experimenting with an instance document containing two 105KB inline xs:base64String files, I found that I got an exception as soon as I interacted with the form:

org.orbeon.oxf.common.OXFException: Unable to retrieve XForms engine state from session cache.
    at org.orbeon.oxf.xforms.processor.XFormsServer.doIt(XFormsServer.java:158)
    at org.orbeon.oxf.xforms.processor.XFormsServer.access$000(XFormsServer.java:48)
    at org.orbeon.oxf.xforms.processor.XFormsServer$1.readImpl(XFormsServer.java:77)
    at org.orbeon.oxf.processor.ProcessorImpl$6.read(ProcessorImpl.java:1012)
    at org.orbeon.oxf.processor.ProcessorImpl$ProcessorOutputImpl$ConcreteProcessorFilter$ForwarderProcessorOutput.read(ProcessorImpl.java:966)
    at org.orbeon.oxf.processor.ProcessorImpl.readInputAsSAX(ProcessorImpl.java:348)
    at org.orbeon.oxf.processor.validation.MSVValidationProcessor.access$700(MSVValidationProcessor.java:44)
    at org.orbeon.oxf.processor.validation.MSVValidationProcessor$5.readImpl(MSVValidationProcessor.java:219)
    at org.orbeon.oxf.processor.ProcessorImpl$6.read(ProcessorImpl.java:1012)
    at org.orbeon.oxf.processor.ProcessorImpl$ProcessorOutputImpl.read(ProcessorImpl.java:1195)
    at org.orbeon.oxf.processor.ProcessorImpl$ProcessorOutputImpl$ConcreteProcessorFilter.read(ProcessorImpl.java:990)
    at org.orbeon.oxf.processor.ProcessorImpl$ProcessorOutputImpl.read(ProcessorImpl.java:1195)
    at org.orbeon.oxf.processor.ProcessorImpl.readInputAsSAX(ProcessorImpl.java:348)
    at org.orbeon.oxf.processor.serializer.legacy.XMLSerializer.readInput(XMLSerializer.java:58)
    at org.orbeon.oxf.processor.serializer.HttpTextSerializer.readInput(HttpTextSerializer.java:54)
    at org.orbeon.oxf.processor.serializer.HttpSerializerBase$1.read(HttpSerializerBase.java:147)
    at org.orbeon.oxf.processor.ProcessorImpl.readCacheInputAsObject(ProcessorImpl.java:470)
    at org.orbeon.oxf.processor.serializer.HttpSerializerBase.start(HttpSerializerBase.java:138)
    at org.orbeon.oxf.processor.pipeline.PipelineProcessor$11.run(PipelineProcessor.java:652)
    at org.orbeon.oxf.processor.ProcessorImpl.executeChildren(ProcessorImpl.java:536)
    at org.orbeon.oxf.processor.pipeline.PipelineProcessor.start(PipelineProcessor.java:649)
    at org.orbeon.oxf.pipeline.InitUtils.runProcessor(InitUtils.java:86)
    at org.orbeon.oxf.webapp.ProcessorService.service(ProcessorService.java:95)

The form works fine with just a single 100KB inline string - it turns out the problem is that the form was exceeding my oxf.xforms.cache.session.size (which was set to 500,000). Increasing this to 1,000,000 prevents the exception.

A couple of points:

1. I'm wondering whether the cache should handle this scenario a little better.  Surely even though we cannot  *cache* the oversized state for performance/repeat hits, it's still required for the form to function. ie, I can accept that the state is too large for the cache, and so there might be a performance hit - but shouldn't it still function? Realistically though I suppose making this distinction between the two is difficult: perhaps at least the error reported might be more indicative? (particularly considering the symptom on the client is that the throbber just spins forever, without any error reported).

2. It seems that the resulting memory usage in the session is quite inflated relative to the actual string length. With some rough testing I found adding a 105KB string to my instance document (which in turn is a base64 encoding of a 77KB original) meant that I had to increase the session.size by ~350KB. Overall that's about a 4.5x increase. So even with small files, the memory impact is pretty severe. Obviously using xs:anyURI is the preferred approach, and there's probably little point trying to optimize memory usage for the inline-file scenario - but perhaps the memory impact of using xs:base64Binary should be stressed a little more in the documentation, and the link to oxf.xforms.cache.session.size mentioned?

Adrian


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Adrian,

You're making good points:

On 10/29/06, Adrian Baker <[hidden email]> wrote:
>  1. I'm wondering whether the cache should handle this scenario a little
> better.  Surely even though we cannot  *cache* the oversized state for
> performance/repeat hits, it's still required for the form to function. ie, I
> can accept that the state is too large for the cache, and so there might be
> a performance hit - but shouldn't it still function? Realistically though I
> suppose making this distinction between the two is difficult: perhaps at
> least the error reported might be more indicative? (particularly considering
> the symptom on the client is that the throbber just spins forever, without
> any error reported).

Maybe there could be an error right away when the page is generated in
this case, since the state doesn't even fit in cache. Do you have a
better idea? Do you have a test case we can use to reproduce this?
Makes sense. Feel free to write a paragraph about this, send it here,
and we'll include this in the documentation.

Alex
--
Blog (XML, Web apps, Open Source):
http://www.orbeon.com/blog/



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Alessandro Vernet wrote: As a somewhat related side note, there is clearly a lot of work that
could go into improving session state handling.

-Erik

--
Orbeon - XForms Everywhere:
http://www.orbeon.com/blog/



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws