> Ralf,
>
> Mmh, I fear that oxf:xforms-submission does not deal very well with
> very large files.
>
> Can you use oxf:url-generator instead here?
>
> -Erik
>
> On Thu, Dec 16, 2010 at 7:11 AM, Ralf Jung <
[hidden email]> wrote:
> > Hi Erik,
> >
> > thanks, that was helpful!
> > However, the data is sent to the DB using the xforms-submission
> > processor. I assume the reason for the large RAM usage is that the
> > stream it creates is cached, too, but how can I disable this? I could
> > not find an option in the processor nor in the submission doc.
> >
> > Kind regards,
> > Ralf Jung
> >
> > Am Donnerstag 16 Dezember 2010, um 06:45:42 schrieb Erik Bruchez:
> >> I would expect a 45 MB file to take at least 10x in RAM, so 450 MB. If
> >> that is just copied once or twice, that would explain why you run out
> >> of memory.
> >>
> >> Now that doesn't mean things can't be improved.
> >>
> >> Both the URL generator and the HTTP serializer have configurations to
> >> disable caching:
> >>
> >> *
> >>
http://wiki.orbeon.com/forms/doc/developer-guide/processors-url-generato> >> r# TOC-Cache-control *
> >>
http://www.orbeon.com/orbeon/doc/processors-serializers-http#d6e26> >>
> >> I would try to load and serialize large documents by setting
> >> use-local-cache to false in both cases.
> >>
> >> -Erik
> >>
> >> On Tue, Dec 14, 2010 at 1:45 AM, Ralf Jung <
[hidden email]> wrote:
> >> > Hi,
> >> >
> >> > not sure if this is related, but I am also under the impression that
> >> > there is some memory issue somewhere: I am using an XPL pipe to load a
> >> > document, using XSLT to split the nodes from the root into segments of
> >> > max. 500 nodes, and then sending each of those segments to the DB. The
> >> > XML file in 45MiB in size (>9k children of the root, i.e. 19
> >> > segments), and if I give less than 1200MiB RAM to Orbeon, it crashes
> >> > with "GC overhead limit exceeded" around the 10th segment. It feels
> >> > like it's keeping much more data during that loop than it needs.
> >> > (I previously had it send the whole file to the DB in a batch, which
> >> > would behave even worse).
> >> > This is with Orbeon 3.8 CE.
> >> >
> >> > Kind regards,
> >> > Ralf Jung
> >> >
> >> > Am Dienstag 14 Dezember 2010, um 10:03:25 schrieb ncrofts:
> >> >> Hi Erik,
> >> >>
> >> >> The tests were done with the official 3.8 CE release,
> >> >> orbeon-3.8.0.rc1.201005041622-CE.
> >> >>
> >> >> I've not tried with a recent nightly build, but with respect all
> >> >> previous builds I've tried have been similarly memory intensive. I'm
> >> >> not sure yet what the specific bottleneck(s) are and there is
> >> >> obviously quite a lot of processing going on just to read from a db
> >> >> and process XML, but it is curious that an XPL based application
> >> >> (that also processes db/XML) seems lighter on the memory footprint.
> >> >> It does feel rather a disproportionate overhead just to generate a
> >> >> few K's worth of HTML and I can't help but feel something is not
> >> >> quite right.
> >> >>
> >> >> Regards,
> >> >> Neil
> >> >
> >> > --
> >> > You receive this message as a subscriber of the
[hidden email]
> >> > mailing list. To unsubscribe: mailto:
[hidden email]
> >> > For general help: mailto:
[hidden email]?subject=help
> >> > OW2 mailing lists service home page:
http://www.ow2.org/wws> >
> > --
> > You receive this message as a subscriber of the
[hidden email] mailing
> > list. To unsubscribe: mailto:
[hidden email]
> > For general help: mailto:
[hidden email]?subject=help
> > OW2 mailing lists service home page:
http://www.ow2.org/wws