> Hi Erik,
>
> thanks, that was helpful!
> However, the data is sent to the DB using the xforms-submission processor. I
> assume the reason for the large RAM usage is that the stream it creates is
> cached, too, but how can I disable this? I could not find an option in the
> processor nor in the submission doc.
>
> Kind regards,
> Ralf Jung
>
> Am Donnerstag 16 Dezember 2010, um 06:45:42 schrieb Erik Bruchez:
>> I would expect a 45 MB file to take at least 10x in RAM, so 450 MB. If
>> that is just copied once or twice, that would explain why you run out
>> of memory.
>>
>> Now that doesn't mean things can't be improved.
>>
>> Both the URL generator and the HTTP serializer have configurations to
>> disable caching:
>>
>> *
>> http://wiki.orbeon.com/forms/doc/developer-guide/processors-url-generator#
>> TOC-Cache-control *
>> http://www.orbeon.com/orbeon/doc/processors-serializers-http#d6e26
>>
>> I would try to load and serialize large documents by setting
>> use-local-cache to false in both cases.
>>
>> -Erik
>>
>> On Tue, Dec 14, 2010 at 1:45 AM, Ralf Jung <[hidden email]> wrote:
>> > Hi,
>> >
>> > not sure if this is related, but I am also under the impression that
>> > there is some memory issue somewhere: I am using an XPL pipe to load a
>> > document, using XSLT to split the nodes from the root into segments of
>> > max. 500 nodes, and then sending each of those segments to the DB. The
>> > XML file in 45MiB in size (>9k children of the root, i.e. 19 segments),
>> > and if I give less than 1200MiB RAM to Orbeon, it crashes with "GC
>> > overhead limit exceeded" around the 10th segment. It feels like it's
>> > keeping much more data during that loop than it needs.
>> > (I previously had it send the whole file to the DB in a batch, which
>> > would behave even worse).
>> > This is with Orbeon 3.8 CE.
>> >
>> > Kind regards,
>> > Ralf Jung
>> >
>> > Am Dienstag 14 Dezember 2010, um 10:03:25 schrieb ncrofts:
>> >> Hi Erik,
>> >>
>> >> The tests were done with the official 3.8 CE release,
>> >> orbeon-3.8.0.rc1.201005041622-CE.
>> >>
>> >> I've not tried with a recent nightly build, but with respect all
>> >> previous builds I've tried have been similarly memory intensive. I'm
>> >> not sure yet what the specific bottleneck(s) are and there is obviously
>> >> quite a lot of processing going on just to read from a db and process
>> >> XML, but it is curious that an XPL based application (that also
>> >> processes db/XML) seems lighter on the memory footprint. It does feel
>> >> rather a disproportionate overhead just to generate a few K's worth of
>> >> HTML and I can't help but feel something is not quite right.
>> >>
>> >> Regards,
>> >> Neil
>> >
>> > --
>> > You receive this message as a subscriber of the [hidden email] mailing
>> > list. To unsubscribe: mailto:[hidden email]
>> > For general help: mailto:[hidden email]?subject=help
>> > OW2 mailing lists service home page: http://www.ow2.org/wws
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws
>
>