Orbeon Forms community mailing list

Multiple or parallel pipeline instances

Classic

List

Threaded

4 messages Options

Steve McCoole

Multiple or parallel pipeline instances

I'm looking into using the Pipeline API to process a large volume of XML information. The information is composed of individual records that can be processed in any order. What I would like to do is to be able to split up the input and use multiple pipeline instances to process the input set. Other than using a separate PipelineDefinition and PipelineContext would there be any other possible issues that I would have to be aware of?

Thanks,
Steve

Arun Batchu

Re: Multiple or parallel pipeline instances

If your processes could be async then try using Mule or similar rather
than Orbeon.

Thx via my iPhone ,
Arun 612-501-6345 [hidden email]

On Oct 1, 2008, at 11:47 AM, Steve McCoole <[hidden email]> wrote:

>
> I'm looking into using the Pipeline API to process a large volume of
> XML
> information. The information is composed of individual records that
> can be
> processed in any order. What I would like to do is to be able to
> split up
> the input and use multiple pipeline instances to process the input
> set.
> Other than using a separate PipelineDefinition and PipelineContext
> would
> there be any other possible issues that I would have to be aware of?
>
> Thanks,
> Steve
> --
> View this message in context: http://www.nabble.com/Multiple-or-parallel-pipeline-instances-tp19764586p19764586.html
> Sent from the ObjectWeb OPS - Users mailing list archive at
> Nabble.com.
>
>
> --
> You receive this message as a subscriber of the [hidden email]
> mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws

Alessandro Vernet

Re: Multiple or parallel pipeline instances

Administrator

In reply to this post by Steve McCoole

Steve,

On Wed, Oct 1, 2008 at 9:47 AM, Steve McCoole <[hidden email]> wrote:
> I'm looking into using the Pipeline API to process a large volume of XML
> information. The information is composed of individual records that can be
> processed in any order. What I would like to do is to be able to split up
> the input and use multiple pipeline instances to process the input set.
> Other than using a separate PipelineDefinition and PipelineContext would
> there be any other possible issues that I would have to be aware of?

You are planning to write Java code for this and invoke multiple
pipeline engine; is that right? Then yes, you should be fine as long as
each concurrent executing has its own PipelineContext. As you can
imagine this is what happens when you have multiple HTTP requests
processed at the same time and each one of them is processed by a pipeline.

Alex
--
Orbeon Forms - Web 2.0 Forms, open-source, for the Enterprise
Orbeon's Blog: http://www.orbeon.com/blog/
Personal Blog: http://avernet.blogspot.com/
Twitter - http://twitter.com/avernet

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws

Steve McCoole-2

Re: Re: Multiple or parallel pipeline instances

That was the direction that I was going in. Thanks Alex!

On Thu, Oct 2, 2008 at 3:39 PM, Alessandro Vernet <[hidden email]> wrote:

On Wed, Oct 1, 2008 at 9:47 AM, Steve McCoole <[hidden email]> wrote:

I'm looking into using the Pipeline API to process a large volume of XML
information. The information is composed of individual records that can be
processed in any order. What I would like to do is to be able to split up
the input and use multiple pipeline instances to process the input set.
Other than using a separate PipelineDefinition and PipelineContext would
there be any other possible issues that I would have to be aware of?

You are planning to write Java code for this and invoke multiple pipeline engine; is that right? Then yes, you should be fine as long as each concurrent executing has its own PipelineContext. As you can imagine this is what happens when you have multiple HTTP requests processed at the same time and each one of them is processed by a pipeline.