RE: Implementing utility pipelines as custom processors, with a migration path to java implementation

> Alex
>
> I did try this approach, with some success.
>
> The easiest way (in terms of coding) is to inherit off the existing
> pipeline processor and override the values for the name of the config
> input.  That way any code changes, bug fixes etc in the main pipeline
> processor don't require code changes in the new processor.
>
> However there's a problem here, in that the config input name is given
> as
> public static final String INPUT_CONFIG = "config";
> in ProcessorImpl.java; so we can't override it.
>
> So what I did was implement a new method, PipelineConfigName() in the
> PipelineProcessor class; change other code in the PipelineProcessor
> class to use this instead of INPUT_CONFIG; then in my inherited class
> override this method to use a different name for the config input.
>
> However this does mean modifying the base OPS code, rather than just
> adding to it.
>
> This technique is very useful to us as it means that we can
> - implement a new pipeline component as an XPL initially
> - declare it in custom-processors.xml
> - call it as if we were calling a custom processor
> - if we decide to write our own custom processor to implement the
> functionality we then need to make no application code changes.
>
> It's also a useful way of declaring utility pipelines, so that if

> have an idea for a more elegant approach?
>
> Steve
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of
> Alessandro Vernet
> Sent: 17 February 2006 01:19
> To: [hidden email]
> Subject: Re: [ops-users] Implementing utility pipelines as custom
> processors, with a migration path to java implementation
>
> Stephen,
>
> I guess doing your own processor that uses the code of the Pipeline
> processor is the way to go. You can then even call your own processor
> "oxf:pipeline" and override the built-in Pipeline processor, if you
> don't think that might be confusing.
>
> Alex
>
> On 1/27/06, Stephen Bayliss <[hidden email]> wrote:
> > Eric
> >
> > I had a go at this, ie customising the existing pipeline processor

> > input.
> >
> > Got it working fine....
> >
> > But...
> >
> > The only way to get it working was to not have
> > addInputInfo(new ProcessorInputOutputInfo(INPUT_CONFIG,
> > PIPELINE_NAMESPACE_URI));
> > in the constructor (as at this point it's not known if a "config"
> input
> > or a "pipeline" input will be created).
> >
> > However, this means that the XPL input is not validated, which is
> > obviously a Bad Thing.
> >
> > And I can't work out any way of plugging validation into the correct
> > input at a later point, ie after it has been created and added (this
> > could be done in the start method).
> >
> > Any ideas?
> >
> > My fallback is to create a pipeline processor proxy, ie a new

> > processor that expects its XPL input on a "pipeline" input.
> >
> > Steve
> >
> >
> >
> > -----Original Message-----
> > From: Erik Bruchez [mailto:[hidden email]] On Behalf Of Erik
> Bruchez
> > Sent: 25 January 2006 23:16
> > To: [hidden email]
> > Subject: Re: [ops-users] Implementing utility pipelines as custom
> > processors, with a migration path to java implementation
> >
> > Stephen Bayliss wrote:
> >
> >  > Disadvantages with this approach
> >  > --------------------------------
> >  > - oxf:pipeline uses its config input to provide the xpl pipeline

> >  > call.  This means that the utility xpl pipeline cannot have a
> config
> >  > input (and if you want to avoid code changes later, your java
> >  > implementation will not have a config input), which is not great
> >  > (currently we use a naming convention of xconfig as the input to
> > utility
> >  > pipelines to deal with this, but this is not ideal)
> >
> > Granted, that's a drawback.
> >
> >  > - maybe OPS's caching mechanism won't cope very well with this
> > approach?
> >
> > No, it should cope just fine.
> >
> >  > One suggestion I have is to change the code for the current
> > oxf:pipeline
> >  > processor:
> >  > - have a new optional input called "pipeline"
> >  > - if this input is present, then use this in place of the current
> >  > "config" input
> >  > - if this input is not present, then use the config input as

> > in
> >  > called pipelines is also allowed.
> >  >
> >  > The processor declaration in custom-processors.xml would use an
> input
> >  > named "pipeline" to reference the xpl pipeline to be called, and
> you
> > are
> >  > then free to use the config input as per other processors.
> >  >
> >  > What do people think of this approach?
> >
> > In the early days, we kind of decided to standardize on a "config"
> > name for certain inputs. Now it hasn't proven a perfect idea.

> > to "pipeline" and implementing the above behavior would be probably
> > ok, but somebody would have to work on it ;-)
> >
> > -Erik
> >
> >
> >
> >
> >
> >
> > --
> > You receive this message as a subscriber of the
> [hidden email] mailing list.
> > To unsubscribe: mailto:[hidden email]
> > For general help: mailto:[hidden email]?subject=help
> > ObjectWeb mailing lists service home page:
> http://www.objectweb.org/wws
> >
> >
> >
>
>
> --
> Blog (XML, Web apps, Open Source):
> http://www.orbeon.com/blog/
>
>
>
>
>
>
> --
> You receive this message as a subscriber of the

> Alex
> I've had a go at what you suggested, following examples in other
> processors including IMProcessor.
>
> I can't claim to understand the OPS object model fully, and I wouldn't
> class myself as an expert Java programmer, but I have got something that
> works fine now.
>
> I'm pasting in my code for PipelineProcessorProxy.java below, and
> attaching it.
>
> I would welcome your thoughts, criticisms etc on this.
>
> Would you consider including it in core OPS?  I'm happy to write
> documentation, unit tests etc and submit those.
>
> It does allow a nice level of indirection for utility pipelines, in that
> you can declare them in custom-processors.xml and call them as a
> processor.  Then if you convert them to java processors in the future
> you have no code change; and importantly you can use the config input in
> your utility pipelines, rather than it being reserved for the XPL.
>
> Steve
>
> ===== PipelineProcessorProxy.java =====
> /*
>  *
>  */
> package org.orbeon.oxf.processor.pipeline;
>
> import java.util.ArrayList;
> import java.util.Iterator;
> import java.util.List;
>
> import org.apache.log4j.Logger;
> import org.orbeon.oxf.cache.OutputCacheKey;
> import org.orbeon.oxf.debugger.api.Breakpoint;
> import org.orbeon.oxf.debugger.api.Debuggable;
> import org.orbeon.oxf.pipeline.api.PipelineContext;
> import org.orbeon.oxf.processor.ProcessorImpl;
> import org.orbeon.oxf.processor.*;
> import org.orbeon.oxf.util.LoggerFactory;
> import org.xml.sax.ContentHandler;
>
> /**
>  * Proxy for PipelineProcessor - use instead of PipelineProcessor when
> XPL to run appears on pipeline input rather than config input
>  * Creates a PipelineProcessor from the pipeline input, connects inputs
> and outputs to this processor's inputs and outputs
>  */
> public class PipelineProcessorProxy extends ProcessorImpl implements
> Debuggable {
>
>     private static Logger logger =
> LoggerFactory.createLogger(PipelineProcessorProxy.class);
>
>     private static final String INPUT_PIPELINE = "pipeline";
>
>     public PipelineProcessorProxy() {
>         addInputInfo(new ProcessorInputOutputInfo(INPUT_PIPELINE));
>     }
>
>     /**
>      * create the PipelineProcessor, from the config input, or from
> state if already created
>      */
>     protected PipelineProcessor Pipeline(PipelineContext context) {
>         State state = (State) getState(context);
>
>         if (state.pipeline == null ) {
>                 PipelineReader reader = new PipelineReader();
>                 ProcessorInput pipelineReaderInput =
> reader.createInput("pipeline");
>
>                 final ProcessorInput config =
> getInputByName(INPUT_PIPELINE);
>
>                 pipelineReaderInput.setOutput(new
> ProcessorImpl.ProcessorOutputImpl(getClass(), "dummy") {
>                     public void readImpl(PipelineContext context,
> ContentHandler contentHandler) {
>                         ProcessorImpl.readInputAsSAX(context, config,
> contentHandler);
>                     }
>
>                     public OutputCacheKey getKeyImpl(PipelineContext
> context) {
>                         return getInputKey(context, config);
>                     }
>
>                     public Object getValidityImpl(PipelineContext
> context) {
>                         return getInputValidity(context, config);
>                     }
>
>                 });
>
>                 reader.start(context);
>
>                 state.pipeline = new
> PipelineProcessor(reader.getPipeline());
>
>                 state.pipeline.reset(context);
>
>                 // connect inputs of this processor to the pipeline
> we've created
>                 Iterator it =
> getConnectedInputs().entrySet().iterator();
>                 while (it.hasNext()) {
>                         String name =
> (String)((java.util.Map.Entry)it.next()).getKey();
>
>                         // don't connect this pipeline's config
> (pipeline) input
>                         if (name != INPUT_PIPELINE)
>                                 state.pipeline.addInput(name,
> getInputByName(name));
>                 }
>         }
>
>                 return state.pipeline;
>     }
>
>         /*
>          * for pipelines with no outputs, call start on the pipeline
>          */
>         public void start(PipelineContext pipelineContext) {
>                 State state = (State) getState(pipelineContext);
>                 if (!state.started) {
>
> Pipeline(pipelineContext).start(pipelineContext);
>                         state.started = true;
>                 }
>         }
>     public void reset(final PipelineContext context) {
>         setState(context, new State());
>     }
>
>         public ProcessorOutput createOutput(final String name) {
>
>                 ProcessorOutput output = new
> ProcessorImpl.ProcessorOutputImpl(getClass(), name) {
>                         public void readImpl(PipelineContext
> pipelineContext, ContentHandler contentHandler) {
>                                 // read the pipeline's output
>
> Pipeline(pipelineContext).createOutput(name).read(pipelineContext,
> contentHandler);
>                         }
>                 };
>
>                 addOutput(name, output);
>                 return output;
>         }
>
>     private static class State {
>         public PipelineProcessor pipeline = null;
>         public boolean started = false;
>     }
>
>     private List breakpoints;
>
>     public void addBreakpoint(Breakpoint breakpoint) {
>         if (breakpoints == null)
>             breakpoints = new ArrayList();
>         breakpoints.add(breakpoint);
>     }
>
>     public List getBreakpoints() {
>         return breakpoints;
>     }
>
> }
>
>
>
>
>
>
>
>
>
>
>
> =======================================
>
>
>
>
>
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of
> Alessandro Vernet
> Sent: 28 February 2006 01:47
> To: [hidden email]
> Subject: Re: [ops-users] Implementing utility pipelines as custom
> processors, with a migration path to java implementation
>
> Hi Stephen,
>
> I am reluctant to make the code more complex and add this indirection
> (protected method returning the name of the input for the pipeline).
> One way to avoid adding the indirection and modifying the
> PipelineProcessor code is to use composition instead of inheritance in
> your processor.
>
> In your new processor, use the PipelineReader to parse & analyze the
> "pipeline" input. You will get an instance of ASTPipeline which
> represents your pipeline. Then instantiate and run the
> PipelineProcessor with that representation of the pipeline. You can
> see code that does this in the IMProcessor.
>
> Alex
>
> On 2/23/06, Stephen Bayliss <[hidden email]> wrote:
> > Alex
> >
> > I did try this approach, with some success.
> >
> > The easiest way (in terms of coding) is to inherit off the existing
> > pipeline processor and override the values for the name of the config
> > input.  That way any code changes, bug fixes etc in the main pipeline
> > processor don't require code changes in the new processor.
> >
> > However there's a problem here, in that the config input name is given
> > as
> > public static final String INPUT_CONFIG = "config";
> > in ProcessorImpl.java; so we can't override it.
> >
> > So what I did was implement a new method, PipelineConfigName() in the
> > PipelineProcessor class; change other code in the PipelineProcessor
> > class to use this instead of INPUT_CONFIG; then in my inherited class
> > override this method to use a different name for the config input.
> >
> > However this does mean modifying the base OPS code, rather than just
> > adding to it.
> >
> > This technique is very useful to us as it means that we can
> > - implement a new pipeline component as an XPL initially
> > - declare it in custom-processors.xml
> > - call it as if we were calling a custom processor
> > - if we decide to write our own custom processor to implement the
> > functionality we then need to make no application code changes.
> >
> > It's also a useful way of declaring utility pipelines, so that if
> paths
> > to the xpl change in a new project we don't have to change any code,
> > just the custom-processors.xml declaration.
> >
> > If you're interested I can submit the code changes I made; or maybe
> you
> > have an idea for a more elegant approach?
> >
> > Steve
> >
> > -----Original Message-----
> > From: [hidden email] [mailto:[hidden email]] On Behalf Of
> > Alessandro Vernet
> > Sent: 17 February 2006 01:19
> > To: [hidden email]
> > Subject: Re: [ops-users] Implementing utility pipelines as custom
> > processors, with a migration path to java implementation
> >
> > Stephen,
> >
> > I guess doing your own processor that uses the code of the Pipeline
> > processor is the way to go. You can then even call your own processor
> > "oxf:pipeline" and override the built-in Pipeline processor, if you
> > don't think that might be confusing.
> >
> > Alex
> >
> > On 1/27/06, Stephen Bayliss <[hidden email]> wrote:
> > > Eric
> > >
> > > I had a go at this, ie customising the existing pipeline processor
> so
> > > that it would accept either a "config" input or a "pipeline" input;
> > and
> > > if it gets a "pipeline" input it uses that instead of the the
> "config"
> > > input.
> > >
> > > Got it working fine....
> > >
> > > But...
> > >
> > > The only way to get it working was to not have
> > > addInputInfo(new ProcessorInputOutputInfo(INPUT_CONFIG,
> > > PIPELINE_NAMESPACE_URI));
> > > in the constructor (as at this point it's not known if a "config"
> > input
> > > or a "pipeline" input will be created).
> > >
> > > However, this means that the XPL input is not validated, which is
> > > obviously a Bad Thing.
> > >
> > > And I can't work out any way of plugging validation into the correct
> > > input at a later point, ie after it has been created and added (this
> > > could be done in the start method).
> > >
> > > Any ideas?
> > >
> > > My fallback is to create a pipeline processor proxy, ie a new
> pipeline
> > > processor that expects its XPL input on a "pipeline" input.
> > >
> > > Steve
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Erik Bruchez [mailto:[hidden email]] On Behalf Of Erik
> > Bruchez
> > > Sent: 25 January 2006 23:16
> > > To: [hidden email]
> > > Subject: Re: [ops-users] Implementing utility pipelines as custom
> > > processors, with a migration path to java implementation
> > >
> > > Stephen Bayliss wrote:
> > >
> > >  > Disadvantages with this approach
> > >  > --------------------------------
> > >  > - oxf:pipeline uses its config input to provide the xpl pipeline
> to
> > >  > call.  This means that the utility xpl pipeline cannot have a
> > config
> > >  > input (and if you want to avoid code changes later, your java
> > >  > implementation will not have a config input), which is not great
> > >  > (currently we use a naming convention of xconfig as the input to
> > > utility
> > >  > pipelines to deal with this, but this is not ideal)
> > >
> > > Granted, that's a drawback.
> > >
> > >  > - maybe OPS's caching mechanism won't cope very well with this
> > > approach?
> > >
> > > No, it should cope just fine.
> > >
> > >  > One suggestion I have is to change the code for the current
> > > oxf:pipeline
> > >  > processor:
> > >  > - have a new optional input called "pipeline"
> > >  > - if this input is present, then use this in place of the current
> > >  > "config" input
> > >  > - if this input is not present, then use the config input as
> before
> > >  >
> > >  > Thus existing behaviour is preserved, but usage of the config
> input
> > > in
> > >  > called pipelines is also allowed.
> > >  >
> > >  > The processor declaration in custom-processors.xml would use an
> > input
> > >  > named "pipeline" to reference the xpl pipeline to be called, and
> > you
> > > are
> > >  > then free to use the config input as per other processors.
> > >  >
> > >  > What do people think of this approach?
> > >
> > > In the early days, we kind of decided to standardize on a "config"
> > > name for certain inputs. Now it hasn't proven a perfect idea.
> Changing
> > > to "pipeline" and implementing the above behavior would be probably
> > > ok, but somebody would have to work on it ;-)
> > >
> > > -Erik
> > >
> > >
> > >
> > >
> > >
> > >
> > > --
> > > You receive this message as a subscriber of the
> > [hidden email] mailing list.
> > > To unsubscribe: mailto:[hidden email]
> > > For general help: mailto:[hidden email]?subject=help
> > > ObjectWeb mailing lists service home page:
> > http://www.objectweb.org/wws
> > >
> > >
> > >
> >
> >
> > --
> > Blog (XML, Web apps, Open Source):
> > http://www.orbeon.com/blog/
> >
> >
> >
> >
> >
> >
> > --
> > You receive this message as a subscriber of the
> [hidden email] mailing list.
> > To unsubscribe: mailto:[hidden email]
> > For general help: mailto:[hidden email]?subject=help
> > ObjectWeb mailing lists service home page:
> http://www.objectweb.org/wws
> >
> >
> >
>
>
> --
> Blog (XML, Web apps, Open Source):
> http://www.orbeon.com/blog/
>
>
>
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>
>
>
>