Execute processor help needed

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Execute processor help needed

Tom Grahame
Using examples from this list and the .rng schema for the execute processor accessed from CVS I have managed to build an execute processor that looks like this:

<p:processor name="oxf:execute-processor">
    <p:input name="config">
                <exec executable="/usr/local/bin/program">
                        <arg line="argument"/>
                </exec>
        </p:input>
    <p:output name="stdout" id="stdout"/>
    <p:output name="stderr" id="stderr"/>
    <p:output name="result" id="result"/>
</p:processor>

'/usr/local/bin/program' is a binary that produces an output given a file as 'argument'. The processor outputs something like this on stdout (the contents of document are entity refs not printed <>, I couldn't work out how to escape them for this post):

<document xsi:type="xs:string" content-type="text/plain">
        <?xml version="1.0" encoding="UTF-8"?>
        <some-xml/>
</document>

I also get this on the result output:

<result>0</result>

It is really xml inside the <document/> element that I want to send to an instance though.

I don't understand the processor fully and would like to configure it more if I could. Does anyone have any pointers as to how I can output text/xml, or is there another processor I can put after the execute-processor that might help?

In addition, can anyone tell me:
What <result>0</result> means and what I should expect on the result output;
What the dir attribute on exec does, I have seen other people using it but don't know what it is for;
What the other attributes on the exec element can be used for?

Many thanks,

Tom
Reply | Threaded
Open this post in threaded view
|

Re: Execute processor help needed

claude felizardo
Hey, I think I can answer some of this...

the 'dir' attribute specifies where your program will run from whereas
the 'executable' attribute is either the full path to your program or it
can just be the name assuming it can be found in your PATH.  So this
would be like doing a cd before it executes your program.  Otherwise it
assumes the current directory which it inherits from where ever you were
when you launched tomcat/orbeon.   When your program exists, the status
or return code is captured in the 'return' output.

Here's an example I've been using that calls 'env' and then converts it
to text:

<p:config xmlns:p="http://www.orbeon.com/oxf/pipeline"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns:oxf="http://www.orbeon.com/oxf/processors">

    <!-- Execute command -->
    <p:processor name="oxf:execute-processor">
        <p:input name="config">
            <exec executable="env"/>
        </p:input>
        <p:output name="stdout" id="stdout"/>
        <p:output name="stderr" id="stderr"/>
        <p:output name="result" id="result"/>
    </p:processor>

    <!-- Output stdout -->
    <p:processor name="oxf:http-serializer">
        <p:input name="config">
            <config/>
        </p:input>
        <p:input name="data" href="#stdout"/>
    </p:processor>

</p:config>



Tom Grahame wrote:

> Using examples from this list and the .rng schema for the execute processor
> accessed from CVS I have managed to build an execute processor that looks
> like this:
>
> <p:processor name="oxf:execute-processor">
>     <p:input name="config">
> <exec executable="/usr/local/bin/program">
> <arg line="argument"/>
> </exec>
> </p:input>
>     <p:output name="stdout" id="stdout"/>
>     <p:output name="stderr" id="stderr"/>
>     <p:output name="result" id="result"/>
> </p:processor>
>
> '/usr/local/bin/program' is a binary that produces an output given a file as
> 'argument'. The processor outputs something like this on stdout (the
> contents of document are entity refs not printed <>, I couldn't work out how
> to escape them for this post):
>
> <document xsi:type="xs:string" content-type="text/plain">
> &lt;?xml version="1.0" encoding="UTF-8"?&gt;
> &lt;some-xml/&gt;
> </document>
>
> I also get this on the result output:
>
> <result>0</result>
>
> It is really xml inside the <document/> element that I want to send to an
> instance though.
>
> I don't understand the processor fully and would like to configure it more
> if I could. Does anyone have any pointers as to how I can output text/xml,
> or is there another processor I can put after the execute-processor that
> might help?
>
> In addition, can anyone tell me:
> What <result>0</result> means and what I should expect on the result output;
> What the dir attribute on exec does, I have seen other people using it but
> don't know what it is for;
> What the other attributes on the exec element can be used for?
>
> Many thanks,
>
> Tom
>  


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Execute processor help needed

claude felizardo
In reply to this post by Tom Grahame
Here's another example using properties to specify both program and data
paths as well as passing an argument via the submit and then using the
aggregate function to combine stdout, stderr and results into xml that
can be used as needed:

<p:config xmlns:p="http://www.orbeon.com/oxf/pipeline"
xmlns:pipeline="java:org.orbeon.oxf.processor.pipeline.PipelineFunctionLibrary"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:oxf="http://www.orbeon.com/oxf/processors">

<p:param name="instance" type="input" />

<p:processor name="oxf:unsafe-xslt">
<p:input name="config">
<xsl:stylesheet version="2.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
<xsl:variable name="args">
<xsl:value-of select="pipeline:property('my-app.args')"/>
<xsl:value-of select="/*/filename"/>
</xsl:variable>
<exec>
<xsl:attribute name="dir" select="pipeline:property('my-app.data-path')"/>
<xsl:attribute name="executable" select="pipeline:property('my-app.cmd')"/>
<xsl:element name="arg"><xsl:attribute name="line"
select="$args"/></xsl:element>
</exec>
</xsl:template>
</xsl:stylesheet>
</p:input>
<p:input name="data" href="#instance"/>
<p:output name="data" id="exec-config"/>
</p:processor>

<!-- Execute command -->
<p:processor name="oxf:execute-processor">
<p:input name="config" href="#exec-config"/>
<p:output name="stdout" id="stdout"/>
<p:output name="stderr" id="stderr"/>
<p:output name="result" id="result"/>
</p:processor>

<!-- serialize the result, force content type to be text/xml -->
<p:processor name="oxf:xml-serializer">
<p:input name="config">
<config>
<cache-control>
<use-local-cache>false</use-local-cache>
</cache-control>
<content-type>text/xml</content-type>
</config>
</p:input>
<p:input name="data" href="aggregate('run', #result, #stdout, #stderr)"/>
</p:processor>

</p:config>

Example when called directly from web browser:

- <run>
<result>0</result>
− <document xsi:type="xs:string" content-type="text/plain">
sample output would go here
</document>
− <document xsi:type="xs:string" content-type="text/plain">
error message would go here
</document>
</run>

Oh, I also ran into a snag where I can't pass a file ending with xml so
I had to append a slash to the end. Here's part of my page-flow.xml

<page id="test" path-info="/myapp/test/(.+)/"
matcher="oxf:perl5-matcher" default-submission="default-form.xml"
model="test.xpl">
<setvalue ref="/form/filename" matcher-group="1"/>
</page>

and default-form.xml looks like this:

<form>
<filename/>
</form>

this part took me a while to figure out - apparently you can't change a
value/attribute unless it exists before hand.

Here's how I'm using it in my view.xhtml file:

<xforms:submission id="test" method="get" serialize="false"
action="/myapp/test/{instance('control-instance')/filename}/"
serialization="none" replace="instance" instance="results-instance"/>
...
<xforms:trigger xxforms:modal="true">
<xforms:label>Validate</xforms:label>
<xforms:action ev:event="DOMActivate">
<xforms:setvalue ref="instance('control-instance')/filename"
value="context()/@filename"/>
<xforms:send submission="validate-fits-delivery"/>
<xforms:setvalue ref="context()/results"
value="instance('results-instance')/document"/>
</xforms:action>
</xforms:trigger>

I did a lot of cut and pasting, hopefully I didn't leave out anything
critical.

claude


Tom Grahame wrote:

> Using examples from this list and the .rng schema for the execute processor
> accessed from CVS I have managed to build an execute processor that looks
> like this:
>
> <p:processor name="oxf:execute-processor">
>     <p:input name="config">
> <exec executable="/usr/local/bin/program">
> <arg line="argument"/>
> </exec>
> </p:input>
>     <p:output name="stdout" id="stdout"/>
>     <p:output name="stderr" id="stderr"/>
>     <p:output name="result" id="result"/>
> </p:processor>
>
> '/usr/local/bin/program' is a binary that produces an output given a file as
> 'argument'. The processor outputs something like this on stdout (the
> contents of document are entity refs not printed <>, I couldn't work out how
> to escape them for this post):
>
> <document xsi:type="xs:string" content-type="text/plain">
> &lt;?xml version="1.0" encoding="UTF-8"?&gt;
> &lt;some-xml/&gt;
> </document>
>
> I also get this on the result output:
>
> <result>0</result>
>
> It is really xml inside the <document/> element that I want to send to an
> instance though.
>
> I don't understand the processor fully and would like to configure it more
> if I could. Does anyone have any pointers as to how I can output text/xml,
> or is there another processor I can put after the execute-processor that
> might help?
>
> In addition, can anyone tell me:
> What <result>0</result> means and what I should expect on the result output;
> What the dir attribute on exec does, I have seen other people using it but
> don't know what it is for;
> What the other attributes on the exec element can be used for?
>
> Many thanks,
>
> Tom
>  


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Execute processor help needed

Tom Grahame
Claude,

Thanks for your help! Your explanation of the dir and executable attributes make sense, and that #result gives the result/exit status of the executable makes even more sense. I should have known that...

I'm still having trouble accessing the xml inside the document tag however. The XML declaration inside the document element really messes things up for me.

The only serializer I seem to be able to make produce the contents of the document tag (thus avoiding the xml declaration) is the file-serializer. For now I'm writing the document to disk and then accessing it again using the url-generator. My pipeline is now arranged like this:

pipeline:input
execute-processor
file-serializer
url-generator
pipeline:output

This is far less than perfect, but it will do for now.

Thanks and regards,

Tom

cfeliz wrote
Here's another example using properties to specify both program and data
paths as well as passing an argument via the submit and then using the
aggregate function to combine stdout, stderr and results into xml that
can be used as needed:

<p:config xmlns:p="http://www.orbeon.com/oxf/pipeline"
xmlns:pipeline="java:org.orbeon.oxf.processor.pipeline.PipelineFunctionLibrary"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:oxf="http://www.orbeon.com/oxf/processors">

<p:param name="instance" type="input" />

<p:processor name="oxf:unsafe-xslt">
<p:input name="config">
<xsl:stylesheet version="2.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
<xsl:variable name="args">
<xsl:value-of select="pipeline:property('my-app.args')"/>
<xsl:value-of select="/*/filename"/>
</xsl:variable>
<exec>
<xsl:attribute name="dir" select="pipeline:property('my-app.data-path')"/>
<xsl:attribute name="executable" select="pipeline:property('my-app.cmd')"/>
<xsl:element name="arg"><xsl:attribute name="line"
select="$args"/></xsl:element>
</exec>
</xsl:template>
</xsl:stylesheet>
</p:input>
<p:input name="data" href="#instance"/>
<p:output name="data" id="exec-config"/>
</p:processor>


<p:processor name="oxf:execute-processor">
<p:input name="config" href="#exec-config"/>
<p:output name="stdout" id="stdout"/>
<p:output name="stderr" id="stderr"/>
<p:output name="result" id="result"/>
</p:processor>


<p:processor name="oxf:xml-serializer">
<p:input name="config">
<config>
<cache-control>
<use-local-cache>false</use-local-cache>
</cache-control>
<content-type>text/xml</content-type>
</config>
</p:input>
<p:input name="data" href="aggregate('run', #result, #stdout, #stderr)"/>
</p:processor>

</p:config>

Example when called directly from web browser:

- <run>
<result>0</result>
− <document xsi:type="xs:string" content-type="text/plain">
sample output would go here
</document>
− <document xsi:type="xs:string" content-type="text/plain">
error message would go here
</document>
</run>

Oh, I also ran into a snag where I can't pass a file ending with xml so
I had to append a slash to the end. Here's part of my page-flow.xml

<page id="test" path-info="/myapp/test/(.+)/"
matcher="oxf:perl5-matcher" default-submission="default-form.xml"
model="test.xpl">
<setvalue ref="/form/filename" matcher-group="1"/>
</page>

and default-form.xml looks like this:

<form>
<filename/>
</form>

this part took me a while to figure out - apparently you can't change a
value/attribute unless it exists before hand.

Here's how I'm using it in my view.xhtml file:

<xforms:submission id="test" method="get" serialize="false"
action="/myapp/test/{instance('control-instance')/filename}/"
serialization="none" replace="instance" instance="results-instance"/>
...
<xforms:trigger xxforms:modal="true">
<xforms:label>Validate</xforms:label>
<xforms:action ev:event="DOMActivate">
<xforms:setvalue ref="instance('control-instance')/filename"
value="context()/@filename"/>
<xforms:send submission="validate-fits-delivery"/>
<xforms:setvalue ref="context()/results"
value="instance('results-instance')/document"/>
</xforms:action>
</xforms:trigger>

I did a lot of cut and pasting, hopefully I didn't leave out anything
critical.

claude


Tom Grahame wrote:
> Using examples from this list and the .rng schema for the execute processor
> accessed from CVS I have managed to build an execute processor that looks
> like this:
>
> <p:processor name="oxf:execute-processor">
>     <p:input name="config">
> <exec executable="/usr/local/bin/program">
> <arg line="argument"/>
> </exec>
> </p:input>
>     <p:output name="stdout" id="stdout"/>
>     <p:output name="stderr" id="stderr"/>
>     <p:output name="result" id="result"/>
> </p:processor>
>
> '/usr/local/bin/program' is a binary that produces an output given a file as
> 'argument'. The processor outputs something like this on stdout (the
> contents of document are entity refs not printed <>, I couldn't work out how
> to escape them for this post):
>
> <document xsi:type="xs:string" content-type="text/plain">
> <?xml version="1.0" encoding="UTF-8"?>
> <some-xml/>
> </document>
>
> I also get this on the result output:
>
> <result>0</result>
>
> It is really xml inside the <document/> element that I want to send to an
> instance though.
>
> I don't understand the processor fully and would like to configure it more
> if I could. Does anyone have any pointers as to how I can output text/xml,
> or is there another processor I can put after the execute-processor that
> might help?
>
> In addition, can anyone tell me:
> What <result>0</result> means and what I should expect on the result output;
> What the dir attribute on exec does, I have seen other people using it but
> don't know what it is for;
> What the other attributes on the exec element can be used for?
>
> Many thanks,
>
> Tom
>  



--
You receive this message as a subscriber of the ops-users@ow2.org mailing list.
To unsubscribe: mailto:ops-users-unsubscribe@ow2.org
For general help: mailto:sympa@ow2.org?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Execute processor help needed

Alessandro Vernet
Administrator
Tom,

Tom Grahame wrote
The only serializer I seem to be able to make produce the contents of the document tag (thus avoiding the xml declaration) is the file-serializer. For now I'm writing the document to disk and then accessing it again using the url-generator. My pipeline is now arranged like this:

pipeline:input
execute-processor
file-serializer
url-generator
pipeline:output
Maybe the To-XML converter will do what you are looking for. See: http://www.orbeon.com/ops/doc/processors-converters#to-xml-converter

You'll let us know how it works for you.

Alex
Reply | Threaded
Open this post in threaded view
|

Re: Execute processor help needed

Tom Grahame
Alex,

The To-XML converter works very well. I am convinced that I had tried it before but not configured the pipeline properly. Your assurance that it should work helped me revisit and solve the problem.

I see from the execute processor source that a potential todo is handle standard input too. This would be nice as I believe I could chain together several  execute processors and pipe the inputs/outputs of binaries and shell scripts together for example.

Now I have to handle invalid xml characters in the data generated by my binary. Normally the pipeline crashes out when it encounters these, so I have tried using the validation processor to avoid crashing and report the error, but so far this has not worked. Here is my validation processor:

<p:processor name="oxf:validation">
    <p:input name="data" href="#xml-file"/>
    <p:input name="schema" href="refman.xsd"/>
    <p:input name="config">
        <config>
            <decorate>true</decorate>
        </config>
    </p:input>
    <p:output name="data" id="validated-xml"/>
</p:processor>

Can the validation processor handle character validation such as this? Is there a recommended way to handle non-xml characters inside xml streams, inside a pipeline?

I keep being surprised and impressed by Orbeon Forms. Apart from it meeting the challenge, it is great fun to work with.

Thanks and regards,

Tom


Alessandro Vernet wrote
Tom,

Tom Grahame wrote
The only serializer I seem to be able to make produce the contents of the document tag (thus avoiding the xml declaration) is the file-serializer. For now I'm writing the document to disk and then accessing it again using the url-generator. My pipeline is now arranged like this:

pipeline:input
execute-processor
file-serializer
url-generator
pipeline:output
Maybe the To-XML converter will do what you are looking for. See: http://www.orbeon.com/ops/doc/processors-converters#to-xml-converter

You'll let us know how it works for you.

Alex
Reply | Threaded
Open this post in threaded view
|

Re: Execute processor help needed

Alessandro Vernet
Administrator
Tom,

Tom Grahame wrote
The To-XML converter works very well. I am convinced that I had tried it before but not configured the pipeline properly. Your assurance that it should work helped me revisit and solve the problem.
Excellent.

Tom Grahame wrote
I see from the execute processor source that a potential todo is handle standard input too. This would be nice as I believe I could chain together several  execute processors and pipe the inputs/outputs of binaries and shell scripts together for example.
We haven't used the execute processor here for quite a while, and don't have plans to extend it in the near future. So I can only encourage you to extend that processor to handle standard input/output, or otherwise. And if you want to contribute back your enhancements (with documentation), we'll be happy to fold them into the codebase.

Tom Grahame wrote
Now I have to handle invalid xml characters in the data generated by my binary. Normally the pipeline crashes out when it encounters these, so I have tried using the validation processor to avoid crashing and report the error, but so far this has not worked.
When you have this problem, it is because the XML you are parsing is not well-formed; is that right? The validation processor checks if well-formed XML is valid according to a schema. It doesn't check if some "text" well-formed (i.e. if it is really XML). Maybe you can use the oxf:exception-catcher processor to catch an exception when one comes up. See:

http://www.nabble.com/Catching-Exceptions-in-XPL-td2720251.html

(Ideally, you'd be able to use a p:try/p:catch in XPL, but this construct isn't implemented at this point, hence the exception catcher processors.)

Alex
Reply | Threaded
Open this post in threaded view
|

Re: Execute processor help needed

Tom Grahame
In reply to this post by Alessandro Vernet
Alex,

My analysis/description of the problem was simply incorrect. The output of the binary is well formed xml, but some elements contain invalid Unicode characters. I am managing those by connecting an exception-catcher as you suggest and applying further logic on that output using a p:choose. I was attempting to use the validation processor incorrectly, but I have implemented it later in the pipeline in combination with another exception-catcher.

My pipeline now does what I want, just not how I originally planned it!

Thanks,

Tom

Alessandro Vernet wrote
Tom,

Tom Grahame wrote
The only serializer I seem to be able to make produce the contents of the document tag (thus avoiding the xml declaration) is the file-serializer. For now I'm writing the document to disk and then accessing it again using the url-generator. My pipeline is now arranged like this:

pipeline:input
execute-processor
file-serializer
url-generator
pipeline:output
Maybe the To-XML converter will do what you are looking for. See: http://www.orbeon.com/ops/doc/processors-converters#to-xml-converter

You'll let us know how it works for you.

Alex