Submission Processor and eXist REST, debugging issue

My pipeline calls submission processor to post an
update using the eXist REST service.  When I
create the data using the identity processor
and point my browser at the service url, it
works correctly.

When I use the instance data and execute the
pipeline from my a submission in my application,
it "freezes" eXist for some time -- even the exist
admin jnpl application does not respond.

Mostly, I am asking how to debug further.

Below are queries, first from identity processor,
second from instance -- I don't see any problems.
Below that, the pipeline.

Has anyone seen such a problem, locking up eXist?

Best,
Hank
Institute for Crustal Studies

*** A working query *****

<?xml version="1.0" encoding="utf-8"?>
<xu:modifications xmlns:xu="http://www.xmldb.org/xupdate" version="1.0">
 <xu:append select="/changes">
  <change>
   <date>2007-01-30</date>
   <hour>17</hour>
   <min>02</min>
   <sec>02</sec>
   <action>replace</action>
   <network>SB</network>
   <station>GVDA</station>
   <loc>03</loc>
   <lat>32.0</lat>
   <lon>-112.0</lon>
   <depth>0.0</depth>
   <sensor-model-new>episensor_10vg</sensor-model-new>
   <sensor-serial-new>101</sensor-serial-new>
   <sensor-model-old>fba23_0_5g</sensor-model-old>
   <sensor-serial-old>100</sensor-serial-old>
   <explanation>test 52</explanation>
 </change>
 </xu:append>
</xu:modifications>


**** a query that locks up exist ******

<?xml version="1.0" encoding="utf-8"?>
<xu:modifications xmlns:xu="http://www.xmldb.org/xupdate" version="1.0">
 <xu:append select="/changes">
  <change>
        <date>2007-01-30-08:00</date>
        <hour>11</hour>
        <min>11</min>
        <sec>11</sec>
        <action>add</action>
        <network>SB</network>
        <station>HEO</station>
        <loc>01</loc>
        <lat>36.758</lat>
        <lon>-121.609</lon>
        <depth>0</depth>
        <sensor-model-new/>
        <sensor-serial-new>1341234</sensor-serial-new>
        <sensor-model-old/>
        <sensor-serial-old/>
        <explanation>test 54</explanation>
  </change>
 </xu:append>
</xu:modifications>

The pipeline that works as an external url,
but freezes eXist when called from a local submission:

<p:config xmlns:p="http://www.orbeon.com/oxf/pipeline"
    xmlns:oxf="http://www.orbeon.com/oxf/processors">

    <p:param name="instance" type="input" />

    <!-- I do a lot of work to remove all namespaces -->
    <p:processor name="oxf:xslt">
      <p:input name="data" href="#instance" />
      <p:input name="config">
        <xsl:stylesheet version="2.0"
           xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
           xmlns:xforms="http://www.w3.org/2002/xforms"
           xmlns:f="http://orbeon.org/oxf/xml/formatting"
           xmlns:xxforms="http://orbeon.org/oxf/xml/xforms"
           xmlns:xi="http://www.w3.org/2001/XInclude"
           xmlns:widget="http://orbeon.org/oxf/xml/widget"
           xmlns:ev="http://www.w3.org/2001/xml-events"
           xmlns:exist="http://exist.sourceforge.net/NS/exist"
           xmlns:fn="http://www.w3.org/2005/02/xpath-functions"
           exclude-result-prefixes="oxf p xforms f xxforms xi widget ev exist
fn">

           <xsl:template match="/">
             <xu:modifications version="1.0"
                xmlns:xu="http://www.xmldb.org/xupdate">
                <xu:append select="/changes">
                    <!-- <xsl:copy-of select="." /> -->
                    <xsl:apply-templates/>
                </xu:append>
             </xu:modifications>
           </xsl:template>

           <xsl:template match="*">
              <xsl:element name="{local-name()}">
                 <xsl:apply-templates/>
              </xsl:element>
           </xsl:template>

         </xsl:stylesheet>

        </p:input>
     <p:output id="query"/>
  </p:processor>

  <p:processor name="oxf:xforms-submission" >
    <p:input name="submission"
      xmlns:xforms="http://www.w3.org/2002/xforms"
      xmlns:xxforms="http://orbeon.org/oxf/xml/xforms">
       <xforms:submission method="post"
          action="http://localhost/exist/rest/db/runtime/sensor-changes.xml"
          xxforms:user="********"
          xxforms:password="*********" />
   </p:input>
   <p:input name="request" href="#query" />
   <p:output name="response" id="response" />
 </p:processor>

 <p:processor name="oxf:xml-serializer">
   <p:input name="config">
     <config>
       <content-type>text/xml</content-type>
     </config>
  </p:input>
 <p:input name="data" href="#response"/>
  </p:processor>

</p:config>


----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Hi Hank,

I have never seen this behavior. You might want to try to use tcpmon
(http://ws.apache.org/commons/tcpmon/), pointing your application to
tcpmon and pointing tcpmon to eXist, so you can see exactly what goes
on with those requests.

Since you have 2 similar cases, where one works but the other does
not, using tcpmon you might be able to pin point the difference
between those 2 cases.

Alex

On 1/30/07, [hidden email] <[hidden email]> wrote:
--
Orbeon Forms - Web Forms for the Enterprise, Done the Right Way
http://www.orbeon.com/



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

tcpmon - that's a great idea.

I can reproduce the behavior, with one difference, the
call is extremely long.  (Previously, the successful
attempts were short, a few seconds ).

In the failed conversation, there is no authorization
request!  But the submission is the same.  The difference
is that in the successful case, the pipeline is executed
in response to a GET from a browser and it generates the
data internally.  The failed case is executed by POSTing
from an xforms control/submission and gets the data from
the instance parameter.

In both cases the xxforms:user and xxforms:password are
the same, but looking at the http conversation, the failed
case does not have an authentication header!.

What is the next step?

Thank you,
Hank

Here is a successful conversation (24006 ms) :

POST /exist/rest HTTP/1.1
authorization: Basic cnQ6NHJ0QHJhdGwx
Cookie: JSESSIONID=7BA992149E027B477817B0BB61DF0C2A
content-type: application/xml
User-Agent: Jakarta Commons-HttpClient/3.0.1
Host: 127.0.0.1:8888
Content-Length: 1583

<?xml version="1.0" encoding="UTF-8"?><xu:modifications xmlns:xu="http://www.xmldb.org/xupdate"
xmlns:oxf="http://www.orbeon.com/oxf/processors" xmlns:xxforms="http://orbeon.org/oxf/xml/xforms"
xmlns:context="java:org.orbeon.oxf.pipeline.StaticExternalContext" xmlns:p="http://www.orbeon.com/oxf/pipeline"
xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:ev="http://www.w3.org/2001/xml-events" xmlns:saxon="http://saxon.sf.net/"
xmlns:xforms="http://www.w3.org/2002/xforms" version="1.0">                <xu:append
select="doc('/db/runtime/sensor-changes.xml')/changes">                    <change>
  <date>2007-02-01</date>
  <hour>02</hour>
  <min>02</min>
  <sec>02</sec>
  <action>replace</action>
  <network>SB</network>
  <station>WLA</station>
  <loc>05</loc>
  <lat>32.0</lat>
  <lon>-112.0</lon>
  <depth>1.0</depth>
  <sensor-model-new>episensor_10vg</sensor-model-new>
  <sensor-serial-new>101</sensor-serial-new>
  <sensor-model-old>fba23_0_5g</sensor-model-old>
  <sensor-serial-old>100</sensor-serial-old>
  <explanation>test 200</explanation>
 </change>
</xu:append>
</xu:modifications>

----------------------------

HTTP/1.1 200 OK
Date: Thu, 01 Feb 2007 19:14:20 GMT
Server: Apache/2.0.53 (Unix) mod_ssl/2.0.53 OpenSSL/0.9.7a mod_jk2/2.0.0
Content-Type: text/xml;charset=UTF-8
Content-Length: 152

<?xml version='1.0'?>
<exist:modifications xmlns:exist='http://exist.sourceforge.net/NS/exist' count='1'>1modifications processed.</exist:modifications>

==========================================

Here is a failed conversation (240138 ms):

POST /exist/rest HTTP/1.1
Cookie: JSESSIONID=7BA992149E027B477817B0BB61DF0C2A
content-type: application/xml
User-Agent: Jakarta Commons-HttpClient/3.0.1
Host: 127.0.0.1:8888
Content-Length: 1479

<?xml version="1.0" encoding="UTF-8"?><xu:modifications xmlns:xu="http://www.xmldb.org/xupdate"
xmlns:oxf="http://www.orbeon.com/oxf/processors" xmlns:xxforms="http://orbeon.org/oxf/xml/xforms"
xmlns:context="java:org.orbeon.oxf.pipeline.StaticExternalContext" xmlns:p="http://www.orbeon.com/oxf/pipeline"
xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:ev="http://www.w3.org/2001/xml-events" xmlns:saxon="http://saxon.sf.net/"
xmlns:xforms="http://www.w3.org/2002/xforms" version="1.0">                <xu:append
select="doc('/db/runtime/sensor-changes.xml')/changes">
   <change>
     <date>2007-02-01-08:00</date>
     <hour>10</hour>
     <min>10</min>
     <sec>10</sec>
     <action>add</action>
     <network>SB</network>
     <station>5210</station>
     <loc>00</loc>
     <lat>33.0969</lat>
     <lon>-115.5304</lon>
     <depth>0</depth>
     <sensor-model-new/>
     <sensor-serial-new>12341234</sensor-serial-new>
     <sensor-model-old/>
     <sensor-serial-old/>
     <explanation>test 102</explanation>
    </change>
  </xu:append>
</xu:modifications>

-----------------

HTTP/1.1 400 Query exception while parsing request: Permission denied to read collection '/db/runtime' [at line 1, column 1]
Date: Thu, 01 Feb 2007 19:06:58 GMT
Server: Apache/2.0.53 (Unix) mod_ssl/2.0.53 OpenSSL/0.9.7a mod_jk2/2.0.0
Content-Type: text/html;charset=ISO-8859-1
Content-Language: en-US
Connection: close
Transfer-Encoding: chunked

3f9
<html><head><title>Apache Tomcat/4.1.30 - Error report</title><STYLE><!--H1{font-family : sans-serif,Arial,Tahoma;color :
white;background-color : #0086b2;} H3{font-family : sans-serif,Arial,Tahoma;color : white;background-color : #0086b2;}
BODY{font-family : sans-serif,Arial,Tahoma;color : black;background-color : white;} B{color : white;background-color : #0086b2;}
HR{color : #0086b2;} --></STYLE> </head><body><h1>HTTP Status 400 - Query exception while parsing request: Permission denied to read
collection '/db/runtime' [at line 1, column 1]</h1><HR size="1" noshade><p><b>type</b> Status report</p><p><b>message</b> <u>Query
exception while parsing request: Permission denied to read collection '/db/runtime' [at line 1, column
1]</u></p><p><b>description</b> <u>The request sent by the client was syntactically incorrect (Query exception while parsing
request: Permission denied to read collection '/db/runtime' [at line 1, column 1]).</u></p><HR size="1" noshade><h3>Apache
Tomcat/4.1.30</h3></body></html>
0


[snip old message]




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

I can summarize this and add one more bit of
information.  The post is to a realm that requires
authorization, but the authorization header is
not within the post that is the result of the
<xforms:send> submission.

Succeeded:

> POST /exist/rest HTTP/1.1
> authorization: Basic cnQ6NHJ0QHJhdGwx
> Cookie: JSESSIONID=7BA992149E027B477817B0BB61DF0C2A
> content-type: application/xml
> User-Agent: Jakarta Commons-HttpClient/3.0.1
> Host: 127.0.0.1:8888
> Content-Length: 1583

Failed:

> POST /exist/rest HTTP/1.1
> Cookie: JSESSIONID=7BA992149E027B477817B0BB61DF0C2A
> content-type: application/xml
> User-Agent: Jakarta Commons-HttpClient/3.0.1
> Host: 127.0.0.1:8888
> Content-Length: 1479

Further, I was looking in XFormsSubmissionUtils.java and I see
where the logger, (appx. line 160) records the url connection,
cookie forwarding, and setting request body:

XForms - opening URL connection for: http://localhost/exist/rest
XForms - forwarding cookie: JSESSIONID=7708FFD746F3201B60AEE42C32099DAC
XForms - setting request body: <?xml version="1.0" encoding="......

It does not show the authorization header, and my brief attempts to force
it to have not worked.  

It is unclear why my test, in which I point a browser to the
pipeline does authenticate, but the same post does not when
it originates from an xforms submission.

Thanks,
Hank

Intitute for Crustal Studies
UCSB




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

On 2/2/07, Hank Ratzesberger <[hidden email]> wrote:
> It does not show the authorization header, and my brief attempts to force
> it to have not worked.

Hank,

I fixed tonight a bug where the symptom is exactly the same as the one
you describe. Could you try to get a nightly build and try this again?

Alex
--
Orbeon Forms - Web Forms for the Enterprise, Done the Right Way
http://www.orbeon.com/



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Hi Alex,

Thank you, but there is no difference in my case.
Below, the log shows that no authentication header
is submitted and the tcpmon trace after that confirms it.

One issue with my configuration is that eXist is deployed on the
same server/container, but in a different application context.
The idea is that the db providing rest services could be moved
or changed (e.g. oracle).  But deploying it to the same container
appears to be a problem even with xforms-submit optimization off.

Regards,
Hank


XForms - dispatching event: DOMActivate - trigger - at XFormsEventFactory.java, line 34, column -1
XForms - executing action: send
XForms - dispatching event:   xforms-submit - default-submission - at XFormsSendAction.java, line 45, column -1
XForms - dispatching event:     xforms-submit-serialize - default-submission - at XFormsModelSubmission.java, line 450, column -1
XForms - opening URL connection for: http://localhost/exist/rest/db/runtime/sensor-changes.xml
XForms - forwarding cookie: JSESSIONID=0CE1CC07BD503E6555D6A32D10307391
XForms - setting request body: <?xml version="1.0" encoding="UTF-8"?>....

-----------------

POST /exist/rest/db/runtime/sensor-changes.xml HTTP/1.1
Cookie: JSESSIONID=0CE1CC07BD503E6555D6A32D10307391
Content-type: application/xml
User-Agent: Jakarta Commons-HttpClient/3.0.1
Host: 127.0.0.1:8888
Content-Length: 1446


----- Original Message -----
From: "Alessandro Vernet" <[hidden email]>
To: <[hidden email]>
Sent: Sunday, February 04, 2007 2:56 AM
Subject: Re: [ops-users] Submission Processor and eXist REST, debugging issue


--------------------------------------------------------------------------------


>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Hank,

Can you try to set the oxf.xforms.optimize-local-submission to "false"
and see if the Authorization header gets passed in this case?

This type of problems are almost impossible to fix without being able
to reproduce them. Could you put together an example that shows this?

Alex

On 2/5/07, Hank Ratzesberger <[hidden email]> wrote:
--
Orbeon Forms - Web Forms for the Enterprise, Done the Right Way
http://www.orbeon.com/



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws