Serving plain text

Hi,

I have an instance now where sometimes I will need to transform some
content to plain text. I'm still responding to an http request, the
source is still in XML, it's dynamically generated (collated, sorted,
filtered, etc), so I am of course thinking to do all this in OPS - I
just need to deliver plain text.

Since I wasn't sure I would get the format of the plain text file
right, and I wasn't sure how it would make it to a browser, I added
debug to my pipeline. So it took me a while to realize that when I got
the error "Null document for debug 'dataout'" then it was not saying
there was much wrong with my xslt, it was the debug itself that was the
problem - it seems to be insisting on xml.

Anyway, so now I can see my plain text in the browser.
My xslt starts with <xsl:output method="text
omit-xml-declaration="yes"/>
and yet when the output reaches the browser, the first line is the xml
declaration.
It's the only "change" from what I am outputting.
Could there be something in the epilogue doing this?

I can't be sure if the declaration is there for a non-browser
application, since the intended recipient is still complaining that the
format isn't right, and there are indeed still other errors in the
format I am outputting that I need to fix. But when I get it all done,
I'd like to know there was nothing else "interfering" ;-)

Thanks & regards
Colin




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

I Colin,

Le jeudi 17 novembre 2005 à 15:00 -0500, Colin O'Brien a écrit : I am not sure how you're describing that in your pipeline, but OPS
pipelines are by definition XML and they always carry XML and only XML
even to convey non XML documents :-)..

Text documents are embedded in XML root elements such as:

<document xsi:type="xs:string" content-type="text/plain; charset=iso-8859-1">
This is line one of the input document!
This is line two of the input document!
This is line three of the input document!  
</document>

http://www.orbeon.com/ops/doc/reference-formats#text-documents

The standard epilogue doesn't filter these documents and send them as
XML...

If you want to serve them as text, you need to either modify the
epilogue of shortcut it bu using only a model in your page definition
and use the HTTP serializer to send your document to the browser (see
http://www.orbeon.com/ops/doc/processors-serializers-http#text-mode).

Hope this helps.

Eric

--
Don't you think all these XML schema languages should work together?
                                                         http://dsdl.org
------------------------------------------------------------------------
Eric van der Vlist       http://xmlfr.org            http://dyomedea.com
(ISO) RELAX NG   ISBN:0-596-00421-4 http://oreilly.com/catalog/relax
(W3C) XML Schema ISBN:0-596-00252-1 http://oreilly.com/catalog/xmlschema
------------------------------------------------------------------------




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Hi Eric

thanks for the quick response.

On Nov 17, 2005, at 3:25 PM, Eric van der Vlist wrote:
On a temporary basis, if nothing else, wrapping the text in an element
allows me to use debug to confirm what is coming out of my xslt, and
that it does not have the xml declaration at this point.

Of course, if I do nothing else, then the element is still there when
the text reaches the browser.

And the xml declaration has been added.

> The standard epilogue doesn't filter these documents and send them as
> XML...
>
> If you want to serve them as text, you need to either modify the
> epilogue of shortcut it bu using only a model in your page definition
> and use the HTTP serializer to send your document to the browser (see
> http://www.orbeon.com/ops/doc/processors-serializers-http#text-mode).

Well, of course, that's very interesting.
Like you, I am trying to avoid editing any files that come in the ops
distribution
(all my page flow is in a separate file - so far I have only edited
web.xml).
So I wouldn't want to edit the epilogue.

On the other hand, what I am doing is just an alternative view of the
data being generated.
Other times it wants to go as xml, or rss (which happily is in xml).
So I am deep in the V of the MVC when I make the decision that it needs
to be output as text and call the relevant transform.
So doing it in page flow would not only not be easy, I'm not sure it
conform to a key architectural concept ;-)

I will have to give this some more thought.

Thanks & regards
Colin




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Le jeudi 17 novembre 2005 à 16:14 -0500, Colin O'Brien a écrit :
The epilogue pipelines (there is one for servlets and one for portlets)
are very straightforward (especially in 3.0) and I don't think that
you'd be taken a high risk by modifying it.

I'd even say that a modification that serialize non XML documents
according to the spec (http://www.orbeon.com/ops/doc/reference-formats)
would probably a good thing for most users and you might be able to
convince Erik and Alex to make it standard :-) ...

Eric

--
Le premier annuaire des apiculteurs 100% XML!
                                                http://apiculteurs.info/
------------------------------------------------------------------------
Eric van der Vlist       http://xmlfr.org            http://dyomedea.com
(ISO) RELAX NG   ISBN:0-596-00421-4 http://oreilly.com/catalog/relax
(W3C) XML Schema ISBN:0-596-00252-1 http://oreilly.com/catalog/xmlschema
------------------------------------------------------------------------




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Hi Eric

yes, I had already given in and started looking at the epilogue.
I was happy to see that it has the instance as input, since that is  
where I know what view to use anyway.
So it is easy to just test and do my own http-serializer.

Except that the browser still shows the output arriving with the xml  
declaration.

So I am now configuring my system so I have something at a lower level  
than the browser to inspect what I am receiving from the server.

Thanks again
Colin

On Nov 17, 2005, at 4:24 PM, Eric van der Vlist wrote:



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Le jeudi 17 novembre 2005 à 16:41 -0500, Colin O'Brien a écrit :
> Hi Eric
>
> yes, I had already given in and started looking at the epilogue.
> I was happy to see that it has the instance as input, since that is  
> where I know what view to use anyway.
> So it is easy to just test and do my own http-serializer.
>
> Except that the browser still shows the output arriving with the xml  
> declaration.

Weird...

In 3.0, when I add the following "when" in the main p:choose:

        <!-- Non XML documents -->
        <p:when test="/document[@xsi:type]">
            <p:processor name="oxf:http-serializer">
                <p:input name="config">
                    <config/>
                </p:input>
                <p:input name="data" href="#xformed-data"/>
            </p:processor>
        </p:when>

I get my document served as text as expected.

It's not 100% safe, though since the "document" element belongs to no
namespace and there is a risk (at least in theory) that someone might
use this element in a custom vocabulary with a different meaning in
mind. That's why I have added a test that there is a xsi:type attribute
to reduce this risk.

I attach the pipeline that I have updated.

Eric
--
Carnet web :
           http://eric.van-der-vlist.com/blog?t=category&a=Fran%C3%A7ais
------------------------------------------------------------------------
Eric van der Vlist       http://xmlfr.org            http://dyomedea.com
(ISO) RELAX NG   ISBN:0-596-00421-4 http://oreilly.com/catalog/relax
(W3C) XML Schema ISBN:0-596-00252-1 http://oreilly.com/catalog/xmlschema
------------------------------------------------------------------------


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Yes, it was weird.
Even though my http-serializer was assigning a mime type (text/plain),
the document was still arriving as application/xml.
Since you had it working (many thanks for taking that effort) I knew
where to look next.

The comments in the epilogue say the instance is the xml submission.
That must mean an xforms submission, because in this case it is empty.
My "instance" was built from the url using default-submission in the
page flow.
Hence my test was failing, and I was following normal epilogue
processing.

If I ignore that detail and force the epilogue to execute my
http-serializer, then I do indeed send down a plain text file.

But to get that to work, it seems it wanted an xml document, so I am
still receiving my text wrapped in an xml element and the receiving
application won't want that.
Which is weird, because if I leave the element off and let the epilogue
do what it wants, it can process the text OK (apart from adding the xml
declaration).

I think that's enough for me for today ;-)

best regards
Colin

PS I am running this in Eclipse, so I am just going through epilogue.xpl
I don't think that makes any difference, but just to be complete.

On Nov 17, 2005, at 4:58 PM, Eric van der Vlist wrote:



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Some more searching in the documentation!
leads me to the text-serializer.
Reasonable enough I guess.
The doc talks about csv files, and if OPS can produce them, it can do
what I need.
Sure can.
Except that it's now legacy.
But the Employee example illustrates very well how to use it.
But not how to achieve the same with the http-serializer.
So now I really can go home, and look for full enlightenment tomorrow.

Thanks & regards
Colin




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Le jeudi 17 novembre 2005 à 19:16 -0500, Colin O'Brien a écrit : :-) ...

I have noticed that Firefox (and it might be the case of other browsers)
tends to ignore a change of media type when you reload a document and
that you often need to do a "shift reload" so that it sees that the
media type has changed let's say from application/xml to text/plain.

Could it be one of your problems?

Eric

--
Le premier annuaire des apiculteurs 100% XML!
                                                http://apiculteurs.info/
------------------------------------------------------------------------
Eric van der Vlist       http://xmlfr.org            http://dyomedea.com
(ISO) RELAX NG   ISBN:0-596-00421-4 http://oreilly.com/catalog/relax
(W3C) XML Schema ISBN:0-596-00252-1 http://oreilly.com/catalog/xmlschema
------------------------------------------------------------------------




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Colin,

The Text serializer is legacy, because you should use the Text converter
+ HTTP serializer combination instead.

-Erik

Colin O'Brien wrote:

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Hi Eric

On Nov 18, 2005, at 2:08 AM, Eric van der Vlist wrote:

> I have noticed that Firefox (and it might be the case of other
> browsers)
> tends to ignore a change of media type when you reload a document and
> that you often need to do a "shift reload" so that it sees that the
> media type has changed let's say from application/xml to text/plain.
>
> Could it be one of your problems?
>
Thanks for the tip - I will try it as when I get back into testing this
morning.
As for yesterday, as soon as the text serializer did what I wanted, the
text showed up correctly everywhere.

Colin




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Hi Erik,

On Nov 18, 2005, at 5:14 AM, Erik Bruchez wrote:

> Colin,
>
> The Text serializer is legacy, because you should use the Text
> converter
> + HTTP serializer combination instead.

yes, I realize that, but
I had already used the http-serializer and it hadn't worked
the example to hand used the text serializer
it worked
and I had had enough for one day ;-)

Hopefully I will get the http serializer working today.

Perhaps you can comment on what I seemed to experience yesterday
* the text serializer required that the text be wrapped in a <text>
element, which it stripped
* the http serializer requires that the text be wrapped in a <document>
element, but it does not strip it

Oh wait - you're saying use the text converter as well as the http
serializer.
I didn't try that (didn't see a mention of the converter on the
serializer page)
So the text converter will strip the <document> element.
Except that the doc on text converter shows the <document> element
still there in the output.
But the doc does say this is how to get csv now.
I will have to try it when I get to the office.

Thanks & regards
Colin




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Hello Erik and Eric

to wrap this up... The combination of text converter and http serializer did produce just
the text I wanted.
I took the <docuemnt> element from the output of the text converter and
made it the wrapper element in my xslt and removed the text converter
and I still got what I wanted.
I'm sure I tried that combination yesterday and it didn't work - the
only difference I can detect is that when I defined the content-type I
omitted the semi-colon from content-type="text/plain;".
It's a harsh world :-)

So the docs are probably right in as much as what they say, but they
don't spell out that the serializers will strip wrapper elements when
they conform to the OPS non-xml document spec.

One thing that seems to be missing in the converter/serializer combo
compared to the legacy text serializer is the ability to suggest a name
for the file being downloaded.
It doesn't matter for what I am doing, since the file isn't intended to
be downloaded to the client file system, but I could see it being more
important with examples like csv.

Thanks once again for your help and the ever greater OPS system
Colin




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Colin O'Brien wrote:

> The combination of text converter and http serializer did produce just
> the text I wanted.
> I took the <docuemnt> element from the output of the text converter and
> made it the wrapper element in my xslt and removed the text converter
> and I still got what I wanted.

Yes, you can do that.

> So the docs are probably right in as much as what they say, but they
> don't spell out that the serializers will strip wrapper elements when
> they conform to the OPS non-xml document spec.

It kind of goes without saying, since there is no such thing as an
"element" in a plain text document ;-)

> One thing that seems to be missing in the converter/serializer combo
> compared to the legacy text serializer is the ability to suggest a name
> for the file being downloaded.
> It doesn't matter for what I am doing, since the file isn't intended to
> be downloaded to the client file system, but I could see it being more
> important with examples like csv.

You can, as shown in the existing epilogues for PDF handling:

<p:processor name="oxf:xslfo-converter">
     <p:input name="config"><config/></p:input>
     <p:input name="data" href="#xformed-data"/>
     <p:output name="data" id="converted"/>
</p:processor>
<p:processor name="oxf:http-serializer">
     <p:input name="config">
         <config>
             <header>
                 <name>Cache-Control</name>
                 <value>post-check=0, pre-check=0</value>
             </header>
             <!-- This is what causes a file name to be suggested -->
             <header>
                 <name>Content-Disposition</name>
                 <value>attachment; filename=document.pdf</value>
             </header>
         </config>
     </p:input>
     <p:input name="data" href="#converted"/>
</p:processor>

> Thanks once again for your help and the ever greater OPS system

You are welcome :-)

-Erik




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

OK, if I have to maintain a custom version of one post-processiing file
in the distribution, I might as well have them all ;-)
So now I have
* config/epilogue.xpl
+ use http serializer for text documents
+ skip xforms processing when no xforms
* config/epilogue-servlet.xpl
+ produce xhtml documents
   (not clear when you would want to return html to a client that says
it can handle xhtml,
    so don't know why xhtml isn't produced by default in such situations
but I suspect it is in the archive since we've discussed it before,
    but I haven't made all clients receive xhtml, yet, but I probably
will since that is what we are already doing with static sites and
indeed with our 2.8 site)
* config/theme/theme.xsl
+ added exclude-results-prefixes
+ skip xforms processing when no xforms

Thanks & regards
Colin




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

On 11/17/05, Eric van der Vlist <[hidden email]> wrote:
> I'd even say that a modification that serialize non XML documents
> according to the spec (http://www.orbeon.com/ops/doc/reference-formats)
> would probably a good thing for most users and you might be able to
> convince Erik and Alex to make it standard :-) ...

Eric,

So that would be a modification to the config/epilogue-servlet.xpl.
Just like PDF is handled in that file right now, we would look for a
root element <document xsi:type="xs:string" content-type="text/plain;
charset=utf-8"> and use the text-converter in that case.

Colin, have you already modified this file for that purpose.

For reference:
http://forge.objectweb.org/tracker/index.php?func=detail&aid=304308&group_id=168&atid=350207

Alex
--
Blog (XML, Web apps, Open Source): http://www.orbeon.com/blog/



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Le lundi 21 novembre 2005 à 18:23 -0800, Alessandro Vernet a écrit : Yes. The patch I have proposed:

        <!-- Non XML documents -->
        <p:when test="/document[@xsi:type]">
            <p:processor name="oxf:http-serializer">
                <p:input name="config">
                    <config/>
                </p:input>
                <p:input name="data" href="#xformed-data"/>
            </p:processor>
        </p:when>

is more general than just plain text and would also support binary
formats.

The only drawback I can see is the one mentioned in my previous mail :

> It's not 100% safe, though since the "document" element belongs to no
> namespace and there is a risk (at least in theory) that someone might
> use this element in a custom vocabulary with a different meaning in
> mind. That's why I have added a test that there is a xsi:type
> attribute
> to reduce this risk.

Eric
--
Le premier annuaire des apiculteurs 100% XML!
                                                http://apiculteurs.info/
------------------------------------------------------------------------
Eric van der Vlist       http://xmlfr.org            http://dyomedea.com
(ISO) RELAX NG   ISBN:0-596-00421-4 http://oreilly.com/catalog/relax
(W3C) XML Schema ISBN:0-596-00252-1 http://oreilly.com/catalog/xmlschema
------------------------------------------------------------------------




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Eric van der Vlist wrote:
All right, the change is in!

-Erik



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Hi Alex

apologies for just getting back to you on this.
I was traveling for the holiday and been in catch up ever since!

As I think I said in an earlier note, maybe a different thread, I put  
the change right at the start of config/epilogue.xpl.
I figured if we had finished processing for this response, why bother  
letting it go any further.
Hope I'm not missing anything.

I am doing a more specific test and a more specific mime type in the  
response, so wherever the test is, I will always need to add my own  
custom one.

I think it is good though to have a general test in the out of the box  
system.
That will hopefully save others from my initial confusion.

I still think the documentation needs to be explicit in saying that the  
<document> element will be automagically removed.

I also think that document is an unfortunate name to use for this  
purpose, as it is so easy for a developer to use such an element in xml  
structures of their own.
Checking through my system, I found that I had!
Probably already in too many people's systems to change by now though.
In which case again, I would suggest more prominent warning in the  
documentation.

Anyway, you've covered the basics, so thank you for that.
And also for Beta 4 - I think that was definitely the right thing to do.
And thanks also for keeping the focus on getting the XForms code  
complete and documented
- just imagine what we'll all think of to do after that ;-)

Well, thanks for it all.
Best regards
Colin

On Nov 21, 2005, at 9:23 PM, Alessandro Vernet wrote:



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws