Orbeon Forms community mailing list

International characters

Classic

List

Threaded

4 messages Options

mob-2

International characters

Hello,

I am tracking this issue forward but I progress quiet slowly. Here is where I currently am (the logs below were put in these places with the assumption that the characters get corrupted between two calls as seen from the pipeline codes, therefore I tried to find an exit point and an entry point in debug messages):

I put a log message in Group.java (under the XForms.output.element package) in "public void end(){}"
where it pushes the $instance element as a hidden field. The instance with my miserable characters are OK.

Next I put a log message inside RequestParameters.java constructor after the lines:
// Parse XML and store as instance
LocationSAXContentHandler saxContentHandler = new LocationSAXContentHandler();
XMLUtils.stringToSAX(xmlText, null, saxContentHandler, false);
instance = saxContentHandler.getDocument();
and logged instance here as well.

The characters at this point are no more OK.

Have you got some hints that could help me get closer to where the characters get mixed up ?

regards,
Bal??zs

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Erik Bruchez

Re: International characters

Administrator

[hidden email] wrote:
> Hello,
>
> I am tracking this issue forward but I progress quiet slowly. Here
> is where I currently am (the logs below were put in these places
> with the assumption that the characters get corrupted between two
> calls as seen from the pipeline codes, therefore I tried to find an
> exit point and an entry point in debug messages):
>
> I put a log message in Group.java (under the XForms.output.element
> package) in "public void end(){}" where it pushes the $instance
> element as a hidden field. The instance with my miserable characters
> are OK.
>
> Next I put a log message inside RequestParameters.java constructor
after the lines:
> // Parse XML and store as instance
> LocationSAXContentHandler saxContentHandler = new
LocationSAXContentHandler();
> XMLUtils.stringToSAX(xmlText, null,
saxContentHandler, false);
> instance = saxContentHandler.getDocument();
> and logged instance here as well.
>
> The characters at this point are no more OK.
>
> Have you got some hints that could help me get closer to where the
> characters get mixed up ?

Can you see whether the "xmlText" string is ok? In which case, that
would point to XMLUtils.stringToSAX() as the culprit.

-Erik

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

mob-2

Re: International characters

In reply to this post by mob-2

Hello,

It is almost certain that Bas64 encoding does something wrong. I modified XFormsUtils as the followings:

- xmlAsString is OK
- the log saying .../decoded:... is no more OK
- then I left out gzip and simply Base64 coded and decoded and printed the result which was also wrong.

From the above I assumed that the issue has to be with Bas64 encoding. The strange thing is the error is not deterministical - the same document fed into a command line test utility I wrote is not necessarily wrong.

Have some ideas what next ?

regards,
Balazs

***********************
public static String instanceToString(PipelineContext pipelineContext, String password, Document instance) {
removeXXFormsAttributes(instance);
try {
String xmlAsString = XMLUtils.domToString(instance); //mob
logger.info("mob->(XFormsUtils::instanceToString, instance="+xmlAsString); //mob
ByteArrayOutputStream gzipByteArray = new ByteArrayOutputStream();
GZIPOutputStream gzipOutputStream = null;
gzipOutputStream = new GZIPOutputStream(gzipByteArray);
// gzipOutputStream.write(XMLUtils.domToString(instance).getBytes("UTF-8")); //mob commented out
gzipOutputStream.write(xmlAsString.getBytes("UTF-8")); // mob replaced above with this one
gzipOutputStream.close();
String compressed = Base64.encode(gzipByteArray.toByteArray());
logger.info("mob/XFormsUtils::instanceToString()/compressed->"+compressed); //mob
logger.info("mob/XFormsUtils::instanceToString()/decoded:"+XFormsUtils.decode(compressed)); //mob
String base64string = Base64.encode(xmlAsString.getBytes("UTF-8")); //mob
ByteArrayInputStream bais = new ByteArrayInputStream(Base64.decode(base64string)); //mob
byte[] buffer = new byte[1024]; //mob
int size;
StringBuffer xml = new StringBuffer(); //mob
try {
while ((size = bais.read(buffer)) != -1)
xml.append(new String(buffer, 0, size, "UTF-8"));
logger.info("mob/XFormsUtils::instanceToString()/base64encode+decode->:"+xml.toString()); //mob
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

return XFormsUtils.isHiddenEncryptionEnabled()
? SecureUtils.encrypt(pipelineContext, password, compressed)
: compressed;
} catch (IOException e) {

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

Erik Bruchez

Re: Re: International characters

Administrator

Balazs,

This is surprising since Base64 encoding only deals with bytes, not
characters. Could there be an issue with the end of the string? Does
your character encoding issue occur with characters at the end of the
string?

-Erik

[hidden email] wrote:

> Hello,
>
> It is almost certain that Bas64 encoding does something wrong. I modified XFormsUtils as the followings:
>
> - xmlAsString is OK
> - the log saying .../decoded:... is no more OK
> - then I left out gzip and simply Base64 coded and decoded and printed the result which was also wrong.
>
> From the above I assumed that the issue has to be with Bas64 encoding. The strange thing is the error is not deterministical - the same document fed into a command line test utility I wrote is not necessarily wrong.
>
> Have some ideas what next ?
>
> regards,
> Balazs
>
>
> ***********************
> public static String instanceToString(PipelineContext pipelineContext, String password, Document instance) {
> removeXXFormsAttributes(instance);
> try {
> String xmlAsString = XMLUtils.domToString(instance); //mob
> logger.info("mob->(XFormsUtils::instanceToString, instance="+xmlAsString); //mob
> ByteArrayOutputStream gzipByteArray = new ByteArrayOutputStream();
> GZIPOutputStream gzipOutputStream = null;
> gzipOutputStream = new GZIPOutputStream(gzipByteArray);
> // gzipOutputStream.write(XMLUtils.domToString(instance).getBytes("UTF-8")); //mob commented out
> gzipOutputStream.write(xmlAsString.getBytes("UTF-8")); // mob replaced above with this one
> gzipOutputStream.close();
> String compressed = Base64.encode(gzipByteArray.toByteArray());
> logger.info("mob/XFormsUtils::instanceToString()/compressed->"+compressed); //mob
> logger.info("mob/XFormsUtils::instanceToString()/decoded:"+XFormsUtils.decode(compressed)); //mob
> String base64string = Base64.encode(xmlAsString.getBytes("UTF-8")); //mob
> ByteArrayInputStream bais = new ByteArrayInputStream(Base64.decode(base64string)); //mob
> byte[] buffer = new byte[1024]; //mob
> int size;
> StringBuffer xml = new StringBuffer(); //mob
> try {
> while ((size = bais.read(buffer)) != -1)
> xml.append(new String(buffer, 0, size, "UTF-8"));
> logger.info("mob/XFormsUtils::instanceToString()/base64encode+decode->:"+xml.toString()); //mob
> } catch (IOException e) {
> // TODO Auto-generated catch block
> e.printStackTrace();
> }
>
> return XFormsUtils.isHiddenEncryptionEnabled()
> ? SecureUtils.encrypt(pipelineContext, password, compressed)
> : compressed;
> } catch (IOException e) {
>
>
>
> ------------------------------------------------------------------------
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws