Win 8.1, Metro and touch events

We are developing a Win8 client for our system and need to support the display and edit of eforms. This is done through a webview control which embeds IE11 in the app.

We started out by just testing with the browser in both desktop and metro mode.

This seems to work fine for Orbeon 3.9 and 4.7.1 as long as you are not using the touch screen.

With the touch screen, in either desktop mode or metro, two things don't seem to work correctly in the form (this is after a quick test, we haven't exhaustively tested this).

Formatted text: (3.9)

Enter some text, a couple of lines and then touch  the bold (B) button at the top. The cursor is relocated to the start of the first line of text. This occurs for the other buttons as well. If you click with the mouse or touch pad the cursor is not relocated.

Date: (3.9 and 4.7.1)

touch the date field and then touch a date in the date picker. It closes but the field is not updated with the selected date. Again, if you use the touch pad or mouse it works as expected.

Fields Date: (3.9 and 4.7.1)

1) if you click near the edge of one of the fields to popup the calendar then it inserts the whole date string in the first field.
2) if you click on the calendar icon then it doesn't update any of the fields.

We suspect this is to do with the different events used for the touch events. This is discussed here:

https://github.com/hakimel/reveal.js/issues/300
http://blogs.msdn.com/b/ie/archive/2011/09/20/touch-input-for-ie10-and-metro-style-apps.aspx
 
Note that while the blog above for IE10 says to use ‘MSPointerDown’, the following documentation says that as of IE11 the prefix should be dropped and ‘pointerdown’ should be used.

https://msdn.microsoft.com/en-us/library/windows/apps/hh465891.aspx

Do you know of any way to get these to work correctly with the touch interface in Win8 and IE11?

thanks
brian...

Hi Brian,

Do you know if it is possible to replicate those issues with IE11 on a non-touch enabled device? I am asking because I don't have access to a Win8 PC with a touch screen, and it will be hard to deal with those issues if I can see the problem. The Chrome Dev Tools have a "emulate touch screen"; for is there something similar in the IE11 F12 tools?

Alex

I haven't reproduced this on a non-touch device and I don't think it can be. The issue seems to be related to the specific and different events used for touch versus mouse interactions. Would this be YUI related? A quick search shows a number of touch related issues in 2013 and 2014.

Is there some info I can gather from the system using the touch device that might help?

thanks,
brian...

This electronic communication, including any attached documents, may contain confidential and/or legally privileged information that is intended only for use by the recipient(s) named above. If you have received this communication in error, please notify the sender immediately and delete the communication and any attachments. Views expressed by the author do not necessarily represent those of Martin Luther College.

--
You received this message because you are subscribed to the Google Groups "Orbeon Forms" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].

Running developer tools in IE shows me a difference between the touch and mouse click on a calendar date. With the mouse click it works fine, with the touch I get an exception in the JS.

"Unable to get property 'id' of undefined or null reference"

See the attached screen shot for more info.

Can I run Orbeon so that it doesn't use the compress JS code. What you can see in the screen shot is not very useful.

Hi Brian,

Interesting; if you add the following 2 properties, so you have non-minimized, separate JavaScript files, maybe that will help you debug this further, and maybe find where this fails, and how it can be fixed.

<property as="xs:boolean" name="oxf.xforms.minimal-resources" value="false"/>
<property as="xs:boolean" name="oxf.xforms.combine-resources" value="false"/>

Alex

That helped, now I have more readable code. Got the browser to break at the point that the exception was thrown (sorry, another screenshot) and the field "control" from the event is null, this is because the MSEventObj doesn't have a control field as far as I can see. JavaScript is not a language I am very familiar with so I haven't got much further than this after a couple of hours.

Hi Brian, Aaron,

Thank you for the screenshot; it is quite telling. So in the setValue() function, control is null. That control should be initialized on click [1], but I guess that event handler doesn't run on IE as it doesn't consider the touch be like a click (unlike Chrome and Safari - more on this later). This clickEvent is a custom event [2], dispatched when click is called [3], which itself is registered for the whole document here [4].

Several things can be happening there. Maybe click isn't called at all, or it just called "too late" as there is usually a delay between the touch and click, when the OS doesn't yet know if the "touch" was done as a "click" or the start of a "drag". It would be really good for me, or someone else to be able to debug / experiment with this.

Aaron, you mentioned that "It might be possible to reproduce on a non-touch device if touch emulating software is installed." Do you know more about this? I wouldn't mind, but I'm not sure working on this is a good enough reason for me to go and purchase a Surface ;).

[1] https://github.com/orbeon/orbeon-forms/blob/tag-release-4.8.1-ce/src/resources-packaged/ops/javascript/orbeon/xforms/control/Calendar.coffee#L118
[2] https://github.com/orbeon/orbeon-forms/blob/tag-release-4.8.1-ce/src/resources-packaged/ops/javascript/xforms.js#L3096
[3] https://github.com/orbeon/orbeon-forms/blob/tag-release-4.8.1-ce/src/resources-packaged/ops/javascript/xforms.js#L3104
[4] https://github.com/orbeon/orbeon-forms/blob/tag-release-4.8.1-ce/src/resources-packaged/ops/javascript/xforms.js#L4068

Alex

This electronic communication, including any attached documents, may contain confidential and/or legally privileged information that is intended only for use by the recipient(s) named above. If you have received this communication in error, please notify the sender immediately and delete the communication and any attachments. Views expressed by the author do not necessarily represent those of Martin Luther College.

--
You received this message because you are subscribed to the Google Groups "Orbeon Forms" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].

EMAIL DISCLAIMER This email message and its attachments are confidential and may also contain copyright or privileged material. If you are not the intended recipient, you may not forward the email or disclose or use the information contained in it. If you have received this email message in error, please advise the sender immediately by replying to this email and delete the message and any associated attachments. Any views, opinions, conclusions, advice or statements expressed in this email message are those of the individual sender and should not be relied upon as the considered view, opinion, conclusions, advice or statement of this company except where the sender expressly, and with authority, states them to be the considered view, opinion, conclusions, advice or statement of this company. Every care is taken but we recommend that you scan any attachments for viruses.

Hi Brian,

I have seen a reference to that as well, but unfortunately I don't have access to a machine with Visual Studio 2012 or one I can install it on, only to a plain machine running IE11.

Alex

I had a play with the emulator and the date field works fine in the emulator when simulating touch.

We had more of a play with the surface pro 3 and discovered that the date field also works fine with the pen, so the touch and pen events are somehow different. I suspect the pen just uses the mouse events.

You also cannot use the image annotation control with touch since touch and drag scrolls instead of draws. The pen and mouse work fine though.

I also tried IE9 emulation mode but no difference in behaviour.

Not sure how to proceed.

Hi Brian,

I am wondering if I would be able to simulate this with Selenium using TouchActions.singleTap() [1]. So the procedure would be to open the controls forms [2], switch to "Date and Time", click on the text field of the date control, click on one of the dates in the picker, at which point the JavaScript error should occur, correct?

[1] http://selenium.googlecode.com/git/docs/api/java/org/openqa/selenium/interactions/touch/TouchActions.html
[2] http://demo.orbeon.com/orbeon/fr/orbeon/controls/new

Alex

Hi Alex,
  I haven't forgotten this but have been distracted by other work.

I'm trying to get a selenium test working at the moment to reproduce this.

brian...

I've also noticed that the scrollable list doesn't work the same with touch as with the mouse.

You can select a different entry in the list with touch as long as that entry is higher up the list than the one already selected, you cannot select an entry that is below the one that is selected. Very odd behaviour. With the mouse it works as expected.

And the image annotation control also doesn't work with touch since the touch events and gestures scroll the whole screen around and are apparently not sent to the underlying control. Mouse drawing also doesn't seem to work well, the coordinates are offset to the upper right by about 3cm on an surface pro.

--
You received this message because you are subscribed to the Google Groups "Orbeon Forms" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].