performance issue with xpath expressions

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

performance issue with xpath expressions

Myxlyxl
Hi there,

I have the following xforms:

<?xml version="1.0" encoding="UTF-8"?>
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:ev="http://www.w3.org/2001/xml-events" xmlns:xxforms="http://orbeon.org/oxf/xml/xforms" xmlns:xforms="http://www.w3.org/2002/xforms" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <head>
    <xforms:model id="mainModel">
      <xforms:instance id="main-instance">
                <WebACD xmlns="">
                        <Routings>
                                <Routing Event="1" ID="SGR000000107">
                                        <Skill>
                                                <Skillgroup IDREF="SG0000000000"/>
                                        </Skill>
                                        <Default_Ziel_Name ID="DZR000000107">
                                                <Defaultziel IDREF="DEF000000101"/>
                                        </Default_Ziel_Name>
                                        <Default_0Ag_Name ID="D0R000000107">
                                                <Defaultziel IDREF="DEF000000014"/>
                                        </Default_0Ag_Name>
                                        <Default_leer_Name>
                                                <Defaultziel IDREF="DEF000000013"/>
                                        </Default_leer_Name>
                                        <BegrAnsage_Name/>
                                        <Dialog_Name/>
                                        <WF_Name>
                                                <Wartefeld IDREF="WF0000000101"/>
                                        </WF_Name>
                                        <Kommentar/>
                                </Routing>
                        </Routings>
                </WebACD>
          </xforms:instance>
     
          <xforms:instance id="tree-instance">
                <WebACD>
                        <Routings Label="'Routings'" Value="concat('/Routings','')">
                                <Routing Label="for $activeNode in (.) return concat( instance('main-instance')/Routings/Routing[count($activeNode/preceding-sibling::*)+1]/Kommentar,' ', ' (',instance('main-instance')/Routings/Routing[count($activeNode/preceding-sibling::*)+1]/@ID,')', '')" Value="concat('/Routings/Routing','|',count(preceding-sibling::*)+1,'')">
                                        <Skill Label="'Skill'" Value="concat('/Routings/Routing/Skill','|',../count(preceding-sibling::*)+1,'')">
                                                <Skillgroup Label="for $activeNode in (.) return concat(instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::*)+1]/Skill/Skillgroup[count($activeNode/preceding-sibling::*)+1]/Bezeichner ,'')" Value="concat('/Routings/Routing/Skill/Skillgroup','|',../../count(preceding-sibling::*)+1,'|',count(preceding-sibling::*)+1,'')"/>
                                        </Skill>
                                        <Default_Ziel_Name Label="for $activeNode in (.) return concat(instance('main-instance')/Routings/Routing[count($activeNode/../preceding-sibling::*)+1]/Default_Ziel_Name/Bezeichner ,' (',instance('main-instance')/Routings/Routing[count($activeNode/../preceding-sibling::*)+1]/Default_Ziel_Name/@ID,')' ,'')" Value="concat('/Routings/Routing/Default_Ziel_Name','|',../count(preceding-sibling::*)+1,'')">
                                                <Defaultziel Label="for $activeNode in (.) return concat( instance('main-instance')/Defaultziele/Defaultziel[@ID=instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::Routing)+1]/Default_Ziel_Name/Defaultziel[count($activeNode/preceding-sibling::Defaultziel)+1]/@IDREF]/Identname ,' ', '')" Value="concat('/Routings/Routing/Default_Ziel_Name/Defaultziel','|',../../count(preceding-sibling::*)+1,'|',count(preceding-sibling::*)+1,'')"/>
                                        </Default_Ziel_Name>
                                        <Default_0Ag_Name Label="for $activeNode in (.) return concat(instance('main-instance')/Routings/Routing[count($activeNode/../preceding-sibling::*)+1]/Default_0Ag_Name/Bezeichner ,' (',instance('main-instance')/Routings/Routing[count($activeNode/../preceding-sibling::*)+1]/Default_0Ag_Name/@ID,')' ,'')" Value="concat('/Routings/Routing/Default_0Ag_Name','|',../count(preceding-sibling::*)+1,'')">
                                                <Defaultziel Label="for $activeNode in (.) return concat( instance('main-instance')/Defaultziele/Defaultziel[@ID=instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::Routing)+1]/Default_Ziel_Name/Defaultziel[count($activeNode/preceding-sibling::Defaultziel)+1]/@IDREF]/Identname ,' ', '')" Value="concat('/Routings/Routing/Default_Ziel_Name/Defaultziel','|',../../count(preceding-sibling::*)+1,'|',count(preceding-sibling::*)+1,'')"/>
                                        </Default_0Ag_Name>
                                        <Default_leer_Name Label="'Default_leer_Name'" Value="concat('/Routings/Routing/Default_leer_Name','|',../count(preceding-sibling::*)+1,'')">
                                                <Defaultziel Label="for $activeNode in (.) return concat( instance('main-instance')/Defaultziele/Defaultziel[@ID=instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::Routing)+1]/Default_Ziel_Name/Defaultziel[count($activeNode/preceding-sibling::Defaultziel)+1]/@IDREF]/Identname ,' ', '')" Value="concat('/Routings/Routing/Default_Ziel_Name/Defaultziel','|',../../count(preceding-sibling::*)+1,'|',count(preceding-sibling::*)+1,'')"/>
                                        </Default_leer_Name>
                                        <BegrAnsage_Name Label="'BegrAnsage_Name'" Value="concat('/Routings/Routing/BegrAnsage_Name','|',../count(preceding-sibling::*)+1,'')"/>
                                        <Dialog_Name Label="'Dialog_Name'" Value="concat('/Routings/Routing/Dialog_Name','|',../count(preceding-sibling::*)+1,'')"/>
                                        <WF_Name Label="'WF_Name'" Value="concat('/Routings/Routing/WF_Name','|',../count(preceding-sibling::*)+1,'')">
                                                <Wartefeld Label="for $activeNode in (.) return concat(instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::*)+1]/WF_Name/Wartefeld[count($activeNode/preceding-sibling::*)+1]/Bezeichner ,'')" Value="concat('/Routings/Routing/WF_Name/Wartefeld','|',../../count(preceding-sibling::*)+1,'|',count(preceding-sibling::*)+1,'')"/>
                                        </WF_Name>
                                </Routing>
                        </Routings>
                </WebACD>
          </xforms:instance>
     
          <xforms:instance id="tree-select">
        <instance xmlns="">
          <selected></selected>
        </instance>
      </xforms:instance>
     
    </xforms:model>
  </head>
  <body>
    <div class="main">
                   <div class="content">
                                                                                                                                                                                               
                                                  <xforms:select1 incremental="true" id="tree-id" appearance="xxforms:tree" ref="instance('tree-select')/selected">
                                                        <xforms:itemset nodeset="instance('tree-instance')/descendant::*">
                                                          <xforms:label value="xxforms:evaluate(@Label)" />
                                                          <xforms:value value="xxforms:evaluate(@Value)" />
                                                        </xforms:itemset>
                                                  </xforms:select1>
                                                  <xforms:input id="tree-select-input-id" ref="instance('tree-select')/selected" />
                                               
                        </div>
          </div>
        <widget:xforms-instance-inspector id="orbeon-xforms-inspector" xmlns:widget="http://orbeon.org/oxf/xml/widget"/>
  </body>
</html>


As you can see, I have two instances (main-instance, and tree-instance). The tree instance is used for the nodeset of the select1 with a tree-appearance. The tree shows the structure of the main-instance and the labels of each tree node can be very complex. E.g. it may consist of several element-values of the main-instance.
Everything is working qutie nice as long as I dont have more than 50-80 elements (e.g. here Routing-elements).
With more elements, every time I click on a node in the tree a Ajax-request is sent to the orbeon-backend (value-changed-event) and stays there up to 30 sec ( with 130 elements in the tree and main-instance).

I recognized also the following:

- some xpath-expressions in the label-attribute of the tree-nodes are more complex than others:
a complex one:

"for $activeNode in (.) return concat( instance('main-instance')/Defaultziele/Defaultziel[@ID=instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::Routing)+1]/Default_Ziel_Name/Defaultziel[count($activeNode/preceding-sibling::Defaultziel)+1]/@IDREF]/Identname ,' ', '')"

A not so complex one:
"for $activeNode in (.) return concat(instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::*)+1]/WF_Name/Wartefeld[count($activeNode/preceding-sibling::*)+1]/Bezeichner ,'')"

When i remove all the complex expressions with the less complex ones, the time which is spent in the orbeon backend with every things else unchanged (same amount of data ...) drops to about 10 seconds. Also a long time, but better.

- The time spent in the orbeon backend rises non linear but exponential. With 280 elements it takes already 120 seconds.

- It seems that orbeon recalculates every label-node-expression each time I click a single node. This is not necessary, as everything which changes in the model, when i click a node, is the instance with the id="tree-select" the select1 is bound to. As no label-xpath-expression uses this value there no reason to reevaluate...

So my question is, if my assumptions are correct and - perhaps - if somebody has an hint or idea to improve the speed. I also read and tried the tuning-tips in the orbeon docu without success.

Thanks for your help

Cheers Michael
Reply | Threaded
Open this post in threaded view
|

Re: performance issue with xpath expressions

Adrian Baker-3
I'm only guessing, but using the evaluate function might be less efficient than placing the label expressions directly on the xforms:label,xforms:value elements.

I was going to suggest that this would let you use a xxforms:variable inside the itemset, to help simplify the expressions: unfortunately xxforms:variable isn't supported inside an itemset (yet?).

Another alternative to evaluate would be to shift the expressions to binds, calculate the label/value and store it in your tree-instance.

But if xxforms:evaluate isn't the reason for slowness neither of these will help.

It looks like some of the complexity of your expressions is working out the relative positions of nodes using count(... preceding-sibling:: ...) . I assume you have some code to insert/delete nodes into tree-instance as the main-instance changes? If so you could calculate the position at the insertion/deletion time and store this in tree-instance, then you wouldn't need to derive it later. If the structure main-instance doesn't change, then you can just do this once on load perhaps with an XSLT.

Adrian

Myxlyxl wrote
Hi there,

I have the following xforms:

<?xml version="1.0" encoding="UTF-8"?>
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:ev="http://www.w3.org/2001/xml-events" xmlns:xxforms="http://orbeon.org/oxf/xml/xforms" xmlns:xforms="http://www.w3.org/2002/xforms" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <head>
    <xforms:model id="mainModel">
      <xforms:instance id="main-instance">
                <WebACD xmlns="">
                        <Routings>
                                <Routing Event="1" ID="SGR000000107">
                                        <Skill>
                                                <Skillgroup IDREF="SG0000000000"/>
                                        </Skill>
                                        <Default_Ziel_Name ID="DZR000000107">
                                                <Defaultziel IDREF="DEF000000101"/>
                                        </Default_Ziel_Name>
                                        <Default_0Ag_Name ID="D0R000000107">
                                                <Defaultziel IDREF="DEF000000014"/>
                                        </Default_0Ag_Name>
                                        <Default_leer_Name>
                                                <Defaultziel IDREF="DEF000000013"/>
                                        </Default_leer_Name>
                                        <BegrAnsage_Name/>
                                        <Dialog_Name/>
                                        <WF_Name>
                                                <Wartefeld IDREF="WF0000000101"/>
                                        </WF_Name>
                                        <Kommentar/>
                                </Routing>
                        </Routings>
                </WebACD>
          </xforms:instance>
     
          <xforms:instance id="tree-instance">
                <WebACD>
                        <Routings Label="'Routings'" Value="concat('/Routings','')">
                                <Routing Label="for $activeNode in (.) return concat( instance('main-instance')/Routings/Routing[count($activeNode/preceding-sibling::*)+1]/Kommentar,' ', ' (',instance('main-instance')/Routings/Routing[count($activeNode/preceding-sibling::*)+1]/@ID,')', '')" Value="concat('/Routings/Routing','|',count(preceding-sibling::*)+1,'')">
                                        <Skill Label="'Skill'" Value="concat('/Routings/Routing/Skill','|',../count(preceding-sibling::*)+1,'')">
                                                <Skillgroup Label="for $activeNode in (.) return concat(instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::*)+1]/Skill/Skillgroup[count($activeNode/preceding-sibling::*)+1]/Bezeichner ,'')" Value="concat('/Routings/Routing/Skill/Skillgroup','|',../../count(preceding-sibling::*)+1,'|',count(preceding-sibling::*)+1,'')"/>
                                        </Skill>
                                        <Default_Ziel_Name Label="for $activeNode in (.) return concat(instance('main-instance')/Routings/Routing[count($activeNode/../preceding-sibling::*)+1]/Default_Ziel_Name/Bezeichner ,' (',instance('main-instance')/Routings/Routing[count($activeNode/../preceding-sibling::*)+1]/Default_Ziel_Name/@ID,')' ,'')" Value="concat('/Routings/Routing/Default_Ziel_Name','|',../count(preceding-sibling::*)+1,'')">
                                                <Defaultziel Label="for $activeNode in (.) return concat( instance('main-instance')/Defaultziele/Defaultziel[@ID=instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::Routing)+1]/Default_Ziel_Name/Defaultziel[count($activeNode/preceding-sibling::Defaultziel)+1]/@IDREF]/Identname ,' ', '')" Value="concat('/Routings/Routing/Default_Ziel_Name/Defaultziel','|',../../count(preceding-sibling::*)+1,'|',count(preceding-sibling::*)+1,'')"/>
                                        </Default_Ziel_Name>
                                        <Default_0Ag_Name Label="for $activeNode in (.) return concat(instance('main-instance')/Routings/Routing[count($activeNode/../preceding-sibling::*)+1]/Default_0Ag_Name/Bezeichner ,' (',instance('main-instance')/Routings/Routing[count($activeNode/../preceding-sibling::*)+1]/Default_0Ag_Name/@ID,')' ,'')" Value="concat('/Routings/Routing/Default_0Ag_Name','|',../count(preceding-sibling::*)+1,'')">
                                                <Defaultziel Label="for $activeNode in (.) return concat( instance('main-instance')/Defaultziele/Defaultziel[@ID=instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::Routing)+1]/Default_Ziel_Name/Defaultziel[count($activeNode/preceding-sibling::Defaultziel)+1]/@IDREF]/Identname ,' ', '')" Value="concat('/Routings/Routing/Default_Ziel_Name/Defaultziel','|',../../count(preceding-sibling::*)+1,'|',count(preceding-sibling::*)+1,'')"/>
                                        </Default_0Ag_Name>
                                        <Default_leer_Name Label="'Default_leer_Name'" Value="concat('/Routings/Routing/Default_leer_Name','|',../count(preceding-sibling::*)+1,'')">
                                                <Defaultziel Label="for $activeNode in (.) return concat( instance('main-instance')/Defaultziele/Defaultziel[@ID=instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::Routing)+1]/Default_Ziel_Name/Defaultziel[count($activeNode/preceding-sibling::Defaultziel)+1]/@IDREF]/Identname ,' ', '')" Value="concat('/Routings/Routing/Default_Ziel_Name/Defaultziel','|',../../count(preceding-sibling::*)+1,'|',count(preceding-sibling::*)+1,'')"/>
                                        </Default_leer_Name>
                                        <BegrAnsage_Name Label="'BegrAnsage_Name'" Value="concat('/Routings/Routing/BegrAnsage_Name','|',../count(preceding-sibling::*)+1,'')"/>
                                        <Dialog_Name Label="'Dialog_Name'" Value="concat('/Routings/Routing/Dialog_Name','|',../count(preceding-sibling::*)+1,'')"/>
                                        <WF_Name Label="'WF_Name'" Value="concat('/Routings/Routing/WF_Name','|',../count(preceding-sibling::*)+1,'')">
                                                <Wartefeld Label="for $activeNode in (.) return concat(instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::*)+1]/WF_Name/Wartefeld[count($activeNode/preceding-sibling::*)+1]/Bezeichner ,'')" Value="concat('/Routings/Routing/WF_Name/Wartefeld','|',../../count(preceding-sibling::*)+1,'|',count(preceding-sibling::*)+1,'')"/>
                                        </WF_Name>
                                </Routing>
                        </Routings>
                </WebACD>
          </xforms:instance>
     
          <xforms:instance id="tree-select">
        <instance xmlns="">
          <selected></selected>
        </instance>
      </xforms:instance>
     
    </xforms:model>
  </head>
  <body>
    <div class="main">
                   <div class="content">
                                                                                                                                                                                               
                                                  <xforms:select1 incremental="true" id="tree-id" appearance="xxforms:tree" ref="instance('tree-select')/selected">
                                                        <xforms:itemset nodeset="instance('tree-instance')/descendant::*">
                                                          <xforms:label value="xxforms:evaluate(@Label)" />
                                                          <xforms:value value="xxforms:evaluate(@Value)" />
                                                        </xforms:itemset>
                                                  </xforms:select1>
                                                  <xforms:input id="tree-select-input-id" ref="instance('tree-select')/selected" />
                                               
                        </div>
          </div>
        <widget:xforms-instance-inspector id="orbeon-xforms-inspector" xmlns:widget="http://orbeon.org/oxf/xml/widget"/>
  </body>
</html>


As you can see, I have two instances (main-instance, and tree-instance). The tree instance is used for the nodeset of the select1 with a tree-appearance. The tree shows the structure of the main-instance and the labels of each tree node can be very complex. E.g. it may consist of several element-values of the main-instance.
Everything is working qutie nice as long as I dont have more than 50-80 elements (e.g. here Routing-elements).
With more elements, every time I click on a node in the tree a Ajax-request is sent to the orbeon-backend (value-changed-event) and stays there up to 30 sec ( with 130 elements in the tree and main-instance).

I recognized also the following:

- some xpath-expressions in the label-attribute of the tree-nodes are more complex than others:
a complex one:

"for $activeNode in (.) return concat( instance('main-instance')/Defaultziele/Defaultziel[@ID=instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::Routing)+1]/Default_Ziel_Name/Defaultziel[count($activeNode/preceding-sibling::Defaultziel)+1]/@IDREF]/Identname ,' ', '')"

A not so complex one:
"for $activeNode in (.) return concat(instance('main-instance')/Routings/Routing[count($activeNode/../../preceding-sibling::*)+1]/WF_Name/Wartefeld[count($activeNode/preceding-sibling::*)+1]/Bezeichner ,'')"

When i remove all the complex expressions with the less complex ones, the time which is spent in the orbeon backend with every things else unchanged (same amount of data ...) drops to about 10 seconds. Also a long time, but better.

- The time spent in the orbeon backend rises non linear but exponential. With 280 elements it takes already 120 seconds.

- It seems that orbeon recalculates every label-node-expression each time I click a single node. This is not necessary, as everything which changes in the model, when i click a node, is the instance with the id="tree-select" the select1 is bound to. As no label-xpath-expression uses this value there no reason to reevaluate...

So my question is, if my assumptions are correct and - perhaps - if somebody has an hint or idea to improve the speed. I also read and tried the tuning-tips in the orbeon docu without success.

Thanks for your help

Cheers Michael
Reply | Threaded
Open this post in threaded view
|

Re: performance issue with xpath expressions

Myxlyxl
Adrian Baker wrote
I'm only guessing, but using the evaluate function might be less efficient than placing the label expressions directly on the xforms:label,xforms:value elements.

I was going to suggest that this would let you use a xxforms:variable inside the itemset, to help simplify the expressions: unfortunately xxforms:variable isn't supported inside an itemset (yet?).

Another alternative to evaluate would be to shift the expressions to binds, calculate the label/value and store it in your tree-instance.

But if xxforms:evaluate isn't the reason for slowness neither of these will help.

It looks like some of the complexity of your expressions is working out the relative positions of nodes using count(... preceding-sibling:: ...) . I assume you have some code to insert/delete nodes into tree-instance as the main-instance changes? If so you could calculate the position at the insertion/deletion time and store this in tree-instance, then you wouldn't need to derive it later. If the structure main-instance doesn't change, then you can just do this once on load perhaps with an XSLT.

Adrian
Hi Adrian,

thanks for your tips.

I improved the speed a bit by putting the values I want to show in the label also in the tree-instance. Of course this means redundance (doesnt matter for me) and more work, as I have to maintain now also the tree-instance, if a value changed. So my new xpath-expressions look something like this:

Label="for $activeNode in (.) return concat( instance('main-instance')/Defaultziele/Defaultziel[@ID=$activeNode/@IDREF]/Identname)"

Now clicking a node takes 3-4 seconds... inserting new elements (I have to insert in main-instance and tree-instance) still takes up to 20 seconds)... and changing a value from e.g. an input-field which is bound to the main-model and which is further used in the label-node, takes also 6 seconds which is really annoying. However, it is better now :-).

Adrian Baker wrote
Another alternative to evaluate would be to shift the expressions to binds, calculate the label/value and store it in your tree-instance.
Could you explain this in more detail. Remember that each node-label could be completely different (the first may take element Bezeichner and @ID of the main-instance, the second some completely other elements).

The idea to precalculate the positions is interesting. But in this case I will have to update all following Position-values, e.g. if I delete one element in the middle of the list. But perhaps it will help.

Thanks for your tips.

Cheer Michael
Reply | Threaded
Open this post in threaded view
|

Re: performance issue with xpath expressions

Alessandro Vernet
Administrator
In reply to this post by Myxlyxl
Michael,

Myxlyxl wrote
Everything is working qutie nice as long as I dont have more than 50-80 elements (e.g. here Routing-elements).
With more elements, every time I click on a node in the tree a Ajax-request is sent to the orbeon-backend (value-changed-event) and stays there up to 30 sec ( with 130 elements in the tree and main-instance).
That seems awfully long. Does removing the XForms inspector help? The XForms you pasted here doesn't have that many element; could you attach an example that has that many elements so we can reproduce the issue here?

Alex
Reply | Threaded
Open this post in threaded view
|

Re: performance issue with xpath expressions

Myxlyxl
Hi Alessandro,

Alessandro Vernet wrote
That seems awfully long. Does removing the XForms inspector help? The XForms you pasted here doesn't have that many element; could you attach an example that has that many elements so we can reproduce the issue here?

Alex
of course I tested without the Xforms inspector.

The problem is, that the main-instance is quite big (a file with 5000 rows), so I could not paste it here. I hope I find time to attach an example which will show the performance issue this week.

By then
Michael
Reply | Threaded
Open this post in threaded view
|

Re: performance issue with xpath expressions

Alessandro Vernet
Administrator
Michael,

Myxlyxl wrote
of course I tested without the Xforms inspector.
I had to check :).

Myxlyxl wrote
The problem is, that the main-instance is quite big (a file with 5000 rows), so I could not paste it here. I hope I find time to attach an example which will show the performance issue this week.
If you have some time, you can also run your code under a profiler (we like YourKit) and see what in the Java code is taking that much time in this case.

Alex