mysql persistence layer performance issues

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

mysql persistence layer performance issues

khh7112

We are using orbeon forms with mysql persistence layer.  But now that we have about 3000 forms, we are seeing performance issues with the parsing of the XML to populate the summary screens.  Has anyone else seen this issue, and is there a work-around or tuning that can be done?


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: mysql persistence layer performance issues

koenvdk
Hi,

We've had the same problem and already posted some info on the queries that are causing it on this mailing list. We did have the problem a lot sooner, though, around 500 records.
Right now, we're working on our own implementation of the persistence layer in Java, speaking directly to the db.


Bye,



Koen Vanderkimpen

Connect: Twitter





From:        "Dr. Kevin Hunt" <[hidden email]>
To:        [hidden email]
Date:        06/07/2012 16:52
Subject:        [ops-users] mysql persistence layer performance issues






We are using orbeon forms with mysql persistence layer.  But now that we have about 3000 forms, we are seeing performance issues with the parsing of the XML to populate the summary screens.  Has anyone else seen this issue, and is there a work-around or tuning that can be done?

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe:
[hidden email]
For general help:
[hidden email]
OW2 mailing lists service home page:
http://www.ow2.org/wws








Overeenkomstig de bepalingen inzake de vertegenwoordiging van de vzw in haar statuten, kan enkel de gedelegeerde bestuurder, de algemeen directeur of zijn uitdrukkelijke lasthebber verbintenissen aangaan namens Smals.
Indien dit bericht niet voor u bestemd is, verzoeken wij u dit onmiddellijk aan ons te melden en het bericht te vernietigen.

Conformément aux dispositions relatives à la représentation de l'asbl dans ses statuts, seul l'administrateur délégué, le directeur général ou son mandataire exprès est habilité à souscrire des engagements au nom de Smals.
Si ce message ne vous est pas destiné, nous vous prions de nous le signaler immédiatement et de détruire le message.

According to the provisions regarding representation of the non profit association in its bylaws, only the chief executive officer, the general manager or his explicit agent can enter into engagements on behalf of Smals.
If you are not the addressee of this message, we kindly ask you to signal this to us immediately and to delete the message.





--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: mysql persistence layer performance issues

khh7112
we solved the problem by reformatting the way the selects were constructed.  Plus one index.

Working very quickly again.

On Mon, Jul 9, 2012 at 3:24 AM, <[hidden email]> wrote:
Hi,

We've had the same problem and already posted some info on the queries that are causing it on this mailing list. We did have the problem a lot sooner, though, around 500 records.
Right now, we're working on our own implementation of the persistence layer in Java, speaking directly to the db.


Bye,



Koen Vanderkimpen

Connect: Twitter





From:        "Dr. Kevin Hunt" <[hidden email]>
To:        [hidden email]
Date:        06/07/2012 16:52
Subject:        [ops-users] mysql persistence layer performance issues






We are using orbeon forms with mysql persistence layer.  But now that we have about 3000 forms, we are seeing performance issues with the parsing of the XML to populate the summary screens.  Has anyone else seen this issue, and is there a work-around or tuning that can be done?

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe:
[hidden email]
For general help:
[hidden email]
OW2 mailing lists service home page:
http://www.ow2.org/wws








Overeenkomstig de bepalingen inzake de vertegenwoordiging van de vzw in haar statuten, kan enkel de gedelegeerde bestuurder, de algemeen directeur of zijn uitdrukkelijke lasthebber verbintenissen aangaan namens Smals.
Indien dit bericht niet voor u bestemd is, verzoeken wij u dit onmiddellijk aan ons te melden en het bericht te vernietigen.

Conformément aux dispositions relatives à la représentation de l'asbl dans ses statuts, seul l'administrateur délégué, le directeur général ou son mandataire exprès est habilité à souscrire des engagements au nom de Smals.
Si ce message ne vous est pas destiné, nous vous prions de nous le signaler immédiatement et de détruire le message.

According to the provisions regarding representation of the non profit association in its bylaws, only the chief executive officer, the general manager or his explicit agent can enter into engagements on behalf of Smals.
If you are not the addressee of this message, we kindly ask you to signal this to us immediately and to delete the message.






--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: mysql persistence layer performance issues

Erik Bruchez
Administrator
Kevin,

Any way you can share the changes?

Thanks,

-Erik

On Wed, Jul 18, 2012 at 1:12 PM, Dr. Kevin Hunt <[hidden email]> wrote:
we solved the problem by reformatting the way the selects were constructed.  Plus one index.

Working very quickly again.


On Mon, Jul 9, 2012 at 3:24 AM, <[hidden email]> wrote:
Hi,

We've had the same problem and already posted some info on the queries that are causing it on this mailing list. We did have the problem a lot sooner, though, around 500 records.
Right now, we're working on our own implementation of the persistence layer in Java, speaking directly to the db.


Bye,



Koen Vanderkimpen

Connect: Twitter





From:        "Dr. Kevin Hunt" <[hidden email]>
To:        [hidden email]
Date:        06/07/2012 16:52
Subject:        [ops-users] mysql persistence layer performance issues






We are using orbeon forms with mysql persistence layer.  But now that we have about 3000 forms, we are seeing performance issues with the parsing of the XML to populate the summary screens.  Has anyone else seen this issue, and is there a work-around or tuning that can be done?

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe:
[hidden email]
For general help:
[hidden email]
OW2 mailing lists service home page:
http://www.ow2.org/wws








Overeenkomstig de bepalingen inzake de vertegenwoordiging van de vzw in haar statuten, kan enkel de gedelegeerde bestuurder, de algemeen directeur of zijn uitdrukkelijke lasthebber verbintenissen aangaan namens Smals.
Indien dit bericht niet voor u bestemd is, verzoeken wij u dit onmiddellijk aan ons te melden en het bericht te vernietigen.

Conformément aux dispositions relatives à la représentation de l'asbl dans ses statuts, seul l'administrateur délégué, le directeur général ou son mandataire exprès est habilité à souscrire des engagements au nom de Smals.
Si ce message ne vous est pas destiné, nous vous prions de nous le signaler immédiatement et de détruire le message.

According to the provisions regarding representation of the non profit association in its bylaws, only the chief executive officer, the general manager or his explicit agent can enter into engagements on behalf of Smals.
If you are not the addressee of this message, we kindly ask you to signal this to us immediately and to delete the message.






--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Re: mysql persistence layer performance issues

khh7112
Yep, we are working on documenting it now.

On Thu, Jul 19, 2012 at 1:54 AM, Erik Bruchez <[hidden email]> wrote:
Kevin,

Any way you can share the changes?

Thanks,

-Erik


On Wed, Jul 18, 2012 at 1:12 PM, Dr. Kevin Hunt <[hidden email]> wrote:
we solved the problem by reformatting the way the selects were constructed.  Plus one index.

Working very quickly again.


On Mon, Jul 9, 2012 at 3:24 AM, <[hidden email]> wrote:
Hi,

We've had the same problem and already posted some info on the queries that are causing it on this mailing list. We did have the problem a lot sooner, though, around 500 records.
Right now, we're working on our own implementation of the persistence layer in Java, speaking directly to the db.


Bye,



Koen Vanderkimpen

Connect: Twitter





From:        "Dr. Kevin Hunt" <[hidden email]>
To:        [hidden email]
Date:        06/07/2012 16:52
Subject:        [ops-users] mysql persistence layer performance issues






We are using orbeon forms with mysql persistence layer.  But now that we have about 3000 forms, we are seeing performance issues with the parsing of the XML to populate the summary screens.  Has anyone else seen this issue, and is there a work-around or tuning that can be done?

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe:
[hidden email]
For general help:
[hidden email]
OW2 mailing lists service home page:
http://www.ow2.org/wws








Overeenkomstig de bepalingen inzake de vertegenwoordiging van de vzw in haar statuten, kan enkel de gedelegeerde bestuurder, de algemeen directeur of zijn uitdrukkelijke lasthebber verbintenissen aangaan namens Smals.
Indien dit bericht niet voor u bestemd is, verzoeken wij u dit onmiddellijk aan ons te melden en het bericht te vernietigen.

Conformément aux dispositions relatives à la représentation de l'asbl dans ses statuts, seul l'administrateur délégué, le directeur général ou son mandataire exprès est habilité à souscrire des engagements au nom de Smals.
Si ce message ne vous est pas destiné, nous vous prions de nous le signaler immédiatement et de détruire le message.

According to the provisions regarding representation of the non profit association in its bylaws, only the chief executive officer, the general manager or his explicit agent can enter into engagements on behalf of Smals.
If you are not the addressee of this message, we kindly ask you to signal this to us immediately and to delete the message.






--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Re: Re: Re: Re: mysql persistence layer performance issues

Erik Bruchez
Administrator
Great! -Erik

On Thu, Jul 19, 2012 at 4:23 AM, Dr. Kevin Hunt <[hidden email]> wrote:
Yep, we are working on documenting it now.


On Thu, Jul 19, 2012 at 1:54 AM, Erik Bruchez <[hidden email]> wrote:
Kevin,

Any way you can share the changes?

Thanks,

-Erik


On Wed, Jul 18, 2012 at 1:12 PM, Dr. Kevin Hunt <[hidden email]> wrote:
we solved the problem by reformatting the way the selects were constructed.  Plus one index.

Working very quickly again.


On Mon, Jul 9, 2012 at 3:24 AM, <[hidden email]> wrote:
Hi,

We've had the same problem and already posted some info on the queries that are causing it on this mailing list. We did have the problem a lot sooner, though, around 500 records.
Right now, we're working on our own implementation of the persistence layer in Java, speaking directly to the db.


Bye,



Koen Vanderkimpen

Connect: Twitter





From:        "Dr. Kevin Hunt" <[hidden email]>
To:        [hidden email]
Date:        06/07/2012 16:52
Subject:        [ops-users] mysql persistence layer performance issues






We are using orbeon forms with mysql persistence layer.  But now that we have about 3000 forms, we are seeing performance issues with the parsing of the XML to populate the summary screens.  Has anyone else seen this issue, and is there a work-around or tuning that can be done?

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe:
[hidden email]
For general help:
[hidden email]
OW2 mailing lists service home page:
http://www.ow2.org/wws








Overeenkomstig de bepalingen inzake de vertegenwoordiging van de vzw in haar statuten, kan enkel de gedelegeerde bestuurder, de algemeen directeur of zijn uitdrukkelijke lasthebber verbintenissen aangaan namens Smals.
Indien dit bericht niet voor u bestemd is, verzoeken wij u dit onmiddellijk aan ons te melden en het bericht te vernietigen.

Conformément aux dispositions relatives à la représentation de l'asbl dans ses statuts, seul l'administrateur délégué, le directeur général ou son mandataire exprès est habilité à souscrire des engagements au nom de Smals.
Si ce message ne vous est pas destiné, nous vous prions de nous le signaler immédiatement et de détruire le message.

According to the provisions regarding representation of the non profit association in its bylaws, only the chief executive officer, the general manager or his explicit agent can enter into engagements on behalf of Smals.
If you are not the addressee of this message, we kindly ask you to signal this to us immediately and to delete the message.






--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws
Reply | Threaded
Open this post in threaded view
|

RE: Re: Re: Re: mysql persistence layer performance issues

GA Jones
In reply to this post by khh7112

We created an index:

create index orbeon_form_data_ix1 on orbeon_form_data ( form, app, document_id );

 

And patched the search.xpl

 

--- search.xpl.orig     2012-07-13 09:02:14.000000000 -0400                                                                                                                         

+++ search.xpl  2012-07-12 14:47:31.000000000 -0400                                                                                                                                 

@@ -143,19 +143,21 @@                                                                                                                                                              

                                     <sql:query>                                                                                                                                    

                                         select                                                                                                                                     

                                             (                                                                                                                                     

-                                                select count(*) from orbeon_form_data                                                                                             

-                                                where                                                                                                                              

-                                                    (app, form, document_id, last_modified) in (                                                                                  

+                                                select count(*) from orbeon_form_data data, (                                                                                     

                                                         select app, form, document_id, max(last_modified) last_modified                                                            

                                                         from orbeon_form_data                                                                                                     

                                                         where                                                                                                                     

                                                             app = <sql:param type="xs:string" select="/search/app"/>                                                               

                                                             and form = <sql:param type="xs:string" select="/search/form"/>                                                        

-                                                        group by app, form, document_id)                                                                                          

+                                                        group by app, form, document_id) latest                                                                                   

+                                                    where data.last_modified = latest.last_modified                                                                                

+                                                    and   data.app           = latest.app                                                                                         

+                                                    and   data.form          = latest.form                                                                                        

+                                                    and   data.document_id   = latest.document_id                                                                                  

                                                     and deleted = 'N'                                                                                                             

                                             ) total,                                                                                                                              

                                             (                                                                                                                                      

-                                                select count(*) from (<xsl:copy-of select="$query"/>)a                                                                             

+                                                select count(*) from (<xsl:copy-of select="$query"/>) a                                                                           

                                             ) search_total                                                                                                                         

                                     </sql:query>

                                     <sql:result-set>

 

Thanks,
Greg A. Jones

 

From: Dr. Kevin Hunt [mailto:[hidden email]]
Sent: Thursday, July 19, 2012 7:24 AM
To: [hidden email]; GA Jones
Subject: Re: [ops-users] Re: Re: Re: mysql persistence layer performance issues

 

Yep, we are working on documenting it now.

On Thu, Jul 19, 2012 at 1:54 AM, Erik Bruchez <[hidden email]> wrote:

Kevin,

 

Any way you can share the changes?

 

Thanks,

 

-Erik

 

On Wed, Jul 18, 2012 at 1:12 PM, Dr. Kevin Hunt <[hidden email]> wrote:

we solved the problem by reformatting the way the selects were constructed.  Plus one index.

Working very quickly again.

 

On Mon, Jul 9, 2012 at 3:24 AM, <[hidden email]> wrote:

Hi,

We've had the same problem and already posted some info on the queries that are causing it on this mailing list. We did have the problem a lot sooner, though, around 500 records.
Right now, we're working on our own implementation of the persistence layer in Java, speaking directly to the db.


Bye,



Koen Vanderkimpen

Connect:





From:        "Dr. Kevin Hunt" <dr.kevin.hunt@...>
To:        ops-users@...
Date:        06/07/2012 16:52
Subject:        [ops-users] mysql persistence layer performance issues


--
You receive this message as a subscriber of the ops-users@... mailing list.
To unsubscribe:
mailto:ops-users-unsubscribe@...
For general help:
mailto:sympa@...?subject=help
OW2 mailing lists service home page:
http://www.ow2.org/wws



 



Overeenkomstig de bepalingen inzake de vertegenwoordiging van de vzw in haar statuten, kan enkel de gedelegeerde bestuurder, de algemeen directeur of zijn uitdrukkelijke lasthebber verbintenissen aangaan namens Smals.
Indien dit bericht niet voor u bestemd is, verzoeken wij u dit onmiddellijk aan ons te melden en het bericht te vernietigen.

Conformément aux dispositions relatives à la représentation de l'asbl dans ses statuts, seul l'administrateur délégué, le directeur général ou son mandataire exprès est habilité à souscrire des engagements au nom de Smals.
Si ce message ne vous est pas destiné, nous vous prions de nous le signaler immédiatement et de détruire le message.

According to the provisions regarding representation of the non profit association in its bylaws, only the chief executive officer, the general manager or his explicit agent can enter into engagements on behalf of Smals.
If you are not the addressee of this message, we kindly ask you to signal this to us immediately and to delete the message.

 

 



--
You receive this message as a subscriber of the ops-users@... mailing list.
To unsubscribe: mailto:ops-users-unsubscribe@...
For general help: mailto:sympa@...?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws

 



--
You receive this message as a subscriber of the ops-users@... mailing list.
To unsubscribe: mailto:ops-users-unsubscribe@...
For general help: mailto:sympa@...?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws

 



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws