scheduled job to purge old data

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

scheduled job to purge old data

ldeck
Hi there,

I know that OrbeonServletContextListener cleans up expired sessions' data in eXist…

but I'm wondering how to schedule a task that would purge data from the db which is older 28 days, for example?

Second, if the context listener had been turned off, how to purge expired session data still lying around?


code snippets welcome :)

cheers,
Lachlan

--
You received this message because you are subscribed to the Google Groups "Orbeon Forms" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: scheduled job to purge old data

Alessandro  Vernet
Administrator
Hi Lachlan,

You're talking about purging session data from eXist? Since Orbeon Forms 3.9, eXist isn't used anymore to store session data (instead Ehcache is). So are you on 3.8 or earlier? If you're on 3.9 or newer, you don't need to worry about expiring session data from eXist anymore.

Alex
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: scheduled job to purge old data

ldeck
The current production version is 3.7.0beta1, soon to be upgraded to 4.3.x.
I believe 3.7.1 was when there was some bug fixed regarding properly removing old sessions via the listener.

So when we were thinking through the upgrade, backup / restore procedure, was wondering how to purge any data (whether expired sessions or not) that was older than a month and making that a scheduled task (as it's possible for people to fill in foobar forms.

On 21/09/2013, at 9:29 AM, Alessandro Vernet <[hidden email]> wrote:

> Hi Lachlan,
>
> You're talking about purging session data from eXist? Since Orbeon Forms
> 3.9, eXist isn't used anymore to store session data (instead Ehcache is). So
> are you on 3.8 or earlier? If you're on 3.9 or newer, you don't need to
> worry about expiring session data from eXist anymore.
>
> Alex
>
> -----
> --
> Follow Orbeon on Twitter: @orbeon
> Follow me on Twitter: @avernet
> --
> View this message in context: http://discuss.orbeon.com/scheduled-job-to-purge-old-data-tp4657355p4657357.html
> Sent from the Orbeon Forms community mailing list mailing list archive at Nabble.com.
>
> --
> You received this message because you are subscribed to the Google Groups "Orbeon Forms" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> To post to this group, send email to [hidden email].

--
You received this message because you are subscribed to the Google Groups "Orbeon Forms" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: scheduled job to purge old data

Alessandro  Vernet
Administrator
Hi Lachlan,

For session data: try not to migrates this at all (if you do, the data just won't be used). For form data, created by forms you authored in Form Builder, you can go by the last modified date in eXist.

Alex
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: scheduled job to purge old data

ldeck
Hey Alex,

On 24/09/2013, at 4:15 AM, Alessandro Vernet <[hidden email]> wrote:

> Hi Lachlan,
>
> For session data: try not to migrates this at all (if you do, the data just
> won't be used). For form data, created by forms you authored in Form
> Builder, you can go by the last modified date in eXist.

that's the theory, of course.
Now how do I actually achieve this? :-)

Interestingly the backups that are now being scheduled are quite small. e.g., less than 1mb as compared to ~3GB on disk. So perhaps those stale sessions are being excluded already?

In the meantime, looks like the client is happy to just wipe the db and start fresh.

So that'd wipe all the session data for now,
but still leaves the question of how to achieve regular purging of old data?
i.e., if you've got any example code you can point me to that'd be great.

cheers,
Lachlan

--
You received this message because you are subscribed to the Google Groups "Orbeon Forms" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Reply | Threaded
Open this post in threaded view
|

Re: scheduled job to purge old data

Alessandro  Vernet
Administrator
Hi Lachlan,

I would do this in XQuery, recursively iterating over collections xmldb:get-child-collections(), then xmldb:get-child-resources() to get the documents in each collection, then xmldb:last-modified() to get the last modification date for the resource, and if it is older than X calling xmldb:remove() to delete it.

http://exist-db.org/exist/apps/fundocs/view.html?uri=http://exist-db.org/xquery/xmldb&location=java:org.exist.xquery.functions.xmldb.XMLDBModule

Alex
--
Follow Orbeon on Twitter: @orbeon
Follow me on Twitter: @avernet
Reply | Threaded
Open this post in threaded view
|

Re: scheduled job to purge old data

ldeck
cheers, Alex

On 27/09/2013, at 11:29 AM, Alessandro Vernet <[hidden email]> wrote:

> Hi Lachlan,
>
> I would do this in XQuery, recursively iterating over collections
> xmldb:get-child-collections(), then xmldb:get-child-resources() to get the
> documents in each collection, then xmldb:last-modified() to get the last
> modification date for the resource, and if it is older than X calling
> xmldb:remove() to delete it.
>
> http://exist-db.org/exist/apps/fundocs/view.html?uri=http://exist-db.org/xquery/xmldb&location=java:org.exist.xquery.functions.xmldb.XMLDBModule
>
> Alex
>
> -----
> --
> Follow Orbeon on Twitter: @orbeon
> Follow me on Twitter: @avernet
> --
> View this message in context: http://discuss.orbeon.com/scheduled-job-to-purge-old-data-tp4657355p4657374.html
> Sent from the Orbeon Forms community mailing list mailing list archive at Nabble.com.
>
> --
> You received this message because you are subscribed to the Google Groups "Orbeon Forms" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> To post to this group, send email to [hidden email].

--
You received this message because you are subscribed to the Google Groups "Orbeon Forms" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].