Does anyone know of an efficient way to read Apache log files/Orbeon Server log files as XML from an Orbeon pipeline?

>
> I would like to read server log files and create a view of the application
> usage derived from these within my Orbeon application.
>
> Firstly, I tried to process my Apache server logs into XML using XSLT2
> unparsed text function but this fails due to a currently documented issue
> using the unparsed-text() from Orbeon. Even if this gets fixed soon it is
> probably not the best way to do it as XSLT is not going to be the most
> efficient to write the logs into XML.
>
> There do appear to be lots of libraries for Perl and PHP applications to
> process log files for auditing usage and user analytics but I am not sure
> about the the best ways to integrate any of those yet. I did manage to
> integrate XML output from AWSTATS and this gets me some of the way but not
> all of it as in some cases I need to read the actual logs themselves rather
> than a higher level snapshot. I noticed something on using ACK for fast
> search in files at
> http://avernet.blogspot.com/2009/08/ack-powerful-command-line-search.html
> that sounded promising but I wasn't sure how that could get intregrated into
> Orbeon and for this purpose.
>
> Does anyone have any tips or pointers on what might work best?
>
>
> --
> View this message in context: http://n4.nabble.com/Does-anyone-know-of-an-efficient-way-to-read-Apache-log-files-Orbeon-Server-log-files-as-XML-from-an-tp1584330p1584330.html
> Sent from the Orbeon Forms (ops-users) mailing list archive at Nabble.com.
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws
>
>

>
> Thanks Alex.  I will run a cron job/scheduler as suggested, I will structure
> chunks with with XSLT unparsed- text and xsl:analyze-string for time being
> to get exactly what I need outside Orbeon then put into Exist and XQuery
> from XFORMs in Orbeon as suggested.
>
> Essentially what I am trying to do is use this process for auditing the
> application usage and the results need to be at least as structured as the
> path-info descriptions in the page-flow.xml and give access to the other
> standard information you get in server logs. It is important I get the final
> results as XML with hooks and handles on everything described so I can
> easily style results in the interface... Also, I need the ability to
> requests log results from a dropdown of whatever options I have as path-info
> described in the application page-flow.xml.
>
> The biggest challenges seems to be the large size of the log files and the
> challenge of finding a way of moving this analytics logic inside Orbeon and
> as close to the page-flow.xml as possible.
> --
> View this message in context: http://n4.nabble.com/Does-anyone-know-of-an-efficient-way-to-read-Apache-log-files-Orbeon-Server-log-files-as-XML-from-an-tp1584330p1587121.html
> Sent from the Orbeon Forms (ops-users) mailing list archive at Nabble.com.
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> OW2 mailing lists service home page: http://www.ow2.org/wws
>
>

> Gary,
>
> We don't use the command line interface much, and if you extend what
> it currently does and are willing to contribute those changes, we'll
> look into putting them into the codebase.
>
> Alex
>
> On Wed, Mar 10, 2010 at 1:25 AM, Gary C <[hidden email]> wrote:
>>
>> Thanks Alex.  I will run a cron job/scheduler as suggested, I will structure
>> chunks with with XSLT unparsed- text and xsl:analyze-string for time being
>> to get exactly what I need outside Orbeon then put into Exist and XQuery
>> from XFORMs in Orbeon as suggested.
>>
>> Essentially what I am trying to do is use this process for auditing the
>> application usage and the results need to be at least as structured as the
>> path-info descriptions in the page-flow.xml and give access to the other
>> standard information you get in server logs. It is important I get the final
>> results as XML with hooks and handles on everything described so I can
>> easily style results in the interface... Also, I need the ability to
>> requests log results from a dropdown of whatever options I have as path-info
>> described in the application page-flow.xml.
>>
>> The biggest challenges seems to be the large size of the log files and the
>> challenge of finding a way of moving this analytics logic inside Orbeon and
>> as close to the page-flow.xml as possible.
>> --
>> View this message in context: http://n4.nabble.com/Does-anyone-know-of-an-efficient-way-to-read-Apache-log-files-Orbeon-Server-log-files-as-XML-from-an-tp1584330p1587121.html
>> Sent from the Orbeon Forms (ops-users) mailing list archive at Nabble.com.
>>
>>
>> --
>> You receive this message as a subscriber of the [hidden email] mailing list.
>> To unsubscribe: mailto:[hidden email]
>> For general help: mailto:[hidden email]?subject=help
>> OW2 mailing lists service home page: http://www.ow2.org/wws
>>
>>
>
>
>
> --
> Orbeon Forms - Web forms, open-source, for the Enterprise -
> http://www.orbeon.com/
> My Twitter: http://twitter.com/avernet
>