Posted by Jim Cheesman on Mar 22, 2011; 10:15am
URL: https://discuss.orbeon.com/Architecture-question-tp3394205p3395965.html

Erik,

Thanks for the quick reply - I have to say the speed you reply to our questions on this list is most impressive!

I'll check out using REST and Hibernate, as well as the more obvious use of indexes etc.

Cheers,
Jim


On lun, 2011-03-21 at 19:25 -0700, Erik Bruchez wrote:

Jim,

Things like full-text search or search on specific fields can be made fast with a full-text index and proper indexes on the stored data. This applies to Oracle, and probably to MySQL as well.

For more complex stuff, with Oracle, we are going to implement soon support for materialized views, which will make the data quickly available in relational form for reporting and searches. But that will be Oracle-only. However maybe that can serve as a basis for creating tables on the fly with MySQL.

Also, Form Runner reads, writes and searches data through a simple REST API. If you provide your own persistence layer implementation behind this API, you can do anything you want, including using Hibernate. Whether Hibernate is the best (most performant) solution, I have to say that we don't know.

Also, depending on what "large" is, eXist might still be a good option. Here as well database tuning, i.e. indexes, is the key to to performance.

-Erik

On Mon, Mar 21, 2011 at 10:24 AM, Jim Cheesman <[hidden email]> wrote:

Hi,

All this talk of Fedora, eXist and EADitor is directly relevant to one of the more pressing issues I have with Orbeon.

The context: we're building a portal that will allow dynamic form creation which will then be linked to users, allowing the portal administrators to dynamically define what personal information is to be gathered about the users, as well as creating user surveys etc. etc. My main worry is that the number of users could be high, and the user data needs to be searchable - both individual searches and aggregate records . While both MySQL and Oracle offer XML parsing funcionality,  I'm not entirely sure what the performance is like, particuarly when simulating table joins using XPath.

So the question: What's the best way to set up large datasets with Orbeon? Is there any way to dynamically link Orbeon form definitions via Hibernate (or similar) to database tables, creating these tables on the fly? (I'll ignore the obvious security problems this could present for the moment...) If I were able to define the data to be gathered beforehand I could probably set up some kind of BI ELT process, but in this case that's not (apparently) an option.

Cheers,
Jim

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws

plain text document attachment (message-footer.txt)

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: [hidden email]
For general help: [hidden email]
OW2 mailing lists service home page: http://www.ow2.org/wws

--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
OW2 mailing lists service home page: http://www.ow2.org/wws