Forum OpenACS Q&A: Storing java Objects in PostgreSQL

Posted by Joshua Ginsberg on 08/14/01 06:31 AM

I'm looking at using the ns_javablend module to do some really cool
things in AOLServer... what would be absolutely fundamental to what
I'm wanting to do would be to maintain instances of java Objects from
page to page... I'm having a difficult time figuring out a way to do
this, since they aren't exactly passable as form data...

So this is what I was thinking... tell me if this won't work or if
there's a better way somebody can come up with...

I was thinking of creating the Object, serializing it, and then
sticking it in Postgres as a bytea field associated with a particular
session id... then I could pull the bytea out, deserialize it, and
have my Object back the way I left it...

Any comments or criticism?

Thanks!

-jag

2: Response to Storing java Objects in PostgreSQL (response to 1)

Posted by Krzysztof Kowalczyk on 08/14/01 07:17 AM

Assuming that you have an unique object id (which you have to have in the solution involving sticking data into a database), why not use a persistent (across connection) memory cache (say a hash table)?

You'll have the problem of freeing objects when they're not needed anymore but db soltuion has the same problem (the price of not deleting objects is lower, though, than leaking memory). But it'll be way faster (databases are actually very slow).

3: Response to Storing java Objects in PostgreSQL (response to 1)

Posted by Dan Wickstrom on 08/14/01 10:10 PM

The best way to do this would be to maintain a cache of java objects in a separate thread that persists across connections. That way you wouldn't have to the price for serializing/deserializing the objects that you stored/retrieved from the db. Creating a persistant object store wouldn't be too difficult, but to do this, you would probably have to hack on the nsjavablend c-code, as there currently is no support for such a thing.

The big problem would be the management of the instances once they are created. What happens when, for example, a user accesses a page that creates some cached instances, but then for some reason, they walk away from the computer and come back several hours, days, or weeks later. You can't keep all those instances around, because eventually you will run out of memory. In addition you're creating a hole for a denial of service attack. To solve that problem, you could have some code that manages the instances such that objects that have been around for x amount of time are serialized and placed into the db, and then if the user comes back later, you can recreate the instances and carry on. Still you'll have the problem of filling your db with serialized instances of expired objects. You could then modify your instance management code such that serialized objects are eventually purged from the db, but then you need to code your pages so that the page redirects to the start of the page-flow sequence if the instances have been purged from the system. Overall, you're going to end up with writing alot of extra code to manage the created java instances. I imagine that if you want automatic mangement of persistant java objects between page requests, you would be better off using a J2EE compliant application server, as it would already have all of the session management features built into the server.

4: Response to Storing java Objects in PostgreSQL (response to 1)

Posted by John Sequeira on 08/14/01 11:08 PM

Am I incorrect or are you describing Session Beans from the j2ee spec? I haven't used them, but that sounds like what they are.

I can't fault your choice of web server, but if you're doing Java anyway, maybe you want to be using Enhydra or jBoss.

5: Response to Storing java Objects in PostgreSQL (response to 1)

Posted by Joshua Ginsberg on 08/15/01 05:14 AM

Thank you for responding... sorry I've not been, but I've been out of town with a wonderful 16.8K Internet connection, so I've not been checking mail as frequently as I would normally...

Krzysztof and Dan - I would love to be able to have some alternate thread to store Java Objects which could interact with client threads... unfortunately, my C hacking skills are @!$#...

Dan - I'm familiar with the caching issues which you've mentioned... The particular application that I'm discussing right now is a WYSIWYG web-based HTML editor... I've got controls in place to make sure that the cache does not overflow with abandoned objects... this control includes sweeping the database periodically for orphaned sessions, making sure that there does not exist more than one cached Object for each user, and if a user comes back to the editor when there is already a cached object for that user, it is given back to the user under their new session identifier... I'm not too particularly worried about DoS attacks because of the cache control I just mentioned and because access to the editor requires deliberate authorization from the sysadmin...

Dan and John - I've considered moving to a servlet engine, but I'm not wanting to because of a couple of reasons... first, I love TclBlend... it has done incredible things for the applications I've written... second, I'm trying to find a reasonable way to extend the ideal of TclBlend (TCL scripting code gluing together large, reusable, application components written in Java) to AOLServer... Dan's ns_tclblend module is a foot in the door, but I'm looking for a way to really make this a viable choice other than loading a separate application server...

So, all of that having been said...

It's true... db's are a massive bottleneck, and the serialization/stick-in-db/pull-from-db/deserialization process from page to page would be a resource hog... so now I'm left with trying to find some sort of built-in mechanism within AOLServer to preserve values between threads... nsv buckets seem inappropriate... are there other options I'm not considering? Some sort of persistent set where I can make a user's sessionid the key and the Java Object the value? Would the Object be saved from garbage collection by such a method?

Thanks again for your feedback...

-jag

6: Response to Storing java Objects in PostgreSQL (response to 1)

Posted by Dan Wickstrom on 08/15/01 06:54 AM

You might try experimenting with the ns_thread command. You could start a detached thread that runs java code, via ns_javablend, that acts as an object store. That way you could pass java objects off to the detached thread for storage in a HashTable which is indexed by a generated id. The id could then be passed between pages using form variables, and the succeding pages in the page-flow sequence would be able to retreive the object from the object-store. At least in theory, I think this is doable. One thing to remember is that Tcl_Obj's (tcl variables are stored as pseudo multi-ported objects) cannot be passed between threads, so you can't create a tclobject in one thread by doing something like:

set x [java::new foo ...]

and then pass "x" to the detached thread. You could however pass the object from one thread to another from the java side, and retain a label to the object so that you can fetch it later. The label could then be passed around between the aolserver connection threads if necessary.

I've never tested ns_javablend with detached threads, so I'm not sure that it would work. In aolserver everything centers around connection threads, and there might be some quirks with making everything work with a detached thread. ns_javablend does work with scheduled procs, and I don't think there is much of a difference between the thread that does proc scheduling and a detached thread, so hopefully using ns_javablend from a detached thread will also work.