Forum OpenACS Development: Re: Usage of XoTCL objects vs. database queries

Collapse
7: Re: Usage of XoTCL objects vs. database queries (response to 5)
Posted by Andrew Piskorski on 08/31/08 08:14 PM
it is a TCL issue with not having a place to store variables and methods in a global, thread independent place

Malte, of course Tcl has such a place, it's called "nsv" (in AOLserver; in the Tcl Threads Extension the same thing is called "tsv").

Of course, nsv/tsv supports only base Tcl types, strings and in some cases more complicated values like lists (with Zoran's clever perforamance hacks to avoid type shimmering). Can all the XOTcl objects you care about be serialized to from nsv/tsv? Efficiently? Between the two of them, Gustaf and Zoran should be able to tell you for sure.

Another option is to dedicate one single thread (and its associated Tcl interpretor) as your "XOTcl db" thread. Then have other threads send it messages with requests to read/write data. Everything will be serialized through that single "db" thread of course. In theory that could be much less scalable than the hash bucketized multi-threaded design of nsv/tsv, but in practice it might be simpler and faster, especially if the XOTcl objects are large but the read/write requests are small.

Collapse
9: Re: Usage of XoTCL objects vs. database queries (response to 7)
Posted by Gustaf Neumann on 08/31/08 10:00 PM
Can all the XOTcl objects you care about be serialized to from nsv/tsv?
Sure, the serialization of an XOTcl object is a string, such strings can be saved in nsvs, one can use the ttrace mechanism, etc. By serializing and deserializing objects, one creates a copy of the object. So, changing one instance variable in one object does not affect the other. So, serializing to nsv is in this respect the same as ns_cache.

However, it is as well possible to store the instance variables of XOTcl objects in nsvs, so value changes of instance variables effect the values in other threads immediately. I have implemented such an interface a few years ago as an experiment, but so far, i have not seen valuable usage of this. In general, one needs locking and sometime rollbacks, clustering over multiple instances is expensive. In my experience the basic approach via ns_cache is much simpler and is quite fast.

Concerning the "single thread" idea: Yes, having one (or a few) thread(s) dedicated for "object pools", which provide for certain classes an interface to the db is certainly an option, especially together with thread::send and the libthread's event loop. A connection thread can query the object pool and get e.g. the serialized object back. This approach could work similar to ttrace, but provide as well a write interface. Note that one can have multiple such thread for e.g. different classes. Note however, that this does not address the additional need for locking and transactions. Direct database manipulations (inserts, writes, deletes) bypassing the object interface are certainly as well a problem (especially for OpenACS, where this is often the common practice).

... ns_set ...
I am not sure, how ns_set could help here.

However, I don't see any direct mention of this sort of stuff in the XOTcl docs nor its slots tutorial. Is there better info or examples somewhere on using XOTcl slots in this way?
The XOTcl docs do not address aolserver specific interfaces. Slots provide a way to provide accessor methods for instance variables, and more general, manage their state. Some basic introduction is in
http://alice.wu-wien.ac.at:8000/s5-xotcl-core-tutorial/slides (not going more into details than the xotcl docs), examples are certainly in the source code (xotcl-core/tcl/05-db-procs.tcl).