Forum OpenACS Q&A: Update on Jabber Client module

Since I rarely read much beyond the emails from this discussion group and the one for AOLserver, I didn't hear about Jabber until I picked up Linux Magazine in August. In the article, the author discusses building a survey 'bot'. Apparently any automated client is called a bot. This client will let you pick a survey and then answer the questions. Everything sounds good until you investigate further and find out that this relatively simple application is difficult to program, and hasn't even been finished!

I guess it shouldn't be a surprise, since it is written in perl and attempts to maintain a mini-database of where the survey taker is in the process.

My first module relied on two assumptions: presence and iq packets didn't need to be saved, since they are session like pieces of information, and message packets should be saved. So I stuffed the messages into the database inside the C module.

The problem with this approach didn't take long to surface. Flexibility was definitely lost. A single thread was used to respond to every packet. Stuffing messages into the database turned out to be a dead end in the world of async jabber communication.

It became apparent to me that I needed to expose every packet directly to the tcl API as quickly as possible and move on to receiving the next packet.

So here is what I did. A diagram might help. A schedule procedure makes a connection to a jabber server. This thread essentially runs a a loop that maintains an always up connection. This thread receives every packet for this connection, checks to see if it is zero length, which would indicate the jabber server died, and then calls TCL_EVAL, passing the text representation of the packet to a tcl procedure.

The tcl procedure, still being run inside the scheduled procedure thread, passes this data on to another thread, using Rob Mayoff's very useful threadpool module. At this point the scheduled procedure thread reenters the select loop waiting for the next packet.

One of the threadpool procs runs the command that was queued. This command is essentially the head procedure that will decide the ultimate fate of each packet.

So that handles data passing into the AOLserver tcl API. Each procedure that wants to pass data back out, again uses Rob Mayoff's threadpool module, but in an unexpected way. The maxthreads is set to one. This essentially places a ns_cond around the outgoing socket, and provides a queue for the socket as well.

Here is the diagram:

                        pool1.1
            sched:4  |----------> |   pool2.1
sockin      -------> |----------> | -----------> sockout
                     |----------> |

Pool1 will handle the bulk of the work.

Jabber is a stateful protocol, completely unlike http. However web applications have developed means to recover state with each connection. Most jabber clients have very limited means of preserving state between connections. By exposing a jabber connection to the AOLserver API, it should be significantly easier to develop useful applications using the jabber protocol, but there are still significant barriers.

One barrier is that jabber clients are relatively dumb. Most of the functionality from jabber is supposed to come from the use of xml. The main problem with this is that xml is not a programming language. Although clients are required to parse xml, what to do with it once parsed has to be decided in advance. This means that clients have to be developed per application. This is obviously a big problem.

Most user clients (Gabber, WinJab, etc), understand a few message subtags: subject, body, thread, x. What to do with x tags is unknown. The thread tag may turn out to be the most useful. The client is supposed to return the thread tag unchanged. It might be possible to use this feature (or the id attribute of the message tag) to move the user client through the application. It might be able to serve as a crude form of url. Combined with the body data, you can envision a rudimentary browsing application.