Forum OpenACS Q&A: Problem Installing The Jabber Package

Hi

I've downloaded the Jabber package from the contribs section to evaluate it but I'm having a problem compiling the nsjabber module. When I run 'make' I hit a problem with the first compile which seems to be related to the jabber.h include. The output is about 14k but starts like this:

gcc -I../include/xode  -I../../include -I../include/jabber -I../include -I../../nsd  -I../../include -D_REENTRANT=1 -DNDEBUG=1 -g -fPIC -Wall -Wno-unused -mcpu=i586 -DHAVE_CMMSG=1 -DUSE_FIONREAD=1 -DHAVE_COND_EINTR=1  -c -o jpacket.o jpacket.c
In file included from jpacket.c:20:
../include/jabber/jabber.h:59: parse error before "pool"
../include/jabber/jabber.h:59: warning: no semicolon at end of struct or union
../include/jabber/jabber.h:65: parse error before '}' token
../include/jabber/jabber.h:65: warning: type defaults to `int' in declaration of `jid'
../include/jabber/jabber.h:65: warning: data definition has no type or storage class
../include/jabber/jabber.h:67: parse error before "jid_new"
../include/jabber/jabber.h:67: parse error before "p"
../include/jabber/jabber.h:67: warning: type defaults to `int' in declaration of `jid_new'
../include/jabber/jabber.h:67: warning: data definition has no type or storage class
../include/jabber/jabber.h:68: parse error before "id"
../include/jabber/jabber.h:69: parse error before "id"
../include/jabber/jabber.h:70: parse error before "a"
../include/jabber/jabber.h:71: parse error before "a"
../include/jabber/jabber.h:72: parse error before "jid_append"
../include/jabber/jabber.h:72: parse error before "a"
../include/jabber/jabber.h:72: warning: type defaults to `int' in declaration of `jid_append'
../include/jabber/jabber.h:72: warning: data definition has no type or storage class
../include/jabber/jabber.h:73: parse error before "jid_xres"
../include/jabber/jabber.h:73: parse error before "id"
../include/jabber/jabber.h:73: warning: type defaults to `int' in declaration of `jid_xres'
../include/jabber/jabber.h:73: warning: data definition has no type or storage class
../include/jabber/jabber.h:74: parse error before "jid_nodescan"
../include/jabber/jabber.h:74: parse error before "id"
../include/jabber/jabber.h:74: warning: type defaults to `int' in declaration of `jid_nodescan'
../include/jabber/jabber.h:74: warning: data definition has no type or storage class
.
.
.

There's plenty more of that if its required. I'm using gmake version 3.79.1 on Linux. My C skills aren't up to this so any assistance would be appreciated.

Thanks

    Steve

Collapse
Posted by Michel Henry de Generet on
may be a problem of end of file. Fiel mismatch between UNIX and DOS!
Collapse
Posted by Steve Manning on
Thanks but sorry you've lost me there. Do you mean in jpacket.c or in jabber.h? Do you mean ctrl-z cos I can't find any?

  Steve
Collapse
Posted by Andrew Piskorski on
The compiler is saying claiming that you have some kind of mistake in your source files. It wouldn't hurt to go look at line 59 of jabber.h and see if there's anything weird in that general vicinity (like you hit a key by accident when you had that file open in your editor).

What Michel is suggestion above is that perhaps the problem is due to your source files having DOS/Windows line endings on a Unix machine. I'm assuming you're doing this on a Linux box or the like, so dun fromdos or dos2unix on jabber.h and see if that changes the file at all. If it does, you had the wrong line endings.

Collapse
Posted by Steve Manning on
Thanks Andrew

Before I posted I'd already looked around the jabber.h source file at line 59 because I was thinking the same thing. I've looked at a hex dump of the file and I can't see anything other than ASCII 0x20 for the spacing and 0x0A for the line feeds.There is nothing hiding that I can see. The area around line 59 looks like this with line 59 being the  pool p; line:

#define JID_RESOURCE 1
#define JID_USER    2
#define JID_SERVER  4

typedef struct jid_struct {
    pool              p;
    char*              resource;
    char*              user;
    char*              server;
    char*              full;
    struct jid_struct *next; /* for lists of jids */
} *jid;

jid    jid_new(pool p, char *idstr);          /* Creates a jabber id from the idstr */

I'm wondering if there is something hiding above this in the file and this is causing the compiler to barf at line 59.

The file is striaght out of the OACS contribs on the CVS. Prior to getting this problem I did nothing more than untar it, copy into my AOLServer dir and run the make - its weird. Has anyone succesffully compiled this package previously?

Collapse
Posted by Andrew Piskorski on
The problem the compiler is having at line 59 doesn't mean the error, if any, is actually anywhere near line 59. It could be much earlier. Maybe it could even be in another file altogether, although I'm not sure about that.

Since I coincidentally already had it checked out from CVS (even though I never use it), I looked briefly at the contrib/packages/jabber/ stuff (from the oacs-5-0 branch, I think the CVS Head is the same):

The way the nsjabber C code is provided is rather weird, there're a bunch of tar and tar.gz files checked into CVS, and the c/nsjabber-0.1.0.1.tar nsjabber-patch-1.tar.gz tarballs seem to have a lot of overlap in what they contain. That shouldn't be direct directly related to your build problems, but it seems sufficiently odd and disorganized that I'd probably ask the mainters of that jabber contrib package about it.

I went ahead and tried to build it, but the build stopped even sooner than yours did, probably because my machine doesn't have something installed on it that yours does:

$ make
gcc -I../include/xode  -I../../include -I../include/jabber -I../include -I../../nsd  -I../../include -D_REENTRANT=1 -DNDEBUG=1 -g -O2 -fPIC -Wall -Wno-unused -mcpu=i686 -DHAVE_CMMSG=1 -DUSE_FIONREAD=1 -DHAVE_COND_EINTR=1   -c -o jpacket.o jpacket.c
In file included from jpacket.c:20:
../include/jabber/jabber.h:42:21: libxode.h: No such file or directory
make: *** [jpacket.o] Error 1
Collapse
Posted by Steve Manning on
Yes I had problems early on with a missing include dir that turns out to be in the ns-jabber-patch tarball which I'd originally missed. Your compile failed because libxode needs to be present. As you say its a mess but I can't attribute anything in there to my problem. I ran Dos2Unix over the files and it didn't find anything to change so I don't think its a DOS file problem. I'll try an email to the maintainers directly.

Thanks for trying.

    Steve

Collapse
Posted by Tom Ayles on
Hey Steve, did you ever get this to compile? I've hit the same problem myself, I'm trying to work through it at the moment, but if you have the solution it could save me another headache!

Ta,
Tom

Collapse
Posted by Steve Manning on
Hi Tom

Yes I did. It turned out that the version on the cvs won't compile but if you get the version from file storage it works.
By the way there is a caveat that it probably won't compile under aols4 - don't know if thats been resolved.

    - Steve

Collapse
Posted by Tom Ayles on

Many thanks for that! I've got it to compile under AOLServer 4 with the minimum of fuss and no headache.

The problem with compiling the source as provided with AOLServer 4 stems from the fact that the DB pool API was factored out into the nsdb module during the move from 3.x to 4. Assuming your include paths are set up properly, you can make it compile under both AOLServer 3.x (not tested, but should work) and 4 by adding the following to the top (ish) of the files db_jclient.c, lib_jclient.c, conference_jclient.c:

#include "ns.h"

#if NS_MAJOR_VERSION > 3
#include "nsdb.h"
#endif
Collapse
Posted by Bjoern Kiesbye on
Hi,

I have uploaded the nsjabber-0.1.0.2.tar.gz to the file storage, this version works with Aolserver 4 (only).

https://openacs.org/storage/file?file_id=187244

You are able to compile the old version with Aolserver 3 after editing the Makefile but the conferencing System will not work, due to a change in Aolservers ns_set c-api. As well it includes all changes (speed enhancement) and bug fixes, since the last release.

Collapse
Posted by Tom Ayles on
Hi Bjoern,

Thanks for making this update available. Unfortunately, I seem to be having some trouble with the file - gunzip complains about 'invalid compressed data--format violated' and won't decompress it. I've tried a couple of different means of getting it, so I think it's a server-side problem; maybe this is related to the recent hardware failure. Any chance you could check the file? (or maybe even send it to me directly!) I'm evaluating the module for a project right now, so far it looks very promising!

TIA,
Tom

Collapse
Posted by Jeff Davis on
Bjoern, there was a bug in file-storage upload (under aolserver 4 and another thing that has been fixed since 4.6.3) which corrupted binary files. I fixed it so if you could upload your file again that would be great.
Collapse
Posted by Nima Mazloumi on
Bjoern, is there already a postgres port for your jabber package? If not, how difficult do you think could it be? If it is not too difficult I would like to help in porting it.
Collapse
Posted by Tom Ayles on
I've had a (casual) look through the code for this. In terms of the OpenACS part of the package, porting to PostgreSQL would be as easy as for any other package, i.e. a little tedious but not hard. Oracle to PostgreSQL is generally easier than the other direction, IMO.

What could be tricky is removing database-dependence from queries in the C code. Generally, access is done via PL/SQL accessors, however PL/SQL blocks in Oracle are different to those in PostgreSQL (at least from within OpenACS, PostgreSQL db_exec_plsql blocks are really functions that get executed with a select statement, and so have to have a return value), so this would raise an issue. Making it run on PostgreSQL _instead_ of Oracle would be comparatively easy, database independence would be hard (but maybe a fun problem to crack 😊).

If I decide to go ahead with using this package for my current project, I think it quite likely that I'd need to port to PostgreSQL, so I'll try and keep the community informed to avoid duplication of effort.

Nima

This is where I was coming from when I ran into the problem which started this thread. I wanted to get it installed and try porting it to pg. Unfortunately, events overtook me and I never managed to progress it very far - although I did get the  jabber module compiled and installed 😊.

From what I remember there isn't much to convert in the sql. Its mostly tables, selects and calls to existing acs functions. I was fairly confident at being able to convert it and my exposure to Oracle is minimal.

    - Steve

Collapse
Posted by Bjoern Kiesbye on
Hi,

I have uploaded the file again hope it will work this time.

https://openacs.org/storage/file?file_id=187244

As well I have uploaded a tarball that includes a Jabber package and a nsjabber module which both support postgres. It's in early testing state, so if you like you can test it.

I ran into the problem with the db_exec_plsql Tom mentioned above, changing them to db_dml statements which execute a begin ... end; Block would work for oracle, but postgres wont execute a Select statement inside a dml call. And I don't think it's possible to execute pgpsql code directly you always have to compile a function first and then trigger it by calling it with a Select statement.

An approach would be to change all plsql procedures to functions (oracle) which return dummy values, so they can be called inside Select statements as well (select func1 from dual). Does any one know how much more time a db_select would take compared to a db_exec_plsql?

Collapse
Posted by Tom Ayles on
Hi Bjoern,

I've downloaded your new version, and it works fine - thx! Regarding the pl/sql problem, whilst PostgreSQL requires that the functions are called using a SELECT statement, Oracle won't allow you to invoke a function that performs DML from within a SELECT (ORA-14551), which pretty much leaves us between a rock and a hard place as far as a simple solution to dual database compatibility is concerned.

Anyway, I've decided to go ahead with using the various components of the package for a project I'm working on, and will probably be doing quite a bit of work with it. For now, my priority is to get the exact functionality I need up and running on Oracle, later on I'll figure out how best to deal with the PostgreSQL issue. I might end up just having a version of nsjabber for each DB, but I'd prefer to end up with a single set of code that once compiled would run on either.

Thanks again for making this update available,
Tom

Collapse
Posted by martin hebrank on
Why don't you just put a check in the C code to change the SQL for each DB?

I added the extern char * DB_DRIVER, then in tcl_jclient.c:

DB_DRIVER = strdup ( ((cache  = Ns_ConfigGetValue("ns/db/pool/jabber" , "driver"))?cache:"postgres") );

Then, where SQL is used:
if (strcmp(DB_DRIVER,"postgres") == 0) {
    snprintf(query, 4096, "select jb_update_status ('%s' , %s' , '%s' , '%s');", state, jb->resrc, jb->jid , jb->service);
  }
  else {
    snprintf(query, 4096, "begin jb_update_status ('%s' , %s' , '%s' , '%s');  end;", state, jb->resrc, jb->jid , jb->service);
  }

Works for me. . .

Of course, this is all stuff I did because when I first started looking at it I couldn't find any references to the jabber package even being actively developed. . .

** Martin

Collapse
Posted by martin hebrank on
I downloaded the postgres packages. . .didn't work right away, I'm looking into it now, but noticably, the -drop sql files are missing, so when it breaks it's not easy to clean out the db and start again. I'm fixing that stuff now.
Collapse
Posted by Bjoern Kiesbye on
Hi,

the problem isn't nsjabber, during header runs (compilation) I can not just define which SQL should be used (like openacs does it with .xql files) as well I can define which db_api call should be used (dml for oracle, Select for postgres). Even if nsjabber with postgress support doesn't run Bug free jet (because I changed some other things together with the  postgres port (db access)).

This problem  ocures in oacs only, do we have a flag which determines which db an oacs installation is using ?
I could then wrap the db calls into if statemnts (their are only a few which are not called often (only when a screen/conference_room object is droped)).

I've fixed some bugs in the jabber-create.sql

Hi Martin did you already wrote a jabber-drop.sql? If so could you upload it to the file storage, I will include it into the package then.

Collapse
Posted by martin hebrank on
Bjoern,
I've emailed you the jabber-drop.sql directly, if you don't get it, tell me.

It doesn't have to be decided at compile time. It can be decided at run time. In the config.tcl we're told to setup the jabber db pool like this:

ns_section "ns/db/pool/jabber"
ns_param driver postgres
...

We can use this ns_param like we do all others to decide if we're using oracle style SQL and DB calls or postgress. As I've listed it before it worked changing. If you're worried about speed, you can move the strcmp to happen once during configuration and then just use a simple boolean as the condition in the if (). Relative to the total work done, this would add no overhead.

If I get a little time today I'll make the changes and send them to you so you can see.

I do, however, have it working in AOLServer 4, OpenACS 5.1 and Postgres 7.3.4 (I should probably upgrade that), though I haven't checked the conference part, but since that's not what I'm very interested in at the moment I'm not going to bother with it yet.

** Martin

Collapse
Posted by Bjoern Kiesbye on
Hi Martin,

Your Idee isn't bad , but we would have to do a second check when the sql is actualy executed , for oracle I use Ns_DbDML and for Postgres Ns_DbExec this is neseccary for at least some of the queries. As well the first check had to be doen everywhere in the code where a SQL query is created (the extra time loss wouldn't be the big problem but it would blow up the code and makes it less readable).

I #define CONSTANS for each query during precompiler run, they are defined whith  SQL querys for oracle or postgress (db_jclient.h), the CONSTANTS are then replaced (in the second recursive precompiler run) inside the code with the previously defined SQL queries.

db_jclient.h looks like that:

#if JBDB == ORA

//first how to execute dml satements
#define JBDB_EXEC_DML ret =  Ns_DbDML(handle, sql);

// the sql  from db_jclient.c
#define JBDB_UPDATE_STATUS  "begin jb_update_status
('%s' , '%s' , '%s' , '%s');  end;"

//sql from lib_jclient.c
#define JBDB_MESSAGE_LOG "update jb_message_log set conn_state='0' ,
send = sysdate where message = '%s' and screen = '%s'"

#elif JBDB == POSTGRES

//first how to execute dml satements
#define JBDB_EXEC_DML(handle , sql) ret = Ns_DbExec(handle , sql);

//the sql  from db_jclient.c
#define JBDB_UPDATE_STATUS  "Select jb_update_status ('s' , '%s' , '%s' , '%s')"

//sql from lib_jclient.c
#define JBDB_MESSAGE_LOG "update jb_message_log set conn_state='0' ,send = curent_timestamp where message = '%s' and screen = '%s')"

Inside the *.c files it looks like this then (no extra if blocks are nesecary):

snprintf(sql , 761 , JBDB_MESSAGE_LOG  , Ns_SetGet(row , "message") , jid_full(id) );
db =  JABBER_Dbml_Main(sql);

This way it's a very littel faster and the code is better readable, the disadvantage is that you need to compile  one nsjabber.so for oracle and one for postgres.

If you want to run both (ora/post) oacs instalations with one Aolserver installation,( I think this will only happen on development Systems)
just compile one nsjabber.so for oracle and one for postgres , create two sub dirs in
/usr/local/aolserver/bin/, name them 'nsjabber-ora' and
'nsjabber-post' and copy the appropriate nsjabber.so files to this dirs. Then change the modul load section of your aolserver-config.tcl that it looks like this:

ns_section ns/server/${server}/modules
ns_param  nssock            ${bindir}/nssock.so
ns_param  nslog              ${bindir}/nslog.so
ns_param  nsdb              ${bindir}/nsdb.so

if { $database == "oracle" } {
ns_param  ns_jabber              ${bindir}/nsjabber-ora/nsjabber.so
} else {
ns_param  ns_jabber              ${bindir}/nsjabber-post/nsjabber.so
}

This way always the appropriate nsjabber.so will be loaded.

To which email address did you send the jabber-drop.sql file (mailto:bjoern_kiesbye@web.de)? I did't get any, could you try to send it again.

Thanks

Collapse
Posted by martin hebrank on
Bjoern,
I resent the jabber-drop.sql. I hope you got it.
Now, is there someplace where plans for this package are discussed? I have to do quite a bit of work on it and want to make sure that I'm working with the community.

** Martin