Forum OpenACS Development: AOL 3.5 and APM

Collapse
Posted by Jon Griffin on
It appears that AOL 3.5 breaks ns_write. APM (packages-install-4) doesn't show any output on either NS 6.2.3 or IE 6 windows.

This also is the case for bootstrapping. When all is complete the bootstrap prints the page, but packages-install-4 never prints. The package is installed correctly though.

Collapse
2: Re: AOL 3.5 and APM (response to 1)
Posted by Jon Griffin on
It ends up that ns_write breaks certain reverse proxies (i.e. pound). This is because it never sends an http header.

I switched to squid, which seems like a lot of overhead for what I need. I also tried nxunix/vhr under 3.5 and have it working except that the log files show the original ip.

If anyone has a fix for the x-forwarded-for header of ns_log I would really appreciate it.

I am documenting my trials and tribulations with single ip/multiple servers for an advanced config guide.

Collapse
3: Re: AOL 3.5 and APM (response to 1)
Posted by Tom Jackson on

Why would ns_write send an http header? I just sends what you tell it to send. If you want an http header, you should use ns_return instead. Maybe the script using ns_write needs rewriting?

Collapse
4: Re: AOL 3.5 and APM (response to 1)
Posted by Dave Bauer on
Pages that use ns_write correctly either write out the headers themselves or use ReturnHeaders https://openacs.org/api-doc/proc-view?proc=ReturnHeaders
Collapse
5: Re: AOL 3.5 and APM (response to 1)
Posted by Jon Griffin on
I understand that ns_write doesn't and maybe shouldn't send out http headers. It also appears that ad_return_top_of_page is broken.

I of course could be wrong, but the proxy returns with an error about malformed/missing headers.

Since this is a production site I couldn't spend much time troubleshooting. APM for example doesn't output the correct headers (even though it looks like it should).

Collapse
6: Re: AOL 3.5 and APM (response to 1)
Posted by Tom Jackson on

ns_write provides low level access to the socket. You use it when you want to, or have to:

  • Write your own complete response.
  • Use several calls to send information out the socket.
  • Impress your friends :)

ReturnHeaders mentioned above uses ns_write to create a basic response.

My feeling it that ns_write doesn't work as intended with the request processor, which wants to bundle everything up and send it at one time.

Collapse
7: Re: AOL 3.5 and APM (response to 1)
Posted by Andrew Piskorski on
Jon, did you ever solve these AOLserver 3.5 problems?

I just tried 3.5.1, and since it completely lacks the the
ns_startcontent command, ReturnHeaders, ad_return_top_of_page, and
various other related procs in acs-tcl/tcl/utilities-procs.tcl fail
completely.

ns_startcontent is part of Rob Mayoff's 3.3+ad13 (and earlier)
patches, and is implemented in aolserver/nsd/conn8.c

Obviously we should make 3.5.x feature complete with respect to
3.3+ad13, sooner rather than later.  I know some folks are already
working on it...

Collapse
8: Re: AOL 3.5 and APM (response to 7)
Posted by Jamie Rasmussen on
(For anyone who doesn't read the AOLserver mailing list.) I'm working on forward porting the AD13+OACS1 changes to 3.5.2. I think I'm mostly done, but I need some help testing the patches. (I'm on Win32 and haven't ever used the i18n code before.) I'm working from the notes I made earlier - http://panoptic.com/wiki/aolserver/92 If anyone would like "preview" patches, I'd be happy to email them to you. Suggestions on good tests to run are also welcome.
Collapse
12: Re: AOL 3.5 and APM (response to 8)
Posted by Jamie Rasmussen on
The patches are temporarily available here - If you try them, please let me know, as I've done very little testing. I think I've got all of the AD13+OACS1 enhancements incorporated except byte range support.

Tomorrow's AOLserver chat will include a discussion of these and the future of i18n support in AOLserver, so I encourage everyone to join us.

Collapse
9: Re: AOL 3.5 and APM (response to 1)
Posted by Vinod Kurup on
Is the gid stuff fixed in 3.5?
root# /usr/local/aolserver-3.5/bin/nsd -V                          
AOLserver/3.5.2 (aolserver_v35_b2)                                              
   CVS Tag:         $Name: aolserver_v35_b2 $                                   
   Built:           Jan  6 2003 at 19:35:54                                     
   Tcl version:     8.4                                                         
   Thread library:  pthread                                                     
   Platform:        linux 
No matter how I set the "-g" flag, aolserver runs as the default group of my "-u" user So, if I try to run it as group 'nogroup', it still runs as vinod's default group "web"
root# ps -Ao pid,user,group,cmd | grep 'nsd'
  485 vinod    web   /usr/local/aolserver-3.5/bin/nsd -t /usr/local/aolserver-3.5/nsd.tcl -u vinod -g nogroup
or if I try to run it as "-u oracle -g audio", it instead runs under oracle's default group 'dba'
 1427 oracle   dba   /usr/local/aolserver-3.5/bin/nsd -t /usr/local/aolserver-3.5/nsd.tcl -u oracle -g audio
Perhaps there's just something wrong with my installation?
Collapse
10: aolserver 3.5 gid (response to 1)
Posted by Vinod Kurup on
Looks like Jon's patch needs to be reapplied to the 3.5.2 source. Here's the 3.5.2 nsmain.c snippet (CVS-1.22.2.3.2.2 line 313):
if (garg != NULL) {
    gid = Ns_GetGid(garg);
    if (gid < 0) {
        gid = atoi(garg);
        if (gid == 0) {
            Ns_Fatal("nsmain: invalid group '%s'", garg);
        }
    }
}
if (uarg != NULL) {
    uid = Ns_GetUid(uarg);
    gid = Ns_GetUserGid(uarg);
    if (uid < 0) {
        uid = atoi(uarg);
    }
    if (uid == 0) {
        Ns_Fatal("nsmain: invalid user '%s'", uarg);
    }
}
So the gid gets set properly, but then if the uid is not null, gid gets overwritten to the user's gid. Jon's fix is to check the uarg first, then the garg.
Collapse
11: Re: AOL 3.5 and APM (response to 1)
Posted by Tom Jackson on

Probably Jon's patch would allow an illegal group to be specified for a user.

3.5.1 is broken in the sense that the -g switch doesn't work, you can only use the user's main group. But it also allows you to set the root group by setting the main group of a user to root. Looks like Jon's patch allows this as well.

I wrote some code to check that a user is in the group specified after -g, disallowing root group and setting the group to any legal group for the user. I wrote a few extra Ns_ commands to do this, but it looks like the actual code maybe should be in nsmain.c, not in an external function. Lemme dig up the code.

#include <stdio.h>
#include <stdlib.h>
#include <grp.h>
#include <pwd.h>

int Ns_UserGroupCheck( char *user, char *group );
int Ns_GetUserGid(char *user);
int Ns_GetUid(char *user);
int Ns_GetGid(char *group);
char *Ns_GetUserName(int uid);
char *Ns_GetGroupName(int gid);


int
main (int argc, char **argv) 

{

  char *group;
  char *user;

  if ( ( (user = argv[1]) != NULL ) && ( (group = argv[2]) != NULL ) ) {
      printf("check returned: %i\n", Ns_UserGroupCheck(user, group));
  } else {
     printf("usage: %s username groupname\n", argv[0]);
  }
  
  return 0;

} 

/*
 * Ns_UserGroupCheck checks:
 * 1. user uid > 0, else return false
 * 2. group gid > 0, else return false
 * 5. user in group, else return false
 * 6. return true.
 */
 
int
Ns_UserGroupCheck( char *user, char *group ) 
{
  
  int gid, uid;
  int ret;
  char *grp;
  struct group *grent;
  char **members;
  char *member;
  int i = 0;

  
  uid = Ns_GetUid(user);

  if (uid > 0) {
    // valid user
  } else if (uid == 0) {
    // root user
    return 0;
  } else if (uid == -1 && ( uid = atoi(user) ) && ( ( user = Ns_GetUserName(uid) ) != NULL ) ) {
    // valid user
  } else {
    // invalid or root user
    return uid;
  }

  gid = Ns_GetGid(group);

  if (gid > 0) {
    // valid group
  } else if (gid == 0) {
    // root group
    return 0;
  } else if (gid == -1 && ( gid = atoi(group) ) && ( ( group = Ns_GetGroupName(gid) ) != NULL ) ) {
    // valid group
  } else {
    // invalid group or root group
    return gid;
  }

  // valid user and group 
  // check if user in primary group

  if ( gid == Ns_GetUserGid(user) ) {
    return 1;
  }
  // check if user in additional group
  
  grent = getgrnam(group);
  members = grent->gr_mem;
  
  while( members[i] ) {
    printf ("member %i: %s\n", i, members[i]);
    if (!strcmp(members[i], user)) {
      return 1;
    }
    i++;
  }
  return -1;

}

int
Ns_GetUserGid(char *user)
{
    struct passwd  *pw;
    int             retcode;

    //Ns_MutexLock(&lock);
    pw = getpwnam(user);
    if (pw == NULL) {
        retcode = -1;
    } else {
        retcode = pw->pw_gid;
    }
    //Ns_MutexUnlock(&lock);

    return retcode;
}

int
Ns_GetUid(char *user)
{
    struct passwd  *pw;
    int             retcode;

    //Ns_MutexLock(&lock);
    pw = getpwnam(user);
    if (pw == NULL) {
        retcode = -1;
    } else {
        retcode = pw->pw_uid;
    }
    //Ns_MutexUnlock(&lock);

    return retcode;
}

char *
Ns_GetUserName(int uid)
{
    struct passwd  *pw;
    char           *pwname;

    //Ns_MutexLock(&lock);
    pw = getpwuid(uid);
    if (pw == NULL) {
        pwname = NULL;
    } else {
        pwname = pw->pw_name;
    }
    //Ns_MutexUnlock(&lock);

    return pwname;
}

char *
Ns_GetGroupName(int gid)
{
    struct group   *grent;
    char           *grpname;

    //Ns_MutexLock(&lock);
    grent = getgrgid(gid);
    if (grent == NULL) {
        grpname = NULL;
    } else {
        grpname = grent->gr_name;
    }
    //Ns_MutexUnlock(&lock);

    return grpname;
}

int
Ns_GetGid(char *group)
{
    int             retcode;
    struct group   *grent;

    //Ns_MutexLock(&lock);
    grent = getgrnam(group);
    if (grent == NULL) {
        retcode = -1;
    } else {
        retcode = grent->gr_gid;
    }
    //Ns_MutexUnlock(&lock);

    return retcode;
}


You can compile and test without aolserver, I just commented out the Ns_Mutex locks to run it.

Problems is the code doesn't distinguish between invalid or root user/group, which would not matter in nsmain.c.

Collapse
13: Re: AOL 3.5 and APM (response to 1)
Posted by Jon Griffin on
I am using 3.5.1 and my groups/user work. I will check this again when I get home tomorrow.
Collapse
14: Re: AOL 3.5 and APM (response to 1)
Posted by Jon Griffin on
Jamie,
The zip file is corrupt.
Also, where is 3.5.2 I only see 3.5.1 at aolserver.com?
Collapse
15: Re: AOL 3.5 and APM (response to 1)
Posted by Andrew Piskorski on
Jon, 3.5.2 is tagged in CVS, "aolserver_v35_b2".

You're right, that zipfile is corrupt. Must have gotten trashed somewhere in the email -> Jamie's site process. I've temporarily stuck a good copy on my website.

Collapse
16: Re: AOL 3.5 and APM (response to 15)
Posted by Jamie Rasmussen on
Sorry about the corruption, I'm just no good at this.  I opened up the file in 7-Zip, which must have corrupted it somehow, though it still isn't giving me any problems.  I'm trying again, but you should probably just get the file from Andy, he knows what he's doing. 😉
Collapse
17: Re: AOL 3.5 and APM (response to 1)
Posted by Jamie Rasmussen on
The AOLserver chat ended a little while ago.  Here's my summary, the log is at http://empoweringminds.mle.ie/openacs/chatlog/today.html for now.

The AOLserver Core Team had its first meeting yesterday, and a lot happened.  Mark Page will be replacing Shahzad Chaudhri as one of the AOL representatives.  The ACT has decided that everyone should be focusing on AOLserver 4.0, so all work on the 3.x series has ended.

Practically, this means that the AD patches will not be added to 3.5.x and there may not be a 3.5.2 release (though it was tagged). Jim wants to finish up his work on 4.0 within the next week.  4.0 already has some i18n support and Mark is working on the remaining issues.  Zoran and Jeff H. are working on potential changes to Tcl initialization - one goal is to have "package require" work out of the box.  There is internal pressure at AOL to have 4.0 installed at Mapquest etc. in the next month or so.  The current hope is to have a tagged beta in the next few weeks, more specific dates are promised for the next chat.

Tom J. has nsreturnz and nsrewrite ready for addition to SourceForge.  Scott plans to start putting TCL-based modules in SF CVS too.

Collapse
18: Re: AOL 3.5 and APM (response to 1)
Posted by Jon Griffin on
Well I hate to sound like the greasy wheel, but the tone of the chat reminds me of the AOL of old. Do what we say and no, you don't get cvs access.

They are making a huge mistake as the 3.5.x patches are basically done. I just put them on some production boxes and have only one minor acs-lang problem and I am sure that is my fault.

I won't trust my important sites to a .0 release and the attitude of "we want the community to focus on 4.x" is ridiculous. There isn't even any clear instructions on the crappy sourceforge site on how to focus on it.

Collapse
20: Re: AOL 3.5 and APM (response to 18)
Posted by Jamie Rasmussen on
There's nothing stopping us from releasing another version of AOLserver that has been customized for OpenACS and dotLRN; as Jon said, the work is basically done.  The hardest part would probably be naming the thing - AOLserver_3.5.2ish+kindofAD13+OACS2?  I've also been using 4b1 for over a year and have been nagging other people to try it out for almost as long.

That said, I don't really buy the ACT's argument.  If we *did* decide another OpenACS AOLserver distribution was needed in the time between now and 4.0 final, wouldn't that cause as much or more developer distraction?  As Andrew's last message makes clear - it would probably be the same people doing the work either way.  Personally, I predict that the amount of time we have and will spend debating this issue will be greater than the time it would have taken to just produce a 3.6 release. 😊

Jon - if you discover the source of the acs-lang problem is in my patches, I'll try to fix them.  While I'm thinking of it - When I was testing acs-lang I ran into the problem that it depends on ref-timezones...

Collapse
19: Re: AOL 3.5 and APM (response to 1)
Posted by Andrew Piskorski on
Jon, at first my thinking was much like yours.  But, the list of
Developers on aolserver.sourceforge.net is very long.

Dan Wickstrom, Dave Bauer, Don Baccus, Jeff Davis, Jerry Asher, Jamie
Rasmussen, Tom Jackson, Scott Goodwin, David Walker - presumably any
one of those people can commit the 3.5.x patches if they feel the
need, no?