Forum OpenACS Q&A: Beware the latest kernel upgrade

Collapse
Posted by Janine Ohmer on
Mike upgraded all our kernels today because of a new security exploit - and then promptly un-upgraded them.  Turns out part of the "fix" is to hide command line switches from ps.  This is seriously annoying, and causes all our restart-aolserver scripts to break.

Mike said that the grumbling has already started; many people are unhappy with the fix and it is causing other problems, so he expects it to be reissued soon.

Hopefully this will save someone a bit of time...

Collapse
Posted by Roberto Mello on
Janine,

Can you give me an example? I haven't followed the linux-kernel list in a while for lack of time, but I don't see such behavior on my debian box:

rbm@zippy:/usr/src$ uname -a
Linux zippy 2.4.20-k7 #1 Sat Dec 21 18:08:10 EST 2002 i686 unknown unknown GNU/Linux

rbm@zippy:/usr/src$ ps fax
...
 2755 ?        S      1:11 /usr/lib/aolserver/bin/nsd -i -u www-data -g www-data -t /etc/aolserver/dotlrn.tcl
 2759 ?        S      0:01  \_ /usr/lib/aolserver/bin/nsd -i -u www-data -g www-data -t /etc/aolserver/dotlrn.tcl
 2760 ?        S      0:00      \_ /usr/lib/aolserver/bin/nsd -i -u www-data -g www-data -t /etc/aolserver/dotlrn.tcl
...

rbm@zippy:/usr/src$ ps faxwww
...
 2755 ?        S      1:11 /usr/lib/aolserver/bin/nsd -i -u www-data -g www-data -t /etc/aolserver/dotlrn.tcl
 2759 ?        S      0:01  \_ /usr/lib/aolserver/bin/nsd -i -u www-data -g www-data -t /etc/aolserver/dotlrn.tcl
 2760 ?        S      0:00      \_ /usr/lib/aolserver/bin/nsd -i -u www-data -g www-data -t /etc/aolserver/dotlrn.tcl
...

Probably better than relying on the behavior of 'ps' is to use the proc "filesystem".

rbm@zippy:/usr/src$ cat /proc/2755/
cmdline  cwd      environ  exe      fd       maps     mem      mounts   root     stat     statm    status   
rbm@zippy:/usr/src$ cat /proc/2755/cmdline 
/usr/lib/aolserver/bin/nsd-i-uwww-data-gwww-data-t/etc/aolserver/dotlrn.tcl

-Roberto

Collapse
Posted by Mike Sisk on

To be more specific, this is the ptrace local root hole patch for Linux that was discussed yesterday in various places including slashdot.

In our case, we're using Red Hat and this affects our 7.2, 7.3, and 8.0 systems.

One effect of this patch is a subtle change in the way the kernel reports the status of child processes.

Example:

On an unpatched kernel (2.4.18-26.7.x):

# ps -efwww | grep nsd | grep tufte

nsadmin   1410  1378  0 Mar18 ?        00:00:00 /export/nsd3.3+ad13/bin/nsd -u nsadmin -g users -s tufte -it /export/nsd3.3+ad13/tufte.tcl

On the system with the patch (2.4.18-27.7.x):


# ps -efwww | grep nsd | grep tufte

nsadmin   1410  1378  0 Mar18 ?        00:00:00 [nsd]

The ps and top commands on Linux gather their info from /proc. If you cat out the /proc/pid#/cmdline of a child process in a patched kernel you'll find it null. Whereas before, it reported the command line arguments for this process.

This is a problem for us on a shared server where there are many ACS and/or OpenACS instances running--the keepalive scripts depend on being able to read the arguments passed to nsd to determine which site is running on a particular process id.

It's unclear to me if this behaviour is an expected result of this bug fix--I'm not too familiar with this part of the Linux kernel and I don't totally understand this particular exploit, either. But I have seen some discussion on the kernel lists about the fix and other folks having similar problems.

Collapse
Posted by Jun Yamog on
Maybe its RH kernel specific perhaps?  RH does have its own custom kernel.  And also the fix is on 2.4.21, RH uses 2.4.18 + patches.
Collapse
Posted by Mike Sisk on
Maybe its RH kernel specific perhaps?
it doesn't look like it; a patch has been introduced to fix the problem:
Alan Cox wrote:
> Vulnerability: CAN-2003-0127
> The Linux 2.2 and Linux 2.4 kernels have a flaw in ptrace. This hole allows
> local users to obtain full privileges. Remote exploitation of this hole is
> not possible. Linux 2.5 is not believed to be vulnerable.

The patch breaks /proc/[pid]/cmdline and /proc/[pid]/environ for 'non dumpable' processes, even for root.

We need to access theses proc files for processes monitoring.

Included is a patch to restore this functionnality for root.

Any comments ?
(See attached file: cmdline_environ_fix.diff)
--
Mathieu Lafon - Arkoon Network Security
Collapse
Posted by Jerry Asher on
On my servers with multiple aolservers, I always give each aolserver process its own user.  Then I create a soft link to the nsd8x binary with the name of the user.  And when I start aolserver for that user, I start it with the name of the user.

So ps for me reveals the username.

Collapse
Posted by Tom Jackson on

Jerry,

Sounds like a brilliant suggestion.

It would be nice to know why ps developed a security hole in a recent version which required changing the output of ps, which has been constant for quite a long time.