Forum OpenACS Q&A: Up 18 months now only connects on port 22

My production OpenACS 3.x/ Postgres / AolServer system On RH7 has been Up 18 Months. The power failed at the datacenter... UPS batteries didn't hold till the generator started, then the Gen got overloaded and power cycled a few times last night.

I can connect only to port 22. (SSH) --- Slow to connect. Postgres and AOLserver come up fine but there appears to be no communication in or out except this port 22.

[nsadmin@ log]$ netstat -natu
Active Internet connections (servers and established)
    R/S-Q   Local Address   Foreign Address         State
tcp 0    0 0.0.0.0:80           0.0.0.0:*           LISTEN
tcp 0    0 127.0.0.1:9000       0.0.0.0:*           LISTEN
tcp 0    0 208.44.220.148:22    60.186.172.110:48603ESTABLISHED
tcp 0    0 0.0.0.0:54320        0.0.0.0:*           LISTEN
tcp 0    0 0.0.0.0:49724        0.0.0.0:*           LISTEN
tcp 0    0 0.0.0.0:40421        0.0.0.0:*           LISTEN
{cut some lines for brevity)
tcp 0    0 0.0.0.0:143          0.0.0.0:*           LISTEN
tcp 0    0 0.0.0.0:119          0.0.0.0:*           LISTEN
tcp 0    0 0.0.0.0:111          0.0.0.0:*           LISTEN
tcp 0    0 0.0.0.0:79           0.0.0.0:*           LISTEN
tcp 0    0 0.0.0.0:15           0.0.0.0:*           LISTEN
tcp 0    0 0.0.0.0:11           0.0.0.0:*           LISTEN
tcp 0    0 0.0.0.0:1            0.0.0.0:*           LISTEN
tcp 0    0 0.0.0.0:25           0.0.0.0:*           LISTEN
tcp 0 2592 208.44.220.148:22    60.186.172.110:48588ESTABLISHED
tcp 0    0 0.0.0.0:22           0.0.0.0:*           LISTEN
udp 0    0 0.0.0.0:54321        0.0.0.0:*
udp 0    0 0.0.0.0:31337        0.0.0.0:*
udp 0    0 0.0.0.0:32774        0.0.0.0:*

I Can NOT ping this machine... I CAN telnet to port 22 but not 25.

Could it be this machine or EXTERNAL to it.... Could the next upstream router be doing wierd stuff to cause these problems?

I am also running portsentry.

I have SHUTDOWN -r and also -h and had them Power cycle the machine.

Any Good sysadmins willing to help me? Reply here or email me at bob@rocnet.com or 207-445-4140.

THANK YOU.

-Bob

Collapse
Posted by Cathy Sarisky on
Bob - I'm not sure I'm a Good sysadmin (with a capital G and all), but you might want to check your iptables entries, and also check that portsentry didn't grab port 80.  (netstat -antp as root)

Good luck!

Collapse
Posted by Jade Rubick on
See http://www.chkrootkit.org/ just in case.
Collapse
Posted by MaineBob OConnor on

So, I got some help from Argen (sp?) at OpenForce -- THank you!

I stopped portsentry.

[root@ sysconfig]# ipchains -L
Chain input (policy DENY):
target     prot opt     source   destination  ports
acct       all  ------  anywhere  anywhere    n/a
DENY       tcp  ----l-  anywhere  anywhere    2000 -> any
ACCEPT     tcp  ------  anywhere  anywhere    ssh -> any
DENY       tcp  ----l-  anywhere  anywhere    any -> 2000
ACCEPT     tcp  ------  anywhere  anywhere    any -> ssh
ACCEPT     all  ------  localhost.localdomain
localhost.localdomain  n/a
Chain forward (policy ACCEPT):
Chain output (policy ACCEPT):
Chain acct (1 references):

Also, the /etc/sysconfig/ipchains and found a very long 575 line file which I understand is not the small standard that comes with RH 7 and maybe what portsentry created.

I went ahead with the current setup and I ran

/etc/rc.d/init.d/ipchains restart

AND my SSH connections died. required me to call the datacenter to powercycle....

I think portsentry and ipchains are the source of my problem but yet not the solution.

On bootup I ran Cathy's suggested version of Netstat

[root@ sysconfig]# netstat -antp
Proto RS-Q Local Address      Foreign Address      State       PID/Program name
tcp   0   0 208.84.220.148:22 60.186.172.110:52115 ESTABLISHED 642/sshd
tcp   0 320 208.84.220.148:22 60.186.172.110:52113 ESTABLISHED 641/sshd
tcp   0   0 0.0.0.0:54320     0.0.0.0:*            LISTEN      590/portsentry
tcp   0   0 0.0.0.0:49724     0.0.0.0:*            LISTEN      590/portsentry
tcp   0   0 0.0.0.0:40421     0.0.0.0:*            LISTEN      590/portsentry
tcp   0   0 0.0.0.0:32774     0.0.0.0:*            LISTEN      590/portsentry
{More not shown} but include ports
54320 49724 40421 32774 32773 32772 32771 31337 20034 12346 12345 
6667 5742 2000 1524 1080 635 540 143 119 111 79 15 11 1 

tcp   0   0 0.0.0.0:25        0.0.0.0:*            LISTEN      506/master
tcp   0   0 0.0.0.0:22        0.0.0.0:*            LISTEN      346/sshd

AND WHEN I
/etc/rc.d/init.d/portsentry stop 
I get...

Proto RS-Q Local Address      Foreign Address      State       PID/Program name
tcp   0   0 208.84.220.148:22 60.186.172.110:52115 ESTABLISHED 642/sshd
tcp   0 320 208.84.220.148:22 60.186.172.110:52113 ESTABLISHED 641/sshd
tcp   0      0 0.0.0.0:25     0.0.0.0:*            LISTEN      506/master
tcp   0      0 0.0.0.0:22     0.0.0.0:*            LISTEN      346/sshd

Anyone with suggestions to configuring portsentry OR changing the DENY / ALLOW statements. This is 2.2 kernel

Again, no ping in or out, No other access in or out 'cept port 22 SSH.

Thanks.

-Bob

Collapse
Posted by MaineBob OConnor on
YaHooooo it is up again.....  This time I just stopped
ipchains... portsentry is still running and we have access
and everything works... ANYONE have suggestions
for rebuilding the ipchains to keep the hooligans out?

-Bob

Collapse
Posted by Stan Kaufman on
FWIW, here's the portion of my ipchains ruleset running on my firewall box for the servers in my DMZ (the vars $HTTPIP and $SMTPIP are the ip addresses of these servers; $DNSIP1 and $DNSIP2 are dns servers at my ISP).

This follows the "serious example" in the IPCHAINS-HOWTO here: http://www.tldp.org/HOWTO/IPCHAINS-HOWTO-7.html

I presume it would be trivial to convert this from ext-dmz and dmz-ext chains to input and output chains. HTH!

# EXT to DMZ
#   SMTP to external
#   accept SMTP from internal and external
#   accept HTTP and SSL from internal and external

$IPCHAINS -A ext-dmz -p TCP --sport smtp -j ACCEPT
$IPCHAINS -A ext-dmz -p TCP --sport www -j ACCEPT
$IPCHAINS -A ext-dmz -p TCP -d $SMTPIP smtp -j ACCEPT
$IPCHAINS -A ext-dmz -p TCP -d $SMTPIP pop-3 -j ACCEPT
$IPCHAINS -A ext-dmz -p TCP -d $HTTPIP www -j ACCEPT
$IPCHAINS -A ext-dmz -p TCP -d $HTTPIP 443 -j ACCEPT
$IPCHAINS -A ext-dmz -p UDP -d $HTTPIP 443 -j ACCEPT
$IPCHAINS -A ext-dmz -p TCP --sport auth -j ACCEPT 
$IPCHAINS -A ext-dmz -p TCP -d $HTTPIP auth -j ACCEPT
$IPCHAINS -A ext-dmz -p TCP -s $DNSIP1 domain -j ACCEPT
$IPCHAINS -A ext-dmz -p UDP -s $DNSIP1 domain -j ACCEPT
$IPCHAINS -A ext-dmz -p TCP -s $DNSIP2 domain -j ACCEPT
$IPCHAINS -A ext-dmz -p UDP -s $DNSIP2 domain -j ACCEPT
# cvs
$IPCHAINS -A ext-dmz -p TCP --sport 2401 -j ACCEPT
$IPCHAINS -A ext-dmz -p UDP --sport 2401 -j ACCEPT
#
#$IPCHAINS -A ext-dmz -p ICMP --icmp-type ping -j ACCEPT 
$IPCHAINS -A ext-dmz -p ICMP --icmp-type pong -j ACCEPT 
$IPCHAINS -A ext-dmz -p ICMP -j icmp-acc 
$IPCHAINS -A ext-dmz -j DENY -l

# DMZ to EXT
#   allow SMTP to external
#   accept SMTP from external
#   accept HTTP and SSL from external

$IPCHAINS -A dmz-ext -p TCP -s $SMTPIP smtp -j ACCEPT
$IPCHAINS -A dmz-ext -p TCP --dport smtp -j ACCEPT
$IPCHAINS -A dmz-ext -p TCP -s $SMTPIP pop-3 -j ACCEPT
$IPCHAINS -A dmz-ext -p TCP --dport 113 -j ACCEPT
$IPCHAINS -A dmz-ext -p TCP -s $HTTPIP auth -j ACCEPT
$IPCHAINS -A dmz-ext -p TCP --dport www -j ACCEPT
$IPCHAINS -A dmz-ext -p TCP -s $HTTPIP www -j ACCEPT
$IPCHAINS -A dmz-ext -p TCP --dport ssh -j ACCEPT
$IPCHAINS -A dmz-ext -p TCP --dport ftp -j ACCEPT
$IPCHAINS -A dmz-ext -p TCP --dport ftp-data -j ACCEPT
$IPCHAINS -A dmz-ext -p TCP --dport 443 -j ACCEPT
$IPCHAINS -A dmz-ext -p UDP --dport 443 -j ACCEPT
#cvs
$IPCHAINS -A dmz-ext -p TCP --dport 2401 -j ACCEPT
$IPCHAINS -A dmz-ext -p UDP --dport 2401 -j ACCEPT
#
$IPCHAINS -A dmz-ext -p TCP -d $DNSIP1 53 -j ACCEPT
$IPCHAINS -A dmz-ext -p UDP -d $DNSIP1 53 -j ACCEPT
$IPCHAINS -A dmz-ext -p TCP -d $DNSIP2 53 -j ACCEPT
$IPCHAINS -A dmz-ext -p UDP -d $DNSIP2 53 -j ACCEPT
$IPCHAINS -A dmz-ext -p ICMP --icmp-type ping -j ACCEPT 
$IPCHAINS -A dmz-ext -p ICMP --icmp-type pong -j ACCEPT 
$IPCHAINS -A dmz-ext -p ICMP -j icmp-acc 
$IPCHAINS -A dmz-ext -j DENY -l
Collapse
Posted by Stan Kaufman on
Oh yeah, in that example $IPCHAINS just is defined thus:

IPCHAINS=/sbin/ipchains

Collapse
Posted by Jon Griffin on
You probably don't want an ipchains script.

Portsentry will block people for you. I know when I setup your ipchains after your hack, it was version 1.0. This had a bug in that it didn't really set ipchains. You need to get 1.1 and everything will work if you make sure the line is not commented that has ipchains ....

If you do run the ipchains script from redhat you will have to make adjustments for your proxys and etc.