I got this error before in a brand new install of OpenACS and found out that the problem was with Sendmail and the machine itself. Whoever put that box together had not put the full domain name in /etc/HOSTNAME and had missed Sendmail config points.
I fixed the /etc/HOSTNAME and don`t know what else I did to get Sendmail working. Its config files are so freaking cryptic that only opening them in vi made me feel dizzy.
Telneting to your mail server from the shell and replicating what AOLserver would do is what led me to find the answers:
(as your AOLserver user)
nsadmin@brasileiro:/home/roberto$ telnet fslc.usu.edu 25
HELO Trying 129.123.2.40...
Connected to fslc.usu.edu.
Escape character is '^]'.
220 fslc.usu.edu ESMTP Sendmail 8.9.3/8.9.3; Mon, 24 Jul 2000 17:57:55 -0600
HELO brasileiro.net
250 fslc.usu.edu Hello nsadmin@slxwy.dorms.usu.edu [129.123.236.160], pleased to meet you
MAIL FROM:nsadmin@brasileiro.net
250 nsadmin@brasileiro.net... Sender ok
RCPT TO:rmello@cc.usu.edu
250 rmello@cc.usu.edu... Recipient ok
...
