you should check the Hostname parameter in your config.tcl file
(it should just be a domain name or IP w/o port number)
and if that is not it then change the line in the
/usr/local/aolserver/modules/tcl/sendmail.tcl file (or wherever your version lives)
from:
_ns_smtp_send $wfp "HELO AOLserver [ns_info hostname]" $timeout
to
_ns_smtp_send $wfp "HELO [ns_info hostname]"
and see if that fixes it.