Hi Robin,
I'm not an expert but here are a few suggestions.
I think the "ns_param mailhost" was changed to "ns_param smtphost" in AOLserver 4, so you could try that first. If you're using AOLserver 3, mailhost should be fine.
Make sure the ns_param mailhost (or smtphost) entry is in the "ns_section ns/parameters" section.
Can you open a manual mail session to the mail server from your unix prompt? Try this:
[qfs18 aol33]$ telnet 192.168.0.80 25
Trying 192.168.0.80...
Connected to 192.168.0.80.
Escape character is '^]'.
220 qfs01nt.quest.ie ESMTP Server (Microsoft Exchange Internet Mail Service 5.5.
2653.13) ready
HELO
250 OK
QUIT
221 closing connection
Connection closed by foreign host.
Hope this helps.
Brian