I think I have found the file you've been talking about. I think it is the "sample-config.tcl" file. I have also read some the aolserver documentation and found some files about configuration. In that specific file the line for configuring the mail was written as such:
ns_param mailhost smtp.yourcompany.com ;# SMTP host for ns_sendmailSo I replaced it with "sendmail" since that is the SMTP we are using.
ns_param mailhost sendmail;# SMTP host for ns_sendmailBut I am still getting some errors.