I recreated our Oracle database with utf8 but our problems persisted. Turns out if AOLserver doesn't run with NLS_LANG=.UTF8 nothing will really work no matter what your database is created in...
I added a note about NLS_LANG in the i18n developers guide and added it commented out in nsd-oracle. Also, I added OutputCharset and URLcharset commented out in the AOLserver config file template (etc/config.tcl) so that they can be easily commented in for internationalized sites.