Where to place the disconnect between list manager and OpenACS is an interesting architectural question. For ezmlm, it sounds like you're leaving quite a bit up to the list manager to carry things while using OpenACS mostly as a source for users. I could see the opposite in which a list manager is more of an OACS module, and the actual mail handler jus pipes anything sent to the list into the module.
I think allowing users to add/remove themselves from a list over email might be redundant, though if ezmlm allows it, you could have the function just delete them from a group (3.x thinking...sorry, haven't played with 4.x enough), or better yet just toggle a switch about receiving email while being a member. Adding should probably go through the normal registration process.
And that all depends on which piece of the equation you want to lean on more heavily. Rewriting a list manager for OACS is a lot of work, though it probably guarantees good integration with OACS. Reusing an existing one like ezmlm could be nice, if it can do everything we want of it and if the things we don't want can be disabled.