Forum OpenACS Development: acs-mail-lite that receives email with attachments

Hi developers,

I just posted a pre-final draft, where ACS Mail Lite can process full emails with attachments and handle a variety of practical situations where prioritization of incoming queue, or special filtering is necessary.

This version works with MailDir and nsIMAP. Most code is shared. The two paradigms can co-exist, so that one might expect a smooth transition between service shared with NaviServer and moving email service to another server to distribute load or work in an environment where incoming email is shared between user email clients and OpenACS for example.

It passes regression tests. Yet, implementation has not been checked yet via an external package.

I'm posting now, because it might take me a week or two to make a package to test everything 100% --I don't want to get in the way of this development. Also, I want to give you an opportunity to influence the changes before it is fully integrated.

Post here what features or changes you'd like to see (or if you spot bugs), and I'll see if it can be accommodated --if it hasn't been already.

Regarding existing code, I took the liberty to upgrade the unique message-id usage to not publish OpenACS data that is passed, such as package_id or object_id. Otherwise, I tried not to break the existing incoming email process by creating a new one in new files to help keep the changes more obvious.

Submission is at https://openacs.org/xowf/package-submissions/PackageSubmit44

cheers,
Ben

Collapse
Posted by Benjamin Brink on
Hi again,

I just fixed a bug for this, thanks to Claudio for finding it:
https://openacs.org/forums/message-view?message_id=5389186#msg_5389203

If it's okay, instead of re-submitting, download/clone from: github.com/tekbasse/acs-mail-lite.git

It's written as acs-mail-lite version 5.9.1b3. Let me know if the version number should be bumped up to ease testing..

Ben

Collapse
Posted by Gustaf Neumann on

Dear Ben,

first, man thanks for the work invested into the new version. The best thing would be to merge this with acs-mail-lite in the main code repository. i can do this, but would like to clarify the following first:

  • the code can co-exist with the "traditional" acs-mail-lite sending mechanism, that is, that switching to an nsimap based version is an "opt-in".

  • the point above means as well, that the code will continue to work with classical AOLserver based installations

  • it is my understanding that the code is so far PostgreSQL only (which is not a show-stopper, when it is optional)

  • are there other know shortcomings in the current version?

Collapse
Posted by Benjamin Brink on
Hi Gustaf,

This revised version was coded to touch as few points in the prior code as possible. This is meant to avoid compatibility issues with an upgrade, while adding more flexible feature set that is consistent between connection types imap/MailDir and smtp/ns_sendmail.

There's only one point where an upgrade may require a change, and that is with the bounce feature.

If a system is relying on bounce, two ways are provided to fix it. The intervention is minimal, considering the flux of bounce code prior to this version, and the discussion of changes suggested to make to it in OpenACS wiki and forums.

The bounce issue should be remedied by adapting to the new callbacks or, by replacing the invoking of acs_mail_lite::parse_bounce_address with a new proc acs_mail_lite::inbound_email_context. The new proc will try to parse the old way, if there is a transition period where an email using the prior way is replied to and intercepted in the new way. The new callbacks use the new parse method, also.

Outbound email was only touched to standardize message-id and originator headers.



Diff'ing through the files of the prior version, here is a summary of the changes:

1. message-id header value is created and parsed using a more general system. ad_sign is used to sign message-id when integrity of package_id etc is passed to email.

2. originator header now uses the same value provided by message-id header.

3. Bounce address may now be a fixed sender email address. (Not default).

4. On server start, scheduled procs to process bounces automatically turn off after failing twice. This saves having to modify acs-mail-lite-init.tcl locally to change email configuration. Just change parameters etc and maybe restart server.

5. acs_mail_lite::parse_bounce_address is deprecated instead of trying to incorporate its logic into the new paradigm. It's replacement acs_mail_lite::inbound_email_context handles both prior and new cases.

6. The way of figuring out the email domain is standardized. Prior code uses multiple methods that could result in inconsistent values.

7. FixedSenderEmail parameter was added as an option in context of some code that relies on configuration and determining domain.

8. cases of 'file delete $var' were changed to 'file delete -- $var'

9. acs-mail-lite-callback-procs.tcl has new callbacks that provide more flexible integration with packages. Prior callbacks are unchanged (but may break without a minor fix. See item 5 in this list).

I should also mention that a good deal of time was spent parsing rfc's related to email to make sure that the new code is as flexible as possible to adapting to custom requirements considering the variety of ways email is used.

cheers,
Ben

Collapse
Posted by Benjamin Brink on
Gustaf,

Here are specific answers to your first four questions:

There should be no issues with the "traditional" acs-mail-lite sending mechanism after an upgrade of acs-mail-lite package. nsimap is completely optional, as is a mailDir configuration and smtp.

An upgrade should not change the behavior of acs-mail-lite, except for the bounce issue presented in my prior post.

You are right, I only provided postgresql sql. It's really basic sql, so will work with Oracle with minimal tweaking.

cheers,
Ben

Collapse
Posted by Gustaf Neumann on
Sounds good! i've merged your changes into CVS HEAD, please double-check, if i have not missed something.
Collapse
Posted by Benjamin Brink on
Hi Gustaf,
Some unit tests error without nsimap installed, so I added catch to them, and a couple of other minor refinements.

If it's okay, I'll post patches to head version in bugtracker. If you prefer another way, let me know.

thank you,
Ben

Collapse
Posted by Gustaf Neumann on
perfect.
Collapse
Posted by Benjamin Brink on
Done. Posted patches 960 to 963 in bugtracker.

No need to reply. I'm posting here in case I goofed with the patch posting process or notification is not made elsewhere.
cheers,
Ben