Forum OpenACS Development: imap4 for ACS Mail Lite incoming email
A current project needs to integrate a custom email system with ACS Mail Lite's incoming email feature.
The road to hacking the suggested implementation seems steep, and fragile to any changes in the underlying component versions.
I'm going to try adding imap4 as a general alternate. It might scale better by allowing the imap server to be on another system, just as outgoing email has SMTP.
There are at least two IMAP4 APIs available:
imap4 via tcllib: https://core.tcl.tk/tcllib/doc/trunk/embedded/www/tcllib/files/modules/imap4/imap4.html
NaviServer's nsimap: https://bitbucket.org/naviserver/nsimap/src
Any reason why one should be used over the other?
nsimap seems the natural choice..
Any suggestions on integrating with acs-mail-lite?
Performance- and feature-wise, i would expect nsimap to be quite good. It was implemented by Vlad as a relative thin module based on the UW IMAP c-client library from washington.edu, which is used in many imap clients and other scripting languages. It is actually the reference implementation for IMAP (see ). Feature-wise, this is very complete . To remove some hard edges, I have just updated the module to compile on recent systems. However, as nsimap is not used widely, expect a few hick-ups when using this. The code compiles as well with panda-imap .
One other advantage is that the NaviServer code scales in general better than Tcl since it does not run into the problems like e.g. the 1024 open file descriptors. There is some work done in the Tcl community to overcome this, but it will take a while until it hits mainstream.
One further option is to use NaviServer nssmtpd module, which allows to act as a SMTP deamon. One of my students did some work last year to add STARTTLS support to it.
In general, when using (Tcl) client packages, one problem is a scalable integration, which might worry people or not. When e.g. a client (e.g. a Tcl function) is called within a connection thread, it might occupy the connection quite a long time. Therefore, the server might run out of connection threads, when e.g. a high number of clients is doing this at the same time. Therefore, it is a goal to move out such operations into the background and handle this in a multi threaded fashion. This is more feasible in C than in Tcl, especially when code is depends on the state of a certain Tcl interpreter. The nsimap code supports multiple sessions, so i would expect advantages in this respect.
https://gitlab.com/cognovis/intranet-mail has the code. Maybe this helps, the project was somewhat abandoned though I probably start working on it again in October as we need to include (and log) incoming reply-to E-Mails in for a customer.
Thank you for offering the code example. Examples are handy.
At this point, my obstacle is in understanding how OpenACS callbacks work.
I understand the theory. Callbacks register events. And contracts are triggered by events. And I'm familiar with even driven programming. And yet, in my own typical way, I'm unable to yet make sense of how it's implemented in OpenACS. I'm relying heavily on existing callback cases in OpenACS, and document you wrote for Incoming email https://openacs.org/xowiki/incoming_email to help sort it out. Thank you for that documentation.
callback the::contract::name -args $args
Basically the system will loop through every registered implementation and fire it one at a time, lappending each result to the final return value of the callback call.
This I guess you have seen already, but for reference:
Thank you for the explanation. Indeed. The main point of confusion for me has been the concept that a callback collects outputs of contracts instead of each contract terminating as a separate thread.
I was expecting a callback to execute contracts independently via new threads.
Fully sequential execution within the same thread seems prone to having a contract adversely affect execution of contracts that follow (in same thread), since contracts are made and operate outside of the callback environment.
This arrangement has a variety of implications. Consequences include a contract possibly affecting input parameters or having a procedure susceptible to timeout conditions that delay processing of some contracts beyond a scheduled callback's expected execution time --when the callback is triggered as part of a scheduled proc.
Designing callbacks (and contracts) that work within this environment is critical for a centralized service, such as ACS Mail Lite.
Questions need answered in the design of a callback: Should a callback 'learn' to ignore a contract with a history of taking 'too long' to process or when a contract errors a certain number of times?
Now that the confusion is cleared, I'm set to resolving the challenges.