Forum OpenACS Development: webmail status?
Has anyone thought/started to port webmail?
<ul><li>It is thoroughly Oracle/SQLJ'd with no pretense of portability.
<li>It's not integrated with acs-mail or acs-messaging so there's duplication of effort.
<li>I've not heard many good things about the UI but I've not used it myself.
<p>A good first step would be to get someone to agree to fire it up under Oracle and do an evaluation of the UI. If it's good enough to keep we could consider ripping out the guts of the datamodel and all the SQLJ code and look into doing an acs-mail based variant.
Since webmail in the ACS is by all accounts a bit on the crappy side, Oracle based and written in Java, it might make sense to simply rebuild the thing, no?
Kapil, can you weigh in here?
SquirrelMail, a PHP-based system for IMAP, with OpenACS:
That said, I have managed to run PHP (and IMP as my webmail) on my home box.
WebMail is awful. Everything stored in Oracle and users can't POP or IMAP. My feeling is that it is a complete waste of time to port. Many, MUCH better alternatives exist and at some point we have to realize not everything is a good fit for OpenACS.
And, don't forget to install keepalive if you use PHP+AOLserver.
I would put a limit on message size if I were hosting webmail. 20
meg per message if I like the people, much smaller if they were
It's possible that unencoding is expending a lot of time, too, which could be fixed by hacking up an ns_uudecode.
Of course, I also kinda take Jon's comment to heart.
Then should your python/php server die, or when it dies, it will only take down the python/php server (until keepalive kick starts it), and leaves your other services running fine.
Side note: yes, I found the tcl-lib mime stuff wonderfully easy to use, and too slow to be of use. Without profiling it, I just suspected the tcl interpreter based mime encoding was way slow compared to some honest op codes.
the browser? Wouldn't that be needed to return attachments without
saving them to disk first?
I heard of someone patching aolserver to return gzipped content.
Would that version be able to return other binary content as well?
Yeah, the tcl code that I'm using is from tcllib. Unfortunately, I made a mistake when I tested the code initially. So, while I told Don that the fixed ns_uuencode was MUCH, MUCH faster than tcllib, it's actually only about 60 times faster. (Sorry, Don!). Most of the time is spent while splitting the string into 48-byte pieces to be sent to ns_uuencode. When I move that task inside ns_uuencode so that you can send arbitrary length strings to ns_uuencode, then ns_uuencode is about 2000 times faster.
Does anyone have the ability to return binary data from aolserver to the browser?
I think this requires that ns_return and ns_write be rewritten as Tcl_ObjCmdProcs. This is what I did for ns_uuencode, and it wouldn't be too hard to make this change for those procs as well (I think).
release? What are the chances of the modified ns_return making it
i totally agree with john that the ad webmail implementation is rather broken (design) and inflexible. i haven't used phpwebmail, so i'll restrict my comments to python.
python has wonderful support for the mail protocols that comes with every python distribution in the python standard library and brings alot of additional functionality to aolserver from unittests to unicode to xml processing.
for example here's a session to check how many messages and how large my mailbox is using pop
server = poplib.POP3('mail.wm.edu')
(you can try this @ home in any python interpreter shell)
# pythons standard lib internet protocols
# python std lib internet data handling
basically the libraries take care of the protocol communication while exposing the protocol functions. its also comes with a mime library (to be updated in the standard lib by www.sf.net/projects/mimelib) and is used by one of the most widely used mailing list softwares, mailman (www.list.org)
as for mime-encoding speed. i agree its important, but i think imap serves to mitigate a large subset of mime-parsing, by preparsing entities into envelopes and message bodies. i think really think webmail should focus on supporting imap not pop, since while pop can be used in a poor man's imap mode, it really isn't designed for this and might be problematic for large boxes/messages, concurrency due to mail acct locks, security.
i've been working with python in aolserver (http://pywx.idyll.org), and i've found it to be remarkably stable. i've stress tested it, done wierd and wacky things like embedding zope in aolserver, and my dev server has been up for over a month with a stable footprint. i haven't yet done much interaction with openacs but after wrassling with the request processor i expect it will be straightforward (hopefully). i'm not sure about the php integration for aolserver, but ns_python can execute tcl commands and is pretty much a full wrapping of the aolserver c api minus some of the socket and callback functionality (python has its own socket library). the ability to execute tcl functionality means some of the session handling functionality and openacs api can be reused by a python based package.
i don't think the picture is entirely rosy though, one problem i've run into is finding a good gpl templating package for python, to date i've mainly used the ones from zope (DTML, and Zope Page Templates), but they aren't compatible with the GPL (yet). ns_python can be used from adp, but adp on a whole feels clunky compared to the relative separation of logic and presentation offered by the acs templating system.
probably the larger problem is acceptance in the openacs community. the question is are people willing to accept a package that requires the installation of the python aolserver bindings?
My major concern is complexity. Python's a great language. At the moment to understand everything in an OpenACS 4 release you need to be able to read: Perl, SQL, Tcl, the templating system/.adp. That's one language less than ACS 4 Classic (which tossed Java into the mix). Perl's only used for a couple of external scripts, so you can heavily customize an OpenACS 4 installation without knowing it. But a Python webmail package would require one to know it if you were to want to customize it. Also ... not only do you have the problem of there being no good GPL'd template system but you'd be forcing folks to learn *two* templating systems to effectively customize the look-and-feel of their site.
So I'm not thrilled with the notion of raising the bar in this way.
On the other hand, integration with the OpenACS 4 datamodel would certainly make it more attractive than an independent, third-party webmail package...
"integrated with the datamodel"? What parts would benefit from
integration? Is it permissions? Do you want to stuff the emails
themselves into the content repository? Is it possible to simply
build some kind of a bridge tool that would make it easy for
people to integrate their webmail package of choice with the right
pieces of OpenACS?
Stuff like that.
I've had some success with returning images and binary files out of mime encoded message files using the reformime utility from sqwebmail and the tcl exec command with ns_returnbinary and ns_writebinary. I'm still working on doing the same thing from C. (I'm new to C though. This could take a while)
I think webmail should store messages in the file system but store information about mime sections/ attachments in the database along with the text/plain or text/html section for easy searching.
I'd like to see the system coexist with pop and imap as well and I think the maildir storage method would cover all my requirements.
don, i agree that an integrated solution would be a much better approach, esp. one that unifies the template handling. as for complexity , i feel that programming imap w/ python would be much simpler and more maintainable than the alternatives.
in effort to solve the template issue, i did some more investigation of the ats (and the upvar hell that it is) and was able to integrate python with the ats (with a little more rp work). so python a based solution can also share the same templates that will be used throughout a site:)
i'd like to do some more testing of persistent imap connections and load testing, but i think that a python based webmail is definitely feasible. i don't think its going to nesc. be a single logon system, as most people don't keep won't have their acs passwords the same as their email accnt passwds, and afaik acs passwords aren't stored clear text.
i seriously dislike the approaches of using storing a users email on the server, mail systems like imap were designed for this type of usage. i don't see any reason to reinvent the wheel, by purposefully burdening the openacs with the complexities and overhead of user mail management and the implied restrictions on user access.