Forum OpenACS Q&A: The webmail problem

Collapse
Posted by Talli Somekh on
Hi guys,

IMHO, an intranet solution is not complete without good email integration, which includes a webmail system. At least, the ideal solution is not complete without a good webmail system. I would like to be able to offer clients the option of webmail but not if it would cost an arm and a leg to build it out.

We have the best of the best for most other tools, but we've never really implemented a good webmail system that has been native to the OACS. aD did something with Java that was Oracle specific and I think never worked in a general sense.

However, it's not clear to me that we necessarily need to have NIH syndrome for every problem. IMP has a large installed base and quite a nice interface. It's written in PHP and should be able to connect with PG or Oracle backends. The external authentication work Lars and Peter will be doing will be a big help in third party integration as well.

At the same time, we might lose a lot of what makes the OACS cool if we were to go with a third party product like integration with the datamodel and use of the CR. Perhaps that stuff could be hacked in, but wouldn't it be better to build it from scratch?

I suppose my questions are the following:

  • Has anyone researched this issue in detail?
  • Has anyone tried the package that Rafa Calvo's student did?
  • Is there a compelling need to have a native OACS webmail tool?
  • What is the state of nsIMAP? Is it production quality and ready for a webmail package to use?
  • If a webmail package were to be built, what kind of architectural features should it have within the context of the OACS?
  • Any other issues?

Collapse
2: Re: The webmail problem (response to 1)
Posted by Malte Sussdorff on
Hi Talli,

a) We run IMP 3.1 with ACS 3.5 for over a year know, tightly integrated into a portlet.
b) Yes, not good enough for running an Intranet Webmail System for more than 10K users.
c) No, and never was. Integration is the way to go, especially as there are excellent systems out there. You might want to bother about having not the same look and feel, but seriously, users don't bother.
d) dunno
e) I'd request nothing that can be handled better by third party applications. This means, leave IMAP, Sendmail and POP server out of the Scope. Take a look at IMP or Squirrelmail. Look at fastmail.fm for enhanced features. <sarcasm>Take a lot of work and even more time to reprogramm theses things in OpenACS and start wondering afterwards, why you did it.</sarcasm>

Sorry to be sarcastic here, but the problem has come up a multiple times and so far nothing has come up, for a very good reason. Creating a good webmail system is complex and takes a lot of effort. Effort, this community should not try to put it's resources into at the moment. Interfacing the basics of a webmail system is easy, relatively speaking. People don't bother switching to a different system. They don't even bother to have to signone twice (though we try to get around this). As long as it works, is untuitive to use and looks sleek, they are happy. At least our experience so far.

That said, setting up IMP, Cyrus, CyrusSASL, Postfix, OpenLDAP and integration into AOLserver is a PITA. One we did not even bother to write a documentation for so far (otherwise I'd have said right in the beginning, here is the URL, that's how we did it, go figure out for yourself). Once we have it, I'll announce, how long will it take, I don't know. How long will it take an average person to setup: 3 days. Maybe more. Or less, if you start without RPM's, take bleeding edge software and pray it works together.

LDAP is a crucial part into integration in general, though you can play around it for sure (or keep databases in sync as we do with Jabber). Therefore we look forward to Collaboraid's effort in this matter, as it will allow us to integrate even more different systems that clients happen to use (Next on our target list is Lotus Notes, probably).

Collapse
3: Re: The webmail problem (response to 1)
Posted by Chris Johnson on
Talli,
I am interested in an eventual (re)write of openacs-integrated (web)mail. E.g. I understand that right now there is outgoing notifications through email, some sort of SMS and Jabber plugins, and the webmail system mentioned above. There was some interest a while back in implementing mailing list management (or had it been done/contributed??) as well, using e.g. qmail.

I want to integrate all messenging and notifications and webmail/forums into a unified architecture. I will settle probaby in the short-medium term with whatever webmail is easiest (e.g. start with IMP or the student's oacs webmail), get together the oacs mlm software, then think of grand schemes later.

The main drawback to my enthusiasm is that my time to donate will be slim because my projects are not customer or business driven but are just my own hobby projects.

Just my $0.02, so you know "I'm in" if a project is on :)

--Chris

Collapse
4: Re: The webmail problem (response to 1)
Posted by Lars Pind on
I played with the webmail package in December, and it's not great, but it's not too bad either. With some guidance, he or another student or novice could probably make it a lot better.

I made some improvements to webmail, and I should probably stick the result in CVS under contrib.

I don't know if ns_imap is production-quality, but it was pretty simple to work with, and it worked for me to check my mail.

I undoubtedly think, however, that using IMAP is the way to go. We don't want to make our own mail storage in the DB, the way Jin did with the old webmail package. Let an IMAP server deal with that, and we'll just integrate with that IMAP server. Everybody already has an IMAP server anyway.

Then if you want to reuse an email as a knowledge object, say, we could just offer a "post to KM" link, which would copy the content. The same thing could be done with forums and other content, for that matter.

/Lars

Collapse
Posted by Malte Sussdorff on
At the moment we are thinking about creating an Multi-Agent system for retrieving knowledge that will access multiple sources. This way you could store the information in multiple systems without the need for "post to KM" eventually.
Collapse
6: Re: The webmail problem (response to 1)
Posted by Vlad Seryakov on
As owner of ns_imap i can say that it in production-quality for about 90-99%. There are some issues with fatal error handling because of the way UW IMAP c-client handles them, but they are very rare. We use it our Webmail based on this module for almost year now without any problems.
More complete Webmail solution is now available in opensource package(ftp://ftp.crystalballinc.com/pub/vlad/oss2.tar.gz) and it does many features of IMP, not everything because we did not need all of them.
Before it we used NeoMail, IMP, SquirrelMail and others, but we wanted same environment(including GUI) where all tools can be easily connected to each other, reuse same database and data, easily configured and maintained. If everything runs in AOLserver it may be pretty easy. Supporting apache/php and other tools can be easy too, but they are additional components.

There is one more product which looks interesting, dbMail(www.dbmail.org), they store email in the database and building webmail client is just writing/reading from database, they have couple webmail clients, writing in OACS may be very easy task.

Collapse
7: Re: The webmail problem (response to 1)
Posted by Jonathan Ellis on
I experimented with IMP a year ago or so, and either php sucks, aolserver's php support sucks, or imp sucks, because it was INCREDIBLY slow.  We're talking pages that do lots of ACS 4 permissioning without all the OACS optimizations slow. :P

Of course that could be past tense now but I was very underwhelmed.

Collapse
8: Re: The webmail problem (response to 7)
Posted by Rich Graves on
I would not expect aolserver/php to be well tested, let alone optimized. Why would someone use php when tcl is right there?

IMP works well in our environment (http://web.brandeis.edu/pages/view/Network/EmailArchitecture).

IMP talking to UW IMAPd or any other server that needs to fork a new heavyweight process per connection can suck. IMP flaps a new process for EVERY OPERATION, which isn't really how IMAP is supposed to work. ImapProxy (http://www.horde.org/imapproxy/) is intended to address this, but I don't trust it yet. Large mbox-formatted mailboxes suck in general. With UW IMAP, read drivers.txt and use mbx format. Cyrus and Maildir have their fans, too. Exchange tends to do reasonably well because it's multithreaded.

Collapse
9: Re: The webmail problem (response to 8)
Posted by Vlad Seryakov on
That's why ns_imap caches all connections to IMAP server, so
you do not open new imap connection with every HTTP request like it is done in almost all PHP Webmail products.
Once the IMAP session is created, it lasts as long as needed and
garbage collection thread closes unused sessions periodically. In this case acess to IMAP server is very fast, c-client IMAP caches all messages and so does IMAP server on the other end, so standard
mailbox format is not an issue here, once mailbox file is open and parsed, you do not need to scan directory and close/open new messages.