Forum OpenACS Q&A: webmail thoughts

In talking with a few people today (all day teleconfernces lead to
interesting conversations) the question of webmail in OpenACS came up
today.

Now I haven't really looked into the OpenACS/ACS webmail modules, but
for what I've seen on the list (and the lack of questions about it)
and the comments on the OpenACS 4.x Porting about it, it seems webmail
is not a particulary useful tool right now.

My question/comment/thought is, is it something that really needs to
be looked at (i.e: does anybody really want it?).

I like the idea of having webmail involved intergrated with OpenACS,
but I'm wondering about the real need/usefulness of such a feature.

I really don't like the ACS java solution for it (with a TCL software
based, I hate adding in another language).  Also, it ties to much to
the database which makes portability a real bear.

If it is determined that having webmail is a needed, and since it
appears to me that re-writing it is needed.  If that is done I'm
thinking a few things would have to be looked at:

Instead of delivering the mail to the OpenACS box (and database) would
it just be better to write a more conventional webmail interface
(using POP3/IMAP, possibly adding SSL Enabled pop3/imap for ISP that
support it), maybe offering the possibility of saving messages in the
database?

Using the filesystem and using virtual hosting then adding mbox or
maildir support to OpenACS, with a webmail interface on top of that?

Writing/using existing TCL code to parse a mail message, stuff it in
the database, and then having a webmail interface using that.  While I
think this is an interesting idea, this would be really hard on a high
volume site.

Possibly a combination (or all) of the above?

Just throwing this out there (and okay, I manage the SMTP
infrastruture for about 100,000 people, so I'm always looking for new
ways to present mail access to people on a more controlled field).

Thoughts/comments ?

here is the architecture I have thought up:

you want 2 physical machines , a mail server and a web server.
this can be done on 1 machine, but if you have more than
just a handful of users, you are going to want to have a dedicated
mailserver.

mail server:

qmail mail server

courier imap

web server:

your openACS installation +

a module which provides for webmail user administration

either machine: you want some AOL TCL scripts acting as
an imap client. i suggest writing these in a manner so
that they don't necessarily rely upon a bunch of ACS procs.
this way, you could even have a separate physical machine
somewhere (mail.yourdomain.com) that only acts as
an imap client for your users.

There are some links on qmail.org to hacks which
allow a qmail installation to authenticate with a postgres
database. I think that if you have the proper postgres
client libraries on the email server, you can do this
authentication between two machines. This would provide
for authentication of the mail users against the openACS
users table.

I use a service called POP3Now!, at http://www.pop3now.com. It
basically gives me a webmail interface for my existing email
accounts. I haven't a clue about how it works, other than to say
that it can gather email from multiple accounts without actually
removing the email from those accounts, so I can fetch the same
messages with my mail client of choice when I'm able. It's not a
perfect solution but it does what I need it to do.

Ola, As we are about to start looking at the webmail options, I would be interested in your solution. Ben koot... chief non techno experimental OpenACS user, and moderator initiator of Klessebes in Holland. By the way greetings to all of you from our lovely tulip country a nd thanks for the great suport so far, also on behalf of Hans, our chief nerd, and Renny, our senior business advisor.

Some very interesting ideas in there.  I think that it might be best to start off by creating interface for displaying messages, controlling who can use it, controlling how people use it etc.

Maybe at first a POP3/IMAP interface could be placed underneth that interface.  Then work out a SMTP->DB interface to handle the messages.

It would be nice to intergrate OpenACS's user authentication with the virtual hosting of various mail servers OR allow the remote mail retrevial.

Okay, well looks like I should really start looking at some code.

Most people who just want a webmail solution do not want to
store mail in postgres, unless you are writing a specialized app
such as a knowledge management system which lets people
search over everyone's email to find the answer to a specific
problem. Something like this would be very valuable, it just  is
beyond the scope of what most people want when they think
about a 'webmail' system.

note, it isn't impossible to use a database as the backend to a
standard webmail system...in fact, AOL stores all of its user mail
in sybase. it comes in handy when you need to find out which of
your users are using dirty words like 'breast' and 'pants.'

Actually the way that I'm currently thinking is developing a webmail interface, and basically setting up the expectations of "Get a list of all my mail", "Delete mail", "Store this mail here" etc.  Then develop the underlying processes to handle it.  In that manner it gives people the option of adding what every type of processes they want to read the mail.  I would suggesting looking at POP3/IMAP support first since there is quite a bit of code out there already that can be at the process layer.

I think this might be a good approach and give quite a bit of flexability, while being a good base for people to expand on give an interface to the mail display and allow intergration into the rest of the system.

With the current webmail on both 3.x and 4.x in a state of "flux" and I'm more than willing to start attacking the problem.  I would just like to start approaching the problem in a manner that people would find useful.

I also think have a good approach to offering webmail (which is becoming an increasing more desired feature from my perspective) I would like to move OpenACS's webmail to (hopefully) something that seems a bit more intergrated, functional, and easier to setup.

For the most part, I think people will already have mail servers and POP3/IMAP access and bring mail into the other features of OpenACS might be helpful in the "community marketing" thoughts that have been discussed lately.

It seems that this thread has been helpful in opening the dicussion about webmail:  what people want, approaches, ideas.  Now we just have to continue to go forward and hopefully develop something useful.

One of the things the ACS and OpenACS community has always seemed to do is try and build a package in the current system rather than use best of breed applications from other environments.

I don't think that this is all bad most of the time (see content management) but I think that for something as complex as webmail it may be easier to try and integrate one of the other systems with OpenACS.

We are currently using Squirrell Mail (http://www.squirrellmail.org). It's a little buggy and not entirely mature. We are going to try IMP (http://www.horder.org/imp) as a substitute. Both are built in PHP and use Apache.

I am admittedly not qualified to speak about how hard it would be to mix these systems. I will only say that it seems like a real bitch to build a good webmail system from scratch, even with the headstart that OpenACS gives.

Mat, I know you said you hate integrating two languages, but would it all be easy to integrate systems that solve your problems but aren't in the same language?

talli

If you are going to use something like IMP, TWIG, etc. then you would be much better off using and apache server and using user authentication from the OpenACS database.  (At least I think).  There are all good products, but my hope is to have something intergrated with OpenACS (Use the calendars, bookmarks, etc.) along with webmail.

I'm looking right now to see how much of the current webmail interface can be pulled out of both 3 and 4 and how much it would take to retrofit POP3/IMAP support (to begin with) into it.  There are some respectable TCL libs out there we can use for the networking, so really I think just an inteface would be a good start.

In the next day or so I'll write up a design spec.  Since there is already a webmail interface and there really isn't a current tcl web interface, a simple interface will not be that hard to put together.  Heck, I'm even willing to do the work, I just would like to make sure I'm going along a path that others will find useful.

I am working on a C module to display MIME parts from a file or the database and then combining it with a qmail maildirs configuration and some integration between qmail and the OpenACS user database and theoretically have a robust, high performance, fully OpenACS integrated webmail solution.

This would mean ACS and the mail server would run on the same machine or at least share a file system but it would allow you to use any existing qmail solutions for pop, imap, and smtp.

I can already read and display mime parts from a stored message using the reformime utility from the sqwebmail package and the ns_returnbinary command from my nsbinarysupport module. (it is at http://www.vorteon.com/download/)

Hi Ben,

Wow, I started reading this thread and didn't realize until the end that it is over a year old!

I just started a new thread about my bulk mail experiences.

https://openacs.org/bboard/q-and-a-fetch-msg.tcl?msg_id=0005N0

-Bob

Though I'm still against dublicating the efforts for a webmail system (there are really good ones out there that are developed with a significant amount of effort) itself, I am pleased that your students are going to work on it.

They might take a look at IMP 3.0, which is the webmail system we are using for 30000 people and they are very pleased so far. And yes, we do have integration into ACS working, with tools to signup users to the webmail system and other integration work (isn't HTTP cool 😊).

If you want surely Venkatesh can give your students a demo otherwise look at http://www.horde.org/imp/3.1/

How would such a webmail be integrated with OpenACS? Would the emails be full-text searchable? I suppose one obvious but small advantage would be that you would have immediate access to all the email addresses of all the users on the site.

On the other hand, we have always encouraged using discussion forums rather than email for correspondance that can be of general interest. It happens very often that I see email exchanges that should really be in the Bboards. We need to be wary to not overuse email.

I rely on Fastmail.fm for email. FastMail.fm is the ultimate Hotmail killer. It is easily the best email service that I have seen (has POP and IMAP, virus and spam protection and loads of other features) and an extremely efficient and usable webmail interface. Fastmail also has a very active Discussion Forum of users and developers. In fact, Fastmail is so good that I wonder why anyone would go to the trouble of hosting their own webmail, unless there are some significant integration benefits that I don't know of.