Forum OpenACS Q&A: Email alerts grouped by topic

Does anyone else follow this bboard and those at aD via email? I find it hard to believe people really follow questions via the web -- AFAIK there's no way to tell what questions have responses I haven't read or for such questions which responses I haven't read other then through lots of clicking, scrolling, and eyeballing.

So I subscribe to this forum and several of those at aD via email alerts. This is ok, but due to the lack of In-Reply-To headers, and the values given to From: and Subject:, reading bboard responses in email is still a pain.

My solution is to accumulate hundreds or thousands of responses in a mbox, then when I feel like catching up on ACS happenings, run some regexps on the file so that my mail reader can group responses by question and display the author in From:.

Each of these makes reading much, much better: grouping by question allows me to easily concentrate on one question at a time, and delete a whole group of responses if they don't interest me with one ^d (using mutt). Showing the author in From: allows me to at a glance read the people who I know write interesting stuff and delete responses from people who aren't worth reading.

It would be trivial to have email alerts for bboard postings automatically generated with the necessary headers, which would save me from having to occasionally munge my mailbox and make it easier for everyone else to follow bboards via email as well:

I'd just post a patch, but unfortunately I don't know exactly what versions of [Open]ACS/bboard are running here and at aD, and it seems that different bboards are set up to generate slightly differently formatted alerts, so I'll describe generally what needs to be done in hope that people with access to the OpenACS and aD forums might make the necessary (trivial) changes:

Subject:

Currently, the subject line is formatted along the lines of

Subject: bboard_name: {Response to }question_subject

This should be changed to

Subject: {Re: }[bboard_name] question_subject

(I'm using curly braces to indicate optional fields, as [] should be used to enclose the bboard name following a commonly-used mailing list convention).

This change allows some mail readers to group messages by subject, appearing as a psuedo-thread. To get real threding, you'd need In-Repy-To:, and I'm assuming that may be harder, as I'm not sure at what point message-ids are generated.

From:

Currently all bboard response alerts are sent with From: set to "bboard@openacs.org" or "bboard-robot@arsdigita.com". This isn't done to protect the email address of the poster, as this information is included in Reply-To: (on aD, the email address and full name of the poster are also included in the message body). It also needn't be done to identify the email as originating from a bboard, as there are plenty of other markers of that in the header and in a trailer instructing the reader how to reply, etc.

How about just setting From: as follows:

From: "Full_Poster_Name" <poster_email_address>

Is this this silly? Should I be posting it elsewhere?

I find your suggestions very welcome. I posted this same question on the aD bboards a while back, but nothing was done (though there was positive feedback).

I find myself in these same frustrations... it's just hard to read a mailbox full of bboard postings the way they are setup right now, especially after you got used to mutt's excellent threaded interface. It's hard to follow posts and it's hard to know who is answering what.

It's also hard to post a response to someone's post when that post is not the original question. I find myself having to open two browser windows: one to post the response and another to look at the response and copy/paste. Or I copy/paste from my mutt window into the browser response window.

I could make a patch to address these issues on the client side (easy). But IMHO this should be addressed on the web interface side too (harder). For example, the user should be given an option of threaded or regular interface. And you should be able to post to a certain response within a thread, instead of just the original question.

It seems to me that (a) bboard is absolutely one of the core, core ACS modules, and (b) everybody agrees that it could use some enhancements in terms of email handling and other capabilities. The company I work for has invested some money into adding email enhancements to bboard already. Unfortunately, due to circumstances that aren't worth going into, these enhancements were added to a highly non-standard (and somewhat ugly) version of bboard in an ACS Classic 3.4 installation. At some point in the future, I anticipate we will able to invest more money into beefing up the email and other capabilities in the OpenACS 4.x bboard.

What I'm getting at here is that there seems to be a lot of interest in bboard right now and it might be worth setting up a forum devoted to it, much the way CMS has its own forum. Has anyone yet volunteered to be the lead on porting bboard?

I'd like to see much of the ACS (but bboard especially) unified with a mailing list manager.

I have found that there are folks that only want to interact with the bboards through email, much as they do their current mailing lists.  And yet, uniformly, mailing list managers out there today all have horrid and very weak web interfaces, and don't let you post new messages.  I've also heard complaints/rumours/suspicion, etc. that ACS mailing list is great compared to the a mailing list manager specifically written to batch up emails, etc., and of course as we all know, the ACS does a particularly poor job at creating the headers our mail user agents (and us) like to see for threading, filtering, (and the removal of vacation messages.)

A unification would benefit both the ACS and mailing lists.

A few weeks ago I looked into integration with Mailman.  Mailman is opensource and reasonably popular.  I discovered then that an employee of GreatBridge is working on a mailman/postgres integration, and I am waiting for that...

The biggest disadvantage is that it would however create a python dependency on the ACS.  (Which now requires Tcl and Perl.)

Anyone interested in pursing this?

It's certainly true that the ACS could be better integrated with qmail (and simply too), regardless of any other effort.  Simply setting a few headers correctly would help identify to *us* who is writing a message, and would direct many vacation programs to not spam us all with the travel news of our various community friends.

But I believe integration of bboard with a maillist program has a wonderful benefit.  It gets the ACS into communities that have members that ONLY use email and won't otherwise get on the net.  That could be existing, "stodgy" communities, it could be folks with mobile email access (pager, pda, or phone/vxml), or it could be folks buying new, email-only devices to attach to their phone systems.

This means supporting people that never want to (or can) click on a link.  That's supporting subscribe/unsubscribe via email.  Creating a new thread via email, and replying to an old thread.  It means supporting these people by letting them choose instant responses or digests, etc., again, without requiring them to come back to a website.

That's what I'd like to see.

This does come up alot.  (But not much has ever happened, right?)

Just to be precise, I think we're talking about two different things.

Mailman doesn't take over qmail's job.  It's not an SMTP mailer.  Mailman is more of a combination procmail/acs-lite targeting subscription/operation/distribution/digestification(yum!) of emails.

Mailman can use qmail, and if bboard were to use mailman (or a new bboardman that is) it doesn't mean that the rest of the ACS should not be better integrated with qmail (which it should be).  It's just saying that it would be nice to be able to let users manage bboard properties (including creating and replying to messages) entirely through the use of email.

This could certainly be done with qmail integration directly.  To do so would mean learning and reimplementing what other mailing list manager programs have been doing for years.  It would simplify ACS requirements to do it directly, as sites would not need Python.  It would probably not be a simple or quick task.  I think the quick task is to integrate with Mailman, especially after (this guy from)GreatBridge finishes the Mailman/PostgreSQL integration effort.  When that happens, presumably, what one would need to do is just to populate certain PostgreSQL tables with new web based posts and read from various tables to determine new email based posts.

Put another way, I would trust myself to integrate with Mailman properly.  I wouldn't trust myself to better integrate with qmail in a manner that would mean that users could post new messages that get distributed to other users AND I wouldn't get called at 3am with calls of how my errant bboard/qmail integration was responsible for a mail loop/DOS/imminent collapse of the net.

I believe identifying posters is trivial, threading is simple, creating new topics is hard, and getting rid of quoted nonsense and signatures is downright difficult.

Posters are identified with their email address which is typically the same as their ACS user identifier. Threading is relatively easy by encoding the thread ID into the reply-to field.

What I think is significantly more difficult is to create a simple way to create new threads. What I fear is:

One, instead of having (relatively( nice, clean threads as we have now, sites will have threads that meander, get off topic, chase two or more topics at a time, etc., as users just hit reply and begin their new topic (as I tried to emulate by changing the subject of this posting.) (And I suspect the answer is in SF since he's supposed to be in Burlingame tomorrow morning.)

Two, how does one best handle replies? Distribution lists angle brackets, right?

> One concern I'd have is the degree to which we can capture bboard web
> input from an incoming email. Can an
> email provide an adequate interface to let users indicate whether they
> are replying to a post or its parent?

>> And what about identifying posters? One of the great things about
>> bboard is that it links all your posts to your profile. We'd want
>>> incoming emails to be reliably recognized by bboard as coming from a
>>> specific registered user. In ACES, at least, making this work required
> some thought. Basically, (as I understand it), the ACS generated a
> unique "reply-to" email address for each user receiving an email alert
> for each post. The system could reliably identify the users because
> only one user could possibly be replying to a given email
> address. Will it be possible to make something like this work with a
> third-party listserver?

Well you get the idea.

Some of this can be attacked by user education much the same way that (as jim Zamboob as been recently educated), a bit of moderation, and the creation of easy ways of doing the *right* thing.

Some can be attacked by semi-intelligent software that knows how to remove signatures, and remove quoted portions.

But I bet such a system will need some tuning. If achievable, I think it would be valuable.

I would have to with Jerry.  When one makes it too easy to reply, the relevance of the messages to the thread would suffer.  I believe this was mentioned somewhere on Phil's website.

I think it would be most effective set the References:, In-Reply_to and Message-ID accordingly.  Isn't the Message-ID set by the MTA? wouldn't a patch to the MTA be needed to put that back (or generated eternally to the MTA) in the bboard  so replies can be threaded? properly?