Forum OpenACS Development: notifications - is there a need for a new package?

Collapse
1: notifications - is there a need for a new package?
Posted by Ben Adida on 05/10/02 05:29 PM
Hi all,

bboard, like other packages, has the concept of user notifications.
Some of these notifications make sense to send instantaneously and in
a scoped manner (you only get a notification about bboard changes).
Other notifications make sense to send in a grouped manner, both in
time and scope.

I haven't completely followed the latest acs-mail/acs-messaging
developments: has anyone worked on a generic notifications package?
This would aggregate all notifications for each user, batching them
when necessary, and tying into user preferences for speed of delivery
and eventually method (email, IM, etc...).

Thoughts?

Collapse
2: Response to notifications - is there a need for a new package? (response to 1)
Posted by Talli Somekh on 05/10/02 05:32 PM
Ben this is a great idea, as it has implications for applications like CRM and calendaring. It would certainly be nice to have something like this.

talli

Collapse
3: Response to notifications - is there a need for a new package? (response to 1)
Posted by Don Baccus on 05/10/02 06:32 PM
Yes, it would be a good thing.  I've thought about it in the past but haven't done anything about it.  I've thought along the lines of a service contract-based package that would let other packages register themselves with it for alerts managment.  I think the trickiest issue is providing a reasonable UI.  People want so many options for configuring alerts!  And so many packages will be more than happy to alert them.  If we're not careful a UI that works well for a couple of packages may become unmanageable for a dozen using teh service.
Collapse
4: Response to notifications - is there a need for a new package? (response to 1)
Posted by Caroline Meeks on 05/13/02 08:55 PM
I think this is a great idea.  Here are some use cases to help with design.

Survey
-When a new response is made some admin want to be notified.
-There is a demand for a "feedback form" where the results will be emailed to the person being evaluated.

File Storage-
I know students would like to be able to choose to be notified when "comments" are uploaded on their homework.
Yahoo provides notification when a new file is upload, people may want that but they would need to control it.

News-
Not sure if Sloan wants it yet but numerous other clients have asked to have news emailed.

Calendar-
Again email reminders are a fairly standard feature people expect.

Group Management -
Sloan needs emails when people request membership to a group and then a confirmation email when they are approved/denied.

LMS-
As we move towards a full LMS we will probably want something like workflow, where students are assigned tasks and various TA/Profs are notified and perhaps assigned tasks. Here is a random example to illustrate the potential complexity. example..student reads a section and then is assigned as problem set. When the problem set is uploaded the TA is notified and assigned a task to grade it.  When the student completes an entire section he is given an Exam and the completed exam is assigned to the professor for grading. Based on the score the student is assigned a remedial section.

This is not an issue for now, but it brings up the question of how a new notifications package would interact with workflow package. Will workflow use general notifications?

Collapse
5: Response to notifications - is there a need for a new package? (response to 1)
Posted by Malte Sussdorff on 05/15/02 01:07 PM
We had this issue up for quite some time and our students came up with some ideas and first very basic scripts. UI is the most crucial issue here, though using various forms of notification does not make the task easier. There has been some discussion going on already in this area, so I think there is some need for a package like this. Anyway, here is the software: https://openacs.org/new-file-storage/one-file.tcl?file_id=367
Collapse
6: Response to notifications - is there a need for a new package? (response to 1)
Posted by Ben Adida on 05/20/02 06:07 PM
Malte,

As a clarification, is your students' work available under the GPL? Under some other license? Good to see these contributions!

Collapse
7: Response to notifications - is there a need for a new package? (response to 1)
Posted by Ben Adida on 05/21/02 03:10 AM
I've put up a high-level spec of notifications at

http://dotlrn.openforce.net/doc/notifications.adp

Noting that I am trying to keep things simple for this first iteration, I would like to get people's comments about the direction I'm proposing. I expect to do a very quick first implementation, so the earlier you can comment on this, the better!

Collapse
8: Response to notifications - is there a need for a new package? (response to 1)
Posted by defunct defunct on 05/21/02 12:43 PM
Ben,

Completely agree that somehting like this is necessary. A single, simple mechanism for delivering notifications.

I'm also pleased to see you're taking other messaging forms into consideration (even if not really supported) such as SMS and IM. Given that these are fast becoming de-facto text standards its vital the OACS has this in it toolkit. Interestingly SMS usage in the UK is now higher than email (apparently ?)...

I'd very much appreciate any early views of what your developing, as we may well be able to contribute in some extensions for SMS and so forth.

However, I'm not entirely convinced by the proposed development. Don't get me wrong cos I can't really think of an alternative, but my gut feeling is that this approach (tabled storage based messaging) might have some issues of scale. One of the biggest headaches we have had in the past is the ability of large volumes of email to swamp the performance of the system.

I know I'm largely discussing implementation here rather than design, but it my pet bugbear. Whatever solution is adopted would make me feel much happier if it put performance and volume at the top of its list.

Also, what about serialisation of messages? You seem to be tacitly assuming that messages are delivered in asynchronous fashion. Is there any case for offering some form of serialised delivery?

I'm thinking of a specific situation (and yes, its cellular phones again). When connecting to some systems (SMSC in particular) the general protocol forces a serialised, acknowledgement based interaction. It therefore becomes important for us to serialise.

So, I guess the next thing you;d say is, well that doesn't really belong in this package, but rather whatever handles SMS delivery.. True I suppose, but I don't think this can be entirely limited to SMS.

In summary then, I personally think some kind of facility to support ordering and queueing might be considered.

And does the same point apply to message delivery and re-transmission? Should this package offer that?

Just a few thoughts. I agree with your general theme Ben, in that I too think packages should be simple building blocks, but the danger is that if they are too simple, or funtionaly thin, then the overhead of learning/deploying them to acheive an overall goal becomes onerous and people will favour custom, ns_sendmail style approaches.

For example, the original Spam package, although never completely finished, was a good example of simple function, small package. However given that it did so little, it was almost never worth using it. Far easier to implement direct.

Cheers

Simon

Collapse
9: Response to notifications - is there a need for a new package? (response to 1)
Posted by Ben Adida on 05/21/02 04:45 PM
Simon,

Thanks for your comments. If anyone else has comments,
please send them along!

seeing what we're developing: since this is a core package, I'm
going to check in my development to the OpenACS tree, so you'll
be able to see this live. I tend to commit files quite often. If you
have significant time to help out, I'm happy to get your help. I will
be working on this very actively in the next couple of days.

scalability: I beg to differ - scalability has to be by design, not only
by implementation! Don't downplay your point,  I think the issue
you raise is very important. Here's the way I see it, and you can
tell me if you agree:

- we need reliable messaging. Reliability, in every OpenACS
application, has been implemented using transactional control
in the DB. That's why we don't do important data storage in RAM.
Do we really want to start implementing reliable messaging
without transactional control?

- a notification message is kept only as long as it needs to be
kept for delivery purposes. If the message needs to be kept for a
week in the case that someone has only weekly notifications, I
can't imagine storing this message in anything *other* than
permanent storage.

- if there is a huge volume of notifications, it seems highly
unlikely that an application will want to offer weekly notifications:
that would be one extremely long notification at the end of the
week!

Thus, in practice, I think that the only time notification messages
will be kept for a while is when the content of the message
notification is already being stored somewhere else (i.e. a
bboard posting). Thus, in the worst of cases, we are temporarily
doubling our storage requirement. Certainly very noticeable, but I
wouldn't call this a scaling problem.

Delivery, retransmission, and synchronous operation. Simon,
I'm glad we have you on board to tell us about SMS. That's
important stuff. As you mentioned, I think these issues are best
resolved by the specific delivery code. The reliability of the
notifications package *prior* to delivery is ensured by the
RDBMS. The protocol for actually performing the SMS
communication should be left to the SMS package. If you need to
do it synchronously, that's no problem. If you need to do retries,
retransmits, and only delete messages from the queue once
you receive an ack, you can do that. If SMS and another protocol
require the same kind of constructs, then building another
subcomponent might be useful. However, I can't imagine this
being a feature to build into the notifications core. That seems
like too much bloat, especially when email (the biggest current
use of this) doesn't require these features.

The goal of notifications is to do:

- easy targetting of recipients

- batching

- formatting

- centralized preference setting and management for users

I think if we accomplish these things, we've got a solid module
that will motivate usage. I certainly don't want to claim that
notifications will do all SMS stuff flawlessly out of the box. I
expect you guys at OpenMSG to build that extension, since you're
the pros :)

Collapse
10: Response to notifications - is there a need for a new package? (response to 1)
Posted by Ben Adida on 05/23/02 06:43 PM
Ahah, an interesting complication about general notifications. What to do if we want to allow bboard email responses? responses in general?

Some initial thoughts:

- responses to batched notifications don't make sense. I'm proposing to ignore these for now.

- responses to single item notifications make sense. The method for responding depends on the method for delivery. Thus, the notification API should take in information relating to how one might respond.

Any additional thoughts?

Collapse
11: Response to notifications - is there a need for a new package? (response to 1)
Posted by defunct defunct on 05/24/02 12:45 PM
Ben,

I'm not sure what you mean by 'responses to batches don't make sense'..

I would certainly want to be able to create some sort of batch (bulk) mail request which might, say. send an email to every user registered at that time. I might then want to tie up responses from those same recipients. If we're talking about the same thing here then I disagree and think the notion of batch response should exist; perhaps even the notion of a response which is the point at which the entire recipient list has responded (formal poll for example)... We've had some similar situations with SMS, so the two here are interchangeable..

Also, I assume your second statement means that it shuld be possible to have potentially any type of delivery response, regardless of original delivery method. i.e. it should be possible to send out an SMS and for a recipient to repsond with an email. This is *defiantely* a requirement, so I'm just confirming thats what you mean.

Cheers
Simon

Collapse
12: Response to notifications - is there a need for a new package? (response to 1)
Posted by Ben Adida on 05/24/02 03:31 PM
Simon,

Sorry for being unclear. Let me explain it more clearly.

If I am subscribed to bboard, and I get an instant notification of
all new messages, then it's quite easy to logically conclude what
should happen when I click "reply" in my mail browser.

However, if I have batched notifications, I may receive one email
with multiple postings that are in different threads. At some
point, I might even receive one email with bboard *and* calendar
notifications. At that point, clicking "reply" in my mail browser
makes no sense. If there is a way to reply to each notification, it
should appear in the text itself of the notification.

Thus, the issue I'm trying to address here is delivery-specific
replies to general notifications: what happens when you use
your delivery mechanism's built-in reply method (as opposed to
looking at the content of the notification for a means of replying).

For the time being, I'm thinking of solving this quickly for email
and instant notifications only.

Collapse
13: Notifcations: update to the design document (response to 1)
Posted by Ben Adida on 05/24/02 03:59 PM
I've updated the design document. Specifically the first paragraph in "Under The Hood".

http:// dotlrn.openforce.net/doc/notifications.adp

In fact, a notification type is an OpenACS object type. The root type of all notifications types is notification_request, because a notification type actually defines the format of what a notification request might look like.

This type/instance stuff can get confusing in the relational model, so feel free to ask me questions.

Collapse
14: One more modification (response to 1)
Posted by Ben Adida on 05/24/02 04:06 PM
One more modification on its way as I just realized it: I don't want
to force packages to have to create their own
notification_request object type, as that would be too much
overhead to start using the notification system.

Thus, notification_types will *not* be a parallel of acs_objects.
You will be able to subtype notification_request if so needed, but
that won't be necessary to use the notifications package.

Collapse
15: Response to notifications - is there a need for a new package? (response to 1)
Posted by Bjorn Thor Jonsson on 05/24/02 11:07 PM
Do you think it makes sense to have the frequency of notifications adjustable on a per notification request basis, as opposed to on a party-wide basis? And also to have the combination of frequencies totally flexible within each request? That is, should one for example be able to batch notifications from threads A, B and C, and also be able to have immediate notifications from thread B (so effectively being notified more than once of the same object)?

Should the definition of frequencies be flexible, or should they be fixed to like daily-, weekly- and monthly summary?

I'm very interested in this, as I'm working on a package that is about nothing but notifications. It is supposed to monitor various categories at various related websites for additions of new material, users will be able to subscribe to the various categories and choose any combination of (fixed) notification intervals they like for each category.

Since I saw this thread, I've been wondering if my package might be an ideal contractor of the Notifications package? Before, I had been planning to hack up something specific for this package (Pind's Rule of Five comes to mind). But it sounds good to not have to "slap your forehead when you note that you've reimplemented bboard notifications".

If you think I'd be doing right by contracting the Notifications package, and if you think a newbie oacs package developer can be of any help, then I'm all for it, if for nothing more than testing.
Collapse
16: Response to notifications - is there a need for a new package? (response to 1)
Posted by Ben Adida on 05/24/02 11:15 PM
I think having frequency adjustable by notification request is fairly
important, simply because there are some things that are more
urgent than others. For example, I want to see posts to an
important design thread right away, but I want information about
OpenACS social activities batched daily (assuming no one
organizes a party on 2 hours' notice).

As for overlapping notifications: I don't see any issue with double
notifications if the user so requests it. The "smarts" for not
overlapping notifications - if necessary - should be in the
application itself (forums), not the generic code (notifications).

Although, your argument does point out a flaw in my current
implementation plan, since I've proposed to track whether a user
has received a particular notification, not specifically tied to a
request. Thus, my current implementation plan inherently
removes duplicate notifications... and I'm not sure that's the right
thing (although i can see the argument for this being the magic
bullet).

Anyways, more thought on this is needed!

As for testing and such: I've *just* imported my first pass at the
code. It's by no means fully functional, but the first data model,
PL/SQL, and Tcl APIs are there. It's called notifications in the dev
branch of the OpenACS tree. It's Oracle-only for now, but I will
definitely be porting to PostgreSQL (since I also need it for PG).