Forum OpenACS Development: new spam package

1: new spam package

Posted by Yonatan Feldman on 05/09/02 08:05 PM

hi all, i have been working on a new spam package, i have begun coding of this package but would appreciate your feedback and guidance regarding its design.

first thing we did was decide to scrap acs-mail in favor of acs-mail-lite (which should really be called mail-queue, since that is all it is). acs-mail is too heavy weight and i would have had to hack it to do what we needed.

we decided to make spam itself a lot more lightweight. it does adds two pieces of functionality though: it is scoped, and it performs variable interpolation in the subject and body of the message.

it is not much more than that right now. it does simple scheduling of spam messages and at the appropriate time a sweeper proc pulls them out, performs the interpolation for each recepient and stuffs them in the acs-mail-lite queue. the spam system requires that the spam query contain an 'email' column. that, and any additional columns will be interpolated into the subject and body of the email. so for example if your spam query is:

    
      select parties.email,
                foo.bar
      from parties,
              foo
      where parties.party_id = foo.user_id

then the system will look for the strings {email} and {bar} and replace them with the values of those columns respectively for each row returned.

you can also select from_addr, subject, and message in the spam query and these values will be used instead of the spam supplied from_addr, subject, and message. this allows you to completely customize the sender, subject, and message on a per recepient basis.

2: Response to new spam package (response to 1)

Posted by Yonatan Feldman on 05/10/02 09:53 PM

oh, and did i forget to mention it is called bulk mail :)

3: Response to new spam package (response to 1)

Posted by John Mileham on 05/10/02 10:11 PM

It appears that you're talking about a pretty lightweight system. Would it support MIME/HTML email? I know that any "solicited bulk email" system that BerkleeMusic would actually end up using will have to support HTML, as well as (yes) inlined telltale images to track readership. The latter part sounds like it would be pretty easy to bolt on assuming the spam system you propose supports MIME email, though I wonder if the code would be more generally useful than just to BerkleeMusic? Whether you want to admit it or not... are your clients asking for this? If they've got marketing departments, the answer is almost certainly "yes". Without making any value judgement, this is an accepted marketing practice, and one that OACS should probably support if we hope to gain the acceptance of a broad spectrum of potential clients without having to hack the functionality on a one-off basis.

Of course, this brings up the obvious user preference issue over HTML v. text. I think a user should be able to pick text only notifications, but the scoping of that preference is probably a very touchy issue depending on the business logic of your site (particularly if your site uses the telltale image approach). Maybe it's safe to say that this pref could be global to all actual "bulk email" as, I assume, this package has nothing to do with bboard notifications, etc, and is more along the lines of newsletters and the like.

4: Response to new spam package (response to 1)

Posted by Yonatan Feldman on 05/10/02 10:45 PM

this spam package is fairly light weight and i would like to keep it that way. it
makes no assumptions about the content of the email, whether it is html or
plain text. this was by design so that you can build a smarter app on top of it
that can decide whether to send html or plain text or aol style email. at the
same time it is pretty flexible, as i mentioned above you can swap out the
entire body of the email on a per-recipient basis, so you can use this
knowledge to create a query that send html email to those who prefer it.

tracking users, via image hits or some other method, should definitely be
done as a higher level service/app. bulk-mail shouldn't know anything about
it.

5: Response to new spam package (response to 1)

Posted by John Mileham on 05/10/02 10:55 PM

That sounds right. Just makin sure. :) Any ideas yet on what the best way to bold functionality on top of the service will be? Just write apps that make use of the data model and API provided? Is there any need for callbacks (service contracts) or other more obscure hooks?

6: Response to new spam package (response to 1)

Posted by Yonatan Feldman on 05/10/02 11:12 PM

it has a pretty simple tcl api that currently looks like this:


    ad_proc -public new {
        {-package_id ""}
        {-send_date ""}
        {-date_format "YYYY MM DD HH24 MI SS"}
        {-from_addr:required} 
        {-subject ""}
        {-reply_to ""}
        {-extra_headers ""}
        {-message:required}
        {-query:required}
    } { 
        create a new bulk_mail message
        
        @param package_id which instance does this bulk_mail belong to
        @param send_date expects a date in the date_format format. if no date
                         is passed it will use the current date and time
        @param date_format how to parse send_date, defaults to
                           "YYYY MM DD HH24 MI SS"
        @param from_addr from email address to bulk_mail from, can be
                                       overridden by a value from the query
        @param subject subject of the bulk_mail message, can be overridden
                              by a value from the query. will be interpolated
                              with values from the query.
        @param reply_to reply_to email address for the bulk_mail, defaults to
                               the value of the from_addr parameter, can be
                               overridden by a value from the query.
        @param extra_headers a list of key, value pairs of extra headers to
                             stick in the email, such as 'X-Mailer' or
                             'Content-type'
        @param message the body of the email, can be overridden by a value
                       selected in the query. will be interpolated with values
                       from the query.
        @param query a query that must select the email address to send to as
                     'email' and can select any other values that will be
                     interpolated into the subject and message of the bulk_mail for
                     each recipient. if column 'foo' is selected it's value
                     will be interpolated anywhere that '{foo}' appears in the
                     subject or message of the bulk_mail. if columns 'from_addr',
                     'reply_to', 'subject', or 'message' are selected, their
                     respective values will also become the 'from_addr',
                     'reply_to', 'subject', and 'message' on a per recipient
                     basis.

        @return bulk_mail_id the id of the newly created bulk_mail message
    }

the pl/sql and pl/pgsql apis are a little bit less functional, basically, they don't default the reply_to address to the from address, don't default the package_id to the current package ([ad_conn package_id]), nor do they check for the presence of the 'email' column in the supplied query.

i haven't been able to come up with a use case for acs-service-contract or regular callbacks, but if necessary we can always add this later. as soon as i write the tcl pages (and somehow test the postgresql port) i will commit it to the openacs tree so anyone can check it out.

7: Response to new spam package (response to 1)

Posted by Yonatan Feldman on 05/10/02 11:16 PM

i forgot to modify the name of the proc for the purpose of the bboard. since i am a pretty heavy advocate of using tcl namespaces (and named parameters for that matter), the full name of the proc is actually bulk_mail::new

8: Response to new spam package (response to 1)

Posted by Yonatan Feldman on 05/16/02 12:16 AM

the first version of the bulk-mail module is now checked into the openacs tree
in openacs-4/packages/bulk-mail

enjoy