Forum OpenACS Development: Problem with notifications from Bug-Tracker

After installing Bug Tracker, entering a ticket and subscribing to notifications I started to receive an email roughly every minute.

A study of the error log indicated that this was because an error was preventing the acs_mail_lite_queue being cleared. So the message was going out but a catch was logging an error and not deleting the message from the queue. Thus every sweep, the message goes out again.

I have spent about three hours now trawling the code to work out what is going on and can't get to the bottom of it.

Here is the error:

[25/Jul/2009:13:05:20][1791.81926][-sched:11-] Notice: dbinit: sql(localhost::webteamworks): '
            select
                  message_id as id,
                  creation_date,
                  locking_server,
                  to_addr,
                  cc_addr,
                  bcc_addr,
                  from_addr,
                  reply_to,
                  subject,
                  body,
                  package_id,
                  file_ids,
                  mime_type,
                  (case when no_callback_p = TRUE then 1 else 0 end) as no_callback_p,
                  extraheaders,
                  (case when use_sender_p = TRUE then 1 else 0 end) as use_sender_p
            from acs_mail_lite_queue
            where locking_server = '' or locking_server is NULL
        '
[25/Jul/2009:13:05:20][1791.81926][-sched:11-] Error: Error while sending queued mail: can't read "from_party_id": no such variable
[25/Jul/2009:13:05:20][1791.81926][-sched:11-] Error: Error while sending queued mail: can't read "from_party_id": no such variable

After nearly five hours of studying the code I have tracked the source of the log entry to the following in mail-tracking-callback-procs.tcl :

ad_proc -public -callback acs_mail_lite::send -impl mail_tracking {
    {-package_id:required}
    {-from_party_id:required}
    {-to_party_id:required}
    {-body:required}
    {-message_id:required}
    {-subject:required}
} {
    create a new entry in the mail tracking table
} {

    set log_id [mail_tracking::new -package_id $package_id \
            -sender_id $from_party_id \
            -recipient_ids $to_party_id \
            -body $body \
            -message_id $message_id \
            -subject $subject]

}

As -from_party_id is a required value I am not clear why I am getting a "can't read" error rather than having ad_proc baulk at the lack of a value being passed in.

Now I have to try :

i) Hard coding a value to see if that solves the problem.

ii) To find the calling proc to establish why the value doesn't get passed in.

Richard

Collapse
Posted by Richard Hamilton on
Ok, at last I think I've found it.

In notifications-email-procs-tcl the proc

ad_proc -public send

calls acs-mail-lite::send

as follows:

    acs_mail_lite::send \
          -to_addr $email \
          -from_addr $from_email \
          -reply_to $reply_to \
          -mime_type $mime_type \
          -subject $subject \
          -body $content \
          -file_ids $file_ids \
          -use_sender \
          -extraheaders $eh_list_of_lists
    }

I think that -from_addr

should be

-from_party_id

but I'm not sure whether the $from_email should be passed or the party_id!

Clearly the 'complex' procs can deal with both internal ids (and look up the email) and simple emails. But the non-complex code seems to be email only.

It looks to me as if recent mods to enhance acs-mail-lite have broken this.

Any guidance eagerly invited! Who is the maintainer for notifications?

Regards
Richard

Collapse
Posted by Dave Bauer on
The bug is in the mail-tracking package, which is apparently not updated to work with the latest acs-mail-lite package.

The problem is not in notifications, which calls the acs-mail-lite apis correctly.

Collapse
Posted by Richard Hamilton on
Dave,

After much forensic study (which wouldn't have taken anything like as long if I had already understood callbacks!) I have found the problem.

Of course , after finding it I then saw your post which would have saved me some time for sure! 😊

In mail-tracking-callback-procs.tcl.......

ad_proc -public -callback acs_mail_lite::send -impl mail_tracking {
    {-package_id:required}
    {-from_party_id:required}
    {-to_party_id:required}
    {-body:required}
    {-message_id:required}
    {-subject:required}
}

should be:

ad_proc -public -callback acs_mail_lite::send -impl mail_tracking {
    {-package_id:required}
    {-from_addr:required}
    {-to_party_id:required}
    {-body:required}
    {-message_id:required}
    {-subject:required}
}

consequently the call to :

set log_id [mail_tracking::new -package_id $package_id \
    -sender_id $from_party_id \
    -recipient_ids $to_party_id \
    -body $body \
    -message_id $message_id \
    -subject $subject]

}

should be:

set log_id [mail_tracking::new -package_id $package_id \
    -sender_id $from_addr \
    -recipient_ids $to_party_id \
    -body $body \
    -message_id $message_id \
    -subject $subject]

}

The silver lining is that I could now write a callback function, and I have also seen all the lovely little mime related calls in the mail API which will help me no end when I re-write my internal mail package to support attachments.

😊

Thanks Dave. Who is the package maintainer for mail-tracking?

R.

Collapse
Posted by Richard Hamilton on
Also:

The parameter:

-to_party_id: required

needs to be changed to:

-to_addr: required

and the variable $to_party_id changed accordingly.

R.

Collapse
Posted by Richard Hamilton on
Now we have a data model problem. The acs-mail-log table has an integer field for sender_id and so is choking on the email address.

My first instinct is to simply look up sender id using the passed in email address rather than make a data model change.

Can anyone think of an occasion when the sender of an email through acs-mail-lite might not have an acs-object ID?

R.

Collapse
Posted by Richard Hamilton on
The next issue is the inserts into acs_mail_log_recipient_map which are also choking.

I am fixing initially by doing party::get_by_email on each of the addresses that are passed in, but I suspect that this also needs to be done for the cc and bcc addresses.

I'll need to find a way to generate outgoing messages with lists of cc and bcc addressees as a test.

Any ideas?

R.

Collapse
Posted by Dave Bauer on
Yes, you'll need to call that proc to get the party_id. There may be cases where the addresses will not have party_ids. I guess that depends on whether any applications let you enter arbitraty addresses instead of using parties.

The old complex_send code did this, you can probably look in the CVS history in complex-procs for some ideas.

Collapse
Posted by Richard Hamilton on
Dave,

The complex_send code still exists. My reading of this is that the simple send process is only now invoked if the message is being sent my a user with a party_id, however I have no idea if this is indeed the case.

I can clearly see the potential for unregistered users to be able to send a message out, although in that case they should probably have to register. Even if they are simply asked for their email address I think the complex_send code would then handle it.

I really need to chat to someone who is close to that code. Who would be able to confirm or refute those assumptions?

R.

Collapse
Posted by Emmanuelle Raffenne on
Hi Richard,

Sorry for being late to this thread.

I'm the one who refactored acs-mail-lite code, so I guess I'm the maintainer since then.

Just to confirm Dave's answer, complex_send is not part of core since openacs 5.4.0. Since then, acs_mail_lite::send doesn't accept a party_id as sender anymore but an email address instead. Thus, an unregistered user can send out an email message.

The signature of the acs_mail_lite::send callback has been modified accordingly (i.e. accepting an email address instead of a party_id). I'm aware that other packages like mail-tracking have implementations for this callback that follow the old signature, but I didn't modify them since I'm not the maintainer and users may use code previous to 5.4.0 or local copy of complex_send. Anyhow, those changes have been described at the forums back then.

Collapse
Posted by Richard Hamilton on
Emmanuelle,

That's great - thanks for the information. That is really helpful and makes good sense.

On that basis then, all that is needed in each case is a check of whether the various arguments to be passed to the mail procs are integer keys or not. If so, look up the email address.

I guess in an ideal world, the mail procs would sort this out for themselves based on whatever value is passed, but the problem is as you say, that the signature of the call itself has changed.

Is it possible for a non-registered user to submit a bug?

I will modify my copy of the files relevant to the bug-tracker notifications and inevitably the mail tracking package but need to work out how to capture that and feed it back to the codebase. I presume I can checkout a single file and then commit it back in isolation?

On that score, does anyone know who the maintainer is for mail-tracking?

Collapse
Posted by Emmanuelle Raffenne on
Richard,

I'm not familiar with the bug tracker package as a developer. However as an user, I need to login to submit a bug, at least on this server.

acs-mail-lite::send won't accept user_id, even in an ideal world :), because 1. it's not responsible for looking up the email address for an user, 2. the sender would need to be a registered user in all cases.

Collapse
Posted by Dave Bauer on
This is changed in openacs 5.5. Complex send is definitely not part of OpenACS anymore.

Anyway the mail tracking package is the problem. That needs to be updated.