Forum OpenACS Q&A: email problem

Collapse
Posted by Claudio Pasolini on

One of our customers has the requirement of sending a special type of encrypted email, that in Italy is called certified electronic mail (PEC) and has the same legal validity of a signed mail.

I set up on a test server a Postfix configuration with relay host and tried to send both regular emails and PEC emails with the canonical acs_mail_lite::send, but the PEC email didn't work (it was received but not accepted, because the sender was not recognized as a valid PEC owner).

I then tried sending the PEC email with 'sendmail':
sendmail oasisoftware@registerpec.it oasisoftware@registerpec.it "Test body"
and it worked, meaning that Postfix was configured correctly.

I also tried these commands via /ds/shell:
package require smtp
package require mime
set body "Test body"
set subject "PEC test"
set token [mime::initialize -canonical text/plain -string $body]
smtp::sendmessage $token \
-header [list Subject $subject] \
-header [list From oasisoftware@registerpec.it] \
-header [list To oasisoftware@registerpec.it]
mime::finalize $tokenk
It also worked and I found that the only difference with the commands prepared by acs_mail_lite::send was the presence of the additional header 'originator', which is built by the following piece of code:
# Build the originator address to be used as enveloppe sender
set rcpt_id 0
if { [llength $to_addr] eq 1 } {
set rcpt_id [party::get_by_email -email $to_addr]
}
set rcpt_id [ad_decode $rcpt_id "" 0 $rcpt_id]

set originator [bounce_address -user_id $rcpt_id \
-package_id $package_id \
-message_id $message_id]

and the result is this string: bounce-0-F2DB00B0792250FEF01178507766F946BC03CF58-550@oasisoftware.it but the recipient doesn't like it.

I don't understand the logic behind the code, but to circumvent the problem I added this line:
set originator $from_addr
and it worked. I tried also sending some regular emails without any problems, but I wonder if problems could arise in different contexts.

Does someone see any potential problems?

Thanks,

Claudio

Collapse
2: Re: email problem (response to 1)
Posted by Benjamin Brink on
Hi Claudio,

Changing originator may break some emails that rely on the incoming or return email features, such as acs-mail-lite bounce handling, notifications bounce handling, and replies to forums posts.

I've been working on a an update to acs-mail-lite that is more general for incoming email.

As a result, outbound email is also modified. I think the use of 'originator' is also different and more in line with rfc recommendations.

I don't remember exactly how originator is changed at the moment, because I essentially finished the code a couple of months ago (and am waiting for feedback).

While waiting, I am working on an api for building apps that will make testing more complex components much easier.

I'd be curious if this version works for you: github.com/tekbasse/acs-mail-lite

Please test only on a development system, as it has *not* yet been reviewed by openacs core developers.

If it does not work, perhaps we can find out exactly what is triggering the issue with PEC, so that changes can be made to acs-mail-lite to make it PEC friendly at the same time.

best wishes,
Ben

Collapse
3: Re: email problem (response to 1)
Posted by Gustaf Neumann on

The bounce address is used for bounce handling, so it depends, how important the OpenACS built-in bounce handling is in your application.

  • one possibility would be to configure a parameter "BounceToSender", which could deactivate the OpenACS bounce handling without much hacking around.

  • another possibility would be to use a custom mail sending proc, in case, you want to send this kind of mail just from one certain application/page/... one could provide an additional parameter to send_immediately for this purpose.

  • a next possibility would be to make postfix accepting the bounce address, which is also configurable via the package parameter EnvelopePrefix of acs-mail-lite, which defines the result of bounce_prefix (see below).

Probably, there are many more options.
-g

    d_proc -public bounce_address {
        -user_id:required
        -package_id:required
        -message_id:required
    } {
        Composes a bounce address
        @option user_id user_id of the mail recipient
        @option package_id package_id of the mail sending package
        (needed to call package-specific code to deal with bounces)
        @option message_id message-id of the mail
        @return bounce address
    } {
        return "[bounce_prefix]-$user_id-[ns_sha1 $message_id]-$package_id@[address_domain]"
    }
Collapse
4: Re: email problem (response to 1)
Posted by Claudio Pasolini on
Thank you for the advice, Benjamin and Gustaf.

I'll try your code and let you know, Benjamin.

Gustaf, for the moment I'll stick with the second possibility. The first two should work, but potentially could break something as suggested by Benjamin, while the third shouldn't work, because a PEC recipient requires a PEC sender.

Collapse
5: Re: email problem (response to 1)
Posted by Claudio Pasolini on
Hi Benjamin,

I tried your acs-mail-lite on oacs-5-9 but I get this error:

can't read "message_id": no such variable
while executing
"acs_mail_lite::unique_id_create -message_id $message_id -package_id $package_id -party_id $rcpt_id"
(procedure "::nsf::procs::acs_mail_lite::send_immediately" line 34)
invoked from within
"acs_mail_lite::send_immediately -to_addr $to_addr -cc_addr $cc_addr -bcc_addr $bcc_addr -from_addr $from_addr -reply_to $reply_to -subject $subj..."

Where should message_id come from?

Claudio

Collapse
6: Re: email problem (response to 1)
Posted by Benjamin Brink on
Hi Claudio,

The quick answer is to remove the parameter '-message_id $message_id ' in the invocation of acs_mail_lite::unique_id_create

I believe that is all (and reload via acs-admin/apm). It will take a few hours for me to formally test to confirm this.

best wishes,
Ben

Collapse
7: Re: email problem (response to 1)
Posted by Benjamin Brink on

Hi Claudio,

Yeah, removing the offending message_id line will work. Or, the actual fix I've posted is to replace both message_ids with object_id. That was what was intended:

github.com/tekbasse/acs-mail-lite/commit/e5ac8b9184cde48ce52bc39b20783b722bec885e

When I test, no originator header is produced. Here is an example:

set body "Test body [clock seconds]"
set subject "PEC test"

acs_mail_lite::send_immediately -to_addr $to_addr -from_addr $from_addr -reply_to $reply_to -subject $subject -body $body

email headers:

Return-Path: <bounce861.9194530178.9-100+1519452391+d961966e2f5ead65629bd33dd655c575befb9e96@a22....>
Received: from ... ([unix socket])
	 by ... (Cyrus 2.5.11) with LMTPA;
	 Sat, 17 Feb 2018 06:14:10 +0000
X-Sieve: CMU Sieve 2.4
Received: from localhost (unknown [n.n.n.n])
	by ... (Postfix) with ESMTP id 1891822274C
	for <admin@...>; Sat, 17 Feb 2018 06:14:10 +0000 (UTC)
Received: from ... ([n.n.n.n])
 by localhost (m...org [n.n.n.n]) (maiad, port 10240)
 with ESMTP id 28819-09 for <admin@...>;
 Sat, 17 Feb 2018 06:14:09 +0000 (UTC)
X-Greylist: delayed 00:07:37 by SQLgrey-1.8.0
Received: from a22.... (a22.... [n.n.n.n])
	by ... (Postfix) with ESMTP id C34A4222741
	for <admin@...>; Sat, 17 Feb 2018 06:14:04 +0000 (UTC)
Received: from localhost (localhost [IPv6:::1])
	by a22.... (Postfix) with ESMTP id 056363A733C
	for <admin@...>; Sat, 17 Feb 2018 01:06:32 -0500 (EST)
MIME-Version: 1.0
Content-ID: <861.1518847592.10@a22...>
message-id: <bounce861.9194530178.9-100+1519452391+D961966E2F5EAD65629BD33DD655C575BEFB9E96@a22...>
date: Sat, 17 Feb 2018 01:06:32 -0500
Subject: =?UTF-8?Q?PEC_test?=
From: bb@a22
Reply-To: bb@a22
To: admin@...
Sender: bb@a22
Content-Type: text/plain;
              charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Test body 1518847591
Collapse
8: Re: email problem (response to 1)
Posted by Benjamin Brink on
Thank you Claudio for catching this bug.

All the new procs have unit tests. Looks like there should be tests made for the existing ones, too. 🤔

best wishes,
Ben

Collapse
9: Re: email problem (response to 1)
Posted by Claudio Pasolini on
Hi Benjamin,

I applied your fix and now regular emails are sent correctly, but not the PEC ones. For these to work I need the originator set to $from_addr and so, for the moment, I'll stick with my customized proc.

All the best,

Claudio

Collapse
10: Re: email problem (response to 1)
Posted by Benjamin Brink on
Hi Claudio,

I don't see an originator requirement in rfc6109 section 2.1.1.2[1].

Importing to a PEC system *seems* to require a minimum amount of consistency of fundamental email structure, consistent with standard email protocol.

I understand the importance of not adding feature creep to a package.
Still, I am up to helping identify a root cause issue if there is one related to a core OpenACS package. The issue may interfere with other systems as well. If so, it would be nice to preemptively provide a general fix sooner than later.

If you (or anyone) is interested in pursuing this and has a PEC system to test against, a question to ask is:

Might there be something else that is triggering the rejection, besides originator?

1. https://tools.ietf.org/html/rfc6109#section-2.1.1.2

cheers,
Ben

Collapse
11: Re: email problem (response to 1)
Posted by Claudio Pasolini on
Thank you for the pointer, Benjamin: I'll take the time to investigate further.

Claudio

Collapse
12: Re: email problem (response to 1)
Posted by Benjamin Brink on
Hi Claudio,

Here's what I've found out.

Originator can be From, Sender, or Reply-To according to rfc2822 section 3.6.2. Given the variety of ways acs-mail-lite is deployed, and possible fall-back values, it seems reasonable to defer the decision of setting its value to sitewide admins.

Subsequently, a package parameter will be set to the current default, with decision tree for practical fallback values determined by switch. It looks like this could reduce the likelihood of emails originating from OpenACS sites becoming stuck in spam filters.

I'm testing now, and will include with some other edits recently added to head version for next release.

best wishes,
Ben

Collapse
13: Re: email problem (response to 1)
Posted by Claudio Pasolini on
Hi Ben,

the idea of adding a new package parameter seems good to me.

Cheers,

Claudio