Forum OpenACS Q&A: ns_sendmail CR/LF lost in some messages

BACKGROUND
We switched to a new server a month ago. Specs:

  • RH Linux 7.0
  • AOLserver/3.2+ad12
  • Postfix
  • Postgresql 7.1.2

I have an admin email browser interface that allows superusers to send messages to over 11,000 users by looping thru the database and calling ns_sendmail each time.

PROBLEM
A few users have reported that our nice text message formatting is scrambled. This was never reported with our older system. It appears as if the CR/LF is Stripped from the messages so that messages like mine would be like the above run together:

...each time. PROBLEM A few users have reported ...

I never see this in my copies of messages.

Users copy the messages to me but I have yet to get the full message with headers to see exactly what they get. I'd look for a missing CR/LF or maybe somehow high bit characters being sent or or.... I've sent myself messages with high bit characters and they come thru fine... Here is some of the header.

Delivered-To: bob@greatestnetworker.com
Received: from AOLserver...
X-UIDL: BZ`"!FG`!!M7M"!E0X"!

=====================
Bob OConnor's Note

I am looking for suggestions as to why and how to fix it.

THANK YOU.
-Bob

Collapse
Posted by Bob OConnor on

Ok, I've checked the output from my admin interface using both ns_log Notice and the (PRE) tag for the body of the message and it is properly formatted with the CR / LF pair.

Yet, some recipients are getting messages where the CR/LF is removed replaced with a space and the longer lines have the a new CR/LF at the end. Other recipients get the message as expected....

Also, it's not the browser as I first suspected.

Is there some trick in formatting or content-type or something to force all email clients to respect plain text formatting?

I could take the sendmail.tcl file from the aolserver source and revise it from ns_sendmail to bob_sendmail and put it in /serverx/tcl IF i can come up with a solution....

Or is it something beyond this....

Thank you
-Bob

Collapse
Posted by Don Baccus on
I think you'll have to answer this on a e-mail client-by-client basis,
if you send them plain text and they mess with it, there's not much you can do.  All you can hope for, I think, is to see if the client has a "don't screw around with the message contents" option and that of course will be very client-specific.
Collapse
Posted by Jonathan Marsden on
If you need to check that the email is being sent the way you think, set up a sendmail (or Postfix) alias that points to a file, and send the message concerned to that address. You'll then get a file containing the message exactly the way sendmail (or postfix) received it. No email client will have got anywhere near it.

You can then examine that file with Emacs or with any hex dumping tools you prefer to make *sure* the body is what you think it is. You could even upload a copy of the file to new-file-storage for others to take a look at, too.

As Don said, if you are sending nice normal standards-compliant plain text email, and email client programs are messing with it, that is a problem that you can't solve at the sending end of things at all.

Collapse
Posted by Bob OConnor on

FOLLOW-UP:
Well, I think I found the solution. Yippi!

It turns out that it only affected Microsoft Outlook IMO Build 9.xxx, and not all of them...
A smaller group who I didn't study had every line double spaced.

I solved it by striping the return from the CR/LF pair in the message body before calling ns_sendmail:

regsub -all " " $body "" body

I don't know why this works... This problem was never reported when we were using sendmail but started showing up when we started using postfix. I tested from the command line by telnetting to the port with HELO etc and found NO problem. So then it occurred to me that the telnet window was only sending new lines... LF's.

Anyway, onto other stuff...