Forum OpenACS Q&A: openssl problem on Mac IE

Collapse
1: openssl problem on Mac IE
Posted by Titi Ala'ilima on 07/10/02 10:15 PM
I've run across an error that IE has trying to connect to openSSL. the supremely helpful error message is "Security failure. Data decryption error". It is apparently pretty well documented. I have the most up-to-date openssl and nsopenssl, and I only have SSLv2 turned on. Here's the relevant portion from my config: 

    # Config for nsopenssl 2.1
    ns_section "ns/server/$service_name/module/nsopenssl"

    # NSD-driven connections:
    ns_param ServerAddress $service_ip_address_ssl
    ns_param ServerPort $service_port_ssl
    ns_param ServerHostname $service_ip_name_ssl
    ns_param ServerCertFile "cert.pem"
    ns_param ServerKeyFile "key.pem"
    ns_param ServerProtocol {SSLv2}
    ns_param ServerCipherSuite
"ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"
    ns_param ServerSessionCache "False"
    ns_param ServerSessionCacheID "1"
    ns_param ServerSessionCacheSize "512"
    ns_param ServerSessionCacheTimeout "300"
    ns_param ServerPeerVerify "false"
    ns_param ServerPeerVerifyDepth "3"
    ns_param ServerCADir "/usr/share/ssl"
    ns_param ServerCAFile "cacert.pem"
    ns_param ServerTrace "false"

    # For listening and accepting SSL connections via Tcl/C API:
    ns_param SockServerCertFile "cert.pem"
    ns_param SockServerKeyFile "key.pem"
    ns_param SockServerProtocol {SSLv2}
    ns_param SockServerCipherSuite
"ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"
    ns_param SockServerSessionCache "False"
    ns_param SockServerSessionCacheID "2"
    ns_param SockServerSessionCacheSize "512"
    ns_param SockServerSessionCacheTimeout "300"
    ns_param SockServerPeerVerify "false"
    ns_param SockServerPeerVerifyDepth "3"
    ns_param SockServerCADir "ca"
    ns_param SockServerCAFile "ca.pem"
    ns_param SockServerTrace "false"

    # Outgoing SSL connections
    ns_param SockClientCertFile "cert.pem"
    ns_param SockClientKeyFile "key.pem"
    ns_param SockClientProtocol {SSLv2}
    ns_param SockClientCipherSuite
"ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"
    ns_param SockClientSessionCache "False"
    ns_param SockClientSessionCacheID "3"
    ns_param SockClientSessionCacheSize "512"
    ns_param SockClientSessionCacheTimeout "300"
    ns_param SockClientPeerVerify "true"
    ns_param SockClientPeerVerifyDepth "3"
    ns_param SockClientCADir "ca"
    ns_param SockClientCAFile "ca.pem"
    ns_param SockClientTrace "false"

    # Typically where you store your certificates
    # Defaults to $AOLSERVER/servers/${service_name}/modules/nsopenssl
    ns_param ModuleDir "$aol_home/servers/$service_name/modules/nsopenssl"

    # OpenSSL library support:
    ns_param RandomFile "/etc/inittab"
    ns_param SeedBytes "1024"

What am I missing?

Collapse
2: self-signed cert (response to 1)
Posted by Titi Ala'ilima on 07/10/02 10:20 PM
one more thing, i'm currently using a self-signed certificate.  i would expect that to fail more explicitly but if anyone knows if that's the/a problem, let me know.
Collapse
3: Response to openssl problem on Mac IE (response to 1)
Posted by Dan Wickstrom on 07/10/02 11:55 PM
Set ServerTrace to true to get more debugging info.

How did you generate your certificate authority certficate (cacert.pem)?  Your server certificate should be signed by the same certificate authority that created your certificate authority certificate.  It is pretty straightforward to become your own certificate authority if necessary for testing.

Collapse
5: My trace from the log (response to 1)
Posted by Titi Ala'ilima on 07/11/02 04:41 PM
I set ServerTrace "true" and got: 
[11/Jul/2002:10:36:53][4331.4101][-conn1-] Notice: nsopenssl: trace: nsdserver: before/accept initialization
[11/Jul/2002:10:36:53][4331.4101][-conn1-] Notice: nsopenssl: trace: nsdserver: before/accept initialization
[11/Jul/2002:10:36:53][4331.4101][-conn1-] Notice: nsopenssl: trace: nsdserver: SSLv3 read client hello A
[11/Jul/2002:10:36:53][4331.4101][-conn1-] Notice: nsopenssl: trace: nsdserver: SSLv3 write server hello A
[11/Jul/2002:10:36:53][4331.4101][-conn1-] Notice: nsopenssl: trace: nsdserver: SSLv3 write certificate A
[11/Jul/2002:10:36:53][4331.4101][-conn1-] Notice: nsopenssl: trace: nsdserver: SSLv3 write server done A
[11/Jul/2002:10:36:53][4331.4101][-conn1-] Notice: nsopenssl: trace: nsdserver: SSLv3 flush data
[11/Jul/2002:10:36:53][4331.4101][-conn1-] Notice: nsopenssl: trace: nsdserver: SSLv3 read client certificate A
[11/Jul/2002:10:36:53][4331.4101][-conn1-] Notice: nsopenssl: trace: nsdserver: SSLv3 read client certificate A
[11/Jul/2002:10:36:53][4331.4101][-conn1-] Notice: nsopenssl: trace: nsdserver: SSLv3 read client certificate A
[11/Jul/2002:10:36:53][4331.4101][-conn1-] Error: nsopenssl: error during SSL handshake: Connection reset by peer

At which point I realized it was using SSLv3 after all even though ServerProtocol was set to SSLv2. Adding SSLv3 to ServerProtocol was no help. So I got rid of that and added ":-SSLv3" to the end of ServerCipherSuite and got "Security Failure. The server reply is invalid." on Mac IE, while my log said: 

[11/Jul/2002:10:42:44][4397.8201][-conn5-] Notice: nsopenssl: trace: nsdserver: before/accept initialization
[11/Jul/2002:10:42:44][4397.8201][-conn5-] Notice: nsopenssl: trace: nsdserver: before/accept initialization
[11/Jul/2002:10:42:44][4397.8201][-conn5-] Notice: nsopenssl: trace: nsdserver: SSLv3 read client hello C; alert type = fatal; alert desc = handshake failure
[11/Jul/2002:10:42:44][4397.8201][-conn5-] Notice: nsopenssl: trace: nsdserver: SSLv3 read client hello C
[11/Jul/2002:10:42:44][4397.8201][-conn5-] Notice: nsopenssl: trace: nsdserver: SSLv3 read client hello C
[11/Jul/2002:10:42:44][4397.8201][-conn5-] Error: nsopenssl: error -1/1 during SSL handshake

Which leads me to believe the browser will only accept SSLv3 connections. So I need it to work under SSLv3. This is consistent with the mod_ssl FAQ explanation of the problem. But how might I implement their keepalive and HTTP version workaround in AOLserver?

One note: the first time I try to load the page from a fresh browser it complains about my certificate not coming from a recognized authority and it lets me continue but only by turning off encryption. Subsequent loads of the page give me the "decryption error", whether or not I allowed the first load to "continue" despite the unrecognized authority. I still don't know if this unrecognized authority is somehow at the root of the problem, and I won't know until I get a real certificate.

Collapse
6: Response to openssl problem on Mac IE (response to 1)
Posted by Dan Wickstrom on 07/12/02 05:52 AM
The error message about the certificate authority shouldn't be a problem, but who knows, each browser probably works differently.  To get rid of this warning message, you need to import the certificate authorities certificate into your browser.  For netscape, I converted the file to .der format and created a page to download it into my browser.  A similar process probably also applies to MSIE.  I'm sure a search on the web will turn up something on how to load the CA's certificate into your browser.

Another thing to check, is to make sure that the common name for the servers certificate matches the FQDN of your server.

Also in your config file above, why do you have ServerPeerVerify set to false?  In my config file, I have it set to true.

Collapse
7: Response to openssl problem on Mac IE (response to 1)
Posted by David Walker on 07/12/02 01:46 PM
Try 
ns_param ServerSessionCache true


I believe ServerPeerVerify controls whether nsopenssl requests a client certificate