Forum OpenACS Development: ad_conn peeraddr

Collapse
Posted by Malte Sussdorff on
I run AOLserver behind a proxy (NGINX) which uses X-Real-IP and X-Forwarded-For headers to communicate to AOLServer what the real IP address is. AOLserver uses the X-Forwarded-For information just as Apache to write the correct IP Address into the logfile.

Sadly this has no effect on ns_conn peeraddr.

Therefore I wrote this tiny patch to ad_conn which get's me the real IP Address based on the X-Forwarded-For.

Question: Is there a different way than patching ad_conn peeraddr to actually make use of this standard (see https://en.wikipedia.org/wiki/X-Forwarded-For)?

If there is not I will TIP this patch. I need it in any case because all our servers run now behind NGINX and therefore it will make it into all of them 😊.

Collapse
2: Re: ad_conn peeraddr (response to 1)
Posted by Gustaf Neumann on
i would not recommend to patch ad_conn, but instead to initialize it differently, when a certain configuration parameter is used. We introduced ReverseProxyMode and check its setting in the request-processor. We use this since a few years without troubles.
ad_conn -set peer_addr [ns_conn peeraddr]

if { [ns_config -bool ns/parameters ReverseProxyMode 0] } {
  set addr [lindex [ns_set iget [ns_conn headers] x-forwarded-for] end]
    if {[string length $addr] > 0} {
        ad_conn -set peeraddr $addr
   }    
} 
Collapse
3: Re: ad_conn peeraddr (response to 2)
Posted by Tom Jackson on
This is a bug in AOLserver, or a limitation. There are two pieces of information. One is the ip address which AOLserver is connected to called the peeraddr in ns_conn. The other is the client ip. AOLserver uses the peeraddr for ns_conn and logging to the access.log file. It is ignorant of the client ip, but sometimes they are the same. If you overwrite the peeraddr with the client ip, it is also a bug. You should probably, for long term stability, create a new name for the new information. Then if you compare the two, you can always tell if there was a proxy or not.

I also wonder how this interacts with SSL. Doesn't SSL have to go straight through, or how does that work?

Collapse
4: Re: ad_conn peeraddr (response to 3)
Posted by Gustaf Neumann on
We use the code above with SSL via pound (client connects to pound via SSL, pound connects to backend via plain HTTP). Pound uses different backends for different tasks. So, the only complete log file is the one provided by pound, which contains as well the correct IP addresses of the clients.

On the backend side everything using ad_conn reports the client ip address as "ad_conn peeraddr" (e.g. request monitor, creation_ip in acs_objects, etc.). Since all but developer traffic (via VLAN) is routed via the reverse proxy, recording the ip address of the proxy is certainly not useful. I would call it rather a bug, seeing always the proxy's ip address, where the code-writer had obviously the intention to report the client IP address. In the parts of OpenACS we are using, we found no place, where having the proxy as the peer address makes sense.

However, I do agree, that in principle one should have two fields, such as peer_addr and client_addr, and that the usages should be changed from "ad_conn peeraddr" to "ad_conn clientaddr". However, the change above is less invasive, and one has actually always both info at hand: one can use still "ns_conn peeraddr" for the bare-bone info.

Collapse
6: Re: ad_conn peeraddr (response to 4)
Posted by Malte Sussdorff on
Gustaf (or anyone else), how do you handle the fact that you might want to force SSL connections to Pound, yet security::secure_conn_p returns 0 always (as the reverse proxy is using http to communicate to the AOLserver). Is there a trick so I can tell security::secure_conn_p that the original request is actually secure ?
Collapse
7: Re: ad_conn peeraddr (response to 6)
Posted by Gustaf Neumann on
Pound adds multiple X-SSL-* request header fields to the request. The backend can query these and could set security::secure_conn_p (see https://www.apsis.ch/pound.html).
This was not an issue for us, since we only allow SSL connections from the outside world.

guess, with nginx one can get the same behaviour by using proxy_set_header.

Collapse
5: Re: ad_conn peeraddr (response to 3)
Posted by Malte Sussdorff on
Tom, That is not entirely correct. If you provide the X-Forward-For header AOLserver correctly logs this into the access.log. So it is probably only an omission.

As for the parameter suggested by Gustaf, that is fine with me as well, could you provide a full diff so we can actually TIP this? (though I cannot see why anyone would want the IP Address of the proxy when calling ad_conn peeraddr and if they want, they can still fallback to ns_conn peeraddr).

Collapse
8: Re: ad_conn peeraddr (response to 1)
Posted by Jay Dubanik on
Hi Gustaf,

Im interested in using your ReverseProxyMode
We have just started using Squid and have same problem with ad_conn peeraddr
Could you please describe in more detail how to implement this patch.

Regards,
Jay

Collapse
9: Re: ad_conn peeraddr (response to 8)
Posted by Gustaf Neumann on
quite simple:
  1. define in your config.tcl file (startup file for the aolserver) something like
    ns_section "ns/parameters"
            ns_param   home            $homedir
            ns_param   debug           false
            ns_param   ReverseProxyMode   true
            ....
    
  2. insert the snippet above into your packages/acs-tcl/tcl/request-processor-procs.tcl (just search for the place, where the peer_addr is set).
  3. make sure, your proxy inserts the x-forwarded-for header field.
When you restart the server, "ad_conn peer_addr" is set to the last x-forwarded-for addr in the header, usually the client. Some proxies have to option to drop incoming x-forwarded-for field, so spoofing this field can be avoided in general.

hope, this helps.

Collapse
10: Re: ad_conn peeraddr (response to 1)
Posted by Jay Dubanik on

Thanks Gustaf,

It worked perfectly, we get correct IPs stored in database now.

Just one correction for those who plan on adding this patch.

  1. There is no peer_addr only peeraddr should be used.
  2. The adconn -set peeraddr should be set in adproc -private rp_filter
ad_proc -private rp_filter { why } {

  This is the first filter that runs for non-resource URLs. It sets up ad_conn and handles
  session security.

} {

    #####
    #
    # Initialize the environment: reset ad_conn, and populate it with
    # a few things.
    #
    #####

    ad_conn -reset
    ad_conn -set request [nsv_incr rp_properties request_count]
    ad_conn -set user_id 0
    ad_conn -set start_clicks [clock clicks -milliseconds]

# Start of patch ReversedProxyMode
    ad_conn -set peeraddr [ns_conn peeraddr]
if { [ns_config -bool ns/parameters ReverseProxyMode 0] } {
  set addr [lindex [ns_set iget [ns_conn headers] x-forwarded-for] end]
    if {[string length $addr] > 0} {
        ad_conn -set peeraddr $addr
   }
}
# End of ReversedProxyMode

    ds_collect_connection_info