Forum OpenACS Q&A: When using the revproxy module, how to rewrite the form

I am setting up the reverse proxy module so I can send a request to another server. I understand and have working the filter and rewrite_url call to add an element and value to the query string of a GET request.

I would like to be able to do this with a POST and a PUT. Is there a way to write to the form/body of the request that gets forwarded?

Thanks,
-Tony Kirkham

Hi Tony,

the reverse proxy module can be used in various setups such as to implement a virtual server, or just to redirect certain requests to a certain server (after having e.g. performed OpenACS permission checking), etc. For example, we have small module for making elasticsearch/kibana charts available in OpenACS, which are e.g. only accessible to community admins.

Below is the full definition of the fisheye integration at openacs.org, where all requests are redirected to a backend service. This happens for GET PUT and POST requests.

all the best -g

########################################################################
# reverse proxy as separate server for CVS Repository Viewer
########################################################################
ns_section "ns/servers" {
        ns_param cvs                    "CVS Repository Viewer"
}
ns_section ns/module/nssock/servers {
        ns_param        cvs             fisheye.openacs.org
        ns_param        cvs             cvs.openacs.org
}
ns_section ns/module/nsssl/servers {
        ns_param        cvs             fisheye.openacs.org
        ns_param        cvs             cvs.openacs.org
}
ns_section ns/server/cvs {
        ns_param        minthreads      5 
}

ns_section ns/server/cvs/tcl {
        ns_param        library         /home/fisheye/naviserver
}

ns_section ns/server/cvs/modules {
    ns_param revproxy   tcl
    ns_param nslog      nslog.so 
}

ns_section "ns/server/cvs/module/revproxy" {
    ns_param filters {
        ns_register_filter postauth GET    /* ::revproxy::upstream -target http://127.0.0.1:8060/
        ns_register_filter postauth POST   /* ::revproxy::upstream -target http://127.0.0.1:8060/
        ns_register_filter postauth PUT    /* ::revproxy::upstream -target http://127.0.0.1:8060/
  }
}
ns_section ns/server/cvs/module/nslog {
        ns_param        file                    ${logroot}/access-fisheye.log
        ns_param        logpartialtimes true 
        ns_param        logthreadname   true
}

Thank you very much for your quick response and the detail. I have a similar set up in my config.tcl and the redirection works great.

What I am trying to accomplish is to authenticate using a main NaviServer instance, get the authenticated users person_id and inject that person_id into the original request as it is redirected to another server. In the ns_register_filter call I am using the -url_rewrite_callback switch to include the person_id. Here is my register call

ns_register_filter postauth GET "/pmd/*" ::revproxy::upstream -target "https://another.server.edu" -url_rewrite_callback pmd::rewrite_url

With my rewritten callback looking like the following

nsf::proc pmd::rewrite_url {
    -target -url {-query ""}
} {
    set query "user_id=[ad_conn user_id][expr {$query eq "" ? "" : "&"}]$query"
    append url ?$query
    return $url
}

This works great for a GET request. What I want is to do the same for a POST and PUT request. Since they are not GET requests I do not want to use the query string. I want to push the "user_id=1234" into the body / form of the request that is sent on to the other server.

Is there a way to accomplish this?

Thanks again,

-Tony

Have you registered the callback also for POST and PUT? The revproxy callbacks work the same way for all kind of HTTP methods. I am not sure, what problem you have. e.g. rewriting the BODY of the request is not want you want to do in the PUT case.

There are some servers, who have problems accepting query variables and form data at the same time (this was a problem with AOLserver). It this the problem you are facing?

Concerning rewriting the form body: everything is possible, but i would not recommend it, since you have to care for the following cases:
a) format application/x-www-form-urlencoded vs. multipart/form-data
b) content spooled to file or memory
c) updating headers (length etc.), handling transport encoding (compressed), ....

so passing the value as a query parameter is easier. If, for whatever reasons this is not possible, consider adding an extra header field.

The experience I have had with AOLServer was exactly what I was concerned about. After doing some other testing it appears that passing the authenticated user_id in the query string is working just fine for a POST. I have set up the ns_register_filter postauth with a * in the place of the method and it is working for both POST and GET. I am expecting it to work for all methods.

GET and POST requests are working well, but we have encountered a problem with a PUT request.

In the callback proc I am obtaining the user_id, to send onto the other server, via a call to [auth::require_login]. This is not working in the case of PUT. With a PUT request the user_id is not obtained inside of the [auth::require_login] via the call to [auth::get_user_id ...]. It returns 0 even though a successful GET or POST was just performed so I know I am logged in. (A subsequent POST after the failed PUT works fine as well.) The cookies are all sent the same as well.

Inside the [auth::require_login] after [auth::get_user_id ...] returns 0, a call to [ad_conn auth_level] is made and fails with an error stating that "auth_level" is not a valid option. This is true. I cannot find "auth_level" as a valid option in ad_conn nor in ns_conn so I do not understand how this call ever succeeds, even though I can call [ad_conn auth_level] from the shell and it returns "ok". This is baffling to me.

With all of this said, is there something different about the PUT method that is different from a GET and POST that would cause the request processor to not authenticate the user (as well as cause auth::require_login to fail)? Is there a way to overcome this?

Thanks,
-Tony

Let me summarize my understanding for your installation:
   INTERNET ------ OpenACS ------ Backend
You want to do all authentication on "OpenACS" and forward some requests to the "Backend" with authentication information.

When the reverse proxy is registered with a postauth filter as a indicated above for PUT requests, I would be really surprised, if it does not grab the PUT requests.

However, as authentication is implemented in OpenACS as a preauth filter, you have to make sure that this preauth filter is called also for PUT requests before the postauth filter, since stock OpenACS just registers the preauth filter for GET, HEAD, and POST requests (see e.g. packages/acs-tcl/tcl/request-processor-init.tcl.

The OpenACS error message about "auth_level" is not a valid option" hints this way.

That worked! Thanks!

Here is what I did. In my package tcl directory I have a revproxy-init.tcl file with the filter registrations, rather than changing the request-processor-init.tcl file.

# pulled from acs-tcl/tcl/request-processor-init.tcl
    foreach httpMethod {PUT} {
        ns_register_filter preauth $httpMethod /resources/* rp_resources_filter
        ns_register_filter preauth $httpMethod * rp_filter
        ns_register_proc $httpMethod / rp_handler
    }
# the postauth filter registration is here

Do I need all three of those registrations? Also, in the request-processor-init.tcl the above filter registration calls are surrounded by a check and comment about making sure that they only run once.

if {[nsv_exists rp_properties request_count] == 0} {
    #
    # Run this only once at startup, and not on re-inits
    #
    nsv_set rp_properties request_count 0

    foreach httpMethod {GET HEAD POST} {
        ns_register_filter preauth $httpMethod /resources/* rp_resources_filter
        ns_register_filter preauth $httpMethod * rp_filter
        ns_register_proc $httpMethod / rp_handler
    }
}

I tried to put this same check around my register calls, but it failed the check and never entered to execute that code so my filters did not get registered. By the time the system is loading my package init file the request_count has incremented to 1. Do I need to worry about that? I do not know when "re-inits" occur. I can only assume that if the one filter registration is concerned about it I should also worry about it.

Do I need all three of those registrations?

no. rp_resources_filter is just needed for special handling of urls containing resources (probably not the case for you application), the ns_register_proc is not needed, since the revproxy handles the request.

. Do I need to worry about that?
no
I do not know when "re-inits" occur.

They should not. probably a left-over from old times, where nobody dared to remove it.

Great, that it works!