Forum OpenACS Q&A: Re: NaviServer cluster running inside of docker questions

3: Re: NaviServer cluster running inside of docker questions (response to 1)

Posted by Gustaf Neumann on 05/29/22 09:07 PM

You are addressing multiple points, and some of these look like feature requests.

However, whenever the [::acs::Cluster info instances] is called it returns nothing

This indicates that the kernel parameters of the cluster configuration are not set up correctly. You have to set "ClusterEnabledP" to "1" and you have to provide "ClusterPeerIP" in the required format, i.e., a list of IP address with optional ports, like e.g., "127.0.0.1:8100 127.0.0.1:8101".

Per design, all cluster specifications are IP addresses plus optional ports (no name resolving) and HTTP only (for performance reasons), therefore the scheme is omitted.

behind nginx using https port 443 I ran into a problem where the code would only support http port 80.

Probably you meant that the intra-cluster talk is only HTTP and not HTTPS. There is no restriction to port 80; the intra-cluster talk is implemented via "ns_http run http://${:host}:${:port}" (note the hard-coded "http:").

A possible way to address these points would be a feature request to allow a list of server locations instead of just IP addresses and ports in "ClusterPeerIP" (e.g. 127.0.0.1:8100 https://localhost:8444). Still, I would not recommend the usage of HTTPS or the use of domain names, but if you have to do so...

Would this help for your major requirements?

Use nslookup to resolve docker hostname to ip

One should use the NaviServer built-in command "ns_addrbyhost" [1] instead. Be aware, that in general, multiple IP addresses can be returned from the DNS lookup (might be different hosts or IPv4 and IPv6 of the same host).

Also, I found a little issue in the cluster-init.tcl

Thanks, fixed.

[1] https://naviserver.sourceforge.io/n/naviserver/files/ns_addrbyhost.html

4: Re: NaviServer cluster running inside of docker questions (response to 3)

Posted by Marty Israelsen on 06/02/22 04:43 PM

Thanks Gustaf,

We do have the ClusterEnabledP set to 1, but you are right, we are trying to put docker DNS names into the ClusterPeerIP, CanonicalServer and even in the ClusterAuthorizedIP.

Your point is well taken that this really belongs under a feature request to make clustering work with docker. Inside Docker we cannot know the IP addresses that are assigned because they are dynamic. Is the best way to request this enhancement, to submit an improvement proposal?

In the meantime cachingmode=none will work for us if we manually restart the backend naviservers ourselves by using 'docker restart {container names}'

We do want to talk https to the backend clusters for a number of reasons. Besides the fact that our company wants us to run https for everything, there also is the reason that the Canonical Server is the only one that can run cronjobs. So when we go to the /cronjobs URL we cannot leave it up to docker which server gets that request. We currently are redirecting to the external port 444 which is our Canonical Server port that we have exposed to the outside with docker. Since only admins can get to /cronjobs this seems to work out fine. There may be another way to do this using nginx, but currently we like getting for each NaviServer directly from the ouside ports for debug purposes and especially the Canonical Server. Also, when we go to acs-admin/install we have also redirected to the Canoncial Server so that if someone restarts the server we know it is the Canonical Server they restarted - then we can manually restart the other two after the Canonical has come all the way back up. Doing it this way, we have a zero downtime restart for our users for non-schema change upgrades.

##  I have added this code to both /cronjob and acs-admin/install to allow redirection to the canonical server
util::split_location [util_current_location] proto host port
if {[info exists ::env(CANONICAL_OUTSIDE_PORT)] && $::env(CANONICAL_OUTSIDE_PORT) != $port} {
    ad_returnredirect -allow_complete_url "$proto://$host:$::env(CANONICAL_OUTSIDE_PORT)[util_current_directory]"
}

Thanks for pointing out ns_addrbyhost. I was hoping there was such a utility but could not find it.

I sure appreciate your insights and expertise

Thanks, Marty