Forum OpenACS Q&A: Multiple domains on same server/ip

Collapse
Posted by Ivan Histand on
Hi,

Has anyone written a top-level overview of how to host multiple
domains with OACS on the same machine with a single IP address?

I currently have two test domains running on separate IPs, but I'd
like to add 5 more domains to the server, and I'm out of static IPs.

These will be low-traffic personal sites, but I'm a bit concerned
about running 7 nsd/oacs processes as well.  What's the best way to do
this?

Thanks,
Ivan

Collapse
Posted by David Walker on
My solution is a series of tcl hacks to [ns_info pageroot], [ns_url2file],
ns_schedule, and a few other functions.  If you don't mind getting your
hands dirty I could send them to you and see if they work for you.

The /web/yourserver/tcl directory of the main site is shared with all the
children but each child has their own copies of all the other files.

Collapse
Posted by Jason Khong on
Start by going here: http://www.theashergroup.com/tag/articles/nsvhr/virtual-hosting-howto.adp
That's the right way to start if off! :)

On the other hand, I'm also interested in the best ways of running multiple low-traffic sites on one machine. Right now it looks like it's 1 nsd per site minimum. But I've seen a server running Apache for hosted sites, and it seems to have less number of threads than the number of sites hosted. Any idea how they did it?
Collapse
Posted by Dave Bauer on
The host-node-map is supposed to handle this to allow subsites of one OpenACS install to repsond to different domain names.
This would keep all the various sites' data in one database with one main nsd process and one thread for each connection being served.

Last I heard there was a user registration problem with this.

Collapse
Posted by David Walker on
Just to be clear.  My hacks are for OpenACS 3.2.5.  You didn't say which
version you are running.

Apache handles virtual hosting internally.  It doesn't need a thread to be
dedicated to any one site.

Collapse
Posted by Tom Jackson on

Can it be done? Yes: checkout

Both sites use the same ACS4.2 install. Both use mostly the same set of scripts, but separate template (.adp) directories. Actually they even share a lot of adp files, but with different configurations based on the domain name. This involves quite a lot of hacking on top of acs, and a few changes to the templating system.

For a number of small sites on one OACS4.5b1, only one using OACS, checkout:

These sites use my Virtual Abstract Templating module. Caffeumbria is using the templating part at http://caffeumbria.com/blends/. The VAT module is available from http://zmbh.com/vat/

One thing that was possible in the 3.x series of OACS was to use separate database pools per server with my module. This is because my module would set the server parameter, and pools in ACS are pulled from [ns/server/$server/db] and ACS parameters are pulled from [ns/server/$server/acs]. I have not checked it this is still possible in 4.5, probably it is not.

Collapse
Posted by David Walker on
See http://www.vorteon.com/download/ for my hacks. It is called virt_host_hacks v.0.3. I've only set this up on one machine so I can't tell you if I put everything there that you need.

The hacks intercept calls to ns_db, ns_info, ns_url2file, ns_config, and ns_schedule and act based on the Host header.

The start_primary_site.tcl create pools for each of the sites (enough sites and Postgres may require you up some of it's setting to support that many open connections).

I've been wanting to make it prettier and probably will when I get time but here's my work so far
Collapse
Posted by David Walker on
Put the tcl files in the /web/primary_site/tcl except for start_primary.tcl.  That
is the aolserver config file.

Make sure your web sites are in /web

Collapse
Posted by Don Baccus on
Caffee Umbria's website is an OpenACS 4.5b1-based site?  I am *so* PSYCHED!  Their old brand, Terrafazione, went downhill after Popeye's bought them out and it's so cool to see the family's new endeavor running on our toolkit.

I work for about three hours almost every afternoon in a little coffee shop named Common Grounds on Hawthorne street here in Portland, Oregon.

It's my afternoon office ... they dumped Terrafazione recently and switched to Caffee Umbria, in fact one of the owners of Caffee Umbria was there yesterday when I was working.  What a nice coincidence, one that will humor the folks who own Common Grounds.  They all know I work in there on web/db stuff ...

Seattle's not a bad coffee town, other than Starbucks and Seattle's Best.

Seattle had one coffee shop listed a couple of weeks ago in US Today's list of the Top Ten best coffee shops in the United States.

Portland had two, of course, including my "morning office", Stumptown, just around the corner (Bruno Mattarollo of Greenpeace and I figure the reporter must've visited when Bruno and I were hanging out there a month ago).

And on average Portland has one more sunny day per year than Seattle ...

Collapse
Posted by Talli Somekh on
vinod asks: "where's the best beer?"

talli

Don, so that means Seattle only has one sunny day per year? Wow, us Portlanders are blessed :)

So what's the other best 10 coffee shop?

Collapse
Posted by Don Baccus on
Talli: there's no "best beer" here.  That's the beauty living in the cradle of the modern microbrew revival!

Jade - I think it was the Fresh Pot but I didn't see the article myself.  Personally I'm surprised that Seattle made the list at all :)

Collapse
Posted by Hamilton Chua on

Hi,

I had more or less the same problem but it was in our office. I was working on several projects at the same time and , at the time, I was only allocated one static IP.

I had read something about hacking OpenACS as mentioned above to host several OpenACS installs in one static IP. However, I was also experimenting with Apache and PHP.

What I decided to do was to follow the procedure for creating virutal hosts on Apache :

http://www.osmosislatina.com/java/javaaolserver.htm

I have been test driving this set up on my development machine for a while and everything seems Ok until recently where I have been experiencing occassional slow downs which I attribute to my hard disk doing a lot of swapping.

Am not certain whether this has anything to do with it or with VMware ,which I am also test driving on my machine, I have yet to determine the cause. Nevertheless, I hope this helps.

Regards,

Collapse
Posted by Ivan Histand on
Thanks all for your responses. I've done quite a bit of research and experimentation the past few days, and these are my findings:

- nsvhr/nssock works. However, I don't like it, because it doesn't proxy, it redirects. For example, if I run a master server on port 80 and slave on 81, a request for http://foo.com/ immediately gets redirected to http://foo.com:81/ This is ugly and really not doing anything more than I could do with a META REFRESH= tag in an index.html page.

- nsvhr/nsunix is buggy. Sometimes it works, sometimes it doesn't...core dumps, etc. Dead end unless someone smarter than I fixes it.

- SQUID seems like it would be an ideal solution, but the documentation and examples for implementing it as a reverse proxy are slim and I gave up trying to get it to work.

- I didn't try any of the tcl-based solutions, as I decided I would prefer a proxy solution giving each domain it's own OACS instance.

- AOLserver4 supposedly has built-in virtual servers. I downloaded the beta, compiled it, installed it, no mention of virtual servers at all that I could find. Dead end.

- Apache works, as advertised. I simply set up an httpd listening on port 80, and used the NameVirtualHost directive as per the examples here: http://httpd.apache.org/docs/vhosts/examples.html This was a last resort for me because It seemed dumb to have to set up two brands of http servers to do what seemed like a simple task, but I guess that's life.

So my final configuration is:

- All domains have DNS entries pointing to the same IP

- Apache listens on port 80 and proxies via VirtualHost

- Each domain has it's own nsd/oacs instance, listening on localhost:81, localhost:82 etc.

This is probably not very scalable, but for me it's going to be a max of maybe half-dozen low traffic domains, and I'm fairly certain it will scale that far at least. On the plus side it is extremely flexible, if a site starts getting lots of traffic it would be trivial to pass all or some of it off to other machines.

For anyone interested here's a psuedo section from my httpd.conf file, everything else in that file is pretty much standard:

Ivan

Collapse
Posted by Tilmann Singer on
<i>nsvhr/nssock  works.  However, I don't like it, because it doesn't proxy, it redirects.  For example, if I run a master server on port 80 and slave on 81, a request for http:// foo.com/ immediately gets redirected to  http://foo.com:81/ </i>
<p>
I have no idea how it is possible to set it up to do that, but it is certainly not the intended way. It should be no problem to use nsvhr/nssock to serve different servers via the standard port 80. The actual port that the slave servers run on can be completely hidden from the client AFAIK.
<p>
Maybe there are some issues with the request processor of oacs 4.5 - which version do you want to run? At least with 3.x it worked fine.
<p>
<i>nsvhr/nsunix is buggy.  Sometimes it works, sometimes it doesn't...core dumps, etc.</i>
<p>

Maybe something was wrong with your installation - did you follow the instructions from the virtual hosting howto posted above? nsvhr/nsunix works great for me, running a few low traffic sites (one oacs 3.x, the others 4.5), without any crashes at all since i installed it many months ago.

I've read the techniques mentioned at https://openacs.org/bboard/q-and-a-fetch-msg.tcl?msg_id=0004TP&topic_id=11&topic=OpenACS. In review of the thread (and linked info):

  • technique: Tom Jackson's VAT vhosting module
  • notes:
    • multiple sites using 1 AOLserver port sharing the same OpenACS (any version),
    • limited documentation available
  • questions: Allows for different templates with each domain under OpenACS 4.5?
  • technique: NSVHR hosting with AOLserver
  • notes:
    • uses AOLserver as reverse proxy and requires 1 additional AOLserver per domain,
    • each OpenACS site is independent,
    • also integrates with Tom Jackson's VAT technique,
    • apparent future plans include switching the reverse-proxy AOLserver to Apache
  • questions: Does patch work for AOLserver 3.4.x? (this technique uses patch with AOLserver3.3ad13)
  • technique: Histand's Apache/AOLserver implementation
  • notes:
    • uses Apache as reverse proxy and requires 1 AOLserver per domain,
    • each OpenACS site is independent
  • questions: Is this the arrangement used at the test servers openacs.hub.org and openacs1.hub.org?

Has anyone tried using Tom Jackson's VAT hosting module in conjunction with OpenACS 4.5's subsite package, so that for example:

domain1.com resolves to domain1.com (the primary is just another instance of subsite)
domain2.com resolves to domain1.com/subsite2/
domain3.org resolves to domain1.com/subsite3/

For clarification purposes, so that resolving occurs without redirects:

domain2.com appears as serving from domain2.com in browser, search engines etc.
domain3.com appears as serving from domain3.com etc. etc.
Collapse
Posted by Ola Hansson on
About nsvhr you write: "uses AOLserver as reverse proxy and requires 1 additional AOLserver per domain,".

In fact it only requires one additional AOLserver instance regardless of how many domains you host...

Also, one downside of this technique is you can't use https (SSL).

Collapse
Posted by Torben Brosten on

My understanding of the requirement for multiple instances of AOLserver comes from http://www.theashergroup.com/tag/articles/nsvhr/virtual-hosting-howto.adp#nsvhr-how. For clarity, isn't Tom Jackson's VAT required to share multiple domains with one AOLserver instance, when using nsvhr?

Collapse
Posted by Tom Jackson on

I think nsvhr does it's own virtual hosting. The VAT module uses one AOLserver instance to serve multiple domains. The VAT module runs as a filter similar to the OACS request processor but runs before any other filter. The VAT module compares the value of the Host header with its in memory database (nsv arrays). If there is a match VAT starts looking for static files in the configured location, then template files, then returns filter_ok if it doesn't find any files. If there isn't a match it simply returns filter_ok. If you have OACS installed, that means the request will the be picked up by the request processor. VAT doesn't work with or know about the OACS subsite module.

If you load the VAT module, and don't configure any domains, it should have no effect on how any request is served.

Collapse
Posted by Ivan Histand on
One clarification on the Apache Reverse Proxy solution, it is possible to do this:

<blockquote>domain1.com resolves to domain1.com (the primary is just another instance of subsite)
</blockquote>

<blockquote>domain2.com resolves to domain1.com/subsite2/
</blockquote>

<blockquote>domain3.org resolves to domain1.com/subsite3/
</blockquote>

The ProxyPass directive in the httpd.conf file takes an url with a wildcard on the end, this can literally be any url, it doesn't have to be a base url. The only restriction is, of course, that you can't redirect to lower-level urls, for example if a page on a virtual domain  served by domain1.com/subsite2/  redirected or even linked to domain1.com/foo you'd have a problem. domain1.com/subsite2/ needs to be designed as if that was the root level.

Collapse
Posted by Jon Griffin on
Apache reverse has serious limitations not the least of which is that it can't do https.

Also, the logging sucks.

Collapse
Posted by Torben Brosten on
"If you have OACS installed, that means the request will the be picked up by the request processor. VAT doesn't work with or know about the OACS subsite module"--Tom Jackson

Hmm.. very interesting.. It seems like a contradiction to me, though, because I don't get the underlying principles of your explanation (yet).

From the subsites-requirements page:

"Subsites enable a single OpenACS instance to provide each subcommunity with its own 'virtual website' by assembling OpenACS packages that together deliver a feature set tailored to the needs of the subcommunity" (Vision Statement)
..At an implementation level this is primarily accomplished by having an application "scope" its content to a particular package instance. The request processor then figures out which package_id a particular URL references and then provides this information through the ad_conn api. (System Overview, Para 1)
.."The other piece of the subsite system is a subsite package that provides subsite admins a "control panel" for administering their subsite. This is the same package used to provide all the community core functionality available at the 'main' site which is in fact simply another subsite." (System Overview, para 2)

Since both VAT and subsites work with the request processor, I would imagine either the combination would work as hoped, there would be some kind of fatal error, or the virtual request through the VAT would be ignored by subsites... because it thinks it's a different animal (package_id). Ah ha! Is this what you are stating? If so, any suggestions on how I could get it to listen? Could the VAT recognize multiple package_id(s)?

I can see this application of VAT and subsites used to create a boston.officesite.com, newyork.officesite.com or client1.site.com, client2.site.com, or even mgt350.dotlrn.edu, syscidept.dotlrn.edu etc. etc. Using subsites (with templating) there could be all sorts of pre-made templates setup for creating small-communities on the fly...

[note: disregard question regarding openacs.hub.org etc. A link served from apache points to an AOLserver instance at a different port --I wasn't thinking clearly at that point ;)]

Collapse
Posted by Tom Jackson on

Torben, sorry to have produced such a poorly documented piece of software. VAT is an AOLserver module, not an OACS module/package. The OACS subsite module redirects or serves pages from a subdirectory of the main site. I really haven't investigated how it works. The subsite module could be modified to work a little more like a virtual server, it it would serve pages from separate pageroot locations, the www directory.

I wrote an OACS module that was Host aware and used separate templates for each host, but used the same tcl pages. Actually I guess I sym linked in the tcl pages I wanted to reuse.

The main problem is that the data model needs to be Host aware also, and it isn't.

Now I have got two OACS3.x sites to work on one database, using different pageroots. But in OACS3.x, everything is under www, so this was easier. Also, the sites were owned by the same company, and were similar in purpose: ecommerce.

My VAT module makes it easy to do single AOLserver virtual hosting, all you need to do is add two lines to the configuration file, and add the site files. Each virtual host can access different, or the same database pools. The original purpose of VAT was to allow a db backed web service appear static. See http://saleonall.com/ as an example. Back when AltaVista allowed unlimited submission of web pages to their search engine, I had over 150k pages indexed there.

Collapse
Posted by Torben Brosten on
Tom, thank you for the clarification. I'll check into this further (and keep everyone posted), as I think it would be a great way to scale from subsite to supersite size of an online community --as it evolves.
Collapse
Posted by Andrew Piskorski on
So, is it possible to do reverse proxying of SSL connections using AOLserver?  Or do you have to use Squid for that?
Collapse
Posted by Jon Griffin on
I have started testing Pound. It is a reverse proxy that can proxy ssl to non-ssl connections etc. I can't say much about it yet as it just started to have reverse http stuff. Search Google and you will find it.
Collapse
Posted by Andrew Piskorski on
Hm, Pound, also some info on FreshMeat and Gnu.org.

From a quick look at nsvhr.c, it's clear that AOLserver's nsvhr module does not work with SSL, as it only accepts protocols "http", "tcp", and "unix" - anything else, like "https", is an error. But, the entire nsvhr.c is only 700 lines, pretty small. Any idea what it would take to make nsvhr work with SSL? Or why no one's done it yet? E.g., is it hard or tricky for some reason?

Collapse
Posted by David Walker on
I haven't worked with nsvhr.c but I believe that you could possibly install nsopenssl on the proxy instance of aolserver and use nsvhr to make an unencrypted connection to your web server.

The big gotcha with SSL is that each ssl machine name needs a distinct IP address. So while your nsvhr could host hundreds of names on one IP only 1 SSL name could be hosted on that IP.
Collapse
Posted by Michael A. Cleverly on
I imagine the reason is a classic catch-22. nsvhr would look at the host header that the client sends to identify which virtual domain was being requested, but on an SSL connection the SSL handshaking/certificate exchange happens before the client sends any headers, and since the client verifies that the SSL certificate matches the host requested browsers will display warnings/errors when there is a mis-match. And the server would have no way to know which SSL certificate to send if you had multiple domains on a single IP. So in the end everyone is effectively stuck with one SSL host per IP address.
Collapse
Posted by Andrew Piskorski on
So, effectively, no matter what reverse proxy software you use, it should be possible to serve:

https://www.site.com/mysite/ and https://www.site.com/yoursite/

each from their own separate AOLserver process, simply by using a single www.site.com SSL certificate for both. But if you try to do the same for

https://mysite.site.com/ and https://yoursite.site.com/

it may run fine, but there is no way to do it and still have the SSL certificates match. Does that sound right?

Collapse
Posted by C. R. Oldham on
Can't you do this with a wildcard cert?  Don't wildcard certs match *.site.com?

However there's no way to have http://firstsite.com and http://secondcite.com use the same cert, right?

--cro

Collapse
Posted by Jon Griffin on
One gotcha with wildcard certs.

*.site.com is literally that. IOW, if you use site.com, it will show as a bad certificate, but www.site.com or dev.site.com works fine. There are ways around it, but it is something to watch out for.

Also, be prepared to spend $$$

Collapse
Posted by Jon Griffin on
I am thinking that Pound may be able to handle this. I could be wrong though.

One of the reasons I am interested is that I went from a co-lo with many ip's at my disposal to my friends T-1 with 1 ip. So I am still figuring all this out.

Even if Pound doesn't work with multiple ssl certs, just getting rid of Apache as a proxy is worth it for me.

Collapse
Posted by Andrew Piskorski on
Jon, wait, you'd rather not use Apache, but you're currently using it as a reverse proxy rather than AOLserver/nsvhr?  Why, because nsvhr currently doesn't support https at all?  Or for some other reason?
Collapse
Posted by Tom Jackson on

I wrote a tcl only proxy for AOLserver a few years ago. It works with https, because the tcl modules in AOLserver are above the ssl layer. It is a relatively simplistic proxy, but works for virtual hosting. The one file proxy+instructions is at http://zmbh.com/tclvhr/

Collapse
Posted by Jon Griffin on
I don't use nsvhr because:
  • I am not using 3.3ad I am using 3.4 and the patches don't work.
  • Some of my proxies are to other machines which makes it impossible to use nsunix.
  • It was easy to setup Apache and I was using Squirrelmail for a client which is really flaky on AOL/php.
Apache doesn't support HTTPS either AFAIK.

I am getting rid of Squirrel so I don't need the overhead and complexity of Apache/PHP.

If I had time I would probably hack the nsvhr module to allow HTTPS, but I have to many other things on my plate. I would love to use a pure AOL solution, but Pound seems lightweight and secure so I am trying it.

Collapse
Posted by Richard Hamilton on
Michael,

Your comment earlier :

I imagine the reason is a classic catch-22. nsvhr would look at the host header that the client sends to identify which virtual domain was being requested, but on an SSL connection the SSL handshaking/certificate exchange happens before the client sends any headers, and since the client verifies that the SSL certificate matches the host requested browsers will display warnings/errors when there is a mis-match. And the server would have no way to know which SSL certificate to send if you had multiple domains on a single IP. So in the end everyone is effectively stuck with one SSL host per IP address.

Your explanation sounds spot on to me. But is that one certificate per IP address or one certificate per domain name? I think that it should be fine to use a different keyfile and certfile with each running instance of AOLServer (i.e. on 8443, 8444, 8445) on any one machine/IP - the correct ssl negotiations should take place. It is the reverse proxy situation that is obviously problematic because if the proxy instance on AOLServer has to have its own key and certfile, it cannot authenticate on behalf of another AOLServer instance.

Cannot think of an easy way around that - kind of what ssl was designed to protect against!

Regards Richard