Forum OpenACS Q&A: network topology for hosting OpenACS?

Thanks for the responses. I've got things working now (and will soon add my site to the OpenACS site list). Since this situation is common to the many PacBell users out there as well as anyone else who gets a small subnet from their ISP, here's what worked for me.

NOTE: This post is rather off-topic for OpenACS and should be avoided by anyone who is smart enough to have other people do their network admin work for them. It is only for those of us who can't keep from peeking under the hood to understand how things work and fix 'em ourselves.

The problem was to implement this topology from the IPCHAINS HOW-TO Example:

   External Network (BAD)
             |
             |
         ppp0|
      ---------------
      | 192.84.219.1|             Server Network (DMZ)
      |             |eth0
      |             |----------------------------------------------
      |             |192.84.219.250 |             |              |
      |             |               |             |              |
      |192.168.1.250|               |             |              |
      ---------------         --------       -------        -------
             | eth1           | SMTP |       | DNS |        | WWW |
             |                --------       -------        -------
             |           192.84.219.128  192.84.219.129 192.84.218.130
             |
     Internal Network (GOOD)

In the case of a small subnet from the ISP, things are slightly different. Since PPP isn't used, the BAD network interface has an ip address within the range of the DMZ network. This actually matches the diagram from the example, but this diagram is misleading, since the ip address assigned to ppp0 comes from the ISP dynamically and so *isn't* 192.84.219.1. (Actually, as I think about it, I don't understand this diagram, since it shows static, public ip addresses in the DMZ yet uses PPP to connect to the ISP.)

Anyway, the problem is the ambiguity of having the ip addresses given by the ISP in both the BAD and the DMZ networks. The result is that while GOOD and BAD can connect and GOOD and DMZ can connect, DMZ can't get to BAD and vice versa. Searching through the debian archives, I found a number of postings reporting this same problem (debian is an incredible distro not least because of the support available).

So here is the topology I wanted to implement (assuming that the subnet from the ISP is the first n/29 subnet so its addresses run from x.y.z.0 to x.y.z.7):

   External Network (BAD) -> ISP gateway x.y.z.1/29
             |                   network x.y.z.0/29
             |                   broadcast x.y.z.7/29
         eth0|
      ---------------
      | x.y.z.2/29  |             Server Network (DMZ)
      |             |eth1
      |             |----------------------------------------------
      |             |x.y.z.3/29     |             |              |
      |             |               |             |              |
      | 192.168.1.1 |               |             |              |
      ---------------         --------       -------        -------
             | eth2           | box 1|       | box 2|      | box 3 |
             |                --------       -------        -------
             |               x.y.z.4/29     x.y.z.5/29     x.y.z.6/29
             |
     Internal Network (GOOD) [192.168.1.0/24]

Stepping back a bit first, though, there are several possible alternatives worth considering:

1. Just IPMASQ the DMZ (like GOOD) and portforward the services you want to run in the DMZ.

This sidesteps the problem and may be the preferred solution; I'm interested in hearing how many people out there host their servers in this way. However, it fails to use the extra ip addresses (unless you alias them to the firewall box and statically translate them into the DMZ, but it's unclear to me how this is superior to the simple IPMASQ solution).

2. Subnet the subnet from the ISP to get separate ranges for BAD and DMZ.

This would work fine, though not with the small n/29 subnet PacBell (and presumably other ISPs) provides. As it is, of the 8 ip addresses in my n/29 slice, one is the network address, one is the broadcast address, and one is the gateway address -- leaving 5. To subnet this into two would remove 4 more (a network and a broadcast address for each sub-subnet) leaving only a single address to actually assign to a NIC. But this topology needs one each for the EXT NIC, the DMZ NIC on the firewall box, and one for each DMZ host NIC. So this obviously won't work.

3. Forget this topology and just expose the web/mail/etc hosts in EXT.

Very simple and certainly a fallback. By running IPCHAINS on each exposed host, you get basically the same result as the DMZ filtering from the target topology. I suspect a lot (most?) people do this. If I hadn't managed to get things to work, I would have done this. Perhaps this is the preferred solution. Still, I wasn't happy giving up without figuring this out 😉

4. Bridge+firewall.

There is an interesting Mini-HOW-TO (http://www.linuxdoc.org/HOWTO/mini/Bridge+Firewall+DSL.html) that sounded appropriate to this problem. This might be a superior solution, too, though it seemed more complex than necessary. It would be interesting to hear from someone who knows more about this than I do as to its merits vs the other alternatives.

5. Proxy ARP

This finally was the solution that most people reported had worked for them, and this is what I did. There is a helpful Mini-HOW-TO (http://www.linuxdoc.org/HOWTO/mini/Proxy-ARP-Subnet/index.html) that discusses Proxy ARP with subnetting. For the reasons discussed in 2, subnetting isn't an option here. But using Proxy ARP without subnetting is a simple solution.

Basically, the idea is to set up the subnet from the ISP so that it "belongs" to EXT (since it does) but then to use Proxy ARP to tell the firewall box kernel that specific addresses from that subnet actually "belong" in the DMZ. (Sorry for the colloquial terminology, though if you've read this far, it's likely that you are more of an learner like me than a TCP/IP expert who would be offended by this!) How this happens is nicely described in the Mini-HOW-TO; it's complex and I won't repeat it here.

Anyway, to accomplish this is remarkably simple. First you use arp for each host you want in the DMZ:

        arp -v -i eth0 -Ds x.y.z.4 eth0 pub

This puts arp in verbose mode while it publishes a static entry for the x.y.z.4 host to the cache for the EXT interface eth0 (the D flag spares us from looking up and entering the MAC (hardware) address for eth0). The Mini-HOW-TO proxy ARPs a subnet; here we have to arp each host separately since we're *not* doing a subnet.

Then you create the correct routes:

        route add -net x.y.z.0 netmask 255.255.255.248 dev eth0

for the subnet from the ISP, and

        route add -host x.y.z.4 dev eth1

for each of the host boxes in the DMZ.

At the end of all this, the routing table for the firewall box should look like this (after the first DMZ host is added):

# netstat -nr
Kernel IP routing table
Destination  Gateway  Genmask          Flags  MSS Window irtt Iface
x.y.z.4      0.0.0.0  255.255.255.255  UH     0   0      0    eth1
x.y.z.0      0.0.0.0  255.255.255.248  U      0   0      0    eth0
192.168.1.0  0.0.0.0  255.255.255.0    U      0   0      0    eth2
0.0.0.0      x.y.z.0  0.0.0.0          UG     0   0      0    eth0

And the output of arp will show (in addition to lines from other hosts depending on whether there has been traffic to/from them) a line from our arp command:

# arp -an
...other stuff deleted...
? (x.y.z.4) at * PERM PUP on eth0

Anyway, hope this summary is useful to anyone else trying to set up this kind of arrangement. If I've made some errors here (or if there are reasons to prefer one of the other alternatives) I'd like to hear about them.