Giving a jail multiple IPs with pf and NAT

From FBSD_tips

Back to "Admin building blocks"

DRAFT

Edit: There are problems with this method, like connecting to the host. I've addressed them and will update this after work. ;)

First, thanks to jpaetzel, mulvane_ and pUmkInhEd for their help with this. We often see questions in ##FreeBSD on FreeNode relating to using jails with multiple IPs, and usually someone mentions unofficial patches--but there's an easier way! And I'm going to tell you what it is! Holy crap!

Personally, I needed an Apache jail to be able to talk to the Internet and a database on a private network. Now, this works just fine with the usual jail setup. The problem is that (even with this method), the jail only has one IP address. The host machine normally sees traffic from this IP address and routes it as normal. So, for my dilemma, if the jail had IP 1.2.3.4, talking to the Internet through the host machine's default gateway worked fine, but talking on the internal network wasn't. Traffic went from 1.2.3.4 to 10.0.0.2 and the DB server said "zomg u rnt on my netwrk. die!" (I use MySQL-SMSTXT). What this article shows is how to use the fact that the host system routes correctly, slap on some IP address translation, and make traffic come from the desired IPs. There are three main steps to this:

1. Create the jail.
2. Enable pf and NAT.
3. There is no step three!

First, create your jail. How you do this is beyond the scope of this article. We are going to use our loopback interface and give it the IP address 127.0.0.2/32. Get the jail up and running. The host machine will see traffic on lo0 and use it's routing table as usual, just like any other jail. The problem is that this traffic, while reaching it's destination, is coming from this 127.0.0.2 address, so now we configure pf and NAT. First, enable pf in rc.conf (and disable any other firewall you may be using):

pf_enable="YES"
pf_rules="/etc/pf.conf.local"

Now create the rule set file for pf. This is a very basic rule set:

scrub in all
binat on xl0 from 127.0.0.2 to any -> 192.168.0.21
binat on xl1 from 127.0.0.2 to any -> 10.50.0.1
block in log all
pass in all
pass out all

The jail's real (or "assigned") IP is, as stated above, 127.0.0.2. The external interface and IP address for this jail are xl0 and 192.168.0.21, and the internal versions are xl1 and 10.50.0.1, respectively. The first "binat" line tells pf to take traffic from 127.0.0.2 to anywhere that gets routed to the external interface and translate it to the external IP address. The second line is the same, except using the internal variations. Start pf by running "/etc/rc.d/pf start" and you're done!

Well, almost. The configuration above will open your system up completely; that is, it will allow all traffic in and out. Usually incoming traffic is controlled. You can use your usual rules (pf style), but one very important fact to remember is that pf translates addresses before it filters traffic. This means, using the example above, traffic sent to 192.168.0.21 will be translated (on interface xl0) to 127.0.0.2 before filters are applied. So, given this fact and assuming you have a default deny rule, how would you allow in port 80 to this jail? Well, we want to allow incoming traffic on the external interface from anyone to our jail which, after transaltion, is 127.0.0.2 (using port 80). This gives us:

pass in on xl0 inet proto tcp from any to 127.0.0.2 port 80 keep state

Keep this in mind when adjusting your current rules, and you should be good to go.