VNET Jail HowTo Part 2: Networking

Posted on Apr 20, 2021

As always, Dan has been tweeting about VNET Jail issues, which means it’s time for another VNET Jail post.

This post assumes that you’ve read the original post on VNET Jail HowTo.

In Part two we will discuss Networking.

We will use PF as a firewall to do things like NAT.

If you need more help please check the FreeBSD Handbook: Chapter - Firewalls or send me an email/tweet.

At this point (from the last post) we were able to ping from the Jail to the Host.

root@www:/ # ping -c 1 10.0.0.1
PING 10.0.0.1 (10.0.0.1): 56 data bytes
64 bytes from 10.0.0.1: icmp_seq=0 ttl=64 time=0.087 ms

--- 10.0.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.087/0.087/0.087/0.000 ms

Now we will setup PF on the host by adding the following to /etc/pf.conf

ext_if="em0"
jailnet="10.0.0.0/24"

nat pass on $ext_if inet from $jailnet to any -> ($ext_if)

set   skip on { lo0, bridge0 }
pass  inet proto icmp
pass  out all keep state

We also need to enable IP Forwarding in the kernel

Add the following in /etc/sysctl.conf

net.inet.ip.forwarding=1

And now execute

sysctl -f /etc/sysctl.conf
service pf restart

That should be it, now your Jail should be able to ping the outside world

root@zvartnots:~ # jexec -l www
You have mail.
root@www:~ # ping -c 1 9.9.9.9
PING 9.9.9.9 (9.9.9.9): 56 data bytes
64 bytes from 9.9.9.9: icmp_seq=0 ttl=61 time=2.566 ms

--- 9.9.9.9 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.566/2.566/2.566/0.000 ms
root@www:~ # 

If you setup a resolver, you should also be able to ping domain names as well.

root@www:~ # echo 'nameserver 9.9.9.9' > /etc/resolv.conf 
root@www:~ # ping -c 1 freebsd.org
PING freebsd.org (96.47.72.84): 56 data bytes
64 bytes from 96.47.72.84: icmp_seq=0 ttl=53 time=133.851 ms

--- freebsd.org ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 133.851/133.851/133.851/0.000 ms

Now, for a more complicated setup that assumes no firewalls and multiple IP addresses, where each Jail has its own IP address. I have a similar setup at home where my ZNC server Jail has its own IP address by connecting the physical NIC to the same bridge as the ZNC Jail.

In my rc.conf on the host

ifconfig_em0="inet 192.168.0.34 netmask 255.255.255.0"
defaultrouter="192.168.0.1"

cloned_interfaces="bridge0"
ifconfig_bridge0="addm em0"

Here’s an example with jail.conf

znc {
	$id		= "52";
	$addr		= "192.168.0.252";
	$mask		= "255.255.255.0";
	$gw		= "192.168.0.1";
	vnet;
	vnet.interface	= "epair${id}b";

	exec.prestart	= "ifconfig epair${id} create up";
	exec.prestart	+= "ifconfig epair${id}a up descr vnet-${name}";
	exec.prestart	+= "ifconfig bridge0 addm epair${id}a up";

	exec.start	= "/sbin/ifconfig lo0 127.0.0.1 up";
	exec.start	+= "/sbin/ifconfig epair${id}b ${addr} netmask ${mask} up";
	exec.start	+= "/sbin/route add default ${gw}";
	exec.start	+= "/bin/sh /etc/rc";

	exec.poststop   = "ifconfig bridge0 deletem epair${id}a";
	exec.poststop  += "ifconfig epair${id}a destroy";

	host.hostname = "${name}.bsd.am";
	path = "/usr/local/jails/${name}";
 	exec.consolelog = "/var/log/jail-${name}.log";
	persist;
}

And that’s pretty much it!

That’s all folks.