So Dan has been tweeting that there’s no good example to get started with VNET Jails with jail.conf, I thought it’s time to write one.

In this example I’ve used FreeBSD 12.1-RELEASE

root@jail-host:~ # freebsd-version
12.1-RELEASE
root@jail-host:~ # uname -a
FreeBSD jail-host 12.1-RELEASE FreeBSD 12.1-RELEASE r354233 GENERIC  amd64
root@jail-host:~ #

First thing first, let’s setup a bridge on our host

root@jail-host:~ # sysrc cloned_interfaces="bridge0"
cloned_interfaces:  -> bridge0
root@jail-host:~ # sysrc ifconfig_bridge0="inet 10.0.0.1 netmask 0xffffff00 descr jails-bridge"
ifconfig_bridge0:  -> inet 10.0.0.1 netmask 0xffffff00 descr jails-bridge

Restart the network interface to apply changes

root@jail-host:~ # service netif restart

Good! let’s setup a ZFS dataset for Jails ;)

root@jail-host:~ # zfs create -o mountpoint=/usr/local/jails zroot/jails

Good! now let’s fetch the base.txz file. I will be using my closest mirror, you should use yours.

root@jail-host:~ # mkdir /usr/local/jails/.dist-files
root@jail-host:~ # fetch -o /usr/local/jails/.dist-files/FreeBSD-12.1-RELEASE-base.txz http://mirror.yandex.ru/freebsd/releases/amd64/12.1-RELEASE/base.txz

Perfect!

Now, we will extract the base into the jail.

root@jail-host:~ # zfs create zroot/jails/www
root@jail-host:~ # tar xf /usr/local/jails/.dist-files/FreeBSD-12.1-RELEASE-base.txz -C /usr/local/jails/www/

Nicely done! Now let’s setup our /etc/jail.conf :)

Here’s my configuration.

# vim: set syntax=sh:
exec.stop  = "/bin/sh /etc/rc.shutdown";
exec.clean;
allow.raw_sockets;
allow.mount.tmpfs;
mount.devfs;


www {
    $id     = "10";
    $ipaddr = "10.0.0.${id}";
    $mask   = "255.255.255.0";
    $gw     = "10.0.0.1";
    vnet;
    vnet.interface = "epair${id}b";

    exec.prestart   = "ifconfig epair${id} create up";
    exec.prestart  += "ifconfig epair${id}a up descr vnet-${name}";
    exec.prestart  += "ifconfig bridge0 addm epair${id}a up";

    exec.start      = "/sbin/ifconfig lo0 127.0.0.1 up";
    exec.start     += "/sbin/ifconfig epair${id}b ${ipaddr} netmask ${mask} up";
    exec.start     += "/sbin/route add default ${gw}";
    exec.start     += "/bin/sh /etc/rc";

    exec.prestop    = "ifconfig epair${id}b -vnet ${name}";

    exec.poststop   = "ifconfig bridge0 deletem epair${id}a";
    exec.poststop  += "ifconfig epair${id}a destroy";

    host.hostname = "${name}.jail-host";
    path = "/usr/local/jails/${name}";
    persist;
}

Now let’s start our Jail!

root@jail-host:~ # service jail enable
jail enabled in /etc/rc.conf
root@jail-host:~ # service jail start www
Starting jails: www.
root@jail-host:~ # jls
   JID  IP Address      Hostname                      Path
     1                  www.jail-host                 /usr/local/jails/www

Let’s check the networking :)

root@jail-host:~ # ping -c 1 10.0.0.10
PING 10.0.0.10 (10.0.0.10): 56 data bytes
64 bytes from 10.0.0.10: icmp_seq=0 ttl=64 time=0.164 ms

--- 10.0.0.10 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.164/0.164/0.164/0.000 ms

We can do the same from the jail.

root@jail-host:~ # jexec www
root@www:/ # ping -c 1 10.0.0.1
PING 10.0.0.1 (10.0.0.1): 56 data bytes
64 bytes from 10.0.0.1: icmp_seq=0 ttl=64 time=0.087 ms

--- 10.0.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.087/0.087/0.087/0.000 ms

We can also stop all the jails.

root@jail-host:~ # service jail stop
Stopping jails: www.

Okay! Couple of notes :)

You can have jail.conf at /etc/jail.conf or /etc/something.jail.conf. The problem with the latter is that if you have jail_enable="YES" in rc.conf without defining jail_list then it will run only the jails in /etc/jail.conf

There are more ways to configure VNET Jails, either with jib or jng, an example is here.

Ideally, it would be nice to have /etc/jail.d/myjail.conf, and I wrote a patch for that (D24570), if you are a FreeBSD developer, please have a look :-)