Tag Archives: Jailer

The FreeBSD-native-ish home lab and network

For many years my setup was pretty simple: A FreeBSD home server running on my old laptop. It runs everything I need to be present on the internet, an email server, a web server (like the one you’ve accessed right now to see this blog post) and a public chat server (XMPP/Jabber) so I can be in touch with friends.

For my home network, I had a basic Access Point and a basic Router.

Lately, my setup has become more… intense. I have IPv6 thanks to Hurricane Electric, the network is passed to my home network (which we’ll talk about in a bit), a home network with multiple VLANs, since friends who come home also need WiFi.

I decided to blog about the details, hoping it would help someone in the future.

I’ll start with the simplest one.

The Home Server

I’ve been running home servers for a long time. I believe that every person/family needs a home server. Forget about buying your kids iPads and Smartphones. Their first devices should be a real computer (sorry Apple, iOS devices are still just a toy) like a desktop/laptop and a home server. The home server doesn’t need to be on the public internet, but mine is, for variety of reasons. This blog being one of them.

I get a static IP address from my ISP, Ucom. After the management change that happened couple of years ago, Ucom has become a very typical ISP (think shitty), but they are the only ones that provide a static IP address, instead of setting it on your router, where you have to do port forwarding.

My home server, hostnamed pingvinashen (meaning the town of the penguins, named after the Armenian cartoon) run FreeBSD. Historically this machine has run Debian, Funtoo, Gentoo and finally FreeBSD.

Hardware wise, here’s what it is:

root@pingvinashen:~ # dmidecode -s system-product-name
Latitude E5470
root@pingvinashen:~ # sysctl hw.model
hw.model: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz
root@pingvinashen:~ # sysctl hw.physmem
hw.physmem: 17016950784
root@pingvinashen:~ # zpool list
zroot   420G   178G   242G        -         -    64%    42%  1.00x    ONLINE  -

While most homelabbers use hardware virtualization, I think that resources are a tight thing, and should be managed properly. Any company that markets itself as “green/eco-friendly” and uses hardware virtualization should do calculations using a pen and paper and prove if going native would save power/resources or not. (sometimes it doesn’t, usually it does)

I use containers, the old-school ones, Jails to be more specific.

I manage jails using Jailer, my own tool, that tries to stay out of your way when working with Jails.

Here are my current jails:

root@pingvinashen:~ # jailer list
NAME        STATE    JID  HOSTNAME              IPv4               GW
antranig    Active   1    antranig.bsd.am
antranigv   Active   2    antranigv.bsd.am
git         Stopped
huginn0     Active   4    huginn0.bsd.am
ifconfig    Active   5    ifconfig.bsd.am
lucy        Active   6    lucy.vartanian.am
mysql       Active   7    mysql.antranigv.am
newsletter  Active   8    newsletter.bsd.am
oragir      Active   9    oragir.am   
psql        Active   10   psql.pingvinashen.am
rss         Active   11   rss.bsd.am  
sarian      Active   12   sarian.am   
syuneci     Active   13   syuneci.am  
znc         Active   14   znc.bsd.am  

You already get a basic idea of how things are. Each of my blogs (Armenian and English) has its own Jail. Since I’m using WordPress, I need a database, so I have a MySQL jail (which ironically runs MariaDB) inside of it.

I also have a Git server, running gitea, which is down at the moment as I’m doing maintanence. The Git server (and many other services) requires PostgreSQL, hence the existence of  a PostgreSQL jail. I run huginn for automation (RSS to Telegram, RSS to XMPP). My sister has her own blog, using WordPress, so that’s a Jail of its own. Same goes about my fiancée.

Other Jails are Newsletter using Listmonk, Sarian (the Armenian instance of lobste.rs) and a personal ZNC server.

As an avid RSS advocate, I also have a RSS Jail, which runs Miniflux. Many of my friends use this service.

Oragir is an instance of WriteFreely, as I advocate public blogging and ActivityPub. Our community uses that too.

The web server that forwards all this traffic from the public to the Jails is nginx. All it does is proxy_pass as needed. It runs on the host.

Other services that run on the host are DNS (BIND9), an email service running OpenSMTPd (which will be moved to a Jail soon), the chat service running prosody (which will be moved to a Jail soon) and finally, WireGuard, because I love VPNs.

Finally, there’s a IPv6-over-IPv4 tunnel that I use to obtain IPv6 thanks to Hurricane Electric.

Yes, I have a firewall, I use pf(4).

For the techies in the room, here’s what my rc.conf looks like.

# cat /etc/rc.conf
# Defaults
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable


# Networking

vlans_em0="37 1000" # 1000 -> WAN; 37 -> Home Router

ifconfig_em0_1000="inet netmask"
ifconfig_em0_37="inet netmask"

route_home="-net -gateway"

cloned_interfaces="bridge0 bridge6 bridge10"
ifconfig_bridge10="inet netmask"

## IPv6

ifconfig_gif0="inet6 2001:470:1f14:ef::2 2001:470:1f14:ef::1 prefixlen 128"

ifconfig_em0_37_ipv6="inet6 2001:470:7914:7065::2 prefixlen 64"
ipv6_static_routes="home guest"
ipv6_route_home="-net 2001:470:7914:6a76::/64 -gateway 2001:470:7914:7065::1"
ipv6_route_guest="-net 2001:470:7914:6969::/64 -gateway 2001:470:7914:7065::1"

ifconfig_bridge6_ipv6="inet6 2001:470:1f15:e4::1 prefixlen 64"

ifconfig_bridge6_aliases="inet6 2001:470:1f15:e4::25 prefixlen 64 \
inet6 2001:470:1f15:e4::80 prefixlen 64      \
inet6 2001:470:1f15:e4::5222 prefixlen 64    \
inet6 2001:470:1f15:e4:c0fe::53 prefixlen 64 \


# Firewall

# Jails


# Mail


# Web

The gif0 interface is a IPv6-over-IPv4 tunnel. I have static routes to my home network, so I don’t go to my server over the ISP every time. This also gives me the ability to get IPv6 in my home network that is routed via my home server.

As you have guessed from this config file, I do have VLANs setup. So let’s get into that.

The Home Network

First of all, here’s a very cheap diagram

I have the following VLANs setup on the switch.

VLAN ID Purpose
1 Switch Management
1000 pingvinashen (home server) WAN
1001 evn0 (home router) WAN
37 pingvinashen ↔ evn0
42 Internal Management
100 Home LAN
69 Home Guest

Here are the active ports

Port VLANs Purpose
24 untagged: 1 Switch management, connects to Port 2
22 untagged: 1000 pingvinashen WAN, from ISP
21 untagged: 1001 Home WAN, from ISP
20 tagged: 1000, 37 To pingvinashen, port em0
19 untagged: 1001 To home router, port igb1
18 tagged: 42, 100, 69, 99 To home router, port igb2
17 untagged: 37 To home router, port igb0
16 tagged: 42, 100, 69 To Lenovo T480s
15 untagged: 100 To Raspberri Pi 4
2 untagged: 99 From Port 24, for switch management
1 untagged: 42; tagged: 100, 69; PoE To UAP AC Pro

The home router, hostnamed evn0 (named after the IATA code of Yerevan’s Zvartnots International Airport) runs FreeBSD as well, the hardware is the following

root@evn0:~ # dmidecode -s system-product-name
root@evn0:~ # sysctl hw.model
hw.model: AMD GX-412TC SOC                               
root@evn0:~ # sysctl hw.physmem
hw.physmem: 4234399744
root@evn0:~ # zpool list
zroot  12.5G  9.47G  3.03G        -         -    67%    75%  1.00x    ONLINE  -

The home router does… well, routing. It also does DHCP, DNS, SLAAC, and can act as a syslog server.

Here’s what the rc.conf looks like

syslogd_flags="-a '*' -H"




# Get an IP address from the ISP's GPON

# Internal routes with pingvinashen
ifconfig_igb0="inet netmask"
ifconfig_igb0_ipv6="inet6 2001:470:7914:7065::1 prefixlen 64"

route_pingvinashen="-net -gateway"


# Home Mgmt, Switch Mgmt, Home LAN, Home Guest
vlans_igb2="42 99 100 69"
ifconfig_igb2_42="inet netmask"
ifconfig_igb2_99="inet netmask"

ifconfig_igb2_100="inet netmask"
ifconfig_igb2_100_ipv6="inet6 2001:470:7914:6a76::1 prefixlen 64"

ifconfig_igb2_69="inet netmask"
ifconfig_igb2_69_ipv6="inet6 2001:470:7914:6969::1 prefixlen 64"

# DNS and DHCP



# Router Advertisement and LLDP

Here’s pf.conf, because security is important.



nat pass on $ext_if from $int_if:network to any -> ($ext_if)
nat pass on $ext_if from $mgmt_if:network to any -> ($ext_if)
nat pass on $ext_if from $guest_if:network to any -> ($ext_if)

set skip on { lo0 }

block in all

pass on $int_if   from $int_if:network   to any
pass on $mgmt_if  from $mgmt_if:network  to any
pass on $sw_if    from $sw_if:network    to any
pass on $guest_if from $guest_if:network to any

block quick on $guest_if from any to { $int_if:network, $mgmt_if:network, $ill_net, $sw_if:network }

pass in on illuria0 from $ill_net to { $ill_net, $mgmt_if:network }

pass inet  proto icmp
pass inet6 proto icmp6
pass out   all   keep state

I’m sure there are places to improve, but it gets the job done and keeps the guest network isolated.

Here’s rtadvd.conf, for my IPv6 folks



For DNS, I’m running BIND, here’s the important parts

listen-on     {;;;;; };
listen-on-v6  { 2001:470:7914:6a76::1; 2001:470:7914:6969::1; };
allow-query   {;;;; 2001:470:7914:6a76::/64; 2001:470:7914:6969::/64;};

And for DHCP, here’s what it looks like

subnet netmask {
        option domain-name-servers;
        option subnet-mask;
        option routers;
        option domain-name "evn0.loc.illuriasecurity.com";
        option domain-search "loc.illuriasecurity.com evn0.loc.illuriasecurity.com";

host zvartnots {
    hardware ethernet d4:57:63:f1:5a:36;

host unifi0 {
    hardware ethernet 58:9c:fc:93:d1:0b;
[…] subnet netmask { range; option domain-name-servers; option subnet-mask; option routers; } subnet netmask { range; option domain-name-servers; option subnet-mask; option routers; }

So you’re wondering, what’s this unifi0? Well, that brings us to


This laptop has been gifted to me by [REDACTED] for my contributions to the Armenian government (which means when a server goes down and no one knows how to fix it, they called me and I showed up)

Here’s the hardware

root@t480s:~ # dmidecode -s system-version
ThinkPad T480s
root@t480s:~ # sysctl hw.model
hw.model: Intel(R) Core(TM) i5-8350U CPU @ 1.70GHz
root@t480s:~ # sysctl hw.physmem
hw.physmem: 25602347008
root@t480s:~ # zpool list
zroot   224G   109G   115G        -         -    44%    48%  1.00x    ONLINE  -

The T480s has access to VLAN 100, 42, 69, but the host itself has access only to VLAN 100 (LAN), while the jails can exist on other VLANs.

So I have a Jail named unifi0 that runs the Unifi Management thingie.

Here’s what rc.conf of the host looks like

# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable


ifconfig_em0="up -rxcsum -txcsum"
vlans_em0="100 42 69"

cloned_interfaces="bridge0 bridge100 bridge42 bridge69"

create_args_bridge100="ether 8c:16:45:82:b4:10"
ifconfig_bridge100="addm em0.100 SYNCDHCP"
ifconfig_bridge100_ipv6="inet6 auto_linklocal"
rtsold_flags="-i -F -m bridge100"

create_args_bridge42=" ether 8c:16:45:82:b4:42"
create_args_bridge69=" ether 8c:16:45:82:b4:69"

ifconfig_bridge42="addm em0.42"
ifconfig_bridge69="addm em0.69"


ifconfig_bridge0="inet up"


I used Jailer to create the unifi0 jail, here’s what the jail.conf looks like

# vim: set syntax=sh:

unifi0 {
  $id             = "6";
  devfs_ruleset   = 10;
  $bridge         = "bridge42";
  $domain         = "evn0.loc.illuriasecurity.com";
  vnet.interface = "epair${id}b";

  exec.prestart   = "ifconfig epair${id} create up";
  exec.prestart  += "ifconfig epair${id}a up descr vnet-${name}";
  exec.prestart  += "ifconfig ${bridge} addm epair${id}a up";

  exec.start      = "/sbin/ifconfig lo0 up";
  exec.start     += "/bin/sh /etc/rc";

  exec.stop       = "/bin/sh /etc/rc.shutdown jail";
  exec.poststop   = "ifconfig ${bridge} deletem epair${id}a";
  exec.poststop  += "ifconfig epair${id}a destroy";

  host.hostname   = "${name}.${domain}";
  path            = "/usr/local/jailer/unifi0";
  exec.consolelog = "/var/log/jail/${name}.log";

Here are the important parts inside the jail

root@t480s:~ # cat /usr/local/jailer/unifi0/etc/rc.conf
root@t480s:~ # cat /usr/local/jailer/unifi0/etc/start_if.epair6b 
ifconfig epair6b ether 58:9c:fc:93:d1:0b

Don’t you love it that you can see what’s inside the jail from the host? God I love FreeBSD!

Did I miss anything? I hope not.

Oh, for the homelabbers out there, the T480s is the one that runs things like Jellyfin if needed.

Finally, the tiny 

Raspberry Pi 4, Model B

I found this in a closed, so I decided to run it for TimeMachine.

I guess all you care about is rc.conf

ifconfig_DEFAULT="DHCP inet6 accept_rtadv"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable

And the Samba Configuration

# Network settings
workgroup = WORKGROUP
server string = Samba Server %v
netbios name = RPi4

# Logging
log file = /var/log/samba4/log.%m
max log size = 50
log level = 0

# Authentication
security = user
encrypt passwords = yes
passdb backend = tdbsam
map to guest = Bad User

min protocol = SMB2
max protocol = SMB3

# Apple Time Machine settings
vfs objects = catia fruit streams_xattr
fruit:metadata = stream
fruit:resource = stream
fruit:encoding = native
fruit:locking = none
fruit:time machine = yes

# File System support
ea support = yes
kernel oplocks = no
kernel share modes = no
posix locking = no
mangled names = no
smbd max xattr size = 2097152

# Performance tuning
read raw = yes
write raw = yes
getwd cache = yes
strict locking = no

# Miscellaneous
local master = no
preferred master = no
domain master = no
wins support = no

comment = Time Machine RPi4
path = /usr/local/timemachine/%U
browseable = yes
read only = no
valid users = antranigv
vfs objects = catia fruit streams_xattr
fruit:time machine = yes
fruit:advertise_fullsync = true
fruit:time machine max size = 800G  # Adjust the size according to your needs
create mask = 0600
directory mask = 0700

That’s pretty much it.


I love running homebrew servers, home networks and home labs. I love that (almost) everything is FreeBSD. The switch itself runs Linux, and the Unifi Access Point also runs Linux, both of which I’m pretty happy with.

While most homelabbers used ESXi in the past, I’m happy to see that most people are moving to open source solutions like Proxmox and Xen, but I think that FreeBSD Jails and bhyve is much better. I still don’t have a need for bhyve at the moment, but I would use it if I needed hardware virtualization.

Most homelabbers would consider the lack of Web/GUI interfaces as a con, but I think that it’s a pro. If I need to “replicate” this network, all I need to do is to copy some text files and modify some IP addresses / Interface names.

I hope this was informative and that it would be useful for anyone in the future.

That’s all folks… 

Reply via email.

Installing DFIR-IRIS on FreeBSD using Jails

This is a live blogging of the installation process of DFIR-IRIS on FreeBSD 14.0-RELEASE using Jails and Jailer.

The main requirements are:

  • Nginx
  • PostgreSQL
  • Python
  • Some random dependencies we saw in the Dockerfile

I assume you already have nginx up and running, we will just be setting up a vhost under the domain name dfir.cert.am. Don’t worry, this is INSIDE our infrastructure, you will not be able to connect to it 🙂

Initial Setup

First we create a jail named iris0, using Jailer:

jailer create iris0

Next we install the required software inside of the jail. Looks like everything is available in FreeBSD packages:

jailer console iris0
pkg install \ nginx \ python39 \ py39-pip \ gnupg \ 7-zip \ rsync \ postgresql12-client \ git-tiny \ libxslt \ rust \ acme.sh

Installing DFIR-IRIS

Since we’re using FreeBSD, we’ll be doing things the right way instead of the Docker way, so we will be running IRIS as a user, not as root.

pw user add iris -m

Next we setup some directories and checkout the repo

root@iris0:~ # pw user add iris -m
root@iris0:~ # su - iris iris@iris0:~ $ git clone --branch v2.4.7 https://github.com/dfir-iris/iris-web.git iris-web

Finally, we install some python dependencies using pip.

iris@iris0:~ $ cd iris-web/source
iris@iris0:~/iris-web/source $ pip install -r requirements.txt

Now we have to configure the .env file based on our needs, I will post my version of it, I hope it helps

export POSTGRES_USER=postgres
export POSTGRES_PASSWORD=postgres
export POSTGRES_DB=iris_db
export POSTGRES_ADMIN_PASSWORD=longpassword

export POSTGRES_SERVER=localhost
export POSTGRES_PORT=5432

# -- IRIS
export IRIS_SECRET_KEY=verylongsecret
export IRIS_UPSTREAM_SERVER=app # these are for docker, you can ignore

export CELERY_BROKER=amqp://localhost
# Set to your rabbitmq instance

# Change these as you need them.
# -- AUTH
## optional
# requests the just-in-time creation of users with ldap authentification (see https://github.com/dfir-iris/iris-web/issues/203)
# the group to which newly created users are initially added, default value is Analysts


Configuring HTTPS

We can use acme.sh to issue a TLS certificate from Lets Encrypt.

root@iris0:~ # acme.sh --set-default-ca --server letsencrypt
root@iris0:~ # acme.sh --issue -d dfir.cert.am --standalone
root@iris0:~ # acme.sh -i -d dfir.cert.am --fullchain-file /usr/local/etc/ssl/dfir.cert.am/fullchain.pem --key-file /usr/local/etc/ssl/dfir.cert.am/key.pem --reloadcmd 'service nginx reload'

Setup nginx

DFIR-IRIS provides a nginx configuration template at nginx.conf, we will be using that, with a little bit of modifications.

The final nginx.conf will look like this:

#user  nobody;
worker_processes  1;

# This default error log path is compiled-in to make sure configuration parsing
# errors are logged somewhere, especially during unattended boot when stderr
# isn't normally logged anywhere. This path will be touched on every nginx
# start regardless of error log location configured here. See
# https://trac.nginx.org/nginx/ticket/147 for more info. 
#error_log  /var/log/nginx/error.log;

#pid        logs/nginx.pid;

events {
    worker_connections  1024;

http {
    include       mime.types;
    default_type  application/octet-stream;

    # Things needed/recommended by DFIR-IRIS
    map $request_uri $csp_header {
        default "default-src 'self' https://analytics.dfir-iris.org; script-src 'self' 'unsafe-inline' https://analytics.dfir-iris.org; style-src 'self' 'unsafe-inline';";

    server_tokens off;
    sendfile    on;
    tcp_nopush  on;
    tcp_nodelay on;

    types_hash_max_size             2048;
    types_hash_bucket_size          128;
    proxy_headers_hash_max_size     2048;
    proxy_headers_hash_bucket_size  128;
    proxy_buffering                 on;
    proxy_buffers                   8 16k;
    proxy_buffer_size               4k;

    client_header_buffer_size   2k;
    large_client_header_buffers 8 64k;
    client_body_buffer_size     64k;
    client_max_body_size        100M;

    reset_timedout_connection   on;
    keepalive_timeout           90s;
    client_body_timeout         90s;
    send_timeout                90s;
    client_header_timeout       90s;
    fastcgi_read_timeout        90s;
    proxy_read_timeout          90s;
    uwsgi_read_timeout          90s;

    gzip off;
    gzip_disable "MSIE [1-6]\.";

    proxy_set_header    HOST                $http_host;
    proxy_set_header    X-Forwarded-Proto   $scheme;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;

    add_header          Last-Modified $date_gmt;
    add_header          'Cache-Control' 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
    if_modified_since   off;
    expires             off;
    etag                off;
    proxy_no_cache      1;
    proxy_cache_bypass  1;

    ssl_protocols               TLSv1.2 TLSv1.3;

    ssl_prefer_server_ciphers   on;
    ssl_certificate             /usr/local/etc/ssl/dfir.cert.am/fullchain.pem;
    ssl_certificate_key         /usr/local/etc/ssl/dfir.cert.am/key.pem;
    ssl_ecdh_curve              secp521r1:secp384r1:prime256v1;
    ssl_buffer_size             4k;

    ssl_session_tickets         off;
    ssl_session_cache           none;
    server {
        listen          443 ssl
        server_name     dfir.cert.am;
        root            /www/data;
        index           index.html;
        error_page      500 502 503 504  /50x.html;

        add_header Content-Security-Policy $csp_header;
        add_header X-XSS-Protection             "1; mode=block";
        add_header X-Frame-Options              DENY;
        add_header X-Content-Type-Options       nosniff;
        # max-age = 31536000s = 1 year
        add_header Strict-Transport-Security    "max-age=31536000: includeSubDomains" always;
        add_header Front-End-Https              on;

        location / {
            proxy_pass  http://localhost:8000;

            location ~ ^/(manage/templates/add|manage/cases/upload_files) {
                keepalive_timeout           10m;
                client_body_timeout         10m;
                send_timeout                10m;
                proxy_read_timeout          10m;
                client_max_body_size        0M;
                proxy_request_buffering off;
                proxy_pass  http://localhost:8000;

            location ~ ^/(datastore/file/add|datastore/file/add-interactive) {
                keepalive_timeout           10m;
                client_body_timeout         10m;
                send_timeout                10m;
                proxy_read_timeout          10m;
                client_max_body_size        0M;
                proxy_request_buffering off;
                proxy_pass  http://localhost:8000;
        location /socket.io {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_http_version 1.1;
            proxy_buffering off;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "Upgrade";
            proxy_pass http://localhost:8000/socket.io;

Setup PostgreSQL

I assume you know how to do this 🙂 You don’t need to configure a separate user, by the looks of it, IRIS likes to do that itself. Thanks to Jails I was able to run a separate PostgreSQL instance in the iris0 jail.

P.S. If you are running PostgreSQL inside a jail, make sure that the following variables are set in your jail configuration

  sysvshm         = new;
  sysvmsg         = new;


Now that everything is up and running, we just need to run DFIR-IRIS and it will create the database, needed users, an administration account, etc.

su - iris
cd ~/iris-web/source
. ../.env
~/.local/bin/gunicorn app:app --worker-class eventlet --bind --timeout 180 --worker-connections 1000 --log-level=debug

Assuming everything is fine, now we can setup a rc.d service script to make sure it runs at boot.

For that I wrote two files, the service itself and a helper start.sh script

rc.d script at /usr/local/etc/rc.d/iris


# PROVIDE: iris

. /etc/rc.subr

load_rc_config ${name}

: ${iris_enable:=no}
: ${iris_path:="/usr/local/iris"}
: ${iris_gunicorn:="/usr/local/bin/gunicorn"}
: ${iris_env="iris_gunicorn=${iris_gunicorn}"}



command_args="-P ${pidfile} -T ${name} -o ${logfile} ${iris_command}"

run_rc_command "$1"

and the helper script at /home/iris/iris-web/start.sh


export HOME=$(getent passwd `whoami` | cut -d : -f 6)

. ../.env

${iris_gunicorn} app:app --worker-class eventlet --bind --timeout 180 --worker-connections 128

now we set some variables in rc.conf using sysrc and we can start the service.

sysrc iris_enable="YES"
sysrc iris_path="/home/iris/iris-web"
sysrc iris_gunicorn="/home/iris/.local/bin/gunicorn"

Finally, we can start DFIR-IRIS as a service.

service iris start

Aaaaand we’re done 🙂

Thank you for reading!

There are some issues that I’d like to tackle, for example, service iris stop doesn’t work, and it would be nice if we ported all of the dependencies into Ports, but for now, this seems to be working fine.

Special thanks to the DFIR-IRIS team for creating this cool platform!

That’s all folks…

Reply via email.

Antranig Vartanian

March 29, 2023

After weeks of thinking, I decided that I need to fork Jailer. Yes, I want to fork my own code. There are two reasons to do this.

  1. Keep the promise of Jailer being “very compatible with FreeBSD”
  2. Have a new version that pushes these limits of compatibility.

The fork is going to be named bant, which is Armenian for jail. I think we’re all tired of Greek names at this point 🙂

I’ll share the details of bant as soon as I have a prototype, which means at least couple of weeks.

Meanwhile, Jailer will be the very-compatible-with-FreeBSD version, that doesn’t brake things and allows new users to use Jails with ease.

Fingers crossed…

Reply via email.

Antranig Vartanian

March 29, 2023

Code cleanup is hard. I’ve been trying to make Jailer’s code simpler, more modular for hours now, and the more I try to simplify it, the harder it is.

My biggest issue right now is this massive complex code that does image management. The end goal is to make it so simple, that anyone with basic knowledge if shell can understand it.

Reply via email.

Design Guidelines vs Pushing The Limits

One of the design guidelines of Jailer is don’t break FreeBSD. As in if someone installed and used Jailer, and then deleted the Jailer binary and libraries, their Jails would still run without any issues. We do this with minimal intervention, for example, jailer init patches FreeBSD’s /etc/rc.d/jail, but in a way that you wouldn’t feel the difference much. We don’t create new rc.conf variables, we just change couple of loops. In a way, you can keep these changes even if you delete Jailer so your system would be much improved. Obviously, we do sent these patches to FreeBSD src.

But I’m in front of an issue right now. On one side, I want to keep these guidelines, on the other, pushing the limit will allow me to improve Jailer way more than I expected.

These are the things that I think about before sleep, or during the shower. I gave a promise, that I will not break the Jail ecosystem. But what if, just what if, the ecosystem was broken in the first place?

Some of you might know, that we’ve been working on integrating libucl with Jail. The experiments have been going well, in such that I feel I want to integrate these experiments with Jailer already, even before they get into FreeBSD (and they might even not get in at all).

My dream of Jailer and its ecosystem is complex. I feel that these integration would do good on the long-term, but I want to keep the short term alive as well.

One idea is to fork Jailer, keep two versions of it. One version that’s FreeBSD compliant, and another one that is pushing the limits.

This is going to be an interesting week…

That’s all folks…

Reply via email.

Call For Testing: Jailer v0.1.1

Well, it’s finally here! After a week of sleepless work, I cleaned up the Jailer codebase and added many features (and removed some as well!) that I wanted since last year 🙂

If you are reading this, please consider testing Jailer on FreeBSD. The codebase is at illuria/jailer.

The README.md should have all the info that you need to run Jailer.

If find any issues, please report to illuria/jailer/issues or you can email me personally at antranigv [at] freebsd [dot] am

Here’s the roadmap for what’s coming next

  1. Complete NetGraph support using jng.
  2. Jailerfile, which will be something similar to Dockerfile, allowing developers to create consistent images.
  3. jailerd and jailerctl, for remote jailer automation. This will be an open-source port of what illuria has already developed.
  4. Distributed Jailer, where jailerctl list will show not just what’s on a remote machine, but on a remote datacenter, inspired by Triton. Again, we have this at illuria, but we need to create an open-source port.

This release is dedicated to

Thank you for reading 🙂

That’s all folks…

Reply via email.

Antranig Vartanian

March 3, 2023

I’ve been working very intensively with Jailer the last couple of days.

The first thing I added, which I personally needed it is a dry run, where Jailer says what it would do. Here’s an example.

Screenshot 2023 03 02 at 8 48 08 PM

by adding -D Jailer would display what it would do, including post the jail.conf file, ZFS commands and any network setup commands.

A while back kfv has made a jailer init

subcommand that initializes the system. Things like rc.conf variables, ZFS datasets and applying our own patches. I just added some coloring and confirmation to that!

Screenshot 2023 03 03 at 3 27 35 PM

I think I will be making an official Jailer v0.1.0 release in the coming days!

Reply via email.

VoidLinux in FreeBSD Jail; with init

Two important things happened this week for me.

First, Faraz asked me if I can rename my Jail manager to something other than Jailio because he got that domain for his Jailer manager already. So I named it

Second, I was able to run a complete Linux system using Jailer. While the repo for Jailer is not released yet (we are auditing for possible security issues), I would like to share how I was able to run VoidLinux in a Jail.

Since Jailer is not announced yet, I will give the examples using jail.conf, as most people either are or should be familiar with its concepts.

I went with VoidLinux because I am able to run the init process without its need to be running as PID1.

Let’s start, shall we?

First, ZFS dataset for our jail!

zfs create zroot/jails/voidlinux

Next we need to fetch the base system of VoidLinux. Luckily they do provide it on their website.

fetch https://alpha.de.repo.voidlinux.org/live/current/void-x86_64-ROOTFS-20210218.tar.xz

Now we can extract this into our dataset

tar xf void-x86_64-ROOTFS-20210218.tar.xz -C /usr/local/jails/voidlinux/

You might get an error that ./usr/bin/iputils-ping: Cannot restore extended attributes: security.capability, which is fine, I think?

If you are on FreeBSD 12.2-RELEASE or later, now you need to enable the Linuxulator.

service linux enable; service linux start

Now you can at least chroot into the system.

chroot /usr/local/jails/voidlinux/ /bin/bash

If everything is fine until now, perfect.

Now we need to add a root user into the system.

root@host:~ # cd /usr/local/jails/voidlinux/etc/
root@host:/usr/local/jails/voidlinux/etc # echo "root::0:0::0:0:Charlie &:/root:/bin/bash" > master.passwd
root@host:/usr/local/jails/voidlinux/etc # pwd_mkdb -d ./ -p master.passwd
pwd_mkdb: warning, unknown root shell

Execute the rest of the commands in Void.

root@host:~ # chroot /usr/local/jails/voidlinux/ /bin/bash
bash-5.1# cd /etc/
bash-5.1# pwconv 
bash-5.1# grpconv 
bash-5.1# passwd 
New password: 
Retype new password: 
passwd: password updated successfully
bash-5.1# exit

If all went fine, then the system is ready to be run as a Jail!

First we need to make an fstab for the system.

Create a file at /usr/local/jails/voidlinux/etc/fstab.pre and insert the following inside

devfs       /usr/local/jails/voidlinux/dev      devfs           rw                      0   0
tmpfs       /usr/local/jails/voidlinux/dev/shm  tmpfs           rw,size=1g,mode=1777    0   0
fdescfs     /usr/local/jails/voidlinux/dev/fd   fdescfs         rw,linrdlnk             0   0
linprocfs   /usr/local/jails/voidlinux/proc     linprocfs       rw                      0   0
linsysfs    /usr/local/jails/voidlinux/sys      linsysfs        rw                      0   0
/tmp        /usr/local/jails/voidlinux/tmp      nullfs          rw                      0   0

Next, let’s create a loopback interface for networking. Oh yes, VNET is not supported yet, but I’m working on a patch 🙂

ifconfig lo1 create
ifconfig lo1 inet up # sorry, was unavailable :P

Okay, time to create our Jail conf!


voidlinux {
    $id     = "1";
    $ipaddr = "";
    $mask   = "";
    $domain = "srv0.bsd.am";
    devfs_ruleset  = 4;
    mount.fstab = "${path}/etc/fstab.pre";

    exec.start     = "/bin/sh /etc/runit/2 &";
    exec.stop      = "/bin/sh /etc/runit/3";

    ip4.addr      = "${ipaddr}";
    interface     = "lo1";
    host.hostname = "${name}.${domain}";
    path = "/usr/local/jails/voidlinux";
    exec.consolelog = "/var/log/jail-${name}.log";

Let’s check?

# jls
   JID  IP Address      Hostname                      Path
     1    voidlinux.srv0.bsd.am         /usr/local/jails/voidlinux

And the process tree?

# ps auxd -J voidlinux
root 35182  0.0  0.1 2320 1428  -  SsJ  21:09   0:00.12 runsvdir -P /run/runit/runsvdir/current log: ot set SO_PASSCRED: Protocol not available\ncould not set SO_PASSCRED: Protocol
root 35190  0.0  0.1 2168 1376  -  SsJ  21:09   0:00.02 - runsv agetty-tty6
root 35397  0.0  0.1 2412 1704  -  SsJ  21:10   0:00.00 `-- agetty tty6 38400 linux
root 35191  0.0  0.1 2168 1376  -  SsJ  21:09   0:00.02 - runsv agetty-tty1
root 35396  0.0  0.1 2412 1704  -  SsJ  21:10   0:00.00 `-- agetty --noclear tty1 38400 linux
root 35192  0.0  0.1 2168 1376  -  SsJ  21:09   0:00.02 - runsv agetty-tty5
root 35398  0.0  0.1 2412 1704  -  SsJ  21:10   0:00.01 `-- agetty tty5 38400 linux
root 35193  0.0  0.1 2168 1376  -  SsJ  21:09   0:00.02 - runsv agetty-tty2
root 35393  0.0  0.1 2412 1704  -  SsJ  21:10   0:00.00 `-- agetty tty2 38400 linux
root 35194  0.0  0.1 2168 1396  -  RsJ  21:09   0:00.12 - runsv udevd
root 35195  0.0  0.1 2168 1376  -  SsJ  21:09   0:00.02 - runsv agetty-tty3
root 35394  0.0  0.1 2412 1704  -  SsJ  21:10   0:00.00 `-- agetty tty3 38400 linux
root 35196  0.0  0.1 2168 1376  -  SsJ  21:09   0:00.02 - runsv agetty-tty4
root 35390  0.0  0.1 2412 1704  -  SsJ  21:10   0:00.00 `-- agetty tty4 38400 linux

You may jexec now 🙂

# jexec voidlinux /bin/bash
bash-5.1# uname -a
Linux voidlinux.srv0.bsd.am 3.2.0 FreeBSD 12.2-RELEASE-p6 GENERIC x86_64 GNU/Linux

Let’s check networking?

bash-5.1# ping -c 1
ping: WARNING: setsockopt(ICMP_FILTER): Protocol not available
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=64 time=0.069 ms

--- ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.069/0.069/0.069/0.000 ms

There you go! Well, things that are related to netlink might not work, but other than that it’s okay.

I did have some problems while installing packages, something about too many levels of symbolic links. Here’s the exact output when I was trying to install the curl package

[*] Unpacking packages
libev-4.33_1: unpacking ...
ERROR: libev-4.33_1: [unpack] failed to extract file `./usr/lib/libev.so.4': Too many levels of symbolic links
ERROR: libev-4.33_1: [unpack] failed to extract files: Too many levels of symbolic links
ERROR: libev-4.33_1: [unpack] failed to unpack files from archive: Too many levels of symbolic links
Transaction failed! see above for errors.

Now, I did not find the time to fix this yet, but if you have any idea, please let me know or comment below 🙂

So, what do we have here? A Linux Jail, running VoidLinux, with init, so you can also run services, and basic networking for it.

That’s all folks…

Reply via email.

Two Colons Equals Modules

Days ago I tweeted a shell function which is part of jailio’s code base. Jailio is a project I’ve been working on for the last 6 months. As the name implies, it’s a container management software for FreeBSD Jails.

It has two unique things compared to other Jail management software. First of all, it has no dependencies, it’s written purely in Shell. You can say the same about BastilleBSD, however, Jailio’s second unique thing is that it uses base tools only and requires the base system only. For example, you need to have bastille_enable in BastilleBSD, it also uses its own config files, etc. In Jailio, you need to have jail_enable, because technically Jailio automates jail.conf files. It also uses my patch to automate the jail.confs in /etc/jail.conf.d.

Anyway, back to our topic about Colons and Modules.

I like modules, I got introduced to them when I started programming in school. In Syria, we learn programming at 7th grade but in our school we started a year early, so 6th grade. We always start with block diagrams and then Turbo Pascal!

Yes, 16-bit Turbo Pascal was my first programming language and it had the concept of modules which we called Units.

And then you have languages like C or Shell which don’t have modules. If you use modules you KNOW that it’s hard not to use modules after that.

While reading the source code of vm-bhyve I learned that you can use two colons (::) as part of the function name, which can give you an amazing new superpower to take over the world write cleaner code.

For me this was a life-changer. I write a LOT of Shell code. I ship them to production too. No, you don’t need to write everything in a fancy new language and run it on kubernetes, you can always use simple languages like Shell and run them in a FreeBSD Jail. Or in my case, write in Shell to automate FreeBSD Jails.

Here’s an example code with “modules” in Shell. Note, this works in FreeBSD’s shell, I have not tested other Shells yet.



. ./mod1.sh




  printf "Here I am, rock you like a hurricane\n"
antranigv@pingvinashen:~ % ./main.sh 
Here I am, Rock you like a hurricane

As you can see it all relies on the concept that the function name itself has two colons in its name.

Here’s the code from jailio that I tweeted.

  expr $(
    ( grep -s '$id' /etc/jail.conf.d/* || echo '$id = "0";' ) |
    awk -F '[="]' '{print $3}' |
    sort -h |
    tail -1
  ) + 1

After tweeting the code above Annatar replied that this should NOT work elsewhere and that’s how I got introduced to The Heirloom Project which provides traditional implementations of the original Unix tools from the original Unix source code.

Hopefully, I will see more people using “modules” in Shell scripts. Hopefully this trick works in other Shell implementations like Bash and zsh.

That’s all folks.

Reply via email.