Mirroring OmniOS: The Complete Guide; Part One

Chapter Ⅰ

I know that “Complete Guide” and “Part One” are oxymorons, but hey, be happy that I’m publishing in parts, otherwise I’d completely ignore this blog post.

Two weeks ago I decided to play with illumos again. I was speaking with a friend and we were sharing our frustrations regarding Open-Source contribution. We write the code, we submit, we get feedback, we submit again, and then we’re ghosted. It’s like the LinkedIn or Tinder version of Software Engineering.

Then I asked him about his best open-source experience and he told me “illumos of course!”.

I was amazed. I thought you had to be very technical in order to even build illumos, but turns out they have an amazing documentation on building illumos and OmniOS (an illumos distribution) has done work to make sure that the system can be self-hosted (i.e. The OS can build itself).

So, I decided to fire up OmniOS on our hackerspace server running FreeBSD inside a bhyve VM.

The installation went smoothly, but the IPS packages were slow to download, and I might be wrong (please correct me if I am) but IPS doesn’t seem to be keeping a local copy of the files. It always downloads. Is that configurable?

Regardless. I thought that the best way to contribute is to advocate. In order to do that I needed to make sure that IPS servers are fast in Armenia. Hence the mirroring project started.

Obey!

Requirements

Here are some terminology that I will use in this blog post, just so we are on the same page.

  • OmniOS: an illumos distribution
  • Origin: OmniOS’s IPS servers at pkg.omnios.org
  • Local: A copy of the Origin
  • Repository: A collection of software
  • Core: The Core Repository of OmniOS
  • Extras: The Extra Repository of OmniOS
  • IPS or PKG: The Image Packaging System and its utility, pkg
  • Zone: an illumos Zone (similar to FreeBSD Jails, Linux Containers, chroot) running on OmniOS

Now that we are on the same page, let’s talk about our setup and what we need.

  • An internet connection: duh!
  • A domain name: I decided to use pkg.omnios.illumos.am. Yes, I’m lucky like that.
  • A publicly accessible IP address.
  • A server: I am running OmniOS Stable (r151048) inside a VM. You can use bare-metal or a cloud VM if you want.
  • Storage: I am currently using around 50GB of storage, expect that to go around 300GB when we get to Part Three

Pre-Mirroring Setup

Before we setup our mirror, let’s make sure that we have a good infrastructure that we can maintain.

Here’s what we’ll create

  • A Zone that will act as the HTTP(s) server using nginx at IP address 10.10.0.80
  • A Zone that will do the mirroring using IPS tools at 10.10.0.51
  • An virtual dumb switch (etherstub) that will connect the Zones and the Global-Zone (a.k.a The Host) together. The GZ will have an address of 10.10.0.1
  • ZFS datasets for each Core and Extras Repository (for each release)

Please note that there are many ways to do this, for example, having everything in a Global Zone, running IPS mirroring and nginx in a single Zone, not using etherstub at all, etc. But I like this setup as it will allow us to “grow” in the future.

From now on, omnios# means that we’re in the Global Zone and zone0# means we’re inside a Zone named zone0.

Let’s start with setting up our etherstub and connecting our Global Zone to it

omnios# dladm create-etherstub switch0
omnios# dladm create-vnic -l switch0 vnic0
omnios# ipadm create-if vnic0
omnios# ipadm create-addr -T static -a 10.10.0.1/24 vnic0/switch0

Done!

Now, we will setup our Zones using the zadm utility. Install zadm by running

omnios# pkg install zadm

After installing zadm, we’ll create a dataset for our Zones

omnios# zfs create -o mountpoint=/zones rpool/zones

This assumes that your ZFS pool is named rpool.

Finally, we can create our Zones. Running

omnios# zadm create -b pkgsrc www0

will open your $EDITOR, where you need to modify some JSON, here’s what mine looks like!

{
   "autoboot" : "true",
   "brand" : "pkgsrc",
   "ip-type" : "exclusive",
"dns-domain" : "omnios.illumos.am", "net" : [ { "allowed-address" : "10.10.0.80/24", "defrouter" : "10.10.0.1", "global-nic" : "switch0", "physical" : "www0" } ], "pool" : "", "scheduling-class" : "", "zonename" : "www0", "zonepath" : "/zones/www0" }

After saving the file, zadm will install the Zone.

Now let’s setup our mirroring Zone. Do the same but change the Zone name to repo, the brand to lipkg (and -b lipkg) and set the IP address to 10.10.0.51/24.

All we need now is to forward the HTTP/HTTPS traffic to www0 Zone and allow all Zones to access the internet using NAT.

Create and edit the IPFilter’s NAT file at /etc/ipf/ipnat.conf, here’s an example configuration

map vioif0 10.10.0.0/24 -> 212.34.250.10

rdr vioif0 212.34.250.10/32 port 80 -> 10.10.0.80 port 80 tcp
rdr vioif0 212.34.250.10/32 port 443 -> 10.10.0.80 port 443 tcp

Make sure you set the correct interface name and the correct external IP address.

Finally, we can boot our Zones!

omnios# zadm boot www0
omnios# zadm boot repo

You should see the following output when you run zadm again

omnios# zadm
NAME              STATUS     BRAND       RAM    CPUS  SHARES
global            running    ipkg        56G      12       1
repo              running    lipkg         -       -       1
www0              running    pkgsrc        -       -       1

Great! Let’s setup the mirroring process.

Mirroring Setup

Let’s create a ZFS dataset for repos for each release

repo# zfs create -o mountpoint=/repo rpool/zones/repo/ROOT/repo      
repo# zfs create rpool/zones/repo/ROOT/repo/r151048      
repo# zfs create rpool/zones/repo/ROOT/repo/r151048/core 
repo# zfs create rpool/zones/repo/ROOT/repo/r151048/extra

And then we use the pkgrepo command to create a repository

repo# pkgrepo create /repo/r151048/core
repo# pkgrepo create /repo/r151048/extra

And finally, we can start receiving the packages from Origin to Local

repo# pkgrecv -s https://pkg.omnios.org/r151048/core/  -d /repo/r151048/core  '*'
repo# pkgrecv -s https://pkg.omnios.org/r151048/extra/ -d /repo/r151048/extra '*'

This will take a while depending on your internet connection speed and the load on OmniOS’s Origin. It’s like a good investment, we spend load and time now so we save traffic and time later 🙂

After it’s done, we need to set the publisher of these repos the same as Origin.

repo# pkgrepo set -s /repo/r151048/core   publisher/prefix=omnios
repo# pkgrepo set -s /repo/r151048/extra/ publisher/prefix=extra.omnios

And we’re done!

Now need to serve these repos using IPS’s depot server.

We will create two instances of the depotd server, one for core and one for extra.

  • r151048/core will run on 5148
  • r151048/extra will run on 1148
  • (in the future) r151050/core will run on 5150
  • (in the future) r151050/extra will run on 1150

We start with core

repo# svccfg -s pkg/server add r151048_core
repo# svccfg -s pkg/server:r151048_core addpg pkg application
repo# svccfg -s pkg/server:r151048_core setprop pkg/inst_root = /repo/r151048/core/
repo# svccfg -s pkg/server:r151048_core setprop pkg/port = 5148
repo# svccfg -s pkg/server:r151048_core setprop pkg/proxy_base = https://pkg.omnios.illumos.am/r151048/core

And we do the same for extra

repo# svccfg -s pkg/server add r151048_extra
repo# svccfg -s pkg/server:r151048_extra addpg pkg application
repo# svccfg -s pkg/server:r151048_extra setprop pkg/inst_root = /repo/r151048/extra/
repo# svccfg -s pkg/server:r151048_extra setprop pkg/port = 1148
repo# svccfg -s pkg/server:r151048_extra setprop pkg/proxy_base = https://pkg.omnios.illumos.am/r151048/extra

Finally, we enable the services

repo# svcadm enable  pkg/server:r151048_core pkg/server:r151048_extra
repo# svcadm restart pkg/server:r151048_core pkg/server:r151048_extra

Let’s check!

We’re good! Now let’s setup Nginx 🙂

The Web Server

This part is pretty easy, we login into www0, install nginx, and setup some paths. I will be posting a copy-pasta of my configs, I assume you can do the rest 🙂

www0# pkgin update
www0# pkgin install nginx

Thank you SmartOS! 🧡

In my nginx.conf, I added

include vhosts/*.conf;

and then in /opt/local/etc/nginx/vhosts I created a file
named pkg.omnios.illumos.am.conf, which looks like this

server {
        listen 80;
        server_name pkg.omnios.illumos.am;

        location /.well-known/acme-challenge/ {
          alias /opt/local/www/acme/.well-known/acme-challenge/;
        }

        location / {
            return 301 "https://pkg.omnios.illumos.am";
        }
}

server {
    listen       443 ssl;
    server_name  pkg.omnios.illumos.am;

    ssl_certificate      /etc/ssl/pkg.omnios.illumos.am/fullchain.pem;
    ssl_certificate_key  /etc/ssl/pkg.omnios.illumos.am/key.pem;
    location /r151048/core/ {
                proxy_pass http://10.10.0.51:5148/;
    }

    location /r151048/extra/ {
                proxy_pass http://10.10.0.51:1148/;
    }

    location / {
# This needs to be changed, later... add_header Content-Type text/plain; return 200 "ok..."; } }

Finally, we just need to enable nginx

www0# svcadm enable pkgsrc/nginx

and check!

Using the Local Repos

This part is actually pretty easy. We just need to remove everything that exists and add our own. I will be running this on a computer named dna0.

dna0# pkg set-publisher -M '*' -G '*' omnios
dna0# pkg set-publisher -M '*' -G '*' extra.omnios
dna0# pkg set-publisher -O https://pkg.omnios.illumos.am/r151048/core omnios
dna0# pkg set-publisher -O https://pkg.omnios.illumos.am/r151048/extra extra.omnios
dna0# pkg publisher PUBLISHER TYPE STATUS P LOCATION extra.omnios origin online F https://pkg.omnios.illumos.am/r151048/extra/ omnios origin online F https://pkg.omnios.illumos.am/r151048/core/

We’re good! 🙂

Fetching Updates

By the time I wanted to publish this I noticed that there’s a new OmniOS Weekly Update, so I thought, hey, maybe I should try updating the Local Repo as well… how do we do that?

Turns out I just need to pkgrecv again, and then run a refresh command.

pkgrecv -v -s https://pkg.omnios.org/r151048/core/ -d /repo/r151048/core/ '*'
pkgrepo -s /repo/r151048/core refresh

And looks like we’re good! Maybe we can setup a simple cronjob 🙂

Final Notes

This has been an amazing experience. Since I started using OmniOS two weeks ago, I’ve setup the mirror, I installed two OmniOS deployments on production for two organization, and I talked about it during our Armenian Hackers Radio Podcast. With this mirror completely setup, I can advocate even more!

I’d like to send my thanks (and later, my money) to the OmniOS team for the amazing work they’re doing, special thanks to andyf for answering all of my questions, neirac for pushing me to try more illumos in my life and everyone who contributed to the docs and blog posts that I used. I’ll leave some links below.

Finally, for the coming (two) posts I will talk about mirroring downloads.OmniOS.org (for ISO/USB/ZFS images) and the pkgsrc repository run by SmartOS/MNX.

Thank you for reading and thank you, illumos-community for being so nice ^_^

That’s all folks…

Links

Reply via email.

macOS log(1): Finding out the previous name of BT device

I got a new mouse yesterday to use it with Mac Mouse Fix, an amazing application that “Makes Your $10 Mouse Better Than an Apple Trackpad!”. I can assure you it does.

The mouse connects via Bluetooth, a short-range wireless technology that even after 25 years, it’s either insecure, unstable or both. Sometimes it’s none, but only when the vendors of both sides are aware of each other.

Anyways. I connected the mouse and renamed it to “Antranig’s Mouse”, now all I need is a cat. An hour later a friend asked me which model was the mouse. I had no idea, but I thought, hey, the original name of the BT device was the model name, right? Maybe I can check that.

Luckily, macOS logs everything, and I mean everything, so I used the log(1) command to see what was the previous name.

Here’s the command to run and what the output looks like

log show --style compact --info --last 12h --predicate 'process == "bluetoothd" && subsystem == "com.apple.bluetooth”' | grep setName
2024-01-06 18:57:38.908 Df bluetoothd[375:8d0c1] [com.apple.bluetooth:CBStackController] setName: device 01903735-1591-7A71-C597-CE40C2ACB232, 'Dell Mouse MS5120W' -> 'Antranig's Mouse'

A simple explanation:

  • style compact: log has styles of output, there’s the default, which is long, and there’s compact, which is short. You can also set it to json.
  • info: type if information, it can also be default or debug.
  • last: time range, can be set to m, h, d for minutes, hours or days.
  • predicate: a macOS predicate, for more information check Predicate Programming Guide.
    • process: a process, in this case bluetoothd.
    • subsystem: a macOS subsystem, in this case com.apple.bluetooth. How did I know that? note sure, but my brains contains a lot of information.
  • grep: Unix grep(1), because we party like its 1969.

I also don’t remember how I knew that I should look for setName, but that’s life for you.

And of course, we get the output, the device was previously named Dell Mouse MS5120W

That’s all folks…

Reply via email.

macOS Sonoma’s Keyboard Layout Switching: When Apple needs actual diversity

I did it, I finally upgraded to macOS Sonoma. To my surprise there’s only a single thing that’s bugging me… Switching the keyboard layout.

Multi-lingual people use multiple keyboard layouts. Most of the time we use custom keyboard layouts because Apple doesn’t like listening to its customers on how keyboard layouts should look like.

Here’s what happens when you switch the keyboard layout on macOS Sonoma

(and here’s the GIF version)

This is really bad, as many people might have multiple layouts which have the same icon. In my case, for example, I use both the Armenian Eastern Alternative layout (custom made, as Apple still ships a very bad Armenian layout) and the Armenian Typewriter layout (custom made, as Apple still… you get the point).

They both have the same “icon” so it’s impossible to know which layout I’m choosing.

Compare this with macOS Ventura where you can see exactly which layout you’re choosing. Here’s a screenshot from Lilith’s computer.

Yes, Lilith uses Armenian Phonetic with English, also a custom layout, as Apple still… didn’t we just do this?

Clearly, Apple lacks diversity. They don’t have people there who use multiple layouts, or custom layouts, or maybe they all just use Emojis to communicate. I really don’t know how this happened, but it was clearly a very bad decision for the majority of the planet.

Dear Apple, if you are reading this, please just email/iMessage/call me, I will show you to to make this better again (just “minify” the old version) and show you the proper Armenian layouts. There are 7 of them. Actually, just have a look at Xorg, the community has published the proper layouts there decades ago.

Thank you.

That’s all folks… 

Reply via email.

Generating SSHFP Records

I added a new server to our hackerspace last week. This new server will be used for research in security. When I was adding the new DNS record when I realized that the previous server had a SSHFP record as well!

I remember that I should use the ssh-keygen command, however, like a normal human being, I forgot which flag to use. A simple search in the manual page says that I should use -r flag, so here we go.

antranigv@srv0:~ % ssh-keygen -r srv0
srv0 IN SSHFP 1 1 785b3fa04870e92bf25f4c7f7092733acf586ffb
srv0 IN SSHFP 1 2 847fd4a76ef7dfcef31ac3fa18c139413ab0017fa17014b3884bff161c3364de
srv0 IN SSHFP 3 1 8268aa7b8dccf4c0e7881472c72093589ca46b2e
srv0 IN SSHFP 3 2 ea0c9f0a50a825f5a0a59cebf8637876970a34000e6e0afd46bf269e08294a88
srv0 IN SSHFP 4 1 2fbe9d0e2ecdbd9dd58576e4683ee70858ca3f25
srv0 IN SSHFP 4 2 a34643bdce1ef3042cdd76fb7e46fcaf108dc436f8fcdb55daf993a27da0654b

All I need to do is to add these into the DNS zone. Luckily I run BIND, so I copy-pastad them into the file, did +1 to the SOA’s serial and done!

Now I can try logging in.

antranigv@zvartnots:~ $ ssh srv0.hackerspace.am -v
OpenSSH_9.3p1, OpenSSL 3.1.3 19 Sep 2023
debug1: Reading configuration data /Users/antranigv/.ssh/config
debug1: Reading configuration data /Users/antranigv/.ssh/personal
[…]
debug1: Server host key: ssh-ed25519 SHA256:OCsizTimnJi1grbxSY5LpvpLozfZ2pk+4Jzwg60WKYA debug1: found 6 secure fingerprints in DNS debug1: verify_host_key_dns: matched SSHFP type 4 fptype 1 debug1: verify_host_key_dns: matched SSHFP type 4 fptype 2 debug1: matching host key fingerprint found in DNS

and I logged in properly!

Reply via email.

Antranig Vartanian

November 12, 2023

I spent some time and moved my What I Use page to WordPress. I finally have a good reason to use the details HTML tag.

I also updated the content! My new music player(s) is the iPod! More about that, soon!

Reply via email.

Antranig Vartanian

November 8, 2023

If you’re seeing this then the migration is done! The weblog has moved from weblog.antranigv.am to antranigv.am.

I have also spent some time updating my About page, I hope I haven’t missed anything important. I will be adding more pages soon, such as link to friends, blogs that I read, what I use, etc.

I hope ActivityPub is working properly with the new domain.

Long Live The Web!

Reply via email.

macOS Desktops limit?

If you’ve ever wondered how many Desktops you can have on macOS, the answer, based on my 10 second test is 16. I do, however, have two apps in fullscreen mode (OmniFocus and Music.app).

Foo

I wonder if this is per screen. If any of you has an external monitor, please test and let me know!

Fun fact: you cant do “⌘⇪3” (Command+Shift+3) to capture the screen if you’re in Mission Control, instead I ran the following inside a terminal.

sleep 5 && screencapture /tmp/foo.png

If you like to nerd out on Unix-y stuff, here’s a screenshot from the manual page of screencapture(1).

Screenshot 2023 11 02 at 7 52 29 PM

Better documentation is needed, indeed.

That’s all folks…

Reply via email.

Upcoming migration

In the coming days/weeks, this blog will be migrated to a new address. Notably, the subdomain will be dropped and I will use antranigv.am for my blog.

I’ve been meaning to do these changes for a while now, and it occurred to me there’s no time like the present. Of course, the biggest kicker was discussing about this with my good friend Rubenerd.

Three major changes:

  • weblog.antranigv.amantranigv.am
  • The blog’s name will be changed from “Freedom Be With All” to something… else
  • The theme, while staying the same, will be modified a bit. Mostly the internals

I always had a love and hate relationship with the subdomain weblog., however, I realized that the only thing on my homepage is little information such as projects I’m working on, some contact info and some banners.

While it’s a cool static page with a cool theme, I generate it in a complex way: OPML → XSLT → HTML, to be specific.

By moving the blog to the homepage, it will make it a better “reading experience” (is RX a thing? or is that part of UX as well?)

The name of this blog “Freedom Be With All” has a bit of history.

Initially, I only had an Armenian blog (which used to be blog.antranigv.am moved to antranigv.am/blog and currently settled to անդրանիկ.հայ which is “my name in Armenian dot Armenia’s IDN TLD”), I was too scared to blog in English.

The title of my Armenian blog was and still is “Ազատութիւն Ամենեցուն”, literally meaning “Freedom to all”. It’s a “mod” of the common “Peace to all” phrase mentioned in The Divine Liturgy Of The Armenian Church.

Personally, freedom is the highest value of all, hence I went with it, instead of peace.

The tagline will stay the same: I’m your worst nightmare. You see, when I was a kid, back in school, back in Syria, I was bullied (I mean, who wasn’t?), but not for my nerdiness, instead, for my “sharpness”. It wasn’t cool to know things, read books, talk with strangers on this thing called “the internet”. It wasn’t cool to talk about Star Wars all day (don’t you have history to study? who cares about WHY the Trade Federation attacked Naboo). It wasn’t cool to “know how to run pirated PlayStation games” because we were, well, poor is the right word here.

But I loved the internet, and the people on the internet introduced me to Unix, specifically to Linux. And that got me to (pirate, of course, because Syria, and) watch a documentary named Revolution OS, where the intro starts with Eric S. Raymond telling a story, ending with “I’m your worst nightmare”.

That feeling, of being such a good computer hacker, that you feel like a god in front of the computer and you feel that you can be a whole corporation’s nightmare is what made me feel powerful, is what made me feel “un-bully-able”. After that I would spend my days (well, technically nights, after my father went to sleep) chatting on IRC, reading books about programming, cracking the neighbour’s WiFi, reinstalling X11 3 times, and being nicer when you know you have the ability to be batman a nightmare.

All of this story aside, I don’t know what to name the blog. Maybe “Antranig Vartanian”, maybe “Antranig’s Notes”, maybe “antranigv”. Still not sure… Have a thought? Reply in the comments 🙂

The theme of the website will stay as is, but some nice modifications will be added, like a calendar, maybe a tag cloud, etc.

Finally, and I just remembered about this while I was typing this post: ActivityPub!

Dear lazyweb

I have a WordPress blog that uses the ActivityPub plugin, which means that you can follow it from the fediverse, e.g. from Mastodon!
However, I am planning to migrate the domain of the blog. Do I need to keep a static JSON somewhere meaning the “account” has migrated?

Any tips will be appreciated!

Otherwise, around 40 nice people will need to follow again 🙁

Wow, this post went more than I expected!

Thank you for reading (or skimming!)

That’s all folks… 

Reply via email.

Antranig Vartanian

October 31, 2023

Once a month, the WordPress app on my iPhone stops working. Specifically, it stops loading and sending data. It just “hangs” there. I’m honestly considering using the mobile web interface to blog remotely.

I wish there was MarsEdit for mobile…

Any alternatives or suggestions? Maybe I should use the “Post via Email” feature like back in the good old days 🙂

Reply via email.