Enabling DTrace in Python on FreeBSD (again)

3 Replies

Last night I was running our usual Greybeard AMA on FreeBSD’s Discord server, when someone asked “I’ve been using Linux for years, but I also like FreeBSD. what can I do for FreeBSD, and what can FreeBSD do for me, as a Python Full Stack Developer?”

I started talking about FreeBSD Jails, ZFS, Boot Environments and more, but I also wanted to focus on DTrace. The ability to dynamically trace on production still sounds like magic to me. I know it isn’t, I’ve read the code and the papers, but the fact that I can ask the operating system questions on the fly, without recompiling, is amazing.

So when I was demoing DTrace, I also wanted to show the DTrace integrations with Python. Little did I know that the Python package on FreeBSD was not compiled with the --with-dtrace option.

As a sane person, I setup a Jail (using Jailer, of course), cloned the FreeBSD ports tree and added the --with-dtrace option back. It did not compile.

The first issue that we encountered was the following:

--- Include/pydtrace_probes.h ---
dtrace: option requires an argument -- s

ah yes, looking over the Makefile we see the following

Include/pydtrace_probes.h: $(srcdir)/Include/pydtrace.d
    $(MKDIR_P) Include
    $(DTRACE) $(DFLAGS) -o $@ -h -s $<
    : sed in-place edit with POSIX-only tools
    sed 's/PYTHON_/PyDTrace_/' $@ > $@.tmp
    mv $@.tmp $@

(you can also view the source here)

As far as I can tell, the $< thingie does not work with BSD Make, so how about if we try using gmake instead?

I did the following changes to the Makefile in the FreeBSD Ports tree. Basically adding gmake into USES=.

 USES=          compiler:c11 cpe ncurses pathfix pkgconfig \
-               python:${PYTHON_DISTVERSION:R},env readline shebangfix ssl tar:xz
+               python:${PYTHON_DISTVERSION:R},env readline shebangfix ssl tar:xz gmake

Now let’s try compiling again.

We get the following errors now:

ld: error: undefined symbol: __dtraceenabled_python___function__entry

Ah yes, linking issues.

After an hour of digging we learned that the DTRACE_OBJS= variable is set to… nothing. But it needs to be set to Python/pydtrace.o. I was not able to fix this issue properly (in configure, configure.ac, or whatever madness that GNU Autotools use), so I just changed the line manually in the Makefile.

Now let’s try compiling again.

We get the following error:

./Python/sysmodule.c:223:24: warning: passing 'const char *' to parameter of type 'char *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]

Someone in the Discord chat recommended that I disable the LTO (Link-Time Optimization) option.

I did make clean, I patched that line again, and here we go, one more time.

It’s compiling, it’s compiling, it’s compiling… aaaaand we’re done! Let’s do make install

===>  Installing for python311-3.11.11
===>  Checking if python311 is already installed
===>   Registering installation for python311-3.11.11
[python.srv0.hackerspace.am] Installing python311-3.11.11...

good! now let’s try DTrace with Python!

In one terminal, I ran Python, and in another one I got the following

# dtrace -l -n 'python*:::'
   ID   PROVIDER            MODULE                          FUNCTION NAME
85302 python8737 libpython3.11.so.1.0                         sys_audit audit
85303 python8737 libpython3.11.so.1.0                  sys_audit_tstate audit
85304 python8737 libpython3.11.so.1.0          _PyEval_EvalFrameDefault function-entry
85305 python8737 libpython3.11.so.1.0            dtrace_function_return function-return
85306 python8737 libpython3.11.so.1.0          _PyEval_EvalFrameDefault function-return
85307 python8737 libpython3.11.so.1.0                   gc_collect_main gc-done
85308 python8737 libpython3.11.so.1.0                   gc_collect_main gc-start
85309 python8737 libpython3.11.so.1.0  PyImport_ImportModuleLevelObject import-find-load-done
85310 python8737 libpython3.11.so.1.0  PyImport_ImportModuleLevelObject import-find-load-start
85311 python8737 libpython3.11.so.1.0          _PyEval_EvalFrameDefault line

Woohoo! Now I’m happy!

Okay, so what did we learn?

  • There’s a little bit of GNUMake-ism in Python’s Makefile. Either we have to use gmake in FreeBSD when building Python or we need to submit an alternative to the upstream
  • Die GNU Autotools… I mean… the GNU Autotools usage in Python has an issue, where the generated Makefile does not set the DTRACE_OBJS correctly. Can anyone help with this? As a Pascal guy, reading GNU configure files makes me wanna die.
  • LTO is problematic. Solutions?

The real question is, how can we enable DTrace by default in FreeBSD packages for Python. DTrace is one of our market advantages and we should find a way to enable it everywhere we can.

This was a fun Greybeard AMA, where we touched multiple parts of the system. Looking forward for next week!

That’s all folks…

Reply via email.

Adding ActivityPub Reactions to WordPress themes

1 Reply

You’d think that I would be the last person to do PHP in 2025, but you’d be wrong! Sure, I’ve ignored it for 10 years, but ever since I came back to WordPress couple of years ago, I’ve been a huge PHP fan. I mean, deploying it is easy (compared to the nightmare that’s NodeJS and NPM) and maintaining it is as simple as doing pkg update in a FreeBSD Jail. WordPress and PHP have indeed come a long way.

However, couple of weeks ago I noticed an interesting change in Automattic authored themes (I’m still a huge fan of Twenty Twelve). A good change, that is. It started showing ActivityPub reactions under my posts! Oh god, this is so amazing!

Turns out they integrated this feature with every theme they maintain. So, when I was publishing my photos today on my photo blog (Photos by Antranig) I noticed that someone did like a picture over the Fediverse but the likes were not displayed.

My photo blog uses the Marianne theme. It’s a very minimalist and highly customizable theme. With a bit of digging here and there, I realized that all I need to do is to install and activate the ActivityPub plugin (which is the first thing I always do) and I need to integrate a bit of code where I want to see these reactions.

I used the Theme File Editor in WordPress’ admin panel and I added the following bit of code

<?php echo do_blocks('<!-- wp:activitypub/reactions /-->'); ?>

After that, I started seeing the reactions on my blog!

Gotta say, I love how blogging is becoming (again) central to the web. Text is easy to process (unlike videos) and protocols like HTTP, RSS and AP will be around, hopefully, for the next 100 years.

That’s all folks…

Reply via email.

Top 5 investments of January 2025

Leave a reply

I woke up today thinking “I can’t believe it’s still January”, and it will still be January for one more day. I actually like it when the year moves slowly. I mean, I know every year is (almost) the same, but I like it when it feels like it’s moving slowly.

So, it got me thinking, what did I do in January? Well, a lot of things, I finally upgraded some servers, I finally started a hackerspace, and I finally figured out what I want to do this year.

That being said, I also invested a lot in January, which got me thinking, what are some of the best investments? Here’s a short list.

Sleep

Sleep is underrated. I know everyone talks about how sleeping can make your live longer, be healthier and such, but none of that actually interested me. However, last year when I learned that sleeping at least 7 hours a day makes my brain work like fire in the morning, I realized that it just got interesting! For me, everything is about productivity. How much value can I produce per minute? It’s similar to Apple’s obsession with Performance Per Watt. The more I produce, the more I can die in peace (one day), and looks like sleep is an amazing solution to that.

While I’ve been sleeping properly (as in 7 hours, at least) for (almost) more than a year now, what I didn’t know was about the schedule of sleeping. For example, I would sleep around 5AM, and wake up mid day. These days? not so much. My usual sleep schedule is around 10:30PM now, and 11PM if we have any Call For Testing, Production Users Call, and I wake up at 5AM.

Yesterday, for example, we had an outage at a data-center, and I was helping a customer recover, so I slept a bit later, but still woke up before 5:30AM.

I do think that I’m a night owl (which implies the existence of morning owls), I can easily stay up to 5AM hacking on code, or configuring servers, without any performance hit… while I’m awake! but oh boy am I gonna have a shitty next morning. It will take me around 5 hours the next day to get back my performance, my production per minute.

In today’s case, however, I was very productive until sleeping at night, and I’m productive now as well.

More importantly, when I wake up at 5AM, and get in front of the computer at 5:30AM, no one is awake (in Armenia, at least), which means I can easily get some work done without anyone bothering me via calls and text messages.

Sleep; a very good investment for 30 year old who want to produce as much as possible for the world and their family.

Internet Radio

Couple of years ago, my mentor norayr introduced me to Deep House Radio, a one-man operation from Cork, Ireland. It basically plays one DJ mix after the other, and I cannot work without it anymore. You can’t work without AI? well, sorry to hear that, but at least my online radio is not just made by people, for people, it’s also very accurate 😛

I got a Deep House Radio subscription around new year, for a year. According to their Buy Me A Coffee page, I am in the top 3 supporters from the last 90 days, which makes me very happy. The operator, Alan, provided me with two links to two streams, meaning I was able to share my subscription with my friend as well.

I want to blog? I put in Deep House Radio (like right now). I want to code? I put on DHR? I want to read? I put on DHR.

Overall, I love DHR, totally recommended.

Triode and BMBX

As soon as I got the personalized links for DHR, I pasted them into Music.app (or as I still call it, iTunes), and it worked fine, as expected. But if got me thinking, do I need this massive app just to listen to radio? I mean sure, I use iTunes to listen my music, in my library, on my computer, and to sync music with my iPod (Touch and Classic), but it seems like an overkill for a simple internet radio stream, aye?

Two days in, I needed to leave the house for a customer site, and I usually listen to music when I’m working in a Data Center (I’m sure our security officers even see me on camera dancing when running around the data center), and all I had was my iPhone, running the shittiest operating system on the planet: iOS… Okay that’s a bit extreme, that title goes to Windows, but that’s for another day.

Anyways, so iOS, the toy operating system (that fits much better), which, in 2025, gives you the ability to listen to internet radio streams in one way only: YOU HAVE TO PASTE THE LINK IN SAFARI!

Oh god, you thought using iTunes was too much for an internet radio stream? How about A WHOLE BROWSER!

So I got deep, I started searching, in the only place I could (because Apple reasons), I started digging into the App Store!

After multiple tried, I finally landed on two apps that I like: Triode and BMBX

Triode is made by a company that I like already, iconfactory. I use many of their apps, and I liked this one as too! It has a monthly and yearly subscription, and you can buy it one time as well. I opted for the latter. 

What I didn’t know is that the app was also available on macOS! which made me very, very happy. You know what made me happier? Knowing that Triode’s subscription can be shared with family members.

Another honorable mentions should be BMBX (which should be read as BoomBox, I guess?), an app that I loved so much, I kept it even after installing Triode. It has a very simple interface, it just works, I don’t know what else to say about it.

In an ideal world, I’d like to see any of these apps being open-source. Maybe I should email them about it, and see if that’s possible?

Good January

So, in conclusion, (almost) every night I go to sleep at a reasonable hour, listening to Deep House Radio, using Triode, and this made me very happy.

Invest in people, invest in yourself, be useful for society. Isn’t that what’s all life is about?

Wait, the title says top 5 investments, I only mentioned 3? Oh well, here’s two more:

  • Spend more time with friends and family, you mean a lot to them
  • It’s 2025, why haven’t you learned Unix tools yet? time to read that AWK book!

That’s all folks…

Reply via email.

Setting ZFS ARC maximum on illumos

Leave a reply

I noticed that the memory usage was too high on one of my servers, and as always, the candle was burning from both sides.

I checked the ARC Size, and it was too high

# kstat -p zfs:0:arcstats:size
zfs:0:arcstats:size     15032385536

So I decided to set the ZFS ARC Max, which was new to me, but after going over the docs, all I needed to do was

# echo 'set zfs:zfs_arc_max = 4294967296' >> /etc/system

and then finally reboot

# shutdown -g0 -y -i 6

That’s pretty much it.

Reply via email.

Changing FreeBSD’s rcorder without patching

I was upgrading my jails, when I noticed that the WriteFreely instance for օրագիր.հայ was not running. I jexec’d into the jail and noticed that the writefreely process was not running at all, doing a simple service writefreely start made it work. Why?

Turns out that WriteFreely needs MySQL to be running during startup, and I assume it wasn’t. By running rcorder I was able to see the boot process.

# rcorder /usr/local/etc/rc.d/* 2>/dev/null
writefreely
rsyncd
mysql-server
garb

So, my first instinct was to patch the /usr/local/etc/rc.d/writefreely script and add mysql into the REQUIRE line, but then I thought to myself, I can’t be the only person who had this problem, right? I mean, I know that the script will be overwritten during the next upgrade. What’s the actual solution here?

After searching a bit, I found the article Override rc order in FreeBSD, so based on that, I created the following file: /usr/local/etc/rc.d/__writefreely which has the following content

#!/bin/sh

# PROVIDE: __writefreely
# REQUIRE: mysql
# BEFORE: writefreely

This is a much cleaner way to do things, let’s check the rcorder again

# rcorder /usr/local/etc/rc.d/* 2>/dev/null
mysql-server
rsyncd
garb
__writefreely
writefreely

Much, much better. After restarting the jail, however, I noticed that WriteFreely is still not running… huh?

Oh, of course, I just needed to do chmod +x __writefreely

And now it works.

Reply via email.

FreeBSD-Update and ~200 Jails

Initially, when I heard about freebsd-rustdate I was very skeptical. I have a fear of “Written in <new hip language>”. I thought, however, I’ll wait, and when the time comes, I will try and see how it works.

For the last couple of days I’ve been updating hosts and jails for my customers and my company, and one of the best resources I found was the FreeBSD Update page on FreeBSD’s Wiki, specially the “freebsd-update Reverse Proxy Cache” section. It has saved me hours when updating the hosts. For some hosts we even did an NFS mount of /var/db/freebsd-update/files directory.

But when it came to upgrading the jails, I realized that this is going to take a very long time. Each host has at least 15 jails, up to 50. There’s a host which has 100+ jails.

Upgrading all of them was going to take a very, very long time. So I ended up doing some research. Here were my options.

  • Build FreeBSD once and run make install everywhere else using NFS and DESTDIR (I used to do this years ago)
  • Migrate to PkgBase (we’ve started doing this, but we’re not done yet, and it will take a while)
  • Nuke the Jails, start fresh, and just move the data (this could work, and I will do that in the future, but now I need to update ~200 jails in the coming 3 days)
  • Somehow, make freebsd-update run faster.

As you have guessed, I went for the last option. Uncle Dave reminded me of freebsd-rustdate again, and I decided to give it a try. Even before starting, my good friend Daniel wrote in our group chat:

@dch my guy. You just saved me several hours per year of flipping back and forth between terminals waiting for the next part of a freebsd-update upgrades to finish running on a million systems.

I arrived to my parent’s house, installed freebsd-rustdate on a host, and tested it on a single jail. Here is my initial reaction

holy fuck freebsd-rustdate is fucking fast

Like I said, I hate “rewrite in <new hip language>”, but clearly, this time it’s a winner.

And frankly speaking, my Jail manager, jailer, does have the same problems that freebsd-update has. It’s much, much slower when you have to manage 100+ jails. I will, however, not rewrite it in another language (for now, and if I do, it will be in Oberon). Although I might end up spending some good amount of time optimizing it 🙂

Kudos to Matthew Fuller, amazing work. And I have to mention, when I was thinking about moving to FreeBSD more than a decade ago, his rant “BSD for Linux Users” was the deciding factor for me, and I’ve been using FreeBSD ever since.

That’s all folks…

Reply via email.

Antranig Vartanian

January 3, 2025

Here are some issues I had, which I fixed in the last three days

  • Email client not updating properly on phone. Fixed
  • Too many apps on phone and programs on computer. Deleted
  • Too many notes here and there. Collected and centralized
  • Too many packages outdated. Upgraded
  • Too many messages from friends left unanswered. Answered
  • Too many wires and cables in the cabinet. Organized
  • Too many cameras unused. Pictures Captured
  • Too many things on my TODO lists. Clarified.

Sometimes all you need is to sit down, clean up your laptop’s monitor and get to work.

Wish you all the best in the coming year. And if I ever missed your message, I’m sorry. If I ever texted too much, I am not.

That’s all folks…

Reply via email.

Antranig Vartanian

December 22, 2024

We’ve organized a CTF (again), but this time it was way more interesting than the previous years, not just because of the newly introduced challenges, but because finally we are seeing the effects of LLMs in our industry, both the good and the bad, and we can now predict one of the possibilities of the future.

I need to articulate my thoughts for couple of days, and a long post might be published soon.

Reply via email.

Link

First Router Designed Specifically For OpenWrt Released – Software Freedom Conservancy:

The New OpenWrt One on sale now for $89 — Ultimate Gift for Right-To-Repair Enthusiasts
[…]
This device services your needs as its owner and user.

This news makes me very happy. But here’s the interesting part;

This new product has completed full FCC compliance tests; it’s confirmed that OpenWrt met all of the FCC compliance requirements. Industry “conventional wisdom” often argues that FCC requirements somehow conflict with the software right to repair. SFC has long argued that’s pure FUD. We at SFC and OpenWrt have now proved copyleft compliance, the software right to repair, and FCC requirements are all attainable in one product!

This is even better news. Combining a device like this with something like OpenWISP will make a killer commercial deployment, specially for organizations such as hospitals and low-budget government agencies in the developing world.

Let’s see how it goes.

Reply via email.