Getting Into FreeBSD Kernel Development: A Practical On-Ramp

Every so often a new FreeBSD kernel development course makes the rounds, and the r/freebsd threads fill up with the same two reactions: a wave of genuine curiosity, and a quieter undercurrent of "that seems way over my head." Both are understandable. Kernel work has a reputation for being an exclusive club. But the truth is that FreeBSD, more than almost any other Unix-like system, makes the on-ramp shallower than people expect. If you can build software from source and you are comfortable in a shell, you already have most of what you need to start.

This is not a tutorial that will make you a committer. It is a practical map: how to get the source, how to build it, how to test without wrecking your machine, and how to write a kernel module that actually loads. The goal is to get you to the point where the kernel stops being a black box and starts being just another codebase you can read and modify.

The Source Tree Is the Whole System

Here is the thing that trips up people coming from Linux: on FreeBSD there is no "kernel repo" separate from a "distro." The kernel, the C library, the userland utilities, the build system, and the documentation all live in one tree. That single-repository model is the reason kernel work is more approachable here. You are not chasing a kernel version against a distribution's patched userland. You clone one thing and you have the whole operating system.

git clone https://git.freebsd.org/src.git /usr/src
cd /usr/src
git checkout stable/14   # or 'main' for -CURRENT, or a releng branch

Once it is on disk, spend an hour just walking sys/. That directory is the kernel. A rough orientation:

• sys/kern/ — the core: scheduler, process management, system calls, VFS glue.
• sys/sys/ — the kernel's header files. This is where module.h, kernel.h, and the type definitions you will include live.
• sys/dev/ — device drivers. If you want to read real, production kernel C, this is the deep end.
• sys/amd64/conf/ — kernel configuration files, including GENERIC, the stock config every release ships.
• sys/netinet/, sys/net/ — the TCP/IP stack that made FreeBSD's reputation.

You do not need to understand all of it. But knowing where things are turns "the kernel" into a set of files you can grep.

Building World and Kernel

Building FreeBSD from source is a documented, routine operation, not a rite of passage. The two commands that matter most for kernel work are buildkernel and installkernel. A custom kernel starts by copying GENERIC and editing the copy — never edit GENERIC itself.

cp /usr/src/sys/amd64/conf/GENERIC /usr/src/sys/amd64/conf/MYKERNEL
# edit MYKERNEL: strip drivers you don't need, add options you do

cd /usr/src
make buildkernel KERNCONF=MYKERNEL
make installkernel KERNCONF=MYKERNEL

The KERNCONF variable names your config file. installkernel copies the new kernel to /boot/kernel/kernel and moves the previous one to /boot/kernel.old/kernel, which gives you a fallback at the loader prompt if the new kernel misbehaves. A full buildworld plus buildkernel is the complete source upgrade path, but for pure kernel experimentation you can iterate on buildkernel alone once world is in sync.

Two practical notes. First, kernel builds are CPU-bound and parallelize well — use make -j$(sysctl -n hw.ncpu) buildkernel. Second, before you touch anything a custom kernel is often unnecessary; if the feature you want ships as a loadable module in /boot/kernel, you can kldload it and skip the rebuild entirely. Building a custom kernel is for when you are changing options that cannot be toggled at runtime, or for stripping a kernel down for an appliance.

Test Somewhere You Can Throw Away

The single most important habit in kernel work: never test on a machine you care about. A kernel panic is not a segfault in a process you can restart — it takes the whole box down. FreeBSD gives you two clean ways to isolate the blast radius.

bhyve is the native type-2 hypervisor built into the base system. Spin up a throwaway FreeBSD guest, share the source tree in, and iterate there. If your module panics the guest, you reboot the guest, not your workstation. The vm-bhyve wrapper from ports makes lifecycle management (create, start, console, destroy) a one-liner, but bhyve is entirely usable from the base tools alone.

Boot environments are the second layer of safety, and they are a ZFS feature that has no clean equivalent elsewhere. Before an installkernel or a risky change, snapshot the running system into a new boot environment:

bectl create pre-kernel-hacking
bectl list
# if the new kernel wedges the system, at the loader:
#   select the 'pre-kernel-hacking' environment and boot it

Between bhyve for the fast inner loop and boot environments for the "I am about to install this to real hardware" moment, you can experiment aggressively without ever risking data.

A Kernel Module That Actually Loads

The best way to make the kernel feel real is to load your own code into it. A FreeBSD loadable kernel module (KLD) needs three things: an event handler that runs on load and unload, a moduledata_t that names the module and points at that handler, and a DECLARE_MODULE macro that registers it with the kernel. Here is the canonical skeleton, straight from the Architecture Handbook:

/*
 * KLD Skeleton
 */
#include <sys/types.h>
#include <sys/systm.h>   /* uprintf */
#include <sys/errno.h>
#include <sys/param.h>   /* defines used in kernel.h */
#include <sys/module.h>
#include <sys/kernel.h>  /* types used in module initialization */

static int
skel_loader(struct module *m, int what, void *arg)
{
        int err = 0;

        switch (what) {
        case MOD_LOAD:
                uprintf("Skeleton KLD loaded.\n");
                break;
        case MOD_UNLOAD:
                uprintf("Skeleton KLD unloaded.\n");
                break;
        default:
                err = EOPNOTSUPP;
                break;
        }
        return (err);
}

static moduledata_t skel_mod = {
        "skel",        /* module name */
        skel_loader,   /* event handler */
        NULL           /* extra data */
};

DECLARE_MODULE(skeleton, skel_mod, SI_SUB_KLD, SI_ORDER_ANY);

The Makefile is almost insultingly short, because bsd.kmod.mk knows how to build a module against the running kernel's headers:

SRCS=skeleton.c
KMOD=skeleton

.include <bsd.kmod.mk>

Then build and load it:

make
kldload ./skeleton.ko
kldstat | grep skeleton     # confirm it's resident
kldunload skeleton

A note on the output: uprintf(9) writes to the controlling terminal of the process that triggered the load, so you will see the message on the terminal where you ran kldload. For messages that should land in the system log instead, kernel code uses printf(9), which goes to dmesg and the console. That is the entire lifecycle — your code ran inside the kernel, printed, and cleaned up on unload. Everything more elaborate (sysctls, character devices, sysinit hooks) is a variation on this same three-part structure.

Where to Go From Here

Once the skeleton loads, the natural next steps are all about observation and reading. DTrace is the tool that turns exploration from guesswork into measurement — it lets you trace kernel functions, arguments, and timing on a live system with effectively zero overhead when idle, which makes it far safer than scattering printf calls through kernel paths. Learning to write a few dtrace one-liners against the running kernel teaches you more about how FreeBSD actually behaves than any amount of static reading.

Beyond that, the FreeBSD Architecture Handbook and the Developer's Handbook are the canonical references, and the section 9 man pages (man 9 kldload, man 9 DECLARE_MODULE, man 9 printf) document the in-kernel APIs the way section 2 and 3 document userland. The freebsd-hackers mailing list is where the actual development conversation happens, and reading the archives is a fast way to absorb the project's conventions and taste.

The Consulting Angle

There is a reason this matters beyond hobbyist curiosity. A great deal of what passes for "FreeBSD experience" in the market is operational: installing packages, configuring jails, tuning sysctls someone else documented. That is valuable work. But it is surface-level, and it hits a ceiling the moment a problem lives below the userland boundary — a driver misbehaving, a network path behaving strangely under load, a panic with only a backtrace to go on.

The engineers who can read sys/, build a custom kernel, write a module to instrument a problem, and trace a live system with DTrace are the ones who can actually resolve those cases rather than escalating them into the void. That depth is not a party trick. It is the difference between a shop that runs FreeBSD and a shop that engineers on FreeBSD. If you are curious enough to load that skeleton module, you are already closer to the second category than most.

Need FreeBSD engineering that goes past the userland boundary? See our systems engineering services or schedule a consultation.