Nginx Performance Tuning on FreeBSD

Why Nginx on FreeBSD

FreeBSD's network stack has been refined over decades and remains one of the most performant TCP/IP implementations available. Companies like Netflix stream massive volumes of traffic from FreeBSD servers running Nginx, and WhatsApp famously scaled to hundreds of millions of connections on FreeBSD infrastructure. The combination is not accidental -- FreeBSD's kernel-level event notification system, kqueue, provides an exceptionally efficient mechanism for handling large numbers of concurrent connections with minimal overhead.

Unlike Linux's epoll, which was modeled after kqueue but carries legacy design decisions, kqueue uses a single system call for both registering and retrieving events. This means fewer context switches, lower latency, and better scalability under high connection counts. For production web servers, FreeBSD and Nginx is a battle-tested pairing that rewards proper tuning with outstanding throughput and stability.

Installing Nginx on FreeBSD

The fastest path to a working Nginx installation on FreeBSD is through the package manager. Install the prebuilt binary, enable it at boot, and start the service:

pkg install nginx
sysrc nginx_enable="YES"
service nginx start

If you need custom modules -- such as GeoIP2, Brotli compression, or the headers-more module -- build from ports instead. The ports tree lets you select exactly which modules to compile:

cd /usr/ports/www/nginx
make config
make install clean

The make config dialog presents all available modules. Select what you need, deselect what you do not. Leaner builds consume less memory and reduce the attack surface. After installation, the primary configuration file lives at /usr/local/etc/nginx/nginx.conf.

The kqueue Event Model

On FreeBSD, Nginx automatically uses kqueue as its event notification mechanism. There is no need to add use kqueue; to your configuration -- it is the default and only sensible choice on this platform. The events block in your configuration should focus on connection limits:

events {
    worker_connections 4096;
    multi_accept on;
}

The worker_connections directive controls how many simultaneous connections each worker process can handle. On FreeBSD, kqueue scales efficiently to thousands of connections per worker, so values of 4096 or even 8192 are practical on servers with sufficient RAM. Combined with multi_accept on, each worker will accept all pending connections in a single kqueue call rather than one at a time, reducing event loop iterations under load.

Set the number of worker processes to match your CPU core count. On FreeBSD, check with sysctl hw.ncpu, or simply use worker_processes auto; and let Nginx detect the count at startup.

Kernel Tuning with sysctl

FreeBSD exposes a rich set of kernel parameters through sysctl that directly affect Nginx performance. Add these to /etc/sysctl.conf so they persist across reboots:

# /etc/sysctl.conf - Nginx performance tuning

# Increase listen queue depth for incoming connections
kern.ipc.somaxconn=4096

# Enlarge maximum socket buffer size
kern.ipc.maxsockbuf=2097152

# Increase TCP send and receive buffer maximums
net.inet.tcp.sendbuf_max=2097152
net.inet.tcp.recvbuf_max=2097152

# Reduce TIME_WAIT duration (default 30000 = 30s)
net.inet.tcp.msl=5000

# Enable TCP fast recycling of TIME_WAIT sockets
net.inet.tcp.fast_finwait2_recycle=1
net.inet.tcp.finwait2_timeout=5000

The kern.ipc.somaxconn parameter controls the maximum length of the listen queue. When Nginx receives a burst of new connections, a shallow queue causes dropped connections and client-visible errors. Setting this to 4096 matches the worker_connections value and prevents queue overflow under traffic spikes. The TCP buffer sizes allow the kernel to allocate larger windows for high-bandwidth transfers, while reducing net.inet.tcp.msl frees up sockets stuck in TIME_WAIT faster.

Apply changes immediately without rebooting using sysctl -f /etc/sysctl.conf.

Boot Loader Tuning

Some kernel parameters must be set at boot time through /boot/loader.conf because they cannot be changed on a running system:

# /boot/loader.conf - Boot-time kernel tuning

# Accept queue depth (requires reboot)
kern.ipc.soacceptqueue=1024

# Use CUBIC congestion control algorithm
net.inet.tcp.cc.algorithm=cubic

# Increase maximum open files system-wide
kern.maxfiles=65536

# Increase per-process file descriptor limit
kern.maxfilesperproc=32768

The kern.ipc.soacceptqueue parameter sets the accept queue depth at boot. The kern.maxfiles value is critical for servers handling many simultaneous connections -- each connection consumes a file descriptor, and the default limit is far too low for a busy Nginx instance. These changes require a reboot to take effect.

sendfile and Asynchronous I/O

FreeBSD has one of the most highly optimized sendfile implementations of any operating system. It performs zero-copy transfers from disk to network socket entirely within the kernel, bypassing userspace buffers. Enable it along with asynchronous I/O in your Nginx http block:

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    aio on;
    directio 512;
}

The tcp_nopush directive works with sendfile to batch response headers and the beginning of a file into a single TCP packet, reducing the number of packets sent. The aio on directive enables POSIX asynchronous I/O for serving large files, and directio 512 bypasses the filesystem cache for files larger than 512 bytes when using aio. For typical web workloads with many small static files, sendfile alone provides the biggest performance gain.

Gzip Compression

Compressing responses before transmission reduces bandwidth usage and improves page load times. Add a gzip block to your Nginx configuration at /usr/local/etc/nginx/nginx.conf:

gzip on;
gzip_vary on;
gzip_proxied any;
gzip_comp_level 5;
gzip_min_length 256;
gzip_types
    text/plain
    text/css
    text/javascript
    application/javascript
    application/json
    application/xml
    application/rss+xml
    image/svg+xml
    font/woff2;

A compression level of 5 offers a strong balance between CPU usage and compression ratio. Levels above 6 provide diminishing returns while consuming noticeably more CPU. The gzip_min_length setting prevents compressing tiny responses where the gzip overhead would actually increase the payload size. The gzip_vary directive ensures caching proxies store both compressed and uncompressed versions.

Static File Caching

For static assets served from FreeBSD's filesystem, configure browser caching with appropriate expiration headers. On FreeBSD, the web root is typically under /usr/local/www/:

location ~* \.(jpg|jpeg|png|webp|gif|ico|css|js|woff2|woff|ttf)$ {
    expires 30d;
    add_header Cache-Control "public, immutable";
    access_log off;
}

The expires 30d directive tells browsers to cache these files for 30 days. The immutable flag in the Cache-Control header prevents browsers from revalidating the resource before the cache expires, eliminating conditional GET requests entirely. Disabling access logging for static files reduces disk I/O and log file growth on busy servers.

Testing and Validation

After making configuration changes, always validate the syntax before reloading. FreeBSD uses the service command for daemon management:

# Validate configuration syntax
nginx -t

# Reload without dropping connections
service nginx reload

# Verify Nginx is listening on the correct ports
sockstat -4l | grep nginx

# Monitor network throughput in real time
systat -ifstat

The nginx -t command parses the entire configuration and reports any syntax errors without affecting the running server. The service nginx reload command sends a HUP signal, causing Nginx to gracefully reload -- existing connections finish processing while new connections use the updated configuration. Use sockstat -4l to confirm Nginx is bound to the expected addresses and ports, and systat -ifstat to watch network interface throughput during load testing.

For benchmarking, pkg install wrk provides a modern HTTP benchmarking tool that can generate significant load from a single FreeBSD machine. Run tests from a separate host to avoid measuring loopback performance.

Need help optimizing your FreeBSD web servers? Learn about our DevOps automation services or schedule a consultation.