Linux Kernel Performance Tuning for Network Servers

High-throughput network servers require kernel-level tuning beyond defaults. Here are the key parameters.

TCP Buffer Sizes

BASH

# /etc/sysctl.d/99-network-performance.conf

# Increase TCP buffer sizes (for 10GbE)
net.core.rmem_max = 134217728
net.core.wmem_max = 134217728
net.ipv4.tcp_rmem = 4096 87380 134217728
net.ipv4.tcp_wmem = 4096 65536 134217728

# Increase backlog queue
net.core.netdev_max_backlog = 300000
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 30000

# TCP keepalive
net.ipv4.tcp_keepalive_time = 60
net.ipv4.tcp_keepalive_intvl = 10
net.ipv4.tcp_keepalive_probes = 6

# Faster TIME_WAIT recycling
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15

BASH

sysctl -p /etc/sysctl.d/99-network-performance.conf

IRQ Affinity

Pin network card interrupts to specific CPU cores to reduce cache misses:

BASH

# List IRQs for your NIC
cat /proc/interrupts | grep eth0

# Set affinity (pin IRQ 45 to CPU 2)
echo 4 > /proc/irq/45/smp_affinity  # bitmask: CPU 2 = bit 2 = 0x4

# Or use irqbalance for automatic distribution
apt install irqbalance
systemctl enable --now irqbalance

CPU Governor

BASH

# Set performance governor
apt install cpufrequtils
cpufreq-set -g performance

# Or persist via systemd
cat > /etc/systemd/system/cpufreq.service << EOF
[Unit]
Description=Set CPU performance governor
[Service]
Type=oneshot
ExecStart=/bin/sh -c 'for cpu in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do echo performance > $cpu; done'
[Install]
WantedBy=multi-user.target
EOF

I/O Scheduler

For NVMe drives, use none; for SSDs, use mq-deadline:

BASH

echo none > /sys/block/nvme0n1/queue/scheduler
echo mq-deadline > /sys/block/sda/queue/scheduler

NUMA Awareness

For multi-socket servers, run network applications on the same NUMA node as the NIC:

BASH

numactl --cpunodebind=0 --membind=0 nginx -g 'daemon off;'

Benchmarking

BASH

# Network throughput
iperf3 -s  # on server
iperf3 -c server_ip -P 8 -t 30  # on client

# Disk IOPS
fio --name=randread --ioengine=libaio --iodepth=32 --rw=randread --bs=4k --numjobs=4 --size=1G --runtime=60 --group_reporting