Proxmox VE Cluster Setup for Production Workloads

Building a Production Proxmox VE Cluster

A 3-node Proxmox cluster with Ceph storage provides full HA with no single point of failure. Here's how to build it right.

Hardware Requirements

3× servers (identical specs recommended)
2× 10GbE NICs per server (one for cluster, one for Ceph/storage)
NVMe SSDs for Ceph OSDs (HDDs work but slower)
Dedicated management NIC

Network Design

Network	Purpose	Speed
corosync	Cluster heartbeat	1GbE dedicated
ceph-pub	Ceph public network	10GbE
ceph-cluster	Ceph replication	10GbE separate
vmbr0	VM traffic	10GbE

Corosync Configuration

BASH

# /etc/pve/corosync.conf — set on first node, auto-synced
totem {
    version: 2
    cluster_name: prod-cluster
    transport: knet
    interface {
        linknumber: 0
        bindnetaddr: 10.10.10.0
        mcastport: 5405
    }
}

Initialize Ceph

BASH

# Run on each node
pveceph init --network 10.20.20.0/24

# Create monitors on all 3 nodes
pveceph mon create

# Add OSDs (one per disk per node)
pveceph osd create /dev/nvme0n1

# Create Ceph pool for VMs
pveceph pool create vm-pool --size 3 --min-size 2

# Add pool as Proxmox storage
pvesm add rbd vm-ceph --pool vm-pool --content images,rootdir

HA Configuration

BASH

# Create HA group
ha-manager groupadd production --nodes pve1,pve2,pve3

# Add VMs to HA
ha-manager add vm:100 --group production --max-restart 3

Live Migration Test

BASH

# Migrate VM between nodes (zero downtime)
qm migrate 100 pve2 --online

Monitoring

Install Prometheus + Grafana with the Proxmox exporter:

BASH

apt install prometheus-pve-exporter

Key metrics: CPU/RAM per node, Ceph health, OSD latency, VM counts.

Common Issues

Split-brain: Always use odd number of nodes (3, 5, 7)
Ceph slow: Check OSD latency; NVMe dramatically improves it
HA storm: Set restart delays to prevent cascading failures