Proxmox Network Configuration

Active/Passive Bonding with VLAN

auto bond0
iface bond0 inet static
        bond-mode active-backup
        bond-miimon 100
        bond-slaves eno1 enp65s0f0np0
        bond-primary enp65s0f0np0
        bond-xmit-hash-policy layer2+3

auto vmbr0
iface vmbr0 inet manual
        bridge-ports bond0
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094

auto vmbr0.898
iface vmbr0.898 inet static
        address 10.13.12.161/24
        gateway 10.13.12.254

Simple Routed Setup for Cluster/Ceph (Config from PVE1)

# To PVE02
auto enp65s0f2np2
iface enp65s0f2np2 inet static
        address 10.161.69.1
        netmask 255.255.255.248
        up ip route add 10.161.69.2/32 dev enp65s0f2np2
        down ip route del 10.161.69.2/32

# To PVE03
auto enp65s0f3np3
iface enp65s0f3np3 inet static
        address 10.161.69.1
        netmask 255.255.255.248
        up ip route add 10.161.69.3/32 dev enp65s0f3np3
        down ip route del 10.161.69.3/32