mirror of
https://github.com/ThmsRyn/homelab-proxmox-starter.git
synced 2026-11-20 21:23:38 +03:00
Initial commit
This commit is contained in:
commit
7e1d222cb2
12 files changed
+3334
No files matched your search
+29
@@ -0,0 +1,29 @@
|
||||
# Secrets and credentials — never commit these
|
||||
*.key
|
||||
*.pem
|
||||
*.p12
|
||||
*.pfx
|
||||
pve.yml
|
||||
*password*
|
||||
*credentials*
|
||||
*.env
|
||||
|
||||
# Backup files created by scripts
|
||||
*.bak.*
|
||||
*.bak
|
||||
|
||||
# OS artifacts
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
desktop.ini
|
||||
|
||||
# Editor artifacts
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
|
||||
# Temporary files
|
||||
*.tmp
|
||||
*.log
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Thomas Rayon
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,637 @@
|
||||
# homelab-proxmox-starter
|
||||
|
||||
A complete step-by-step guide to building a homelab on Proxmox VE 9.1.
|
||||
No complex stack, no Kubernetes, no over-engineering. Just a solid foundation.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Hardware Requirements](#1-hardware-requirements)
|
||||
2. [Install Proxmox VE 9.1](#2-install-proxmox-ve-91)
|
||||
3. [Post-Install Configuration](#3-post-install-configuration)
|
||||
4. [Network Configuration](#4-network-configuration)
|
||||
5. [Create Your First VM](#5-create-your-first-vm)
|
||||
6. [Create a VM Template for Fast Cloning](#6-create-a-vm-template-for-fast-cloning)
|
||||
7. [LXC Containers](#7-lxc-containers)
|
||||
8. [Backups with Proxmox Backup Server](#8-backups-with-proxmox-backup-server)
|
||||
9. [Monitoring with Prometheus and Grafana](#9-monitoring-with-prometheus-and-grafana)
|
||||
10. [Basic Hardening](#10-basic-hardening)
|
||||
11. [Useful Daily Commands](#11-useful-daily-commands)
|
||||
12. [Directory Structure of This Repo](#12-directory-structure-of-this-repo)
|
||||
|
||||
---
|
||||
|
||||
## 1. Hardware Requirements
|
||||
|
||||
Proxmox VE is a Type-1 hypervisor. It runs directly on bare metal. Do not install it inside a VM.
|
||||
|
||||
### Minimum viable homelab
|
||||
|
||||
| Component | Minimum | Recommended |
|
||||
|-------------|----------------------|--------------------------|
|
||||
| CPU | 4 cores, VT-x/VT-d | 8+ cores, VT-x/VT-d/IOMMU |
|
||||
| RAM | 16 GB | 32–64 GB ECC |
|
||||
| System disk | 32 GB SSD | 120 GB SSD (dedicated) |
|
||||
| VM storage | 256 GB HDD/SSD | 1 TB+ NVMe or ZFS pool |
|
||||
| NIC | 1x 1GbE | 2x 1GbE or 1x 10GbE |
|
||||
|
||||
### CPU flags to verify
|
||||
|
||||
Before buying or repurposing hardware, confirm your CPU supports:
|
||||
|
||||
- `vmx` (Intel VT-x) or `svm` (AMD-V) — required for VMs
|
||||
- `ept` or `npt` — required for nested virtualisation
|
||||
- `vt-d` (Intel) or `amd-iommu` (AMD) — required for PCIe passthrough (GPU, NIC, HBA)
|
||||
|
||||
Check on a running Linux system:
|
||||
|
||||
```bash
|
||||
grep -E 'vmx|svm' /proc/cpuinfo | head -1
|
||||
```
|
||||
|
||||
### Storage recommendations
|
||||
|
||||
- **System/boot**: a dedicated SSD, even 32 GB is fine. Keep VM storage separate.
|
||||
- **VM storage**: ZFS on NVMe gives you snapshots, checksums, and compression for free.
|
||||
- **Avoid USB boot**: Proxmox writes heavily to the system disk (logs, corosync). USB drives wear out fast.
|
||||
|
||||
---
|
||||
|
||||
## 2. Install Proxmox VE 9.1
|
||||
|
||||
### Download the ISO
|
||||
|
||||
Download the official ISO from [https://www.proxmox.com/en/downloads](https://www.proxmox.com/en/downloads).
|
||||
|
||||
Current version as of April 2026: **Proxmox VE 9.1**.
|
||||
|
||||
Verify the checksum before flashing:
|
||||
|
||||
```bash
|
||||
sha256sum proxmox-ve_9.1-1.iso
|
||||
```
|
||||
|
||||
Compare against the published SHA256 on the download page.
|
||||
|
||||
### Flash to USB
|
||||
|
||||
```bash
|
||||
# Linux / macOS
|
||||
dd if=proxmox-ve_9.1-1.iso of=/dev/sdX bs=4M status=progress oflag=sync
|
||||
|
||||
# Windows: use Rufus (write in DD mode, not ISO mode)
|
||||
```
|
||||
|
||||
### Boot and install
|
||||
|
||||
1. Boot from USB. Select **Install Proxmox VE (Graphical)**.
|
||||
2. Accept the EULA.
|
||||
3. **Target disk**: select your dedicated system SSD. Check **Options** if you want ZFS on the system disk (recommended if you have 2+ disks for a mirror).
|
||||
4. **Location and time zone**: set your country and timezone.
|
||||
5. **Password and email**: set a strong root password. The email is used for alerts.
|
||||
6. **Network configuration**:
|
||||
- Management interface: pick your primary NIC.
|
||||
- Hostname: use a FQDN, e.g. `pve01.lab.local`.
|
||||
- IP: assign a static IP on your LAN, e.g. `192.168.1.10/24`.
|
||||
- Gateway: your router IP, e.g. `192.168.1.1`.
|
||||
- DNS: `192.168.1.1` or `1.1.1.1`.
|
||||
7. Review the summary and click **Install**.
|
||||
8. Remove USB, reboot.
|
||||
|
||||
Access the web UI at `https://192.168.1.10:8006`. Accept the self-signed certificate warning.
|
||||
|
||||
---
|
||||
|
||||
## 3. Post-Install Configuration
|
||||
|
||||
### Disable the enterprise repository (requires paid subscription)
|
||||
|
||||
The enterprise repo is enabled by default but requires a Proxmox subscription. Comment it out:
|
||||
|
||||
```bash
|
||||
# On the Proxmox host via SSH or the web shell
|
||||
sed -i 's/^deb/#deb/' /etc/apt/sources.list.d/pve-enterprise.list
|
||||
```
|
||||
|
||||
Same for the Ceph enterprise repo if it exists:
|
||||
|
||||
```bash
|
||||
if [ -f /etc/apt/sources.list.d/ceph.list ]; then
|
||||
sed -i 's/^deb/#deb/' /etc/apt/sources.list.d/ceph.list
|
||||
fi
|
||||
```
|
||||
|
||||
### Enable the no-subscription repository
|
||||
|
||||
```bash
|
||||
cat >> /etc/apt/sources.list << 'EOF'
|
||||
|
||||
# Proxmox VE No-Subscription Repo
|
||||
deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription
|
||||
EOF
|
||||
```
|
||||
|
||||
### Remove the subscription nag popup
|
||||
|
||||
The web UI shows a nag dialog on login if no subscription is active. Remove it:
|
||||
|
||||
```bash
|
||||
sed -Ezi.bak "s/(Ext.Msg.show\(\{\s+title: gettext\('No valid sub)/void\(\{ \/\/\1/g" \
|
||||
/usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
|
||||
|
||||
# Restart the web service to apply
|
||||
systemctl restart pveproxy
|
||||
```
|
||||
|
||||
Note: this patch needs to be reapplied after each Proxmox update.
|
||||
|
||||
### Update the system
|
||||
|
||||
```bash
|
||||
apt update && apt dist-upgrade -y
|
||||
```
|
||||
|
||||
After updating, reboot if the kernel changed:
|
||||
|
||||
```bash
|
||||
reboot
|
||||
```
|
||||
|
||||
### Set the correct time zone
|
||||
|
||||
```bash
|
||||
timedatectl set-timezone Europe/Brussels
|
||||
timedatectl status
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Network Configuration
|
||||
|
||||
See [network/README.md](network/README.md) for the full guide.
|
||||
|
||||
### Default bridge: vmbr0
|
||||
|
||||
Proxmox creates a Linux bridge `vmbr0` during installation. It maps to your physical NIC and is what VMs use for network access.
|
||||
|
||||
Check the current config:
|
||||
|
||||
```bash
|
||||
cat /etc/network/interfaces
|
||||
```
|
||||
|
||||
Typical output:
|
||||
|
||||
```
|
||||
auto lo
|
||||
iface lo inet loopback
|
||||
|
||||
auto eno1
|
||||
iface eno1 inet manual
|
||||
|
||||
auto vmbr0
|
||||
iface vmbr0 inet static
|
||||
address 192.168.1.10/24
|
||||
gateway 192.168.1.1
|
||||
bridge-ports eno1
|
||||
bridge-stp off
|
||||
bridge-fd 0
|
||||
```
|
||||
|
||||
### Add a second bridge for isolated VM traffic
|
||||
|
||||
```bash
|
||||
# Append to /etc/network/interfaces
|
||||
auto vmbr1
|
||||
iface vmbr1 inet manual
|
||||
bridge-ports none
|
||||
bridge-stp off
|
||||
bridge-fd 0
|
||||
# Internal only — no physical NIC attached
|
||||
```
|
||||
|
||||
Apply:
|
||||
|
||||
```bash
|
||||
ifreload -a
|
||||
```
|
||||
|
||||
### VLAN configuration
|
||||
|
||||
For VLAN tagging across a managed switch, see [network/vlan-setup.sh](network/vlan-setup.sh).
|
||||
Quick summary: you create a VLAN-aware bridge and tag VMs at the VM config level, not at the bridge level.
|
||||
|
||||
---
|
||||
|
||||
## 5. Create Your First VM
|
||||
|
||||
### Download an ISO
|
||||
|
||||
In the Proxmox web UI:
|
||||
|
||||
1. Navigate to your node > **local** storage > **ISO Images**.
|
||||
2. Click **Download from URL**.
|
||||
3. URL: `https://releases.ubuntu.com/24.04/ubuntu-24.04.4-live-server-amd64.iso`
|
||||
4. Click **Query URL**, verify the filename, then **Download**.
|
||||
|
||||
Or via CLI:
|
||||
|
||||
```bash
|
||||
wget -P /var/lib/vz/template/iso/ \
|
||||
https://releases.ubuntu.com/24.04/ubuntu-24.04.4-live-server-amd64.iso
|
||||
```
|
||||
|
||||
### Create the VM via CLI (qm)
|
||||
|
||||
```bash
|
||||
qm create 100 \
|
||||
--name ubuntu-2404 \
|
||||
--memory 2048 \
|
||||
--cores 2 \
|
||||
--net0 virtio,bridge=vmbr0 \
|
||||
--ostype l26 \
|
||||
--scsihw virtio-scsi-pci \
|
||||
--scsi0 local-lvm:32 \
|
||||
--ide2 local:iso/ubuntu-24.04.4-live-server-amd64.iso,media=cdrom \
|
||||
--boot order=ide2 \
|
||||
--agent enabled=1
|
||||
```
|
||||
|
||||
Start it:
|
||||
|
||||
```bash
|
||||
qm start 100
|
||||
```
|
||||
|
||||
Open the console in the web UI to complete the Ubuntu installation.
|
||||
|
||||
### Recommended VM settings
|
||||
|
||||
- **CPU type**: `host` — passes through the real CPU model. Gives better performance and enables all CPU features. Do not use `kvm64` unless migrating between heterogeneous hosts.
|
||||
- **Disk**: `virtio-scsi` controller. Better performance than IDE or SATA.
|
||||
- **Network**: `virtio` model. Always. It is the fastest para-virtual NIC.
|
||||
- **QEMU Guest Agent**: install it inside the VM so Proxmox can report the IP and do clean shutdowns.
|
||||
|
||||
Inside Ubuntu after install:
|
||||
|
||||
```bash
|
||||
apt install -y qemu-guest-agent
|
||||
systemctl enable --now qemu-guest-agent
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Create a VM Template for Fast Cloning
|
||||
|
||||
Templates let you spin up a new VM in seconds instead of installing from scratch every time.
|
||||
|
||||
### Prepare the base VM
|
||||
|
||||
Start from the VM you created in step 5 (ID 100). Do a minimal install, run updates, install the guest agent, then clean up:
|
||||
|
||||
```bash
|
||||
# Inside the VM
|
||||
apt update && apt upgrade -y
|
||||
apt install -y qemu-guest-agent cloud-init
|
||||
systemctl enable qemu-guest-agent
|
||||
|
||||
# Remove SSH host keys (they will be regenerated on first boot)
|
||||
rm /etc/ssh/ssh_host_*
|
||||
truncate -s 0 /etc/machine-id
|
||||
|
||||
# Clean up
|
||||
apt clean
|
||||
cloud-init clean
|
||||
```
|
||||
|
||||
Shut down the VM:
|
||||
|
||||
```bash
|
||||
qm shutdown 100
|
||||
```
|
||||
|
||||
### Add a cloud-init drive and convert to template
|
||||
|
||||
```bash
|
||||
# Add cloud-init drive
|
||||
qm set 100 --ide1 local-lvm:cloudinit
|
||||
|
||||
# Set cloud-init defaults
|
||||
qm set 100 \
|
||||
--ciuser thomas \
|
||||
--sshkeys ~/.ssh/authorized_keys \
|
||||
--ipconfig0 ip=dhcp
|
||||
|
||||
# Convert to template
|
||||
qm template 100
|
||||
```
|
||||
|
||||
See [vms/cloud-init.yaml](vms/cloud-init.yaml) for a full cloud-init configuration example.
|
||||
|
||||
### Clone the template
|
||||
|
||||
```bash
|
||||
# Full clone (independent disk)
|
||||
qm clone 100 201 --name webserver-01 --full
|
||||
|
||||
# Start the clone
|
||||
qm start 201
|
||||
```
|
||||
|
||||
A full clone takes about 10–30 seconds depending on disk speed.
|
||||
|
||||
---
|
||||
|
||||
## 7. LXC Containers
|
||||
|
||||
LXC containers share the host kernel. They are lighter than VMs and start in under a second. Use them for services that do not need a full OS (Nginx, Pi-hole, Home Assistant, etc.).
|
||||
|
||||
### Download a container template
|
||||
|
||||
```bash
|
||||
pveam update
|
||||
pveam available | grep ubuntu
|
||||
pveam download local ubuntu-24.04-standard_24.04-2_amd64.tar.zst
|
||||
```
|
||||
|
||||
### Create an LXC container
|
||||
|
||||
```bash
|
||||
pct create 200 local:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst \
|
||||
--hostname pihole \
|
||||
--memory 512 \
|
||||
--cores 1 \
|
||||
--net0 name=eth0,bridge=vmbr0,ip=dhcp \
|
||||
--storage local-lvm \
|
||||
--rootfs local-lvm:8 \
|
||||
--password changeme \
|
||||
--unprivileged 1 \
|
||||
--start 1
|
||||
```
|
||||
|
||||
Access the console:
|
||||
|
||||
```bash
|
||||
pct enter 200
|
||||
```
|
||||
|
||||
### Privileged vs unprivileged
|
||||
|
||||
- **Unprivileged** (recommended): UIDs are remapped. Root inside the container is not root on the host. Use this by default.
|
||||
- **Privileged**: required for some use cases (NFS mounts, Docker inside LXC, some kernel modules). Only use when needed.
|
||||
|
||||
---
|
||||
|
||||
## 8. Backups with Proxmox Backup Server
|
||||
|
||||
See [backups/README.md](backups/README.md) for the full guide.
|
||||
|
||||
### Option A: Proxmox Backup Server (PBS) — recommended
|
||||
|
||||
PBS is a dedicated backup solution from Proxmox. It supports deduplication, incremental backups, and encryption. Run it on a second machine or a dedicated VM on another host.
|
||||
|
||||
Install PBS from [https://www.proxmox.com/en/downloads](https://www.proxmox.com/en/downloads) on a separate machine.
|
||||
|
||||
### Add PBS as a storage backend in Proxmox
|
||||
|
||||
In the web UI: **Datacenter** > **Storage** > **Add** > **Proxmox Backup Server**.
|
||||
|
||||
| Field | Value |
|
||||
|--------------|------------------------------------|
|
||||
| ID | pbs-01 |
|
||||
| Server | 192.168.1.20 (your PBS IP) |
|
||||
| Datastore | backup-store (your PBS datastore) |
|
||||
| Username | backup-user@pbs |
|
||||
| Password | (your PBS backup user password) |
|
||||
| Fingerprint | (shown in PBS dashboard) |
|
||||
|
||||
### Option B: local backup to a directory
|
||||
|
||||
For a simple setup without PBS, back up to a local directory or NFS share:
|
||||
|
||||
**Datacenter** > **Storage** > **Add** > **Directory**
|
||||
|
||||
Then create a backup job: **Datacenter** > **Backup** > **Add**
|
||||
|
||||
```
|
||||
Storage: local (or your NFS share)
|
||||
Schedule: 03:00 (daily at 3 AM)
|
||||
Mode: Snapshot (requires QEMU agent) or Suspend
|
||||
Compress: zstd
|
||||
Max Backups: 3
|
||||
```
|
||||
|
||||
### Verify backups
|
||||
|
||||
See [backups/backup-job.sh](backups/backup-job.sh) for a script that checks backup age and sends an alert if a backup is stale.
|
||||
|
||||
---
|
||||
|
||||
## 9. Monitoring with Prometheus and Grafana
|
||||
|
||||
See [monitoring/README.md](monitoring/README.md) for the full guide.
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
Proxmox host
|
||||
└── pve-exporter (scrapes Proxmox API)
|
||||
└── node_exporter (host metrics)
|
||||
|
||||
Monitoring VM
|
||||
└── Prometheus (scrapes exporters)
|
||||
└── Grafana (dashboards)
|
||||
```
|
||||
|
||||
### Deploy a monitoring VM or LXC
|
||||
|
||||
Create a small Ubuntu 24.04 LXC (512 MB RAM, 2 cores, 10 GB disk) or VM.
|
||||
|
||||
### Install Prometheus
|
||||
|
||||
```bash
|
||||
apt install -y prometheus
|
||||
```
|
||||
|
||||
Replace the default config with [monitoring/prometheus.yml](monitoring/prometheus.yml).
|
||||
|
||||
```bash
|
||||
cp prometheus.yml /etc/prometheus/prometheus.yml
|
||||
systemctl restart prometheus
|
||||
```
|
||||
|
||||
### Install Grafana
|
||||
|
||||
```bash
|
||||
apt install -y apt-transport-https software-properties-common
|
||||
wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key
|
||||
echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" \
|
||||
> /etc/apt/sources.list.d/grafana.list
|
||||
apt update && apt install -y grafana
|
||||
systemctl enable --now grafana-server
|
||||
```
|
||||
|
||||
Access Grafana at `http://monitoring-vm-ip:3000`. Default credentials: `admin / admin`.
|
||||
|
||||
Import the Proxmox dashboard: ID **10347** (Proxmox via Prometheus) from grafana.com.
|
||||
|
||||
---
|
||||
|
||||
## 10. Basic Hardening
|
||||
|
||||
See [security/README.md](security/README.md) for the full hardening guide.
|
||||
|
||||
### Quick wins
|
||||
|
||||
**Disable root SSH login:**
|
||||
|
||||
```bash
|
||||
sed -i 's/^#*PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
|
||||
systemctl restart sshd
|
||||
```
|
||||
|
||||
**Enable the Proxmox firewall (datacenter level):**
|
||||
|
||||
In the web UI: **Datacenter** > **Firewall** > **Options** > set **Firewall** to `Yes`.
|
||||
|
||||
Then set the default policy to DROP and add explicit ACCEPT rules for:
|
||||
- Port 8006 (web UI) from your management VLAN
|
||||
- Port 22 (SSH) from your management VLAN
|
||||
- Port 3128 (SPICE proxy) if you use SPICE consoles
|
||||
|
||||
**Enable 2FA:**
|
||||
|
||||
**Datacenter** > **Permissions** > **Two Factor** > **Add** > **TOTP**
|
||||
|
||||
Scan the QR code with an authenticator app (Aegis, Bitwarden, etc.).
|
||||
|
||||
**Create a non-root user for daily use:**
|
||||
|
||||
```bash
|
||||
pveum user add thomas@pve --password 'strongpassword'
|
||||
pveum role add HomeLabAdmin --privs \
|
||||
"VM.Allocate VM.Clone VM.Config.CDROM VM.Config.CPU VM.Config.Disk \
|
||||
VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options \
|
||||
VM.Console VM.Monitor VM.PowerMgmt VM.Snapshot Datastore.AllocateSpace \
|
||||
Datastore.Audit Pool.Allocate Sys.Audit Sys.Console"
|
||||
pveum aclmod / -user thomas@pve -role HomeLabAdmin
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Useful Daily Commands
|
||||
|
||||
### VM management
|
||||
|
||||
```bash
|
||||
# List all VMs
|
||||
qm list
|
||||
|
||||
# Start / stop / reset
|
||||
qm start <vmid>
|
||||
qm shutdown <vmid>
|
||||
qm reset <vmid>
|
||||
|
||||
# Show VM config
|
||||
qm config <vmid>
|
||||
|
||||
# Create a snapshot
|
||||
qm snapshot <vmid> snap-before-update --description "Before dist-upgrade"
|
||||
|
||||
# Rollback to snapshot
|
||||
qm rollback <vmid> snap-before-update
|
||||
|
||||
# Delete snapshot
|
||||
qm delsnapshot <vmid> snap-before-update
|
||||
```
|
||||
|
||||
### LXC management
|
||||
|
||||
```bash
|
||||
# List containers
|
||||
pct list
|
||||
|
||||
# Start / stop / enter
|
||||
pct start <ctid>
|
||||
pct stop <ctid>
|
||||
pct enter <ctid>
|
||||
|
||||
# Show config
|
||||
pct config <ctid>
|
||||
```
|
||||
|
||||
### Storage and ZFS
|
||||
|
||||
```bash
|
||||
# List storage
|
||||
pvesm status
|
||||
|
||||
# ZFS pool status
|
||||
zpool status
|
||||
|
||||
# ZFS filesystem list
|
||||
zfs list
|
||||
|
||||
# Show disk usage per dataset
|
||||
zfs list -o name,used,avail,refer
|
||||
```
|
||||
|
||||
### Cluster and node
|
||||
|
||||
```bash
|
||||
# Node status
|
||||
pvesh get /nodes/$(hostname)/status
|
||||
|
||||
# Task log
|
||||
pvesh get /nodes/$(hostname)/tasks --limit 20
|
||||
|
||||
# Check service status
|
||||
systemctl status pve-cluster pveproxy pvedaemon
|
||||
```
|
||||
|
||||
### Logs
|
||||
|
||||
```bash
|
||||
# Proxmox cluster log
|
||||
journalctl -u pve-cluster -f
|
||||
|
||||
# Task log for a specific VM
|
||||
cat /var/log/pve/tasks/active
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12. Directory Structure of This Repo
|
||||
|
||||
```
|
||||
homelab-proxmox-starter/
|
||||
├── README.md # This file — full step-by-step tutorial
|
||||
├── network/
|
||||
│ ├── README.md # Network guide: bridges, VLANs, pfSense
|
||||
│ └── vlan-setup.sh # VLAN configuration script for Proxmox
|
||||
├── vms/
|
||||
│ ├── README.md # VM creation and template guide
|
||||
│ └── cloud-init.yaml # Cloud-init example for Ubuntu 24.04
|
||||
├── backups/
|
||||
│ ├── README.md # Backup guide: PBS and local backup
|
||||
│ └── backup-job.sh # Backup verification script
|
||||
├── monitoring/
|
||||
│ ├── README.md # Monitoring guide: Prometheus + Grafana
|
||||
│ └── prometheus.yml # Prometheus config for Proxmox
|
||||
├── security/
|
||||
│ └── README.md # Hardening guide
|
||||
├── LICENSE # MIT
|
||||
└── .gitignore
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Contributing
|
||||
|
||||
Issues and pull requests are welcome.
|
||||
Keep things simple — this repo is intentionally a starter guide, not an automation framework.
|
||||
|
||||
## License
|
||||
|
||||
MIT — see [LICENSE](LICENSE).
|
||||
@@ -0,0 +1,287 @@
|
||||
# Backups Guide — Proxmox VE 9.1
|
||||
|
||||
This guide covers two backup strategies for a Proxmox homelab:
|
||||
|
||||
- **Option A**: Proxmox Backup Server (PBS) — recommended, supports deduplication and incremental backups
|
||||
- **Option B**: local backup to a directory or NFS share — simpler, no extra server needed
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Backup Concepts in Proxmox](#1-backup-concepts-in-proxmox)
|
||||
2. [Option A: Proxmox Backup Server (PBS)](#2-option-a-proxmox-backup-server-pbs)
|
||||
3. [Option B: Local Directory or NFS Backup](#3-option-b-local-directory-or-nfs-backup)
|
||||
4. [Backup Jobs: Schedule and Retention](#4-backup-jobs-schedule-and-retention)
|
||||
5. [Restore a VM or LXC](#5-restore-a-vm-or-lxc)
|
||||
6. [Verify Your Backups](#6-verify-your-backups)
|
||||
7. [Backup Script](#7-backup-script)
|
||||
|
||||
---
|
||||
|
||||
## 1. Backup Concepts in Proxmox
|
||||
|
||||
### Backup modes
|
||||
|
||||
| Mode | VM state during backup | Requires QEMU agent | Notes |
|
||||
|-----------|------------------------|---------------------|------------------------------------|
|
||||
| Snapshot | Running | Yes | Minimal downtime, best option |
|
||||
| Suspend | Briefly suspended | No | Short freeze, then continues |
|
||||
| Stop | Shut down | No | Safest consistency, longest window |
|
||||
|
||||
Use **Snapshot** mode for production VMs with the QEMU agent installed.
|
||||
Use **Stop** mode for VMs without the agent, or when data consistency is critical.
|
||||
|
||||
### Backup formats
|
||||
|
||||
- **VMA** (`.vma.zst`): native Proxmox format. Stores disk + config. Used by vzdump and PBS.
|
||||
- **tar.zst**: used for LXC containers.
|
||||
|
||||
### Retention policy
|
||||
|
||||
Proxmox supports keeping the last N backups, or defining a pruning schedule (keep-daily, keep-weekly, keep-monthly, keep-yearly). PBS handles this natively.
|
||||
|
||||
---
|
||||
|
||||
## 2. Option A: Proxmox Backup Server (PBS)
|
||||
|
||||
PBS is a separate product from Proxmox. Install it on a second machine or a dedicated VM on a different physical host.
|
||||
|
||||
Download: [https://www.proxmox.com/en/downloads](https://www.proxmox.com/en/downloads)
|
||||
|
||||
PBS ISO: **Proxmox Backup Server 3.x** — verifier la derniere version disponible sur [proxmox.com/en/downloads](https://www.proxmox.com/en/downloads).
|
||||
|
||||
### Install PBS
|
||||
|
||||
Boot from the PBS ISO. Installation is similar to Proxmox VE:
|
||||
- Assign a static IP (e.g. `192.168.1.20/24`)
|
||||
- Set a strong root password
|
||||
- Select a disk (PBS writes backups here — size it accordingly)
|
||||
|
||||
Access the PBS web UI at `https://192.168.1.20:8007`.
|
||||
|
||||
### Create a datastore
|
||||
|
||||
In the PBS web UI: **Administration** > **Datastores** > **Create Datastore**
|
||||
|
||||
| Field | Value |
|
||||
|---------|------------------------|
|
||||
| Name | backup-store |
|
||||
| Backing Path | /mnt/backup (or your disk mount point) |
|
||||
|
||||
### Create a backup user in PBS
|
||||
|
||||
```bash
|
||||
# On the PBS host
|
||||
proxmox-backup-manager user create backup-user@pbs --password 'StrongPass1!'
|
||||
proxmox-backup-manager acl update /datastore/backup-store backup-user@pbs DatastoreBackup
|
||||
```
|
||||
|
||||
### Get the PBS fingerprint
|
||||
|
||||
```bash
|
||||
# On the PBS host
|
||||
proxmox-backup-manager cert info | grep Fingerprint
|
||||
```
|
||||
|
||||
### Add PBS as storage in Proxmox
|
||||
|
||||
**Datacenter** > **Storage** > **Add** > **Proxmox Backup Server**
|
||||
|
||||
| Field | Value |
|
||||
|-------------|-------------------------------------|
|
||||
| ID | pbs-01 |
|
||||
| Server | 192.168.1.20 |
|
||||
| Username | backup-user@pbs |
|
||||
| Password | StrongPass1! |
|
||||
| Datastore | backup-store |
|
||||
| Fingerprint | (paste from PBS) |
|
||||
| Encryption key | Optional — generate in PBS UI |
|
||||
|
||||
### Enable client-side encryption (optional but recommended)
|
||||
|
||||
In the storage config, click **Encryption** > **Generate** to create an encryption key.
|
||||
Store this key somewhere safe (not only on the Proxmox host).
|
||||
|
||||
Backups stored in PBS will be encrypted at rest. The key is required for any restore.
|
||||
|
||||
---
|
||||
|
||||
## 3. Option B: Local Directory or NFS Backup
|
||||
|
||||
### Local directory
|
||||
|
||||
**Datacenter** > **Storage** > **Add** > **Directory**
|
||||
|
||||
| Field | Value |
|
||||
|---------|------------------------|
|
||||
| ID | backup-local |
|
||||
| Directory | /mnt/backup-disk |
|
||||
| Content | VZDump backup file |
|
||||
|
||||
Mount an external disk:
|
||||
|
||||
```bash
|
||||
# Find the disk
|
||||
lsblk
|
||||
# Format if needed (only on first use)
|
||||
mkfs.ext4 /dev/sdb1
|
||||
# Mount
|
||||
mkdir -p /mnt/backup-disk
|
||||
echo '/dev/sdb1 /mnt/backup-disk ext4 defaults,nofail 0 2' >> /etc/fstab
|
||||
mount -a
|
||||
```
|
||||
|
||||
### NFS share
|
||||
|
||||
Mount an NFS share from a NAS:
|
||||
|
||||
```bash
|
||||
apt install -y nfs-common
|
||||
mkdir -p /mnt/nas-backup
|
||||
|
||||
# Add to /etc/fstab
|
||||
echo '192.168.1.30:/volume1/proxmox-backup /mnt/nas-backup nfs defaults,nofail 0 0' >> /etc/fstab
|
||||
mount -a
|
||||
```
|
||||
|
||||
Then add as a Directory storage in Proxmox.
|
||||
|
||||
---
|
||||
|
||||
## 4. Backup Jobs: Schedule and Retention
|
||||
|
||||
### Create a backup job
|
||||
|
||||
**Datacenter** > **Backup** > **Add**
|
||||
|
||||
| Field | Recommended value |
|
||||
|-------------|-------------------------------|
|
||||
| Storage | pbs-01 (or backup-local) |
|
||||
| Schedule | 03:00 (daily at 3 AM) |
|
||||
| Selection | All (or select specific VMs) |
|
||||
| Mode | Snapshot |
|
||||
| Compression | ZSTD |
|
||||
| Send email | your@email.com (for failures) |
|
||||
|
||||
> **Note:** Proxmox VE does not include a mail transfer agent (MTA) by default. To use `--notify-email`, install one first: `apt-get install postfix` (or `msmtp` for a lightweight alternative).
|
||||
|
||||
### PBS retention (prune schedule)
|
||||
|
||||
In PBS: **Datastore** > **backup-store** > **Prune & GC** > **Add Prune Job**
|
||||
|
||||
```
|
||||
keep-last: 3
|
||||
keep-daily: 7
|
||||
keep-weekly: 4
|
||||
keep-monthly: 3
|
||||
```
|
||||
|
||||
This keeps:
|
||||
- 3 most recent backups regardless of age
|
||||
- 1 backup per day for the last 7 days
|
||||
- 1 backup per week for the last 4 weeks
|
||||
- 1 backup per month for the last 3 months
|
||||
|
||||
For local/directory storage, set retention in the backup job:
|
||||
**Max Backups**: 7 (keep last 7).
|
||||
|
||||
---
|
||||
|
||||
## 5. Restore a VM or LXC
|
||||
|
||||
### Restore via the web UI
|
||||
|
||||
1. Navigate to **local** (or **pbs-01**) storage > **Backups**.
|
||||
2. Select the backup file.
|
||||
3. Click **Restore**.
|
||||
4. Assign a new VM ID and storage, then click **Restore**.
|
||||
|
||||
### Restore via CLI
|
||||
|
||||
```bash
|
||||
# List available backups
|
||||
pvesm list pbs-01
|
||||
|
||||
# Restore a VM (adjust backup path and target storage)
|
||||
qmrestore pbs-01:backup/vm/101/2026-04-01T03:00:00Z 101 \
|
||||
--storage local-lvm \
|
||||
--unique 1
|
||||
|
||||
# Restore an LXC
|
||||
pct restore 200 pbs-01:backup/ct/200/2026-04-01T03:00:00Z \
|
||||
--storage local-lvm \
|
||||
--unique 1
|
||||
```
|
||||
|
||||
`--unique 1`: assigns a new unique MAC address and VMID to avoid conflicts.
|
||||
|
||||
### Test restore to a different VM ID
|
||||
|
||||
Always test restores before relying on them in a real incident:
|
||||
|
||||
```bash
|
||||
qmrestore pbs-01:backup/vm/101/2026-04-01T03:00:00Z 999 \
|
||||
--storage local-lvm
|
||||
qm start 999
|
||||
# Verify the VM boots and data is intact
|
||||
qm destroy 999
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Verify Your Backups
|
||||
|
||||
See [backup-job.sh](backup-job.sh) for an automated verification script.
|
||||
|
||||
### Check backup age via PBS API
|
||||
|
||||
```bash
|
||||
# On the PBS host or via API
|
||||
proxmox-backup-client list --repository backup-user@pbs@192.168.1.20:backup-store
|
||||
```
|
||||
|
||||
### Check backup age on local storage
|
||||
|
||||
```bash
|
||||
# List backup files with timestamps
|
||||
ls -lht /mnt/backup-disk/dump/ | head -20
|
||||
```
|
||||
|
||||
### Verify PBS backup integrity
|
||||
|
||||
PBS stores SHA-256 checksums for every chunk. Run a verification job:
|
||||
|
||||
**PBS web UI** > **Datastore** > **backup-store** > **Verify** > **Verify All**
|
||||
|
||||
Or via CLI on the PBS host:
|
||||
|
||||
```bash
|
||||
proxmox-backup-manager verify backup-store
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Backup Script
|
||||
|
||||
The script [backup-job.sh](backup-job.sh) checks the age of the latest backup for each VM and sends an alert if:
|
||||
|
||||
- No backup exists in the last 24 hours
|
||||
- The backup job returned an error
|
||||
|
||||
Usage:
|
||||
|
||||
```bash
|
||||
bash backups/backup-job.sh --storage pbs-01 --max-age-hours 26
|
||||
```
|
||||
|
||||
Configure it as a cron job on the Proxmox host:
|
||||
|
||||
```bash
|
||||
# Edit crontab
|
||||
crontab -e
|
||||
|
||||
# Run every day at 05:30 (after the 03:00 backup job)
|
||||
30 5 * * * /root/homelab-proxmox-starter/backups/backup-job.sh \
|
||||
--storage pbs-01 --max-age-hours 26 >> /var/log/backup-check.log 2>&1
|
||||
```
|
||||
@@ -0,0 +1,268 @@
|
||||
#!/usr/bin/env bash
|
||||
# backup-job.sh — Verify Proxmox backup age and report stale or missing backups
|
||||
#
|
||||
# For each VM and LXC on this node, checks when the last backup was created.
|
||||
# Exits with code 1 and prints an alert if any backup is older than --max-age-hours.
|
||||
#
|
||||
# Usage:
|
||||
# bash backup-job.sh [OPTIONS]
|
||||
#
|
||||
# Options:
|
||||
# --storage <id> Proxmox storage ID to check (e.g. pbs-01, backup-local)
|
||||
# --max-age-hours <n> Alert if last backup is older than N hours (default: 26)
|
||||
# --notify-email <addr> Send alert email via sendmail (optional)
|
||||
# --dry-run Only report, do not exit with error code
|
||||
# --help Show this help message
|
||||
#
|
||||
# Exit codes:
|
||||
# 0 All backups are within the max-age window
|
||||
# 1 One or more backups are stale or missing
|
||||
#
|
||||
# Requirements:
|
||||
# - Run on the Proxmox VE host as root
|
||||
# - pvesh and pvesm must be available (they are on any PVE node)
|
||||
# - sendmail optional (only required for --notify-email)
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Defaults
|
||||
# ---------------------------------------------------------------------------
|
||||
STORAGE=""
|
||||
MAX_AGE_HOURS=26
|
||||
NOTIFY_EMAIL=""
|
||||
DRY_RUN=false
|
||||
SCRIPT_NAME="$(basename "$0")"
|
||||
NODE="$(hostname)"
|
||||
ALERT_LINES=()
|
||||
HAS_STALE=false
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Color output
|
||||
# ---------------------------------------------------------------------------
|
||||
if [ -t 1 ]; then
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[1;33m'
|
||||
GREEN='\033[0;32m'
|
||||
CYAN='\033[0;36m'
|
||||
RESET='\033[0m'
|
||||
else
|
||||
RED='' YELLOW='' GREEN='' CYAN='' RESET=''
|
||||
fi
|
||||
|
||||
log_info() { echo -e "${CYAN}[INFO]${RESET} $(date '+%Y-%m-%d %H:%M:%S') $*"; }
|
||||
log_ok() { echo -e "${GREEN}[OK]${RESET} $(date '+%Y-%m-%d %H:%M:%S') $*"; }
|
||||
log_warn() { echo -e "${YELLOW}[WARN]${RESET} $(date '+%Y-%m-%d %H:%M:%S') $*"; }
|
||||
log_alert() { echo -e "${RED}[ALERT]${RESET} $(date '+%Y-%m-%d %H:%M:%S') $*"; }
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Usage
|
||||
# ---------------------------------------------------------------------------
|
||||
usage() {
|
||||
sed -n '/^# Usage:/,/^# Requirements:/p' "$0" | sed 's/^# //'
|
||||
exit 0
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Argument parsing
|
||||
# ---------------------------------------------------------------------------
|
||||
parse_args() {
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--storage)
|
||||
STORAGE="${2:?--storage requires a value}"
|
||||
shift 2
|
||||
;;
|
||||
--max-age-hours)
|
||||
MAX_AGE_HOURS="${2:?--max-age-hours requires a value}"
|
||||
shift 2
|
||||
;;
|
||||
--notify-email)
|
||||
NOTIFY_EMAIL="${2:?--notify-email requires a value}"
|
||||
shift 2
|
||||
;;
|
||||
--dry-run)
|
||||
DRY_RUN=true
|
||||
shift
|
||||
;;
|
||||
--help|-h)
|
||||
usage
|
||||
;;
|
||||
*)
|
||||
echo "Unknown option: $1" >&2
|
||||
usage
|
||||
;;
|
||||
esac
|
||||
done
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Validation
|
||||
# ---------------------------------------------------------------------------
|
||||
validate() {
|
||||
if [[ -z "$STORAGE" ]]; then
|
||||
echo "Error: --storage is required." >&2
|
||||
usage
|
||||
fi
|
||||
|
||||
if ! [[ "$MAX_AGE_HOURS" =~ ^[0-9]+$ ]] || (( MAX_AGE_HOURS < 1 )); then
|
||||
echo "Error: --max-age-hours must be a positive integer." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ $EUID -ne 0 ]]; then
|
||||
echo "Error: this script must run as root on the Proxmox host." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! command -v pvesh &>/dev/null; then
|
||||
echo "Error: pvesh not found. This script must run on a Proxmox VE node." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! command -v pvesm &>/dev/null; then
|
||||
echo "Error: pvesm not found. This script must run on a Proxmox VE node." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Verify storage exists
|
||||
if ! pvesm status | awk '{print $1}' | grep -q "^${STORAGE}$"; then
|
||||
echo "Error: storage '${STORAGE}' not found on this node." >&2
|
||||
echo "Available storage:" >&2
|
||||
pvesm status | awk 'NR>1 {print " " $1}' >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Get list of VMIDs (VMs and LXCs) on this node
|
||||
# ---------------------------------------------------------------------------
|
||||
get_all_vmids() {
|
||||
{
|
||||
pvesh get /nodes/"$NODE"/qemu --output-format json 2>/dev/null \
|
||||
| python3 -c "import sys,json; [print(v['vmid']) for v in json.load(sys.stdin)]" 2>/dev/null || true
|
||||
pvesh get /nodes/"$NODE"/lxc --output-format json 2>/dev/null \
|
||||
| python3 -c "import sys,json; [print(v['vmid']) for v in json.load(sys.stdin)]" 2>/dev/null || true
|
||||
} | sort -n
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Get the timestamp of the latest backup for a given VMID in the storage
|
||||
# Returns epoch seconds or empty string if no backup found
|
||||
# ---------------------------------------------------------------------------
|
||||
get_latest_backup_epoch() {
|
||||
local vmid="$1"
|
||||
# pvesm list returns lines like:
|
||||
# pbs-01:backup/vm/101/2026-04-01T03:00:00Z ... ctime=1743476400
|
||||
pvesm list "$STORAGE" --vmid "$vmid" 2>/dev/null \
|
||||
| awk 'NR>1 {print $NF}' \
|
||||
| grep -oP 'ctime=\K[0-9]+' \
|
||||
| sort -n \
|
||||
| tail -1
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Send email alert via sendmail
|
||||
# ---------------------------------------------------------------------------
|
||||
send_email_alert() {
|
||||
if [[ -z "$NOTIFY_EMAIL" ]]; then
|
||||
return
|
||||
fi
|
||||
|
||||
if ! command -v sendmail &>/dev/null; then
|
||||
log_warn "sendmail not found — cannot send email alert."
|
||||
return
|
||||
fi
|
||||
|
||||
local subject="[Proxmox Backup Alert] $NODE — stale or missing backups"
|
||||
local body
|
||||
body=$(printf "Backup alert from %s\n\nThe following VMs/LXCs have stale or missing backups:\n\n%s\n\nMax age configured: %d hours\nTimestamp: %s\n" \
|
||||
"$NODE" \
|
||||
"$(printf '%s\n' "${ALERT_LINES[@]}")" \
|
||||
"$MAX_AGE_HOURS" \
|
||||
"$(date)")
|
||||
|
||||
{
|
||||
echo "To: $NOTIFY_EMAIL"
|
||||
echo "Subject: $subject"
|
||||
echo "Content-Type: text/plain; charset=utf-8"
|
||||
echo ""
|
||||
echo "$body"
|
||||
} | sendmail "$NOTIFY_EMAIL"
|
||||
|
||||
log_info "Alert email sent to $NOTIFY_EMAIL."
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Main
|
||||
# ---------------------------------------------------------------------------
|
||||
main() {
|
||||
parse_args "$@"
|
||||
validate
|
||||
|
||||
log_info "Proxmox backup verification"
|
||||
log_info "Node : $NODE"
|
||||
log_info "Storage : $STORAGE"
|
||||
log_info "Max age : ${MAX_AGE_HOURS}h"
|
||||
log_info "Dry-run : $DRY_RUN"
|
||||
echo
|
||||
|
||||
local now
|
||||
now=$(date +%s)
|
||||
local max_age_seconds=$(( MAX_AGE_HOURS * 3600 ))
|
||||
|
||||
mapfile -t vmids < <(get_all_vmids)
|
||||
|
||||
if [[ ${#vmids[@]} -eq 0 ]]; then
|
||||
log_warn "No VMs or LXCs found on node '$NODE'."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
log_info "Found ${#vmids[@]} VMs/LXCs to check."
|
||||
echo
|
||||
|
||||
for vmid in "${vmids[@]}"; do
|
||||
local latest_epoch
|
||||
latest_epoch=$(get_latest_backup_epoch "$vmid")
|
||||
|
||||
if [[ -z "$latest_epoch" ]]; then
|
||||
local msg="VMID $vmid — NO BACKUP FOUND in storage '$STORAGE'"
|
||||
log_alert "$msg"
|
||||
ALERT_LINES+=("$msg")
|
||||
HAS_STALE=true
|
||||
continue
|
||||
fi
|
||||
|
||||
local age_seconds=$(( now - latest_epoch ))
|
||||
local age_hours=$(( age_seconds / 3600 ))
|
||||
local last_backup_str
|
||||
last_backup_str=$(date -d "@$latest_epoch" '+%Y-%m-%d %H:%M:%S' 2>/dev/null || date -r "$latest_epoch" '+%Y-%m-%d %H:%M:%S')
|
||||
|
||||
if (( age_seconds > max_age_seconds )); then
|
||||
local msg="VMID $vmid — last backup: $last_backup_str (${age_hours}h ago — STALE, max ${MAX_AGE_HOURS}h)"
|
||||
log_alert "$msg"
|
||||
ALERT_LINES+=("$msg")
|
||||
HAS_STALE=true
|
||||
else
|
||||
log_ok "VMID $vmid — last backup: $last_backup_str (${age_hours}h ago)"
|
||||
fi
|
||||
done
|
||||
|
||||
echo
|
||||
|
||||
if [[ "$HAS_STALE" == true ]]; then
|
||||
log_alert "RESULT: ${#ALERT_LINES[@]} VM(s) with stale or missing backups."
|
||||
send_email_alert
|
||||
if [[ "$DRY_RUN" == false ]]; then
|
||||
exit 1
|
||||
else
|
||||
log_info "Dry-run mode: exiting with 0 despite alerts."
|
||||
exit 0
|
||||
fi
|
||||
else
|
||||
log_ok "RESULT: All ${#vmids[@]} VM(s) have backups within the ${MAX_AGE_HOURS}h window."
|
||||
exit 0
|
||||
fi
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -0,0 +1,378 @@
|
||||
# Monitoring Guide — Proxmox VE 9.1
|
||||
|
||||
This guide sets up monitoring for a Proxmox homelab using:
|
||||
|
||||
- **prometheus-pve-exporter**: exposes Proxmox API metrics (VMs, nodes, storage, cluster)
|
||||
- **node_exporter**: exposes host-level OS metrics (CPU, RAM, disk, network)
|
||||
- **Prometheus**: scrapes and stores metrics
|
||||
- **Grafana**: dashboards
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Architecture Overview](#1-architecture-overview)
|
||||
2. [Create the Monitoring VM or LXC](#2-create-the-monitoring-vm-or-lxc)
|
||||
3. [Install prometheus-pve-exporter](#3-install-prometheus-pve-exporter)
|
||||
4. [Install node_exporter on Proxmox Hosts](#4-install-node_exporter-on-proxmox-hosts)
|
||||
5. [Install Prometheus](#5-install-prometheus)
|
||||
6. [Configure Prometheus](#6-configure-prometheus)
|
||||
7. [Install Grafana](#7-install-grafana)
|
||||
8. [Import Dashboards](#8-import-dashboards)
|
||||
9. [Alerting with Alertmanager (Optional)](#9-alerting-with-alertmanager-optional)
|
||||
10. [Useful Queries (PromQL)](#10-useful-queries-promql)
|
||||
|
||||
---
|
||||
|
||||
## 1. Architecture Overview
|
||||
|
||||
```
|
||||
Proxmox Node(s)
|
||||
|- node_exporter :9100 (host OS metrics)
|
||||
|- pve-exporter :9221 (Proxmox API metrics, runs on monitoring VM or node)
|
||||
|
||||
Monitoring VM / LXC
|
||||
|- Prometheus :9090 (scrapes exporters, stores TSDB)
|
||||
|- Grafana :3000 (reads Prometheus, renders dashboards)
|
||||
|- Alertmanager :9093 (optional, handles alert routing)
|
||||
```
|
||||
|
||||
All components run on a single monitoring VM or LXC.
|
||||
The Proxmox API is scraped remotely — no agent on the hypervisor beyond node_exporter.
|
||||
|
||||
---
|
||||
|
||||
## 2. Create the Monitoring VM or LXC
|
||||
|
||||
A lightweight LXC is sufficient:
|
||||
|
||||
```bash
|
||||
pct create 150 local:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst \
|
||||
--hostname monitoring \
|
||||
--memory 1024 \
|
||||
--cores 2 \
|
||||
--net0 name=eth0,bridge=vmbr0,ip=192.168.1.15/24,gw=192.168.1.1 \
|
||||
--storage local-lvm \
|
||||
--rootfs local-lvm:10 \
|
||||
--unprivileged 1 \
|
||||
--onboot 1 \
|
||||
--start 1
|
||||
```
|
||||
|
||||
Update the system:
|
||||
|
||||
```bash
|
||||
pct enter 150
|
||||
apt update && apt upgrade -y
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Install prometheus-pve-exporter
|
||||
|
||||
`prometheus-pve-exporter` is a Python application that queries the Proxmox API and exposes metrics in Prometheus format.
|
||||
|
||||
Source: [https://github.com/prometheus-pve/prometheus-pve-exporter](https://github.com/prometheus-pve/prometheus-pve-exporter)
|
||||
|
||||
### Install on the monitoring LXC
|
||||
|
||||
```bash
|
||||
apt install -y python3-pip python3-venv
|
||||
|
||||
python3 -m venv /opt/pve-exporter
|
||||
/opt/pve-exporter/bin/pip install prometheus-pve-exporter
|
||||
```
|
||||
|
||||
### Create a read-only Proxmox user for the exporter
|
||||
|
||||
On the Proxmox host (not the monitoring LXC):
|
||||
|
||||
```bash
|
||||
pveum user add pve-exporter@pve --password 'ExporterReadOnly1!'
|
||||
pveum role add PVEExporterRole --privs "Sys.Audit VM.Audit Datastore.Audit Pool.Audit"
|
||||
pveum aclmod / -user pve-exporter@pve -role PVEExporterRole
|
||||
```
|
||||
|
||||
### Configure the exporter
|
||||
|
||||
On the monitoring LXC:
|
||||
|
||||
```bash
|
||||
mkdir -p /etc/prometheus
|
||||
cat > /etc/prometheus/pve.yml << 'EOF'
|
||||
default:
|
||||
user: pve-exporter@pve
|
||||
password: ExporterReadOnly1!
|
||||
verify_ssl: false
|
||||
EOF
|
||||
|
||||
chmod 600 /etc/prometheus/pve.yml
|
||||
```
|
||||
|
||||
### Create a systemd service
|
||||
|
||||
```bash
|
||||
cat > /etc/systemd/system/prometheus-pve-exporter.service << 'EOF'
|
||||
[Unit]
|
||||
Description=Prometheus Proxmox VE Exporter
|
||||
Documentation=https://github.com/prometheus-pve/prometheus-pve-exporter
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=nobody
|
||||
Group=nogroup
|
||||
ExecStart=/opt/pve-exporter/bin/pve_exporter \
|
||||
--config.file=/etc/prometheus/pve.yml \
|
||||
--web.listen-address=0.0.0.0:9221
|
||||
Restart=on-failure
|
||||
RestartSec=5s
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
|
||||
systemctl daemon-reload
|
||||
systemctl enable --now prometheus-pve-exporter
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
curl -s "http://localhost:9221/pve?target=192.168.1.10&module=default" | head -20
|
||||
```
|
||||
|
||||
Replace `192.168.1.10` with your Proxmox node IP.
|
||||
|
||||
---
|
||||
|
||||
## 4. Install node_exporter on Proxmox Hosts
|
||||
|
||||
`node_exporter` exposes OS-level metrics from the Proxmox host itself.
|
||||
|
||||
Run this on each Proxmox node (not the monitoring LXC):
|
||||
|
||||
```bash
|
||||
# Download node_exporter
|
||||
NODE_EXPORTER_VERSION="1.8.2"
|
||||
wget -q "https://github.com/prometheus/node_exporter/releases/download/v${NODE_EXPORTER_VERSION}/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gz" \
|
||||
-O /tmp/node_exporter.tar.gz
|
||||
|
||||
tar -xzf /tmp/node_exporter.tar.gz -C /tmp
|
||||
cp /tmp/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64/node_exporter /usr/local/bin/
|
||||
chmod +x /usr/local/bin/node_exporter
|
||||
|
||||
# Create system user
|
||||
useradd --no-create-home --shell /bin/false node_exporter
|
||||
|
||||
# Create systemd service
|
||||
cat > /etc/systemd/system/node_exporter.service << 'EOF'
|
||||
[Unit]
|
||||
Description=Prometheus Node Exporter
|
||||
Documentation=https://github.com/prometheus/node_exporter
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
User=node_exporter
|
||||
Group=node_exporter
|
||||
Type=simple
|
||||
ExecStart=/usr/local/bin/node_exporter \
|
||||
--collector.disable-defaults \
|
||||
--collector.cpu \
|
||||
--collector.diskstats \
|
||||
--collector.filesystem \
|
||||
--collector.loadavg \
|
||||
--collector.meminfo \
|
||||
--collector.netdev \
|
||||
--collector.time \
|
||||
--collector.vmstat \
|
||||
--web.listen-address=0.0.0.0:9100
|
||||
Restart=on-failure
|
||||
RestartSec=5s
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
|
||||
systemctl daemon-reload
|
||||
systemctl enable --now node_exporter
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:9100/metrics | grep node_cpu_seconds_total | head -5
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Install Prometheus
|
||||
|
||||
On the monitoring LXC:
|
||||
|
||||
```bash
|
||||
apt install -y prometheus
|
||||
systemctl enable prometheus
|
||||
```
|
||||
|
||||
The default Prometheus config is at `/etc/prometheus/prometheus.yml`.
|
||||
Replace it with the config from this repo.
|
||||
|
||||
---
|
||||
|
||||
## 6. Configure Prometheus
|
||||
|
||||
Copy the provided config:
|
||||
|
||||
```bash
|
||||
cp prometheus.yml /etc/prometheus/prometheus.yml
|
||||
systemctl restart prometheus
|
||||
```
|
||||
|
||||
See [prometheus.yml](prometheus.yml) for the full configuration.
|
||||
|
||||
Key targets in the config:
|
||||
|
||||
| Job | Port | Source |
|
||||
|------------------------|------|----------------------------------|
|
||||
| `pve-node` | 9221 | Proxmox API via pve-exporter |
|
||||
| `node-pve` | 9100 | Proxmox host OS via node_exporter|
|
||||
| `prometheus` | 9090 | Prometheus self-monitoring |
|
||||
|
||||
Verify Prometheus targets at `http://192.168.1.15:9090/targets`.
|
||||
|
||||
All targets should show **UP**.
|
||||
|
||||
---
|
||||
|
||||
## 7. Install Grafana
|
||||
|
||||
On the monitoring LXC:
|
||||
|
||||
```bash
|
||||
apt install -y apt-transport-https software-properties-common wget gnupg
|
||||
|
||||
mkdir -p /usr/share/keyrings
|
||||
wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key
|
||||
|
||||
echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" \
|
||||
> /etc/apt/sources.list.d/grafana.list
|
||||
|
||||
apt update && apt install -y grafana
|
||||
|
||||
systemctl enable --now grafana-server
|
||||
```
|
||||
|
||||
Access Grafana at `http://192.168.1.15:3000`.
|
||||
|
||||
Default credentials: `admin` / `admin`. Change on first login.
|
||||
|
||||
### Add Prometheus as a data source
|
||||
|
||||
1. In Grafana: **Connections** > **Data Sources** > **Add data source** > **Prometheus**
|
||||
2. URL: `http://localhost:9090`
|
||||
3. Click **Save & Test**. Should return "Data source is working".
|
||||
|
||||
---
|
||||
|
||||
## 8. Import Dashboards
|
||||
|
||||
### Proxmox VE dashboard (via pve-exporter)
|
||||
|
||||
Dashboard ID: **10347**
|
||||
|
||||
1. In Grafana: **Dashboards** > **Import**
|
||||
2. Enter ID `10347`, click **Load**
|
||||
3. Select your Prometheus data source
|
||||
4. Click **Import**
|
||||
|
||||
This dashboard shows: node status, VM count, CPU/RAM per VM, storage usage, network I/O.
|
||||
|
||||
### Node Exporter Full (host OS metrics)
|
||||
|
||||
Dashboard ID: **1860**
|
||||
|
||||
Same import process. Shows detailed CPU, memory, disk, and network metrics from node_exporter.
|
||||
|
||||
### Proxmox Cluster Dashboard
|
||||
|
||||
Dashboard ID: **15356** — useful if you have multiple Proxmox nodes in a cluster.
|
||||
|
||||
---
|
||||
|
||||
## 9. Alerting with Alertmanager (Optional)
|
||||
|
||||
Alertmanager handles routing, grouping, and sending of alerts from Prometheus.
|
||||
|
||||
```bash
|
||||
apt install -y prometheus-alertmanager
|
||||
systemctl enable --now prometheus-alertmanager
|
||||
```
|
||||
|
||||
Example alert rule — create `/etc/prometheus/alerts/proxmox.yml`:
|
||||
|
||||
```yaml
|
||||
groups:
|
||||
- name: proxmox
|
||||
rules:
|
||||
- alert: ProxmoxNodeDown
|
||||
expr: up{job="node-pve"} == 0
|
||||
for: 2m
|
||||
labels:
|
||||
severity: critical
|
||||
annotations:
|
||||
summary: "Proxmox node unreachable"
|
||||
description: "node_exporter on {{ $labels.instance }} has been down for 2 minutes."
|
||||
|
||||
- alert: ProxmoxHighCPU
|
||||
expr: 100 - (avg by(instance)(rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 90
|
||||
for: 5m
|
||||
labels:
|
||||
severity: warning
|
||||
annotations:
|
||||
summary: "Proxmox high CPU usage"
|
||||
description: "CPU usage on {{ $labels.instance }} is above 90% for 5 minutes."
|
||||
|
||||
- alert: ProxmoxLowDisk
|
||||
expr: (node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"}) * 100 < 10
|
||||
for: 1m
|
||||
labels:
|
||||
severity: warning
|
||||
annotations:
|
||||
summary: "Proxmox root disk almost full"
|
||||
description: "Less than 10% free disk space on {{ $labels.instance }}."
|
||||
```
|
||||
|
||||
Add the rule file reference to `prometheus.yml`:
|
||||
|
||||
```yaml
|
||||
rule_files:
|
||||
- /etc/prometheus/alerts/*.yml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Useful Queries (PromQL)
|
||||
|
||||
```promql
|
||||
# Proxmox node CPU usage (%)
|
||||
100 - avg by(instance)(rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100
|
||||
|
||||
# Memory used (%)
|
||||
(1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100
|
||||
|
||||
# Disk space used on root (%)
|
||||
(1 - node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"}) * 100
|
||||
|
||||
# Number of running VMs on the cluster
|
||||
pve_up{type="qemu"}
|
||||
|
||||
# VM CPU usage
|
||||
rate(pve_cpu_usage_ratio{type="qemu"}[5m]) * 100
|
||||
|
||||
# Storage usage per pool
|
||||
pve_disk_usage_bytes / pve_disk_size_bytes * 100
|
||||
|
||||
# Network receive rate per interface (bytes/s)
|
||||
rate(node_network_receive_bytes_total[5m])
|
||||
```
|
||||
@@ -0,0 +1,175 @@
|
||||
# prometheus.yml — Prometheus configuration for Proxmox VE 9.1 homelab monitoring
|
||||
#
|
||||
# This config scrapes:
|
||||
# - Proxmox VE API metrics via prometheus-pve-exporter (port 9221)
|
||||
# - Host OS metrics via node_exporter on the Proxmox node (port 9100)
|
||||
# - Prometheus self-monitoring (port 9090)
|
||||
#
|
||||
# Prerequisites:
|
||||
# - prometheus-pve-exporter running on the monitoring VM (192.168.1.15:9221)
|
||||
# - node_exporter running on each Proxmox node (port 9100)
|
||||
# - A read-only Proxmox user configured in /etc/prometheus/pve.yml
|
||||
#
|
||||
# Documentation:
|
||||
# - pve-exporter: https://github.com/prometheus-pve/prometheus-pve-exporter
|
||||
# - node_exporter: https://github.com/prometheus/node_exporter
|
||||
# - prometheus: https://prometheus.io/docs/prometheus/latest/configuration/configuration/
|
||||
|
||||
global:
|
||||
# How often Prometheus scrapes each target
|
||||
scrape_interval: 30s
|
||||
|
||||
# Timeout for each scrape request
|
||||
scrape_timeout: 15s
|
||||
|
||||
# How often Prometheus evaluates alert rules
|
||||
evaluation_interval: 30s
|
||||
|
||||
# Labels added to all time series scraped by this Prometheus instance
|
||||
external_labels:
|
||||
cluster: homelab
|
||||
region: on-premise
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Alertmanager configuration (uncomment if Alertmanager is installed)
|
||||
# ---------------------------------------------------------------------------
|
||||
# alerting:
|
||||
# alertmanagers:
|
||||
# - static_configs:
|
||||
# - targets:
|
||||
# - localhost:9093
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Alert rule files
|
||||
# ---------------------------------------------------------------------------
|
||||
rule_files: []
|
||||
# Uncomment and create the file to enable alerting:
|
||||
# rule_files:
|
||||
# - /etc/prometheus/alerts/*.yml
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Scrape configurations
|
||||
# ---------------------------------------------------------------------------
|
||||
scrape_configs:
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Prometheus self-monitoring
|
||||
# Scrapes Prometheus's own /metrics endpoint.
|
||||
# --------------------------------------------------------------------------
|
||||
- job_name: prometheus
|
||||
static_configs:
|
||||
- targets:
|
||||
- localhost:9090
|
||||
labels:
|
||||
instance: monitoring
|
||||
role: prometheus
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Proxmox VE API metrics via prometheus-pve-exporter
|
||||
#
|
||||
# The pve-exporter uses a "target" parameter to know which Proxmox node to
|
||||
# query. The scrape is sent to the exporter (192.168.1.15:9221), which then
|
||||
# queries the Proxmox API on the specified target.
|
||||
#
|
||||
# This configuration uses relabeling to pass the target address to the
|
||||
# exporter as a URL parameter, following the pve-exporter documentation.
|
||||
#
|
||||
# Add one entry per Proxmox node under targets.
|
||||
# --------------------------------------------------------------------------
|
||||
- job_name: pve-node
|
||||
# The actual scrape target is the exporter, not the Proxmox node directly.
|
||||
# The Proxmox node address is passed as a query parameter.
|
||||
metrics_path: /pve
|
||||
params:
|
||||
# The Proxmox API module defined in /etc/prometheus/pve.yml
|
||||
module: [default]
|
||||
# Include per-VM/LXC metrics (set to 1 to enable, 0 for cluster-level only)
|
||||
# cluster: ["1"]
|
||||
static_configs:
|
||||
- targets:
|
||||
# Replace with your Proxmox node IP(s)
|
||||
# For multiple nodes, add one entry per node.
|
||||
- 192.168.1.10 # pve01
|
||||
# - 192.168.1.11 # pve02 (uncomment for a second node)
|
||||
labels:
|
||||
cluster: homelab
|
||||
relabel_configs:
|
||||
# Copy the scraped address into the __param_target label
|
||||
# so pve-exporter knows which Proxmox node to query.
|
||||
- source_labels: [__address__]
|
||||
target_label: __param_target
|
||||
# Keep the Proxmox node address as the "instance" label in metrics.
|
||||
- source_labels: [__param_target]
|
||||
target_label: instance
|
||||
# Redirect the actual HTTP request to the exporter running locally.
|
||||
- target_label: __address__
|
||||
replacement: localhost:9221
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Node Exporter — host OS metrics on Proxmox node(s)
|
||||
#
|
||||
# node_exporter must be installed and running on each Proxmox node.
|
||||
# See monitoring/README.md for installation instructions.
|
||||
# --------------------------------------------------------------------------
|
||||
- job_name: node-pve
|
||||
static_configs:
|
||||
- targets:
|
||||
# One entry per Proxmox node.
|
||||
- 192.168.1.10:9100 # pve01
|
||||
# - 192.168.1.11:9100 # pve02
|
||||
labels:
|
||||
role: hypervisor
|
||||
cluster: homelab
|
||||
relabel_configs:
|
||||
# Strip the port from the instance label for cleaner display in Grafana
|
||||
- source_labels: [__address__]
|
||||
target_label: instance
|
||||
regex: '([^:]+)(:[0-9]+)?'
|
||||
replacement: '$1'
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Node Exporter — monitoring VM itself
|
||||
# Scrapes the monitoring LXC/VM's own node_exporter.
|
||||
# Install node_exporter on the monitoring LXC the same way as on PVE nodes.
|
||||
# --------------------------------------------------------------------------
|
||||
- job_name: node-monitoring
|
||||
static_configs:
|
||||
- targets:
|
||||
- localhost:9100
|
||||
labels:
|
||||
role: monitoring
|
||||
instance: monitoring
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Optional: Grafana self-monitoring
|
||||
# Uncomment if you want to track Grafana's own health metrics.
|
||||
# --------------------------------------------------------------------------
|
||||
# - job_name: grafana
|
||||
# static_configs:
|
||||
# - targets:
|
||||
# - localhost:3000
|
||||
# labels:
|
||||
# instance: monitoring
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Optional: Proxmox Backup Server (PBS) metrics
|
||||
#
|
||||
# PBS exposes Prometheus metrics natively since PBS 2.0.
|
||||
# No exporter needed — just scrape the PBS node directly.
|
||||
# Enable in PBS: Administration > Options > Metric Server > Add > Prometheus
|
||||
# Or scrape the built-in endpoint:
|
||||
# --------------------------------------------------------------------------
|
||||
# - job_name: pbs
|
||||
# metrics_path: /metrics
|
||||
# scheme: https
|
||||
# tls_config:
|
||||
# insecure_skip_verify: true
|
||||
# basic_auth:
|
||||
# username: pbs-metrics@pbs
|
||||
# password: your-token-here
|
||||
# static_configs:
|
||||
# - targets:
|
||||
# - 192.168.1.20:8007
|
||||
# labels:
|
||||
# instance: pbs-01
|
||||
# role: backup-server
|
||||
@@ -0,0 +1,301 @@
|
||||
# Network Guide — Proxmox VE 9.1
|
||||
|
||||
This guide covers network configuration for a homelab Proxmox setup:
|
||||
bridges, VLANs, trunk ports, and an optional pfSense VM as the network gateway.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [How Proxmox Networking Works](#1-how-proxmox-networking-works)
|
||||
2. [Linux Bridges](#2-linux-bridges)
|
||||
3. [VLAN Overview](#3-vlan-overview)
|
||||
4. [VLAN-Aware Bridge Setup](#4-vlan-aware-bridge-setup)
|
||||
5. [Trunk Port to a Managed Switch](#5-trunk-port-to-a-managed-switch)
|
||||
6. [Assigning VMs to VLANs](#6-assigning-vms-to-vlans)
|
||||
7. [pfSense as a VM Gateway](#7-pfsense-as-a-vm-gateway)
|
||||
8. [Bonding and Link Aggregation](#8-bonding-and-link-aggregation)
|
||||
9. [Useful Network Commands](#9-useful-network-commands)
|
||||
|
||||
---
|
||||
|
||||
## 1. How Proxmox Networking Works
|
||||
|
||||
Proxmox uses standard Linux networking. There is no proprietary abstraction. The main file is:
|
||||
|
||||
```
|
||||
/etc/network/interfaces
|
||||
```
|
||||
|
||||
Changes made in the web UI (Node > Network) write to this file. Changes are applied with `ifreload -a` or on reboot.
|
||||
|
||||
Key concepts:
|
||||
|
||||
- **Physical NIC** (`eno1`, `enp3s0`, etc.): the real hardware interface. Set to `manual` — Proxmox bridges own the IP.
|
||||
- **Linux bridge** (`vmbr0`, `vmbr1`, etc.): virtual switch. VMs plug into bridges, not directly into NICs.
|
||||
- **VLAN interface** (`vmbr0.10`, `eno1.10`): tagged sub-interface for a specific VLAN.
|
||||
- **Bond** (`bond0`): aggregates 2+ physical NICs into one logical link.
|
||||
|
||||
---
|
||||
|
||||
## 2. Linux Bridges
|
||||
|
||||
### Default bridge (vmbr0)
|
||||
|
||||
Created during installation. Carries the Proxmox management IP.
|
||||
|
||||
```
|
||||
auto vmbr0
|
||||
iface vmbr0 inet static
|
||||
address 192.168.1.10/24
|
||||
gateway 192.168.1.1
|
||||
bridge-ports eno1
|
||||
bridge-stp off
|
||||
bridge-fd 0
|
||||
```
|
||||
|
||||
`bridge-stp off` and `bridge-fd 0` are important: they prevent the 15-second STP forwarding delay on startup.
|
||||
|
||||
### Adding a second bridge (isolated internal network)
|
||||
|
||||
```
|
||||
auto vmbr1
|
||||
iface vmbr1 inet manual
|
||||
bridge-ports none
|
||||
bridge-stp off
|
||||
bridge-fd 0
|
||||
# No physical port: internal-only traffic between VMs
|
||||
```
|
||||
|
||||
Apply:
|
||||
|
||||
```bash
|
||||
ifreload -a
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. VLAN Overview
|
||||
|
||||
VLANs let you segment traffic on a single physical link. Each VLAN has an ID (1–4094). Traffic is tagged with an 802.1Q header.
|
||||
|
||||
Common homelab VLAN layout:
|
||||
|
||||
| VLAN ID | Name | Purpose |
|
||||
|---------|--------------|--------------------------------|
|
||||
| 1 | Native/Mgmt | Proxmox management, hypervisor |
|
||||
| 10 | LAN | Trusted devices, desktops |
|
||||
| 20 | IoT | Smart home devices |
|
||||
| 30 | DMZ | Public-facing services |
|
||||
| 40 | Lab | Experimental VMs |
|
||||
|
||||
---
|
||||
|
||||
## 4. VLAN-Aware Bridge Setup
|
||||
|
||||
Proxmox supports VLAN-aware bridges natively since PVE 4.x. This is the recommended approach.
|
||||
|
||||
### Manual edit of /etc/network/interfaces
|
||||
|
||||
```
|
||||
auto vmbr0
|
||||
iface vmbr0 inet static
|
||||
address 192.168.1.10/24
|
||||
gateway 192.168.1.1
|
||||
bridge-ports eno1
|
||||
bridge-stp off
|
||||
bridge-fd 0
|
||||
bridge-vlan-aware yes
|
||||
bridge-vids 2-4094
|
||||
```
|
||||
|
||||
`bridge-vlan-aware yes` enables 802.1Q tagging on this bridge.
|
||||
`bridge-vids 2-4094` declares all VLANs as allowed. You can restrict to specific IDs if needed.
|
||||
|
||||
Apply:
|
||||
|
||||
```bash
|
||||
ifreload -a
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
bridge vlan show
|
||||
```
|
||||
|
||||
For automated setup, use the provided script:
|
||||
|
||||
```bash
|
||||
bash network/vlan-setup.sh --bridge vmbr0 --vlans 10,20,30,40 --dry-run
|
||||
```
|
||||
|
||||
Remove `--dry-run` to apply.
|
||||
|
||||
---
|
||||
|
||||
## 5. Trunk Port to a Managed Switch
|
||||
|
||||
For VLANs to reach other devices on your network, your managed switch must have the Proxmox port configured as a **trunk** (carries all tagged VLANs).
|
||||
|
||||
### Cisco-style configuration (example)
|
||||
|
||||
```
|
||||
interface GigabitEthernet0/1
|
||||
description Proxmox-PVE01
|
||||
switchport mode trunk
|
||||
switchport trunk allowed vlan 1,10,20,30,40
|
||||
spanning-tree portfast trunk
|
||||
```
|
||||
|
||||
### TP-Link / UniFi / Netgear
|
||||
|
||||
Set the port connecting to Proxmox as **Tagged** for all VLANs you want to pass through.
|
||||
|
||||
The Proxmox management VLAN (VLAN 1 by default) should be the **native** VLAN (untagged) on the trunk port so the Proxmox host itself remains accessible.
|
||||
|
||||
---
|
||||
|
||||
## 6. Assigning VMs to VLANs
|
||||
|
||||
With a VLAN-aware bridge, you tag VMs individually. No extra bridge per VLAN needed.
|
||||
|
||||
### In the web UI
|
||||
|
||||
When creating or editing a VM: **Hardware** > **Network Device** > set **VLAN Tag** to the desired VLAN ID.
|
||||
|
||||
### Via CLI (qm set)
|
||||
|
||||
```bash
|
||||
# Assign VM 201 to VLAN 10
|
||||
qm set 201 --net0 virtio,bridge=vmbr0,tag=10
|
||||
|
||||
# Assign LXC 200 to VLAN 20
|
||||
pct set 200 --net0 name=eth0,bridge=vmbr0,tag=20,ip=dhcp
|
||||
```
|
||||
|
||||
The VM sees an untagged interface (the hypervisor handles the tagging). No changes needed inside the VM.
|
||||
|
||||
---
|
||||
|
||||
## 7. pfSense as a VM Gateway
|
||||
|
||||
pfSense or OPNsense running as a VM provides routing and firewall between VLANs.
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
Internet (WAN)
|
||||
|
|
||||
[Physical NIC eno2] -- vmbr-wan (bridge, no IP)
|
||||
|
|
||||
[pfSense VM]
|
||||
|
|
||||
[vmbr0, VLAN-aware] -- internal VLANs (LAN, IoT, DMZ, Lab)
|
||||
```
|
||||
|
||||
pfSense gets:
|
||||
- A WAN interface: plugged into `vmbr-wan` (connected to your router or modem)
|
||||
- A LAN interface: plugged into `vmbr0` with a VLAN trunk
|
||||
|
||||
pfSense then handles inter-VLAN routing and firewall rules.
|
||||
|
||||
### Create the WAN bridge
|
||||
|
||||
```
|
||||
auto vmbr-wan
|
||||
iface vmbr-wan inet manual
|
||||
bridge-ports eno2
|
||||
bridge-stp off
|
||||
bridge-fd 0
|
||||
```
|
||||
|
||||
### pfSense VM configuration
|
||||
|
||||
```bash
|
||||
qm create 300 \
|
||||
--name pfsense \
|
||||
--memory 2048 \
|
||||
--cores 2 \
|
||||
--net0 virtio,bridge=vmbr-wan \
|
||||
--net1 virtio,bridge=vmbr0 \
|
||||
--ostype other \
|
||||
--scsihw virtio-scsi-pci \
|
||||
--scsi0 local-lvm:16
|
||||
```
|
||||
|
||||
Inside pfSense, configure net1 as a VLAN trunk and create VLAN interfaces for each VLAN ID.
|
||||
|
||||
---
|
||||
|
||||
## 8. Bonding and Link Aggregation
|
||||
|
||||
If your server has 2+ NICs, you can bond them for redundancy or increased throughput.
|
||||
|
||||
### Active-backup bond (redundancy)
|
||||
|
||||
```
|
||||
auto bond0
|
||||
iface bond0 inet manual
|
||||
bond-slaves eno1 eno2
|
||||
bond-miimon 100
|
||||
bond-mode active-backup
|
||||
bond-primary eno1
|
||||
|
||||
auto vmbr0
|
||||
iface vmbr0 inet static
|
||||
address 192.168.1.10/24
|
||||
gateway 192.168.1.1
|
||||
bridge-ports bond0
|
||||
bridge-stp off
|
||||
bridge-fd 0
|
||||
bridge-vlan-aware yes
|
||||
bridge-vids 2-4094
|
||||
```
|
||||
|
||||
`bond-miimon 100` checks link state every 100ms. If `eno1` fails, traffic switches to `eno2` automatically.
|
||||
|
||||
### LACP / 802.3ad (requires managed switch support)
|
||||
|
||||
```
|
||||
auto bond0
|
||||
iface bond0 inet manual
|
||||
bond-slaves eno1 eno2
|
||||
bond-miimon 100
|
||||
bond-mode 802.3ad
|
||||
bond-lacp-rate fast
|
||||
bond-xmit-hash-policy layer2+3
|
||||
```
|
||||
|
||||
Configure the corresponding port-channel on your switch.
|
||||
|
||||
---
|
||||
|
||||
## 9. Useful Network Commands
|
||||
|
||||
```bash
|
||||
# Show all interfaces and their state
|
||||
ip link show
|
||||
|
||||
# Show IP addresses
|
||||
ip addr show
|
||||
|
||||
# Show bridge details (ports, VLANs)
|
||||
bridge link show
|
||||
bridge vlan show
|
||||
|
||||
# Reload network config without full reboot
|
||||
ifreload -a
|
||||
|
||||
# Show the current routing table
|
||||
ip route show
|
||||
|
||||
# Test connectivity between VMs on different VLANs
|
||||
ping -c 3 -I vmbr0.10 192.168.10.1
|
||||
|
||||
# Show bond status (if bonding is configured)
|
||||
cat /proc/net/bonding/bond0
|
||||
|
||||
# Capture traffic on a bridge port (tcpdump)
|
||||
tcpdump -i vmbr0 -n -v 'vlan 10'
|
||||
```
|
||||
@@ -0,0 +1,323 @@
|
||||
#!/usr/bin/env bash
|
||||
# vlan-setup.sh — Configure VLAN-aware bridge on Proxmox VE 9.1
|
||||
#
|
||||
# Modifies /etc/network/interfaces to:
|
||||
# 1. Enable bridge-vlan-aware on the target bridge
|
||||
# 2. Set the allowed VLAN range (bridge-vids)
|
||||
# 3. Create VLAN sub-interfaces on the bridge for host-level VLAN access
|
||||
#
|
||||
# Usage:
|
||||
# bash vlan-setup.sh [OPTIONS]
|
||||
#
|
||||
# Options:
|
||||
# --bridge <name> Bridge to configure (default: vmbr0)
|
||||
# --vlans <list> Comma-separated VLAN IDs to configure (e.g. 10,20,30)
|
||||
# --iface-file <path> Path to interfaces file (default: /etc/network/interfaces)
|
||||
# --dry-run Print what would be done, do not modify any file
|
||||
# --help Show this help message
|
||||
#
|
||||
# Examples:
|
||||
# bash vlan-setup.sh --bridge vmbr0 --vlans 10,20,30 --dry-run
|
||||
# bash vlan-setup.sh --bridge vmbr0 --vlans 10,20,30,40
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Defaults
|
||||
# ---------------------------------------------------------------------------
|
||||
BRIDGE="vmbr0"
|
||||
VLANS=""
|
||||
IFACE_FILE="/etc/network/interfaces"
|
||||
DRY_RUN=false
|
||||
BACKUP_SUFFIX=".bak.$(date +%Y%m%d_%H%M%S)"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Color output (only when stdout is a terminal)
|
||||
# ---------------------------------------------------------------------------
|
||||
if [ -t 1 ]; then
|
||||
RED='\033[0;31m'
|
||||
YELLOW='\033[1;33m'
|
||||
GREEN='\033[0;32m'
|
||||
CYAN='\033[0;36m'
|
||||
RESET='\033[0m'
|
||||
else
|
||||
RED='' YELLOW='' GREEN='' CYAN='' RESET=''
|
||||
fi
|
||||
|
||||
log_info() { echo -e "${CYAN}[INFO]${RESET} $*"; }
|
||||
log_ok() { echo -e "${GREEN}[OK]${RESET} $*"; }
|
||||
log_warn() { echo -e "${YELLOW}[WARN]${RESET} $*"; }
|
||||
log_error() { echo -e "${RED}[ERROR]${RESET} $*" >&2; }
|
||||
log_dryrun() { echo -e "${YELLOW}[DRY-RUN]${RESET} $*"; }
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Usage
|
||||
# ---------------------------------------------------------------------------
|
||||
usage() {
|
||||
sed -n '/^# Usage:/,/^$/p' "$0" | head -n 30 | sed 's/^# //'
|
||||
exit 0
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Argument parsing
|
||||
# ---------------------------------------------------------------------------
|
||||
parse_args() {
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--bridge)
|
||||
BRIDGE="${2:?--bridge requires a value}"
|
||||
shift 2
|
||||
;;
|
||||
--vlans)
|
||||
VLANS="${2:?--vlans requires a value}"
|
||||
shift 2
|
||||
;;
|
||||
--iface-file)
|
||||
IFACE_FILE="${2:?--iface-file requires a value}"
|
||||
shift 2
|
||||
;;
|
||||
--dry-run)
|
||||
DRY_RUN=true
|
||||
shift
|
||||
;;
|
||||
--help|-h)
|
||||
usage
|
||||
;;
|
||||
*)
|
||||
log_error "Unknown option: $1"
|
||||
usage
|
||||
;;
|
||||
esac
|
||||
done
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Validation
|
||||
# ---------------------------------------------------------------------------
|
||||
validate_args() {
|
||||
if [[ -z "$VLANS" ]]; then
|
||||
log_error "--vlans is required. Example: --vlans 10,20,30"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ ! -f "$IFACE_FILE" ]]; then
|
||||
log_error "Interfaces file not found: $IFACE_FILE"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Validate VLAN IDs
|
||||
IFS=',' read -ra VLAN_ARRAY <<< "$VLANS"
|
||||
for vlan in "${VLAN_ARRAY[@]}"; do
|
||||
if ! [[ "$vlan" =~ ^[0-9]+$ ]] || (( vlan < 1 || vlan > 4094 )); then
|
||||
log_error "Invalid VLAN ID: $vlan (must be 1–4094)"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# Check we are running on Linux (Proxmox is Linux only)
|
||||
if [[ "$(uname -s)" != "Linux" ]]; then
|
||||
log_error "This script is designed for Linux / Proxmox VE only."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Warn if not root (changes to /etc/network/interfaces require root)
|
||||
if [[ $EUID -ne 0 ]] && [[ "$DRY_RUN" == false ]]; then
|
||||
log_warn "Not running as root. Changes to $IFACE_FILE will likely fail."
|
||||
log_warn "Run with sudo or as root, or use --dry-run to preview changes."
|
||||
fi
|
||||
|
||||
log_ok "Arguments validated."
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Check if the bridge exists in the interfaces file
|
||||
# ---------------------------------------------------------------------------
|
||||
bridge_exists_in_file() {
|
||||
grep -q "^auto ${BRIDGE}$" "$IFACE_FILE" || grep -q "^iface ${BRIDGE} " "$IFACE_FILE"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Check if a specific VLAN sub-interface stanza already exists
|
||||
# ---------------------------------------------------------------------------
|
||||
vlan_iface_exists() {
|
||||
local vlan_id="$1"
|
||||
grep -q "^auto ${BRIDGE}.${vlan_id}$" "$IFACE_FILE"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Build the patch for bridge-vlan-aware and bridge-vids
|
||||
# This function prints the modified block for the target bridge.
|
||||
# We use awk to insert the options after the last existing bridge-* line
|
||||
# inside the bridge stanza.
|
||||
# ---------------------------------------------------------------------------
|
||||
patch_bridge_stanza() {
|
||||
local tmpfile
|
||||
tmpfile=$(mktemp)
|
||||
|
||||
awk -v bridge="$BRIDGE" '
|
||||
BEGIN {
|
||||
in_bridge = 0
|
||||
vlan_aware_added = 0
|
||||
bridge_vids_added = 0
|
||||
}
|
||||
|
||||
/^(auto|iface|allow-hotplug)/ {
|
||||
# Entering a new stanza: flush pending additions for previous bridge
|
||||
if (in_bridge && !vlan_aware_added) {
|
||||
print " bridge-vlan-aware yes"
|
||||
print " bridge-vids 2-4094"
|
||||
vlan_aware_added = 1
|
||||
bridge_vids_added = 1
|
||||
}
|
||||
in_bridge = 0
|
||||
vlan_aware_added = 0
|
||||
bridge_vids_added = 0
|
||||
}
|
||||
|
||||
/^iface[[:space:]]/ && $2 == bridge {
|
||||
in_bridge = 1
|
||||
}
|
||||
|
||||
in_bridge && /bridge-vlan-aware/ {
|
||||
# Already present: normalise the value
|
||||
sub(/bridge-vlan-aware[[:space:]]+.*/, "bridge-vlan-aware yes")
|
||||
vlan_aware_added = 1
|
||||
print
|
||||
next
|
||||
}
|
||||
|
||||
in_bridge && /bridge-vids/ {
|
||||
# Already present: normalise the value
|
||||
sub(/bridge-vids[[:space:]]+.*/, "bridge-vids 2-4094")
|
||||
bridge_vids_added = 1
|
||||
print
|
||||
next
|
||||
}
|
||||
|
||||
in_bridge && /bridge-fd/ && !vlan_aware_added {
|
||||
# Insert after bridge-fd
|
||||
print
|
||||
print " bridge-vlan-aware yes"
|
||||
print " bridge-vids 2-4094"
|
||||
vlan_aware_added = 1
|
||||
bridge_vids_added = 1
|
||||
next
|
||||
}
|
||||
|
||||
{ print }
|
||||
|
||||
END {
|
||||
if (in_bridge && !vlan_aware_added) {
|
||||
print " bridge-vlan-aware yes"
|
||||
print " bridge-vids 2-4094"
|
||||
}
|
||||
}
|
||||
' "$IFACE_FILE" > "$tmpfile"
|
||||
|
||||
echo "$tmpfile"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Build stanzas for VLAN sub-interfaces (host-level VLAN access)
|
||||
# These are manual interfaces — no IP assigned here.
|
||||
# Assign IPs at the VM/LXC level or add them manually afterward.
|
||||
# ---------------------------------------------------------------------------
|
||||
build_vlan_stanzas() {
|
||||
local output=""
|
||||
IFS=',' read -ra VLAN_ARRAY <<< "$VLANS"
|
||||
for vlan in "${VLAN_ARRAY[@]}"; do
|
||||
if ! vlan_iface_exists "$vlan"; then
|
||||
output+="
|
||||
auto ${BRIDGE}.${vlan}
|
||||
iface ${BRIDGE}.${vlan} inet manual
|
||||
# VLAN ${vlan} sub-interface on ${BRIDGE}
|
||||
# Assign an IP here if the Proxmox host needs to communicate on VLAN ${vlan}
|
||||
# Example: address 192.168.${vlan}.1/24
|
||||
"
|
||||
else
|
||||
log_warn "VLAN sub-interface ${BRIDGE}.${vlan} already exists in $IFACE_FILE — skipped."
|
||||
fi
|
||||
done
|
||||
echo "$output"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Main
|
||||
# ---------------------------------------------------------------------------
|
||||
main() {
|
||||
parse_args "$@"
|
||||
|
||||
log_info "Proxmox VE VLAN setup script"
|
||||
log_info "Bridge : $BRIDGE"
|
||||
log_info "VLANs : $VLANS"
|
||||
log_info "File : $IFACE_FILE"
|
||||
log_info "Dry-run : $DRY_RUN"
|
||||
echo
|
||||
|
||||
validate_args
|
||||
|
||||
# Check bridge exists in file
|
||||
if ! bridge_exists_in_file; then
|
||||
log_error "Bridge '$BRIDGE' not found in $IFACE_FILE."
|
||||
log_error "Create the bridge first via the Proxmox web UI or manually."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
log_info "Bridge '$BRIDGE' found in $IFACE_FILE."
|
||||
|
||||
# Build patched bridge stanza
|
||||
local patched_file
|
||||
patched_file=$(patch_bridge_stanza)
|
||||
|
||||
# Build VLAN stanzas
|
||||
local vlan_stanzas
|
||||
vlan_stanzas=$(build_vlan_stanzas)
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
echo
|
||||
log_dryrun "--- Patched $IFACE_FILE (bridge section) ---"
|
||||
cat "$patched_file"
|
||||
echo
|
||||
if [[ -n "$vlan_stanzas" ]]; then
|
||||
log_dryrun "--- VLAN sub-interfaces to append ---"
|
||||
echo "$vlan_stanzas"
|
||||
fi
|
||||
log_dryrun "No changes written (--dry-run mode)."
|
||||
rm -f "$patched_file"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Backup original file
|
||||
cp "$IFACE_FILE" "${IFACE_FILE}${BACKUP_SUFFIX}"
|
||||
log_ok "Backup created: ${IFACE_FILE}${BACKUP_SUFFIX}"
|
||||
|
||||
# Write patched bridge stanza
|
||||
cp "$patched_file" "$IFACE_FILE"
|
||||
rm -f "$patched_file"
|
||||
log_ok "Bridge stanza patched (bridge-vlan-aware yes, bridge-vids 2-4094)."
|
||||
|
||||
# Append VLAN stanzas
|
||||
if [[ -n "$vlan_stanzas" ]]; then
|
||||
echo "$vlan_stanzas" >> "$IFACE_FILE"
|
||||
log_ok "VLAN sub-interfaces appended."
|
||||
fi
|
||||
|
||||
# Apply changes
|
||||
log_info "Applying network configuration with ifreload..."
|
||||
if command -v ifreload &>/dev/null; then
|
||||
ifreload -a && log_ok "Network configuration applied."
|
||||
else
|
||||
# WARNING: never run this on vmbr0 (management bridge) — it will cut your SSH session
|
||||
log_warn "ifreload not found. Reboot to apply changes, or run: ifdown $BRIDGE && ifup $BRIDGE"
|
||||
fi
|
||||
|
||||
echo
|
||||
log_ok "Done. Verify with: bridge vlan show"
|
||||
echo
|
||||
log_info "To assign a VM to a VLAN, run:"
|
||||
log_info " qm set <vmid> --net0 virtio,bridge=${BRIDGE},tag=<vlan_id>"
|
||||
log_info "To assign an LXC to a VLAN, run:"
|
||||
log_info " pct set <ctid> --net0 name=eth0,bridge=${BRIDGE},tag=<vlan_id>,ip=dhcp"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
@@ -0,0 +1,403 @@
|
||||
# Security Hardening Guide — Proxmox VE 9.1
|
||||
|
||||
This guide covers baseline hardening for a Proxmox homelab.
|
||||
Not an enterprise checklist — a practical starting point that covers the most impactful actions.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Web UI and API Hardening](#1-web-ui-and-api-hardening)
|
||||
2. [SSH Hardening](#2-ssh-hardening)
|
||||
3. [Two-Factor Authentication (2FA)](#3-two-factor-authentication-2fa)
|
||||
4. [User and Permission Management](#4-user-and-permission-management)
|
||||
5. [Proxmox Firewall](#5-proxmox-firewall)
|
||||
6. [Remove the Subscription Nag](#6-remove-the-subscription-nag)
|
||||
7. [Kernel and System Hardening](#7-kernel-and-system-hardening)
|
||||
8. [Audit and Logging](#8-audit-and-logging)
|
||||
9. [VM Isolation](#9-vm-isolation)
|
||||
10. [Checklist Summary](#10-checklist-summary)
|
||||
|
||||
---
|
||||
|
||||
## 1. Web UI and API Hardening
|
||||
|
||||
### Change the default port (optional, security by obscurity)
|
||||
|
||||
The web UI runs on port 8006 by default. Changing it does not add real security but reduces automated scanning noise.
|
||||
|
||||
```bash
|
||||
# Edit the pveproxy config
|
||||
cat >> /etc/default/pveproxy << 'EOF'
|
||||
LISTEN_PORT=8443
|
||||
EOF
|
||||
|
||||
systemctl restart pveproxy
|
||||
```
|
||||
|
||||
If you change the port, update your firewall rules accordingly.
|
||||
|
||||
### Restrict web UI access to a specific subnet
|
||||
|
||||
The best approach is to allow the Proxmox web UI port only from your management machine or VLAN, using the Proxmox firewall (see section 5).
|
||||
|
||||
### Use a valid TLS certificate
|
||||
|
||||
The default certificate is self-signed. For a homelab, you can use Let's Encrypt with a DNS challenge if your domain is publicly resolvable — even for internal use.
|
||||
|
||||
```bash
|
||||
# Install acme.sh or use the built-in ACME client in Proxmox
|
||||
# Web UI: Datacenter > ACME > Add Account, then Node > Certificates > Add ACME Domain
|
||||
```
|
||||
|
||||
Alternatively, generate a certificate from your own internal CA (e.g. step-ca, Easy-RSA).
|
||||
|
||||
---
|
||||
|
||||
## 2. SSH Hardening
|
||||
|
||||
### Disable root SSH login
|
||||
|
||||
Root login over SSH should be disabled. Use a non-root user or key-only root if strictly necessary.
|
||||
|
||||
```bash
|
||||
sed -i 's/^#*PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
|
||||
```
|
||||
|
||||
If you need root access via SSH temporarily, prefer: `ssh thomas@pve01 sudo -i`
|
||||
|
||||
### Disable password authentication (keys only)
|
||||
|
||||
```bash
|
||||
sed -i 's/^#*PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
|
||||
sed -i 's/^#*KbdInteractiveAuthentication.*/KbdInteractiveAuthentication no/' /etc/ssh/sshd_config
|
||||
```
|
||||
|
||||
Make sure your SSH public key is in `/root/.ssh/authorized_keys` before applying this, or you will lock yourself out.
|
||||
|
||||
### Additional SSH hardening options
|
||||
|
||||
Create `/etc/ssh/sshd_config.d/99-proxmox-hardening.conf`:
|
||||
|
||||
```
|
||||
MaxAuthTries 3
|
||||
LoginGraceTime 30
|
||||
ClientAliveInterval 300
|
||||
ClientAliveCountMax 2
|
||||
AllowUsers thomas root
|
||||
X11Forwarding no
|
||||
AllowTcpForwarding no
|
||||
```
|
||||
|
||||
Apply:
|
||||
|
||||
```bash
|
||||
sshd -t # test config before reloading
|
||||
systemctl restart sshd
|
||||
```
|
||||
|
||||
### Change SSH port (optional)
|
||||
|
||||
```bash
|
||||
# Uncomment and set in /etc/ssh/sshd_config
|
||||
Port 2222
|
||||
```
|
||||
|
||||
Update firewall rules if you change the port.
|
||||
|
||||
---
|
||||
|
||||
## 3. Two-Factor Authentication (2FA)
|
||||
|
||||
2FA prevents account compromise if credentials are stolen. Proxmox supports TOTP natively.
|
||||
|
||||
### Enable 2FA for a user
|
||||
|
||||
1. In the web UI: **Datacenter** > **Permissions** > **Two Factor**
|
||||
2. Click **Add** > **TOTP**
|
||||
3. Enter the user (e.g. `thomas@pve`)
|
||||
4. Scan the QR code with an authenticator app: Aegis (Android), Bitwarden Authenticator, or FreeOTP
|
||||
5. Enter the current OTP to confirm
|
||||
6. Click **Add**
|
||||
|
||||
### Enforce 2FA at the realm level
|
||||
|
||||
This requires all users in the realm to have 2FA configured before they can log in.
|
||||
|
||||
**Datacenter** > **Permissions** > **Realms** > click **pve** > **Edit** > enable **Two Factor**.
|
||||
|
||||
Note: set up 2FA for all users before enforcing, or you will lock them out.
|
||||
|
||||
### WebAuthn / hardware keys (alternative)
|
||||
|
||||
Proxmox 7.2+ supports WebAuthn (FIDO2/U2F). You can use a YubiKey or another hardware token.
|
||||
|
||||
**Datacenter** > **Permissions** > **Two Factor** > **WebAuthn Settings**
|
||||
|
||||
Configure the Relying Party ID (your Proxmox FQDN) and origin URL, then register your key.
|
||||
|
||||
---
|
||||
|
||||
## 4. User and Permission Management
|
||||
|
||||
### Never use root for daily operations
|
||||
|
||||
Create a dedicated admin user with only the privileges needed:
|
||||
|
||||
```bash
|
||||
# Create a local PVE user
|
||||
pveum user add thomas@pve --password 'ChangeThisPassword!'
|
||||
|
||||
# Create a custom role for homelab administration
|
||||
pveum role add HomeLabAdmin --privs \
|
||||
"VM.Allocate VM.Clone VM.Config.CDROM VM.Config.CPU VM.Config.Disk \
|
||||
VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options \
|
||||
VM.Console VM.Monitor VM.PowerMgmt VM.Snapshot VM.Backup \
|
||||
Datastore.AllocateSpace Datastore.AllocateTemplate Datastore.Audit \
|
||||
Pool.Allocate Sys.Audit Sys.Console Sys.Modify"
|
||||
|
||||
# Grant the role to the user at the root path
|
||||
pveum aclmod / -user thomas@pve -role HomeLabAdmin
|
||||
```
|
||||
|
||||
### Create read-only users for monitoring
|
||||
|
||||
```bash
|
||||
pveum user add monitor@pve --password 'ReadOnlyPassword!'
|
||||
pveum aclmod / -user monitor@pve -role PVEAuditor
|
||||
```
|
||||
|
||||
### List current users and ACLs
|
||||
|
||||
```bash
|
||||
pveum user list
|
||||
pveum acl list
|
||||
pveum role list
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Proxmox Firewall
|
||||
|
||||
Proxmox has a built-in firewall at three levels: datacenter, node, and VM/LXC.
|
||||
|
||||
### Enable the datacenter firewall
|
||||
|
||||
**Web UI**: **Datacenter** > **Firewall** > **Options** > set **Firewall** to enabled.
|
||||
|
||||
Set the default policy:
|
||||
|
||||
| Direction | Policy |
|
||||
|-----------|--------|
|
||||
| Input | DROP |
|
||||
| Output | ACCEPT |
|
||||
| Forward | DROP |
|
||||
|
||||
### Add explicit rules
|
||||
|
||||
Allow only necessary traffic to the node:
|
||||
|
||||
| Type | Action | Source | Dest port | Protocol | Comment |
|
||||
|-------|--------|----------------|-----------|----------|--------------------------|
|
||||
| in | ACCEPT | 192.168.1.0/24 | 8006 | TCP | Proxmox web UI |
|
||||
| in | ACCEPT | 192.168.1.0/24 | 22 | TCP | SSH |
|
||||
| in | ACCEPT | 192.168.1.0/24 | 3128 | TCP | SPICE proxy (optional) |
|
||||
| in | ACCEPT | any | icmp | ICMP | Ping (optional) |
|
||||
|
||||
Add rules in: **Datacenter** > **Firewall** > **Rules** > **Add**
|
||||
|
||||
Or via CLI:
|
||||
|
||||
```bash
|
||||
# Allow SSH from management subnet
|
||||
pvesh create /nodes/pve01/firewall/rules \
|
||||
--type in \
|
||||
--action ACCEPT \
|
||||
--source 192.168.1.0/24 \
|
||||
--dport 22 \
|
||||
--proto tcp \
|
||||
--comment "SSH from LAN"
|
||||
|
||||
# Allow web UI from management subnet
|
||||
pvesh create /nodes/pve01/firewall/rules \
|
||||
--type in \
|
||||
--action ACCEPT \
|
||||
--source 192.168.1.0/24 \
|
||||
--dport 8006 \
|
||||
--proto tcp \
|
||||
--comment "Proxmox UI from LAN"
|
||||
```
|
||||
|
||||
### Enable the node firewall
|
||||
|
||||
After adding rules: **Node** > **Firewall** > **Options** > set **Firewall** to enabled.
|
||||
|
||||
Do this AFTER adding your rules, or you will lock yourself out.
|
||||
|
||||
---
|
||||
|
||||
## 6. Remove the Subscription Nag
|
||||
|
||||
The subscription nag dialog appears on every login without a paid subscription. It is cosmetic only.
|
||||
|
||||
```bash
|
||||
# Patch the proxmox widget toolkit JS
|
||||
sed -Ezi.bak \
|
||||
"s/(Ext.Msg.show\(\{\s+title: gettext\('No valid sub)/void\(\{ \/\/\1/g" \
|
||||
/usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
|
||||
|
||||
systemctl restart pveproxy
|
||||
```
|
||||
|
||||
This patch is reset on each Proxmox update. Reapply after `apt dist-upgrade`.
|
||||
|
||||
---
|
||||
|
||||
## 7. Kernel and System Hardening
|
||||
|
||||
### Apply security-relevant sysctl settings
|
||||
|
||||
```bash
|
||||
cat > /etc/sysctl.d/99-proxmox-hardening.conf << 'EOF'
|
||||
# Prevent IP spoofing
|
||||
net.ipv4.conf.all.rp_filter = 1
|
||||
net.ipv4.conf.default.rp_filter = 1
|
||||
|
||||
# Disable IP source routing
|
||||
net.ipv4.conf.all.accept_source_route = 0
|
||||
net.ipv4.conf.default.accept_source_route = 0
|
||||
|
||||
# Disable ICMP redirects
|
||||
net.ipv4.conf.all.accept_redirects = 0
|
||||
net.ipv6.conf.all.accept_redirects = 0
|
||||
net.ipv4.conf.all.send_redirects = 0
|
||||
|
||||
# Enable TCP SYN cookies (protection against SYN flood)
|
||||
net.ipv4.tcp_syncookies = 1
|
||||
|
||||
# Log martian packets (packets with impossible source addresses)
|
||||
net.ipv4.conf.all.log_martians = 1
|
||||
EOF
|
||||
|
||||
sysctl -p /etc/sysctl.d/99-proxmox-hardening.conf
|
||||
```
|
||||
|
||||
### Keep the system updated
|
||||
|
||||
```bash
|
||||
# Unattended security updates
|
||||
apt install -y unattended-upgrades
|
||||
|
||||
dpkg-reconfigure -f noninteractive unattended-upgrades
|
||||
```
|
||||
|
||||
### Disable unnecessary services
|
||||
|
||||
Check which services are running:
|
||||
|
||||
```bash
|
||||
systemctl list-units --type=service --state=running
|
||||
```
|
||||
|
||||
Disable services you do not use:
|
||||
|
||||
```bash
|
||||
# Example: disable rpcbind if no NFS is used
|
||||
systemctl disable --now rpcbind rpcbind.socket 2>/dev/null || true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Audit and Logging
|
||||
|
||||
### Proxmox task log
|
||||
|
||||
Every action in the web UI is logged to the task log. Access via:
|
||||
- **Datacenter** > **Tasks**
|
||||
- CLI: `pvesh get /nodes/$(hostname)/tasks --limit 50`
|
||||
|
||||
### System log
|
||||
|
||||
```bash
|
||||
# SSH logins
|
||||
journalctl -u sshd --since "24h ago"
|
||||
|
||||
# Authentication log
|
||||
grep -E 'sshd|sudo|pam' /var/log/auth.log | tail -50
|
||||
|
||||
# Proxmox cluster events
|
||||
journalctl -u pve-cluster -u pvedaemon -u pveproxy --since "24h ago"
|
||||
```
|
||||
|
||||
### Install auditd (optional, for compliance)
|
||||
|
||||
```bash
|
||||
apt install -y auditd
|
||||
|
||||
# Basic rules: monitor sensitive files
|
||||
cat > /etc/audit/rules.d/proxmox.rules << 'EOF'
|
||||
# Monitor changes to network interfaces
|
||||
-w /etc/network/interfaces -p wa -k network-config
|
||||
|
||||
# Monitor changes to Proxmox config files
|
||||
-w /etc/pve/ -p wa -k pve-config
|
||||
|
||||
# Monitor SSH config changes
|
||||
-w /etc/ssh/sshd_config -p wa -k sshd-config
|
||||
|
||||
# Monitor authentication events
|
||||
-w /var/log/auth.log -p wa -k auth-log
|
||||
EOF
|
||||
|
||||
systemctl enable --now auditd
|
||||
augenrules --load
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. VM Isolation
|
||||
|
||||
### Use separate VLANs per trust level
|
||||
|
||||
Segment your VMs so that a compromised VM cannot reach the hypervisor management interface.
|
||||
|
||||
- Proxmox management: VLAN 1 or a dedicated management VLAN — no VM traffic here
|
||||
- Production VMs: VLAN 10
|
||||
- Lab/test VMs: VLAN 40
|
||||
- IoT: VLAN 20
|
||||
|
||||
See [../network/README.md](../network/README.md) for VLAN setup.
|
||||
|
||||
### Do not use privileged LXC containers unless required
|
||||
|
||||
Privileged LXC containers can theoretically escape to the host if a kernel vulnerability is exploited. Use unprivileged containers:
|
||||
|
||||
```bash
|
||||
pct create ... --unprivileged 1
|
||||
```
|
||||
|
||||
### Enable SecureBoot for VMs handling sensitive data
|
||||
|
||||
When creating a VM: **System** > **BIOS** > select **OVMF (UEFI)** instead of SeaBIOS. Enable SecureBoot.
|
||||
|
||||
---
|
||||
|
||||
## 10. Checklist Summary
|
||||
|
||||
| Action | Priority | Done |
|
||||
|----------------------------------------------------------|----------|------|
|
||||
| Disable subscription nag | Low | |
|
||||
| Disable enterprise apt repo, enable no-subscription | High | |
|
||||
| Run `apt dist-upgrade` after install | High | |
|
||||
| Disable root SSH login | High | |
|
||||
| Disable SSH password authentication | High | |
|
||||
| Enable 2FA for all admin accounts | High | |
|
||||
| Create a non-root user for daily use | High | |
|
||||
| Enable Proxmox firewall (datacenter + node level) | High | |
|
||||
| Restrict web UI access to management subnet only | High | |
|
||||
| Apply sysctl hardening | Medium | |
|
||||
| Set up VLAN segmentation (management vs VM traffic) | Medium | |
|
||||
| Use unprivileged LXC containers | Medium | |
|
||||
| Enable unattended security updates | Medium | |
|
||||
| Set up auditd | Low | |
|
||||
| Use a valid TLS certificate (internal CA or Let's Encrypt)| Low | |
|
||||
+327
@@ -0,0 +1,327 @@
|
||||
# VMs and Templates Guide — Proxmox VE 9.1
|
||||
|
||||
This guide covers VM creation, cloud-init configuration, template creation, and fast cloning.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [VM Storage Options](#1-vm-storage-options)
|
||||
2. [Create a VM via the Web UI](#2-create-a-vm-via-the-web-ui)
|
||||
3. [Create a VM via CLI (qm)](#3-create-a-vm-via-cli-qm)
|
||||
4. [Cloud-Init Overview](#4-cloud-init-overview)
|
||||
5. [Prepare a Cloud-Init Ready VM](#5-prepare-a-cloud-init-ready-vm)
|
||||
6. [Convert to Template](#6-convert-to-template)
|
||||
7. [Clone a Template](#7-clone-a-template)
|
||||
8. [Import a Cloud Image (Faster Method)](#8-import-a-cloud-image-faster-method)
|
||||
9. [VM Configuration Reference](#9-vm-configuration-reference)
|
||||
10. [Snapshots](#10-snapshots)
|
||||
|
||||
---
|
||||
|
||||
## 1. VM Storage Options
|
||||
|
||||
Proxmox supports multiple storage backends. The most common:
|
||||
|
||||
| Backend | Format | Snapshots | Notes |
|
||||
|---------------|--------------|-----------|----------------------------------|
|
||||
| LVM-Thin | raw | Yes | Default after installation |
|
||||
| ZFS | zvol / qcow2 | Yes | Best option if you have NVMe |
|
||||
| Directory | qcow2, raw | Yes (qcow2)| Simple, works on any filesystem |
|
||||
| NFS/CIFS | qcow2, raw | Yes (qcow2)| For shared storage |
|
||||
|
||||
Use LVM-Thin or ZFS for the best snapshot performance.
|
||||
|
||||
---
|
||||
|
||||
## 2. Create a VM via the Web UI
|
||||
|
||||
1. Click **Create VM** in the top-right of the web UI.
|
||||
2. **General**: set Node, VM ID (e.g. 100), Name.
|
||||
3. **OS**: select ISO image from local storage. Set OS type to Linux, version 6.x kernel.
|
||||
4. **System**: leave defaults (BIOS: SeaBIOS or OVMF for UEFI). Enable QEMU Agent.
|
||||
5. **Disks**: select storage (local-lvm or ZFS pool). Set size. Disk bus: SCSI, controller: VirtIO SCSI Single.
|
||||
6. **CPU**: set cores. Type: `host` for best performance.
|
||||
7. **Memory**: set RAM. Enable ballooning for dynamic memory.
|
||||
8. **Network**: model `VirtIO (paravirt)`, bridge `vmbr0`. Set VLAN tag if needed.
|
||||
9. **Confirm**: review and click **Finish**.
|
||||
|
||||
---
|
||||
|
||||
## 3. Create a VM via CLI (qm)
|
||||
|
||||
The `qm` command manages virtual machines from the shell.
|
||||
|
||||
### Minimal Ubuntu 24.04 VM
|
||||
|
||||
```bash
|
||||
qm create 101 \
|
||||
--name ubuntu-2404-base \
|
||||
--memory 2048 \
|
||||
--balloon 1024 \
|
||||
--cores 2 \
|
||||
--cpu host \
|
||||
--net0 virtio,bridge=vmbr0 \
|
||||
--ostype l26 \
|
||||
--scsihw virtio-scsi-single \
|
||||
--scsi0 local-lvm:32,discard=on,ssd=1 \
|
||||
--ide2 local:iso/ubuntu-24.04.4-live-server-amd64.iso,media=cdrom \
|
||||
--boot order=ide2 \
|
||||
--agent enabled=1 \
|
||||
--onboot 0
|
||||
```
|
||||
|
||||
Key flags:
|
||||
|
||||
- `--cpu host`: exposes full CPU feature set to the VM.
|
||||
- `--balloon 1024`: minimum memory for ballooning (requires balloon driver inside VM).
|
||||
- `--scsihw virtio-scsi-single`: modern, high-performance SCSI controller.
|
||||
- `--discard=on,ssd=1`: enables TRIM/discard passthrough (important for SSDs and LVM-Thin).
|
||||
- `--agent enabled=1`: enables QEMU guest agent socket.
|
||||
- `--onboot 0`: do not start automatically (set to 1 for production VMs).
|
||||
|
||||
### Start the VM and open the console
|
||||
|
||||
```bash
|
||||
qm start 101
|
||||
# Then open the web console or use:
|
||||
pvesh get /nodes/$(hostname)/vms/101/termproxy --shell 1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Cloud-Init Overview
|
||||
|
||||
Cloud-init is the standard for initialising cloud instances. On first boot it:
|
||||
|
||||
- Sets the hostname
|
||||
- Creates users and injects SSH keys
|
||||
- Configures networking (DHCP or static IP)
|
||||
- Runs custom scripts
|
||||
|
||||
Proxmox supports cloud-init natively via a special drive (virtio, IDE, or SCSI). The VM reads this drive on boot like a data CD.
|
||||
|
||||
Supported cloud-init parameters in Proxmox:
|
||||
|
||||
| `qm set` flag | Effect |
|
||||
|---------------------|---------------------------------------------|
|
||||
| `--ciuser` | Default user name |
|
||||
| `--cipassword` | Password for the default user |
|
||||
| `--sshkeys` | Authorised SSH public keys |
|
||||
| `--ipconfig0` | Network config for net0 (ip=dhcp or static) |
|
||||
| `--nameserver` | DNS server |
|
||||
| `--searchdomain` | DNS search domain |
|
||||
| `--cicustom` | Path to a custom cloud-init YAML file |
|
||||
|
||||
---
|
||||
|
||||
## 5. Prepare a Cloud-Init Ready VM
|
||||
|
||||
### Download the Ubuntu 24.04 cloud image
|
||||
|
||||
```bash
|
||||
wget -P /var/lib/vz/template/iso/ \
|
||||
https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
|
||||
```
|
||||
|
||||
This is a pre-built cloud image with cloud-init already installed. It is much faster than installing from the server ISO.
|
||||
|
||||
### Create the base VM
|
||||
|
||||
```bash
|
||||
qm create 9000 \
|
||||
--name ubuntu-2404-cloud \
|
||||
--memory 2048 \
|
||||
--cores 2 \
|
||||
--cpu host \
|
||||
--net0 virtio,bridge=vmbr0 \
|
||||
--ostype l26 \
|
||||
--scsihw virtio-scsi-single \
|
||||
--agent enabled=1
|
||||
```
|
||||
|
||||
### Import the cloud image as the primary disk
|
||||
|
||||
```bash
|
||||
qm importdisk 9000 \
|
||||
/var/lib/vz/template/iso/noble-server-cloudimg-amd64.img \
|
||||
local-lvm
|
||||
|
||||
# Attach it as scsi0
|
||||
qm set 9000 --scsi0 local-lvm:vm-9000-disk-0,discard=on,ssd=1
|
||||
```
|
||||
|
||||
### Add cloud-init drive and configure boot
|
||||
|
||||
```bash
|
||||
# Cloud-init drive on IDE channel 1
|
||||
qm set 9000 --ide1 local-lvm:cloudinit
|
||||
|
||||
# Boot order: disk first
|
||||
qm set 9000 --boot order=scsi0
|
||||
|
||||
# Serial console (needed for cloud-init to work properly)
|
||||
qm set 9000 --serial0 socket --vga serial0
|
||||
```
|
||||
|
||||
### Set cloud-init defaults
|
||||
|
||||
```bash
|
||||
qm set 9000 \
|
||||
--ciuser thomas \
|
||||
--sshkeys /root/.ssh/authorized_keys \
|
||||
--ipconfig0 ip=dhcp \
|
||||
--nameserver 1.1.1.1 \
|
||||
--searchdomain lab.local
|
||||
```
|
||||
|
||||
See [cloud-init.yaml](cloud-init.yaml) for a full custom cloud-init configuration.
|
||||
|
||||
To use a custom file:
|
||||
|
||||
```bash
|
||||
# Place the file in a Proxmox snippets storage (a directory storage with 'snippets' content enabled)
|
||||
cp cloud-init.yaml /var/lib/vz/snippets/ubuntu-base.yaml
|
||||
|
||||
qm set 9000 --cicustom "user=local:snippets/ubuntu-base.yaml"
|
||||
```
|
||||
|
||||
### Resize the disk (cloud images start at ~3 GB)
|
||||
|
||||
```bash
|
||||
qm resize 9000 scsi0 +29G
|
||||
# Result: 32 GB total
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Convert to Template
|
||||
|
||||
Once the base VM is configured and the disk is clean:
|
||||
|
||||
```bash
|
||||
qm template 9000
|
||||
```
|
||||
|
||||
After this, the VM is read-only. You can no longer start it directly — only clone it.
|
||||
|
||||
To update the template in the future: clone it, make changes, re-convert to template (delete the old one first).
|
||||
|
||||
---
|
||||
|
||||
## 7. Clone a Template
|
||||
|
||||
### Linked clone (fast, shares base disk — less storage)
|
||||
|
||||
```bash
|
||||
qm clone 9000 201 --name webserver-01
|
||||
```
|
||||
|
||||
A linked clone shares the base disk blocks and only stores differences. Faster to create, but tied to the template — you cannot delete the template while linked clones exist.
|
||||
|
||||
### Full clone (independent — recommended for production)
|
||||
|
||||
```bash
|
||||
qm clone 9000 202 --name db-01 --full
|
||||
```
|
||||
|
||||
Full clones are completely independent. Delete or update the template freely.
|
||||
|
||||
### Override cloud-init settings per clone
|
||||
|
||||
```bash
|
||||
qm set 202 \
|
||||
--ipconfig0 ip=192.168.1.50/24,gw=192.168.1.1 \
|
||||
--ciuser admin
|
||||
|
||||
# Regenerate the cloud-init drive
|
||||
qm cloudinit dump 202 user
|
||||
```
|
||||
|
||||
Start the clone:
|
||||
|
||||
```bash
|
||||
qm start 202
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Import a Cloud Image (Faster Method)
|
||||
|
||||
If you already have the image locally:
|
||||
|
||||
```bash
|
||||
# Download directly to the right location
|
||||
wget -q --show-progress \
|
||||
https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img \
|
||||
-O /tmp/noble-server-cloudimg-amd64.img
|
||||
|
||||
# Create VM + import in one shot
|
||||
VMID=9001
|
||||
qm create $VMID --name ubuntu-noble-template --memory 2048 --cores 2 --cpu host \
|
||||
--net0 virtio,bridge=vmbr0 --ostype l26 --scsihw virtio-scsi-single --agent enabled=1
|
||||
qm importdisk $VMID /tmp/noble-server-cloudimg-amd64.img local-lvm
|
||||
qm set $VMID \
|
||||
--scsi0 local-lvm:vm-${VMID}-disk-0,discard=on,ssd=1 \
|
||||
--ide1 local-lvm:cloudinit \
|
||||
--boot order=scsi0 \
|
||||
--serial0 socket --vga serial0 \
|
||||
--ciuser thomas \
|
||||
--sshkeys /root/.ssh/authorized_keys \
|
||||
--ipconfig0 ip=dhcp
|
||||
qm resize $VMID scsi0 +29G
|
||||
qm template $VMID
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. VM Configuration Reference
|
||||
|
||||
```bash
|
||||
# Show full config
|
||||
qm config <vmid>
|
||||
|
||||
# Modify CPU and RAM without stopping
|
||||
qm set <vmid> --memory 4096
|
||||
qm set <vmid> --cores 4
|
||||
|
||||
# Add a second disk
|
||||
qm set <vmid> --scsi1 local-lvm:20
|
||||
|
||||
# Hotplug a USB device
|
||||
qm set <vmid> --usb0 host=<vendorid>:<productid>
|
||||
|
||||
# Enable start on boot
|
||||
qm set <vmid> --onboot 1
|
||||
|
||||
# Set startup order (lower = earlier)
|
||||
qm set <vmid> --startup order=1,up=30,down=60
|
||||
|
||||
# Protect from accidental deletion
|
||||
qm set <vmid> --protection 1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Snapshots
|
||||
|
||||
Snapshots capture the VM state (disk + optionally memory) at a point in time.
|
||||
|
||||
```bash
|
||||
# Create a snapshot (disk only — VM can be running with QEMU agent)
|
||||
qm snapshot <vmid> snap-$(date +%Y%m%d) --description "Before update"
|
||||
|
||||
# Create snapshot including RAM state (VM must be running)
|
||||
qm snapshot <vmid> snap-with-ram --vmstate 1
|
||||
|
||||
# List snapshots
|
||||
qm listsnapshot <vmid>
|
||||
|
||||
# Rollback
|
||||
qm rollback <vmid> snap-20260401
|
||||
|
||||
# Delete snapshot
|
||||
qm delsnapshot <vmid> snap-20260401
|
||||
```
|
||||
|
||||
Note: snapshots on LVM-Thin are fast and efficient. On raw LVM without thin provisioning, snapshots are not supported.
|
||||
@@ -0,0 +1,185 @@
|
||||
#cloud-config
|
||||
# cloud-init.yaml — Example cloud-init user-data for Ubuntu 24.04 on Proxmox VE 9.1
|
||||
#
|
||||
# Usage with Proxmox:
|
||||
# 1. Place this file in a snippets-enabled storage directory:
|
||||
# cp cloud-init.yaml /var/lib/vz/snippets/ubuntu-base.yaml
|
||||
# 2. Apply to a VM:
|
||||
# qm set <vmid> --cicustom "user=local:snippets/ubuntu-base.yaml"
|
||||
# 3. Regenerate the cloud-init ISO:
|
||||
# qm cloudinit update <vmid>
|
||||
# 4. Start or reboot the VM.
|
||||
#
|
||||
# This file is the "user-data" section only.
|
||||
# Proxmox handles "meta-data" (instance ID, hostname) automatically.
|
||||
# Proxmox handles "network-config" via --ipconfig0 flags.
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# System locale and timezone
|
||||
# ---------------------------------------------------------------------------
|
||||
locale: en_US.UTF-8
|
||||
timezone: Europe/Brussels
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Hostname
|
||||
# Set by Proxmox via --name / meta-data. Defined here as a fallback.
|
||||
# ---------------------------------------------------------------------------
|
||||
hostname: proxmox-vm
|
||||
manage_etc_hosts: true
|
||||
fqdn: proxmox-vm.lab.local
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Default user
|
||||
# Proxmox sets the user via --ciuser. This section extends that user.
|
||||
# If you set --ciuser thomas in Proxmox, the username below must match.
|
||||
# ---------------------------------------------------------------------------
|
||||
users:
|
||||
- name: thomas
|
||||
gecos: Homelab User
|
||||
groups:
|
||||
- sudo
|
||||
shell: /bin/bash
|
||||
sudo: ALL=(ALL) NOPASSWD:ALL
|
||||
lock_passwd: true
|
||||
# SSH keys are injected by Proxmox via --sshkeys
|
||||
# You can also add them here as a fallback:
|
||||
# ssh_authorized_keys:
|
||||
# - ssh-ed25519 AAAA... thomas@workstation
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# SSH configuration
|
||||
# ---------------------------------------------------------------------------
|
||||
ssh_pwauth: false
|
||||
disable_root: true
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Package management
|
||||
# ---------------------------------------------------------------------------
|
||||
package_update: true
|
||||
package_upgrade: true
|
||||
package_reboot_if_required: true
|
||||
|
||||
packages:
|
||||
# System essentials
|
||||
- curl
|
||||
- wget
|
||||
- git
|
||||
- vim
|
||||
- htop
|
||||
- unzip
|
||||
- gnupg
|
||||
- ca-certificates
|
||||
# QEMU guest agent (required for Proxmox integration)
|
||||
- qemu-guest-agent
|
||||
# Network tools
|
||||
- net-tools
|
||||
- iputils-ping
|
||||
- dnsutils
|
||||
# System monitoring
|
||||
- sysstat
|
||||
- iotop
|
||||
# Optional: remove if not needed
|
||||
- fail2ban
|
||||
- ufw
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Write files
|
||||
# Creates config files on first boot.
|
||||
# ---------------------------------------------------------------------------
|
||||
write_files:
|
||||
# SSH hardening
|
||||
- path: /etc/ssh/sshd_config.d/99-hardening.conf
|
||||
owner: root:root
|
||||
permissions: '0600'
|
||||
content: |
|
||||
PermitRootLogin no
|
||||
PasswordAuthentication no
|
||||
KbdInteractiveAuthentication no
|
||||
UsePAM yes
|
||||
X11Forwarding no
|
||||
PrintMotd no
|
||||
AcceptEnv LANG LC_*
|
||||
Subsystem sftp /usr/lib/openssh/sftp-server
|
||||
MaxAuthTries 3
|
||||
ClientAliveInterval 300
|
||||
ClientAliveCountMax 2
|
||||
|
||||
# UFW default rules
|
||||
- path: /etc/ufw/before.rules.lab
|
||||
owner: root:root
|
||||
permissions: '0644'
|
||||
content: |
|
||||
# Placeholder for custom UFW rules
|
||||
# Applied by runcmd section below
|
||||
|
||||
# System limits
|
||||
- path: /etc/sysctl.d/99-lab.conf
|
||||
owner: root:root
|
||||
permissions: '0644'
|
||||
content: |
|
||||
# Disable IPv6 if not needed in your lab
|
||||
# net.ipv6.conf.all.disable_ipv6 = 1
|
||||
# Reduce swappiness for VM workloads
|
||||
vm.swappiness = 10
|
||||
# Increase file descriptor limits
|
||||
fs.file-max = 100000
|
||||
# Network tuning
|
||||
net.core.somaxconn = 1024
|
||||
net.ipv4.tcp_syncookies = 1
|
||||
|
||||
# Bash aliases for convenience
|
||||
- path: /etc/profile.d/lab-aliases.sh
|
||||
owner: root:root
|
||||
permissions: '0644'
|
||||
content: |
|
||||
alias ll='ls -lah --color=auto'
|
||||
alias gs='git status'
|
||||
alias df='df -h'
|
||||
alias free='free -h'
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Run commands on first boot (as root, in order)
|
||||
# ---------------------------------------------------------------------------
|
||||
runcmd:
|
||||
# Enable and start QEMU guest agent
|
||||
- systemctl enable --now qemu-guest-agent
|
||||
|
||||
# Apply sysctl settings
|
||||
- sysctl -p /etc/sysctl.d/99-lab.conf
|
||||
|
||||
# Configure UFW
|
||||
- ufw default deny incoming
|
||||
- ufw default allow outgoing
|
||||
- ufw allow ssh
|
||||
- ufw --force enable
|
||||
|
||||
# Configure fail2ban for SSH
|
||||
- |
|
||||
cat > /etc/fail2ban/jail.d/sshd.conf << 'EOF'
|
||||
[sshd]
|
||||
enabled = true
|
||||
port = ssh
|
||||
filter = sshd
|
||||
logpath = /var/log/auth.log
|
||||
maxretry = 5
|
||||
bantime = 3600
|
||||
findtime = 600
|
||||
EOF
|
||||
- systemctl enable --now fail2ban
|
||||
|
||||
# Set correct locale
|
||||
- locale-gen en_US.UTF-8
|
||||
- update-locale LANG=en_US.UTF-8
|
||||
|
||||
# Log cloud-init completion
|
||||
- echo "cloud-init provisioning completed at $(date)" >> /var/log/cloud-init-lab.log
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Final message — visible in the cloud-init log (/var/log/cloud-init-output.log)
|
||||
# ---------------------------------------------------------------------------
|
||||
final_message: |
|
||||
Cloud-init setup complete.
|
||||
System: $HOSTNAME
|
||||
Python version: $VERSION
|
||||
Uptime: $UPTIME
|
||||
Run 'sudo cloud-init status' to verify.
|
||||
Reference in new issue
Block a user