Initial commit

This commit is contained in:
Thomas Rayon committed 2026-04-04 10:52:45 +02:00
commit 7e1d222cb2
12 files changed
+3334

No files matched your search

+29
View File
@@ -0,0 +1,29 @@
# Secrets and credentials — never commit these
*.key
*.pem
*.p12
*.pfx
pve.yml
*password*
*credentials*
*.env
# Backup files created by scripts
*.bak.*
*.bak
# OS artifacts
.DS_Store
Thumbs.db
desktop.ini
# Editor artifacts
.vscode/
.idea/
*.swp
*.swo
*~
# Temporary files
*.tmp
*.log
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Thomas Rayon
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+637
View File
@@ -0,0 +1,637 @@
# homelab-proxmox-starter
A complete step-by-step guide to building a homelab on Proxmox VE 9.1.
No complex stack, no Kubernetes, no over-engineering. Just a solid foundation.
---
## Table of Contents
1. [Hardware Requirements](#1-hardware-requirements)
2. [Install Proxmox VE 9.1](#2-install-proxmox-ve-91)
3. [Post-Install Configuration](#3-post-install-configuration)
4. [Network Configuration](#4-network-configuration)
5. [Create Your First VM](#5-create-your-first-vm)
6. [Create a VM Template for Fast Cloning](#6-create-a-vm-template-for-fast-cloning)
7. [LXC Containers](#7-lxc-containers)
8. [Backups with Proxmox Backup Server](#8-backups-with-proxmox-backup-server)
9. [Monitoring with Prometheus and Grafana](#9-monitoring-with-prometheus-and-grafana)
10. [Basic Hardening](#10-basic-hardening)
11. [Useful Daily Commands](#11-useful-daily-commands)
12. [Directory Structure of This Repo](#12-directory-structure-of-this-repo)
---
## 1. Hardware Requirements
Proxmox VE is a Type-1 hypervisor. It runs directly on bare metal. Do not install it inside a VM.
### Minimum viable homelab
| Component | Minimum | Recommended |
|-------------|----------------------|--------------------------|
| CPU | 4 cores, VT-x/VT-d | 8+ cores, VT-x/VT-d/IOMMU |
| RAM | 16 GB | 32–64 GB ECC |
| System disk | 32 GB SSD | 120 GB SSD (dedicated) |
| VM storage | 256 GB HDD/SSD | 1 TB+ NVMe or ZFS pool |
| NIC | 1x 1GbE | 2x 1GbE or 1x 10GbE |
### CPU flags to verify
Before buying or repurposing hardware, confirm your CPU supports:
- `vmx` (Intel VT-x) or `svm` (AMD-V) — required for VMs
- `ept` or `npt` — required for nested virtualisation
- `vt-d` (Intel) or `amd-iommu` (AMD) — required for PCIe passthrough (GPU, NIC, HBA)
Check on a running Linux system:
```bash
grep -E 'vmx|svm' /proc/cpuinfo | head -1
```
### Storage recommendations
- **System/boot**: a dedicated SSD, even 32 GB is fine. Keep VM storage separate.
- **VM storage**: ZFS on NVMe gives you snapshots, checksums, and compression for free.
- **Avoid USB boot**: Proxmox writes heavily to the system disk (logs, corosync). USB drives wear out fast.
---
## 2. Install Proxmox VE 9.1
### Download the ISO
Download the official ISO from [https://www.proxmox.com/en/downloads](https://www.proxmox.com/en/downloads).
Current version as of April 2026: **Proxmox VE 9.1**.
Verify the checksum before flashing:
```bash
sha256sum proxmox-ve_9.1-1.iso
```
Compare against the published SHA256 on the download page.
### Flash to USB
```bash
# Linux / macOS
dd if=proxmox-ve_9.1-1.iso of=/dev/sdX bs=4M status=progress oflag=sync
# Windows: use Rufus (write in DD mode, not ISO mode)
```
### Boot and install
1. Boot from USB. Select **Install Proxmox VE (Graphical)**.
2. Accept the EULA.
3. **Target disk**: select your dedicated system SSD. Check **Options** if you want ZFS on the system disk (recommended if you have 2+ disks for a mirror).
4. **Location and time zone**: set your country and timezone.
5. **Password and email**: set a strong root password. The email is used for alerts.
6. **Network configuration**:
- Management interface: pick your primary NIC.
- Hostname: use a FQDN, e.g. `pve01.lab.local`.
- IP: assign a static IP on your LAN, e.g. `192.168.1.10/24`.
- Gateway: your router IP, e.g. `192.168.1.1`.
- DNS: `192.168.1.1` or `1.1.1.1`.
7. Review the summary and click **Install**.
8. Remove USB, reboot.
Access the web UI at `https://192.168.1.10:8006`. Accept the self-signed certificate warning.
---
## 3. Post-Install Configuration
### Disable the enterprise repository (requires paid subscription)
The enterprise repo is enabled by default but requires a Proxmox subscription. Comment it out:
```bash
# On the Proxmox host via SSH or the web shell
sed -i 's/^deb/#deb/' /etc/apt/sources.list.d/pve-enterprise.list
```
Same for the Ceph enterprise repo if it exists:
```bash
if [ -f /etc/apt/sources.list.d/ceph.list ]; then
sed -i 's/^deb/#deb/' /etc/apt/sources.list.d/ceph.list
fi
```
### Enable the no-subscription repository
```bash
cat >> /etc/apt/sources.list << 'EOF'
# Proxmox VE No-Subscription Repo
deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription
EOF
```
### Remove the subscription nag popup
The web UI shows a nag dialog on login if no subscription is active. Remove it:
```bash
sed -Ezi.bak "s/(Ext.Msg.show\(\{\s+title: gettext\('No valid sub)/void\(\{ \/\/\1/g" \
/usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
# Restart the web service to apply
systemctl restart pveproxy
```
Note: this patch needs to be reapplied after each Proxmox update.
### Update the system
```bash
apt update && apt dist-upgrade -y
```
After updating, reboot if the kernel changed:
```bash
reboot
```
### Set the correct time zone
```bash
timedatectl set-timezone Europe/Brussels
timedatectl status
```
---
## 4. Network Configuration
See [network/README.md](network/README.md) for the full guide.
### Default bridge: vmbr0
Proxmox creates a Linux bridge `vmbr0` during installation. It maps to your physical NIC and is what VMs use for network access.
Check the current config:
```bash
cat /etc/network/interfaces
```
Typical output:
```
auto lo
iface lo inet loopback
auto eno1
iface eno1 inet manual
auto vmbr0
iface vmbr0 inet static
address 192.168.1.10/24
gateway 192.168.1.1
bridge-ports eno1
bridge-stp off
bridge-fd 0
```
### Add a second bridge for isolated VM traffic
```bash
# Append to /etc/network/interfaces
auto vmbr1
iface vmbr1 inet manual
bridge-ports none
bridge-stp off
bridge-fd 0
# Internal only — no physical NIC attached
```
Apply:
```bash
ifreload -a
```
### VLAN configuration
For VLAN tagging across a managed switch, see [network/vlan-setup.sh](network/vlan-setup.sh).
Quick summary: you create a VLAN-aware bridge and tag VMs at the VM config level, not at the bridge level.
---
## 5. Create Your First VM
### Download an ISO
In the Proxmox web UI:
1. Navigate to your node > **local** storage > **ISO Images**.
2. Click **Download from URL**.
3. URL: `https://releases.ubuntu.com/24.04/ubuntu-24.04.4-live-server-amd64.iso`
4. Click **Query URL**, verify the filename, then **Download**.
Or via CLI:
```bash
wget -P /var/lib/vz/template/iso/ \
https://releases.ubuntu.com/24.04/ubuntu-24.04.4-live-server-amd64.iso
```
### Create the VM via CLI (qm)
```bash
qm create 100 \
--name ubuntu-2404 \
--memory 2048 \
--cores 2 \
--net0 virtio,bridge=vmbr0 \
--ostype l26 \
--scsihw virtio-scsi-pci \
--scsi0 local-lvm:32 \
--ide2 local:iso/ubuntu-24.04.4-live-server-amd64.iso,media=cdrom \
--boot order=ide2 \
--agent enabled=1
```
Start it:
```bash
qm start 100
```
Open the console in the web UI to complete the Ubuntu installation.
### Recommended VM settings
- **CPU type**: `host` — passes through the real CPU model. Gives better performance and enables all CPU features. Do not use `kvm64` unless migrating between heterogeneous hosts.
- **Disk**: `virtio-scsi` controller. Better performance than IDE or SATA.
- **Network**: `virtio` model. Always. It is the fastest para-virtual NIC.
- **QEMU Guest Agent**: install it inside the VM so Proxmox can report the IP and do clean shutdowns.
Inside Ubuntu after install:
```bash
apt install -y qemu-guest-agent
systemctl enable --now qemu-guest-agent
```
---
## 6. Create a VM Template for Fast Cloning
Templates let you spin up a new VM in seconds instead of installing from scratch every time.
### Prepare the base VM
Start from the VM you created in step 5 (ID 100). Do a minimal install, run updates, install the guest agent, then clean up:
```bash
# Inside the VM
apt update && apt upgrade -y
apt install -y qemu-guest-agent cloud-init
systemctl enable qemu-guest-agent
# Remove SSH host keys (they will be regenerated on first boot)
rm /etc/ssh/ssh_host_*
truncate -s 0 /etc/machine-id
# Clean up
apt clean
cloud-init clean
```
Shut down the VM:
```bash
qm shutdown 100
```
### Add a cloud-init drive and convert to template
```bash
# Add cloud-init drive
qm set 100 --ide1 local-lvm:cloudinit
# Set cloud-init defaults
qm set 100 \
--ciuser thomas \
--sshkeys ~/.ssh/authorized_keys \
--ipconfig0 ip=dhcp
# Convert to template
qm template 100
```
See [vms/cloud-init.yaml](vms/cloud-init.yaml) for a full cloud-init configuration example.
### Clone the template
```bash
# Full clone (independent disk)
qm clone 100 201 --name webserver-01 --full
# Start the clone
qm start 201
```
A full clone takes about 10–30 seconds depending on disk speed.
---
## 7. LXC Containers
LXC containers share the host kernel. They are lighter than VMs and start in under a second. Use them for services that do not need a full OS (Nginx, Pi-hole, Home Assistant, etc.).
### Download a container template
```bash
pveam update
pveam available | grep ubuntu
pveam download local ubuntu-24.04-standard_24.04-2_amd64.tar.zst
```
### Create an LXC container
```bash
pct create 200 local:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst \
--hostname pihole \
--memory 512 \
--cores 1 \
--net0 name=eth0,bridge=vmbr0,ip=dhcp \
--storage local-lvm \
--rootfs local-lvm:8 \
--password changeme \
--unprivileged 1 \
--start 1
```
Access the console:
```bash
pct enter 200
```
### Privileged vs unprivileged
- **Unprivileged** (recommended): UIDs are remapped. Root inside the container is not root on the host. Use this by default.
- **Privileged**: required for some use cases (NFS mounts, Docker inside LXC, some kernel modules). Only use when needed.
---
## 8. Backups with Proxmox Backup Server
See [backups/README.md](backups/README.md) for the full guide.
### Option A: Proxmox Backup Server (PBS) — recommended
PBS is a dedicated backup solution from Proxmox. It supports deduplication, incremental backups, and encryption. Run it on a second machine or a dedicated VM on another host.
Install PBS from [https://www.proxmox.com/en/downloads](https://www.proxmox.com/en/downloads) on a separate machine.
### Add PBS as a storage backend in Proxmox
In the web UI: **Datacenter** > **Storage** > **Add** > **Proxmox Backup Server**.
| Field | Value |
|--------------|------------------------------------|
| ID | pbs-01 |
| Server | 192.168.1.20 (your PBS IP) |
| Datastore | backup-store (your PBS datastore) |
| Username | backup-user@pbs |
| Password | (your PBS backup user password) |
| Fingerprint | (shown in PBS dashboard) |
### Option B: local backup to a directory
For a simple setup without PBS, back up to a local directory or NFS share:
**Datacenter** > **Storage** > **Add** > **Directory**
Then create a backup job: **Datacenter** > **Backup** > **Add**
```
Storage: local (or your NFS share)
Schedule: 03:00 (daily at 3 AM)
Mode: Snapshot (requires QEMU agent) or Suspend
Compress: zstd
Max Backups: 3
```
### Verify backups
See [backups/backup-job.sh](backups/backup-job.sh) for a script that checks backup age and sends an alert if a backup is stale.
---
## 9. Monitoring with Prometheus and Grafana
See [monitoring/README.md](monitoring/README.md) for the full guide.
### Architecture
```
Proxmox host
└── pve-exporter (scrapes Proxmox API)
└── node_exporter (host metrics)
Monitoring VM
└── Prometheus (scrapes exporters)
└── Grafana (dashboards)
```
### Deploy a monitoring VM or LXC
Create a small Ubuntu 24.04 LXC (512 MB RAM, 2 cores, 10 GB disk) or VM.
### Install Prometheus
```bash
apt install -y prometheus
```
Replace the default config with [monitoring/prometheus.yml](monitoring/prometheus.yml).
```bash
cp prometheus.yml /etc/prometheus/prometheus.yml
systemctl restart prometheus
```
### Install Grafana
```bash
apt install -y apt-transport-https software-properties-common
wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key
echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" \
> /etc/apt/sources.list.d/grafana.list
apt update && apt install -y grafana
systemctl enable --now grafana-server
```
Access Grafana at `http://monitoring-vm-ip:3000`. Default credentials: `admin / admin`.
Import the Proxmox dashboard: ID **10347** (Proxmox via Prometheus) from grafana.com.
---
## 10. Basic Hardening
See [security/README.md](security/README.md) for the full hardening guide.
### Quick wins
**Disable root SSH login:**
```bash
sed -i 's/^#*PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
systemctl restart sshd
```
**Enable the Proxmox firewall (datacenter level):**
In the web UI: **Datacenter** > **Firewall** > **Options** > set **Firewall** to `Yes`.
Then set the default policy to DROP and add explicit ACCEPT rules for:
- Port 8006 (web UI) from your management VLAN
- Port 22 (SSH) from your management VLAN
- Port 3128 (SPICE proxy) if you use SPICE consoles
**Enable 2FA:**
**Datacenter** > **Permissions** > **Two Factor** > **Add** > **TOTP**
Scan the QR code with an authenticator app (Aegis, Bitwarden, etc.).
**Create a non-root user for daily use:**
```bash
pveum user add thomas@pve --password 'strongpassword'
pveum role add HomeLabAdmin --privs \
"VM.Allocate VM.Clone VM.Config.CDROM VM.Config.CPU VM.Config.Disk \
VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options \
VM.Console VM.Monitor VM.PowerMgmt VM.Snapshot Datastore.AllocateSpace \
Datastore.Audit Pool.Allocate Sys.Audit Sys.Console"
pveum aclmod / -user thomas@pve -role HomeLabAdmin
```
---
## 11. Useful Daily Commands
### VM management
```bash
# List all VMs
qm list
# Start / stop / reset
qm start <vmid>
qm shutdown <vmid>
qm reset <vmid>
# Show VM config
qm config <vmid>
# Create a snapshot
qm snapshot <vmid> snap-before-update --description "Before dist-upgrade"
# Rollback to snapshot
qm rollback <vmid> snap-before-update
# Delete snapshot
qm delsnapshot <vmid> snap-before-update
```
### LXC management
```bash
# List containers
pct list
# Start / stop / enter
pct start <ctid>
pct stop <ctid>
pct enter <ctid>
# Show config
pct config <ctid>
```
### Storage and ZFS
```bash
# List storage
pvesm status
# ZFS pool status
zpool status
# ZFS filesystem list
zfs list
# Show disk usage per dataset
zfs list -o name,used,avail,refer
```
### Cluster and node
```bash
# Node status
pvesh get /nodes/$(hostname)/status
# Task log
pvesh get /nodes/$(hostname)/tasks --limit 20
# Check service status
systemctl status pve-cluster pveproxy pvedaemon
```
### Logs
```bash
# Proxmox cluster log
journalctl -u pve-cluster -f
# Task log for a specific VM
cat /var/log/pve/tasks/active
```
---
## 12. Directory Structure of This Repo
```
homelab-proxmox-starter/
├── README.md # This file — full step-by-step tutorial
├── network/
│ ├── README.md # Network guide: bridges, VLANs, pfSense
│ └── vlan-setup.sh # VLAN configuration script for Proxmox
├── vms/
│ ├── README.md # VM creation and template guide
│ └── cloud-init.yaml # Cloud-init example for Ubuntu 24.04
├── backups/
│ ├── README.md # Backup guide: PBS and local backup
│ └── backup-job.sh # Backup verification script
├── monitoring/
│ ├── README.md # Monitoring guide: Prometheus + Grafana
│ └── prometheus.yml # Prometheus config for Proxmox
├── security/
│ └── README.md # Hardening guide
├── LICENSE # MIT
└── .gitignore
```
---
## Contributing
Issues and pull requests are welcome.
Keep things simple — this repo is intentionally a starter guide, not an automation framework.
## License
MIT — see [LICENSE](LICENSE).
+287
View File
@@ -0,0 +1,287 @@
# Backups Guide — Proxmox VE 9.1
This guide covers two backup strategies for a Proxmox homelab:
- **Option A**: Proxmox Backup Server (PBS) — recommended, supports deduplication and incremental backups
- **Option B**: local backup to a directory or NFS share — simpler, no extra server needed
---
## Table of Contents
1. [Backup Concepts in Proxmox](#1-backup-concepts-in-proxmox)
2. [Option A: Proxmox Backup Server (PBS)](#2-option-a-proxmox-backup-server-pbs)
3. [Option B: Local Directory or NFS Backup](#3-option-b-local-directory-or-nfs-backup)
4. [Backup Jobs: Schedule and Retention](#4-backup-jobs-schedule-and-retention)
5. [Restore a VM or LXC](#5-restore-a-vm-or-lxc)
6. [Verify Your Backups](#6-verify-your-backups)
7. [Backup Script](#7-backup-script)
---
## 1. Backup Concepts in Proxmox
### Backup modes
| Mode | VM state during backup | Requires QEMU agent | Notes |
|-----------|------------------------|---------------------|------------------------------------|
| Snapshot | Running | Yes | Minimal downtime, best option |
| Suspend | Briefly suspended | No | Short freeze, then continues |
| Stop | Shut down | No | Safest consistency, longest window |
Use **Snapshot** mode for production VMs with the QEMU agent installed.
Use **Stop** mode for VMs without the agent, or when data consistency is critical.
### Backup formats
- **VMA** (`.vma.zst`): native Proxmox format. Stores disk + config. Used by vzdump and PBS.
- **tar.zst**: used for LXC containers.
### Retention policy
Proxmox supports keeping the last N backups, or defining a pruning schedule (keep-daily, keep-weekly, keep-monthly, keep-yearly). PBS handles this natively.
---
## 2. Option A: Proxmox Backup Server (PBS)
PBS is a separate product from Proxmox. Install it on a second machine or a dedicated VM on a different physical host.
Download: [https://www.proxmox.com/en/downloads](https://www.proxmox.com/en/downloads)
PBS ISO: **Proxmox Backup Server 3.x** — verifier la derniere version disponible sur [proxmox.com/en/downloads](https://www.proxmox.com/en/downloads).
### Install PBS
Boot from the PBS ISO. Installation is similar to Proxmox VE:
- Assign a static IP (e.g. `192.168.1.20/24`)
- Set a strong root password
- Select a disk (PBS writes backups here — size it accordingly)
Access the PBS web UI at `https://192.168.1.20:8007`.
### Create a datastore
In the PBS web UI: **Administration** > **Datastores** > **Create Datastore**
| Field | Value |
|---------|------------------------|
| Name | backup-store |
| Backing Path | /mnt/backup (or your disk mount point) |
### Create a backup user in PBS
```bash
# On the PBS host
proxmox-backup-manager user create backup-user@pbs --password 'StrongPass1!'
proxmox-backup-manager acl update /datastore/backup-store backup-user@pbs DatastoreBackup
```
### Get the PBS fingerprint
```bash
# On the PBS host
proxmox-backup-manager cert info | grep Fingerprint
```
### Add PBS as storage in Proxmox
**Datacenter** > **Storage** > **Add** > **Proxmox Backup Server**
| Field | Value |
|-------------|-------------------------------------|
| ID | pbs-01 |
| Server | 192.168.1.20 |
| Username | backup-user@pbs |
| Password | StrongPass1! |
| Datastore | backup-store |
| Fingerprint | (paste from PBS) |
| Encryption key | Optional — generate in PBS UI |
### Enable client-side encryption (optional but recommended)
In the storage config, click **Encryption** > **Generate** to create an encryption key.
Store this key somewhere safe (not only on the Proxmox host).
Backups stored in PBS will be encrypted at rest. The key is required for any restore.
---
## 3. Option B: Local Directory or NFS Backup
### Local directory
**Datacenter** > **Storage** > **Add** > **Directory**
| Field | Value |
|---------|------------------------|
| ID | backup-local |
| Directory | /mnt/backup-disk |
| Content | VZDump backup file |
Mount an external disk:
```bash
# Find the disk
lsblk
# Format if needed (only on first use)
mkfs.ext4 /dev/sdb1
# Mount
mkdir -p /mnt/backup-disk
echo '/dev/sdb1 /mnt/backup-disk ext4 defaults,nofail 0 2' >> /etc/fstab
mount -a
```
### NFS share
Mount an NFS share from a NAS:
```bash
apt install -y nfs-common
mkdir -p /mnt/nas-backup
# Add to /etc/fstab
echo '192.168.1.30:/volume1/proxmox-backup /mnt/nas-backup nfs defaults,nofail 0 0' >> /etc/fstab
mount -a
```
Then add as a Directory storage in Proxmox.
---
## 4. Backup Jobs: Schedule and Retention
### Create a backup job
**Datacenter** > **Backup** > **Add**
| Field | Recommended value |
|-------------|-------------------------------|
| Storage | pbs-01 (or backup-local) |
| Schedule | 03:00 (daily at 3 AM) |
| Selection | All (or select specific VMs) |
| Mode | Snapshot |
| Compression | ZSTD |
| Send email | your@email.com (for failures) |
> **Note:** Proxmox VE does not include a mail transfer agent (MTA) by default. To use `--notify-email`, install one first: `apt-get install postfix` (or `msmtp` for a lightweight alternative).
### PBS retention (prune schedule)
In PBS: **Datastore** > **backup-store** > **Prune & GC** > **Add Prune Job**
```
keep-last: 3
keep-daily: 7
keep-weekly: 4
keep-monthly: 3
```
This keeps:
- 3 most recent backups regardless of age
- 1 backup per day for the last 7 days
- 1 backup per week for the last 4 weeks
- 1 backup per month for the last 3 months
For local/directory storage, set retention in the backup job:
**Max Backups**: 7 (keep last 7).
---
## 5. Restore a VM or LXC
### Restore via the web UI
1. Navigate to **local** (or **pbs-01**) storage > **Backups**.
2. Select the backup file.
3. Click **Restore**.
4. Assign a new VM ID and storage, then click **Restore**.
### Restore via CLI
```bash
# List available backups
pvesm list pbs-01
# Restore a VM (adjust backup path and target storage)
qmrestore pbs-01:backup/vm/101/2026-04-01T03:00:00Z 101 \
--storage local-lvm \
--unique 1
# Restore an LXC
pct restore 200 pbs-01:backup/ct/200/2026-04-01T03:00:00Z \
--storage local-lvm \
--unique 1
```
`--unique 1`: assigns a new unique MAC address and VMID to avoid conflicts.
### Test restore to a different VM ID
Always test restores before relying on them in a real incident:
```bash
qmrestore pbs-01:backup/vm/101/2026-04-01T03:00:00Z 999 \
--storage local-lvm
qm start 999
# Verify the VM boots and data is intact
qm destroy 999
```
---
## 6. Verify Your Backups
See [backup-job.sh](backup-job.sh) for an automated verification script.
### Check backup age via PBS API
```bash
# On the PBS host or via API
proxmox-backup-client list --repository backup-user@pbs@192.168.1.20:backup-store
```
### Check backup age on local storage
```bash
# List backup files with timestamps
ls -lht /mnt/backup-disk/dump/ | head -20
```
### Verify PBS backup integrity
PBS stores SHA-256 checksums for every chunk. Run a verification job:
**PBS web UI** > **Datastore** > **backup-store** > **Verify** > **Verify All**
Or via CLI on the PBS host:
```bash
proxmox-backup-manager verify backup-store
```
---
## 7. Backup Script
The script [backup-job.sh](backup-job.sh) checks the age of the latest backup for each VM and sends an alert if:
- No backup exists in the last 24 hours
- The backup job returned an error
Usage:
```bash
bash backups/backup-job.sh --storage pbs-01 --max-age-hours 26
```
Configure it as a cron job on the Proxmox host:
```bash
# Edit crontab
crontab -e
# Run every day at 05:30 (after the 03:00 backup job)
30 5 * * * /root/homelab-proxmox-starter/backups/backup-job.sh \
--storage pbs-01 --max-age-hours 26 >> /var/log/backup-check.log 2>&1
```
+268
View File
@@ -0,0 +1,268 @@
#!/usr/bin/env bash
# backup-job.sh — Verify Proxmox backup age and report stale or missing backups
#
# For each VM and LXC on this node, checks when the last backup was created.
# Exits with code 1 and prints an alert if any backup is older than --max-age-hours.
#
# Usage:
# bash backup-job.sh [OPTIONS]
#
# Options:
# --storage <id> Proxmox storage ID to check (e.g. pbs-01, backup-local)
# --max-age-hours <n> Alert if last backup is older than N hours (default: 26)
# --notify-email <addr> Send alert email via sendmail (optional)
# --dry-run Only report, do not exit with error code
# --help Show this help message
#
# Exit codes:
# 0 All backups are within the max-age window
# 1 One or more backups are stale or missing
#
# Requirements:
# - Run on the Proxmox VE host as root
# - pvesh and pvesm must be available (they are on any PVE node)
# - sendmail optional (only required for --notify-email)
set -euo pipefail
# ---------------------------------------------------------------------------
# Defaults
# ---------------------------------------------------------------------------
STORAGE=""
MAX_AGE_HOURS=26
NOTIFY_EMAIL=""
DRY_RUN=false
SCRIPT_NAME="$(basename "$0")"
NODE="$(hostname)"
ALERT_LINES=()
HAS_STALE=false
# ---------------------------------------------------------------------------
# Color output
# ---------------------------------------------------------------------------
if [ -t 1 ]; then
RED='\033[0;31m'
YELLOW='\033[1;33m'
GREEN='\033[0;32m'
CYAN='\033[0;36m'
RESET='\033[0m'
else
RED='' YELLOW='' GREEN='' CYAN='' RESET=''
fi
log_info() { echo -e "${CYAN}[INFO]${RESET} $(date '+%Y-%m-%d %H:%M:%S') $*"; }
log_ok() { echo -e "${GREEN}[OK]${RESET} $(date '+%Y-%m-%d %H:%M:%S') $*"; }
log_warn() { echo -e "${YELLOW}[WARN]${RESET} $(date '+%Y-%m-%d %H:%M:%S') $*"; }
log_alert() { echo -e "${RED}[ALERT]${RESET} $(date '+%Y-%m-%d %H:%M:%S') $*"; }
# ---------------------------------------------------------------------------
# Usage
# ---------------------------------------------------------------------------
usage() {
sed -n '/^# Usage:/,/^# Requirements:/p' "$0" | sed 's/^# //'
exit 0
}
# ---------------------------------------------------------------------------
# Argument parsing
# ---------------------------------------------------------------------------
parse_args() {
while [[ $# -gt 0 ]]; do
case "$1" in
--storage)
STORAGE="${2:?--storage requires a value}"
shift 2
;;
--max-age-hours)
MAX_AGE_HOURS="${2:?--max-age-hours requires a value}"
shift 2
;;
--notify-email)
NOTIFY_EMAIL="${2:?--notify-email requires a value}"
shift 2
;;
--dry-run)
DRY_RUN=true
shift
;;
--help|-h)
usage
;;
*)
echo "Unknown option: $1" >&2
usage
;;
esac
done
}
# ---------------------------------------------------------------------------
# Validation
# ---------------------------------------------------------------------------
validate() {
if [[ -z "$STORAGE" ]]; then
echo "Error: --storage is required." >&2
usage
fi
if ! [[ "$MAX_AGE_HOURS" =~ ^[0-9]+$ ]] || (( MAX_AGE_HOURS < 1 )); then
echo "Error: --max-age-hours must be a positive integer." >&2
exit 1
fi
if [[ $EUID -ne 0 ]]; then
echo "Error: this script must run as root on the Proxmox host." >&2
exit 1
fi
if ! command -v pvesh &>/dev/null; then
echo "Error: pvesh not found. This script must run on a Proxmox VE node." >&2
exit 1
fi
if ! command -v pvesm &>/dev/null; then
echo "Error: pvesm not found. This script must run on a Proxmox VE node." >&2
exit 1
fi
# Verify storage exists
if ! pvesm status | awk '{print $1}' | grep -q "^${STORAGE}$"; then
echo "Error: storage '${STORAGE}' not found on this node." >&2
echo "Available storage:" >&2
pvesm status | awk 'NR>1 {print " " $1}' >&2
exit 1
fi
}
# ---------------------------------------------------------------------------
# Get list of VMIDs (VMs and LXCs) on this node
# ---------------------------------------------------------------------------
get_all_vmids() {
{
pvesh get /nodes/"$NODE"/qemu --output-format json 2>/dev/null \
| python3 -c "import sys,json; [print(v['vmid']) for v in json.load(sys.stdin)]" 2>/dev/null || true
pvesh get /nodes/"$NODE"/lxc --output-format json 2>/dev/null \
| python3 -c "import sys,json; [print(v['vmid']) for v in json.load(sys.stdin)]" 2>/dev/null || true
} | sort -n
}
# ---------------------------------------------------------------------------
# Get the timestamp of the latest backup for a given VMID in the storage
# Returns epoch seconds or empty string if no backup found
# ---------------------------------------------------------------------------
get_latest_backup_epoch() {
local vmid="$1"
# pvesm list returns lines like:
# pbs-01:backup/vm/101/2026-04-01T03:00:00Z ... ctime=1743476400
pvesm list "$STORAGE" --vmid "$vmid" 2>/dev/null \
| awk 'NR>1 {print $NF}' \
| grep -oP 'ctime=\K[0-9]+' \
| sort -n \
| tail -1
}
# ---------------------------------------------------------------------------
# Send email alert via sendmail
# ---------------------------------------------------------------------------
send_email_alert() {
if [[ -z "$NOTIFY_EMAIL" ]]; then
return
fi
if ! command -v sendmail &>/dev/null; then
log_warn "sendmail not found — cannot send email alert."
return
fi
local subject="[Proxmox Backup Alert] $NODE — stale or missing backups"
local body
body=$(printf "Backup alert from %s\n\nThe following VMs/LXCs have stale or missing backups:\n\n%s\n\nMax age configured: %d hours\nTimestamp: %s\n" \
"$NODE" \
"$(printf '%s\n' "${ALERT_LINES[@]}")" \
"$MAX_AGE_HOURS" \
"$(date)")
{
echo "To: $NOTIFY_EMAIL"
echo "Subject: $subject"
echo "Content-Type: text/plain; charset=utf-8"
echo ""
echo "$body"
} | sendmail "$NOTIFY_EMAIL"
log_info "Alert email sent to $NOTIFY_EMAIL."
}
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
main() {
parse_args "$@"
validate
log_info "Proxmox backup verification"
log_info "Node : $NODE"
log_info "Storage : $STORAGE"
log_info "Max age : ${MAX_AGE_HOURS}h"
log_info "Dry-run : $DRY_RUN"
echo
local now
now=$(date +%s)
local max_age_seconds=$(( MAX_AGE_HOURS * 3600 ))
mapfile -t vmids < <(get_all_vmids)
if [[ ${#vmids[@]} -eq 0 ]]; then
log_warn "No VMs or LXCs found on node '$NODE'."
exit 0
fi
log_info "Found ${#vmids[@]} VMs/LXCs to check."
echo
for vmid in "${vmids[@]}"; do
local latest_epoch
latest_epoch=$(get_latest_backup_epoch "$vmid")
if [[ -z "$latest_epoch" ]]; then
local msg="VMID $vmid — NO BACKUP FOUND in storage '$STORAGE'"
log_alert "$msg"
ALERT_LINES+=("$msg")
HAS_STALE=true
continue
fi
local age_seconds=$(( now - latest_epoch ))
local age_hours=$(( age_seconds / 3600 ))
local last_backup_str
last_backup_str=$(date -d "@$latest_epoch" '+%Y-%m-%d %H:%M:%S' 2>/dev/null || date -r "$latest_epoch" '+%Y-%m-%d %H:%M:%S')
if (( age_seconds > max_age_seconds )); then
local msg="VMID $vmid — last backup: $last_backup_str (${age_hours}h ago — STALE, max ${MAX_AGE_HOURS}h)"
log_alert "$msg"
ALERT_LINES+=("$msg")
HAS_STALE=true
else
log_ok "VMID $vmid — last backup: $last_backup_str (${age_hours}h ago)"
fi
done
echo
if [[ "$HAS_STALE" == true ]]; then
log_alert "RESULT: ${#ALERT_LINES[@]} VM(s) with stale or missing backups."
send_email_alert
if [[ "$DRY_RUN" == false ]]; then
exit 1
else
log_info "Dry-run mode: exiting with 0 despite alerts."
exit 0
fi
else
log_ok "RESULT: All ${#vmids[@]} VM(s) have backups within the ${MAX_AGE_HOURS}h window."
exit 0
fi
}
main "$@"
+378
View File
@@ -0,0 +1,378 @@
# Monitoring Guide — Proxmox VE 9.1
This guide sets up monitoring for a Proxmox homelab using:
- **prometheus-pve-exporter**: exposes Proxmox API metrics (VMs, nodes, storage, cluster)
- **node_exporter**: exposes host-level OS metrics (CPU, RAM, disk, network)
- **Prometheus**: scrapes and stores metrics
- **Grafana**: dashboards
---
## Table of Contents
1. [Architecture Overview](#1-architecture-overview)
2. [Create the Monitoring VM or LXC](#2-create-the-monitoring-vm-or-lxc)
3. [Install prometheus-pve-exporter](#3-install-prometheus-pve-exporter)
4. [Install node_exporter on Proxmox Hosts](#4-install-node_exporter-on-proxmox-hosts)
5. [Install Prometheus](#5-install-prometheus)
6. [Configure Prometheus](#6-configure-prometheus)
7. [Install Grafana](#7-install-grafana)
8. [Import Dashboards](#8-import-dashboards)
9. [Alerting with Alertmanager (Optional)](#9-alerting-with-alertmanager-optional)
10. [Useful Queries (PromQL)](#10-useful-queries-promql)
---
## 1. Architecture Overview
```
Proxmox Node(s)
|- node_exporter :9100 (host OS metrics)
|- pve-exporter :9221 (Proxmox API metrics, runs on monitoring VM or node)
Monitoring VM / LXC
|- Prometheus :9090 (scrapes exporters, stores TSDB)
|- Grafana :3000 (reads Prometheus, renders dashboards)
|- Alertmanager :9093 (optional, handles alert routing)
```
All components run on a single monitoring VM or LXC.
The Proxmox API is scraped remotely — no agent on the hypervisor beyond node_exporter.
---
## 2. Create the Monitoring VM or LXC
A lightweight LXC is sufficient:
```bash
pct create 150 local:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst \
--hostname monitoring \
--memory 1024 \
--cores 2 \
--net0 name=eth0,bridge=vmbr0,ip=192.168.1.15/24,gw=192.168.1.1 \
--storage local-lvm \
--rootfs local-lvm:10 \
--unprivileged 1 \
--onboot 1 \
--start 1
```
Update the system:
```bash
pct enter 150
apt update && apt upgrade -y
```
---
## 3. Install prometheus-pve-exporter
`prometheus-pve-exporter` is a Python application that queries the Proxmox API and exposes metrics in Prometheus format.
Source: [https://github.com/prometheus-pve/prometheus-pve-exporter](https://github.com/prometheus-pve/prometheus-pve-exporter)
### Install on the monitoring LXC
```bash
apt install -y python3-pip python3-venv
python3 -m venv /opt/pve-exporter
/opt/pve-exporter/bin/pip install prometheus-pve-exporter
```
### Create a read-only Proxmox user for the exporter
On the Proxmox host (not the monitoring LXC):
```bash
pveum user add pve-exporter@pve --password 'ExporterReadOnly1!'
pveum role add PVEExporterRole --privs "Sys.Audit VM.Audit Datastore.Audit Pool.Audit"
pveum aclmod / -user pve-exporter@pve -role PVEExporterRole
```
### Configure the exporter
On the monitoring LXC:
```bash
mkdir -p /etc/prometheus
cat > /etc/prometheus/pve.yml << 'EOF'
default:
user: pve-exporter@pve
password: ExporterReadOnly1!
verify_ssl: false
EOF
chmod 600 /etc/prometheus/pve.yml
```
### Create a systemd service
```bash
cat > /etc/systemd/system/prometheus-pve-exporter.service << 'EOF'
[Unit]
Description=Prometheus Proxmox VE Exporter
Documentation=https://github.com/prometheus-pve/prometheus-pve-exporter
After=network.target
[Service]
Type=simple
User=nobody
Group=nogroup
ExecStart=/opt/pve-exporter/bin/pve_exporter \
--config.file=/etc/prometheus/pve.yml \
--web.listen-address=0.0.0.0:9221
Restart=on-failure
RestartSec=5s
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable --now prometheus-pve-exporter
```
Verify:
```bash
curl -s "http://localhost:9221/pve?target=192.168.1.10&module=default" | head -20
```
Replace `192.168.1.10` with your Proxmox node IP.
---
## 4. Install node_exporter on Proxmox Hosts
`node_exporter` exposes OS-level metrics from the Proxmox host itself.
Run this on each Proxmox node (not the monitoring LXC):
```bash
# Download node_exporter
NODE_EXPORTER_VERSION="1.8.2"
wget -q "https://github.com/prometheus/node_exporter/releases/download/v${NODE_EXPORTER_VERSION}/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gz" \
-O /tmp/node_exporter.tar.gz
tar -xzf /tmp/node_exporter.tar.gz -C /tmp
cp /tmp/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64/node_exporter /usr/local/bin/
chmod +x /usr/local/bin/node_exporter
# Create system user
useradd --no-create-home --shell /bin/false node_exporter
# Create systemd service
cat > /etc/systemd/system/node_exporter.service << 'EOF'
[Unit]
Description=Prometheus Node Exporter
Documentation=https://github.com/prometheus/node_exporter
After=network.target
[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter \
--collector.disable-defaults \
--collector.cpu \
--collector.diskstats \
--collector.filesystem \
--collector.loadavg \
--collector.meminfo \
--collector.netdev \
--collector.time \
--collector.vmstat \
--web.listen-address=0.0.0.0:9100
Restart=on-failure
RestartSec=5s
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable --now node_exporter
```
Verify:
```bash
curl -s http://localhost:9100/metrics | grep node_cpu_seconds_total | head -5
```
---
## 5. Install Prometheus
On the monitoring LXC:
```bash
apt install -y prometheus
systemctl enable prometheus
```
The default Prometheus config is at `/etc/prometheus/prometheus.yml`.
Replace it with the config from this repo.
---
## 6. Configure Prometheus
Copy the provided config:
```bash
cp prometheus.yml /etc/prometheus/prometheus.yml
systemctl restart prometheus
```
See [prometheus.yml](prometheus.yml) for the full configuration.
Key targets in the config:
| Job | Port | Source |
|------------------------|------|----------------------------------|
| `pve-node` | 9221 | Proxmox API via pve-exporter |
| `node-pve` | 9100 | Proxmox host OS via node_exporter|
| `prometheus` | 9090 | Prometheus self-monitoring |
Verify Prometheus targets at `http://192.168.1.15:9090/targets`.
All targets should show **UP**.
---
## 7. Install Grafana
On the monitoring LXC:
```bash
apt install -y apt-transport-https software-properties-common wget gnupg
mkdir -p /usr/share/keyrings
wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key
echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" \
> /etc/apt/sources.list.d/grafana.list
apt update && apt install -y grafana
systemctl enable --now grafana-server
```
Access Grafana at `http://192.168.1.15:3000`.
Default credentials: `admin` / `admin`. Change on first login.
### Add Prometheus as a data source
1. In Grafana: **Connections** > **Data Sources** > **Add data source** > **Prometheus**
2. URL: `http://localhost:9090`
3. Click **Save & Test**. Should return "Data source is working".
---
## 8. Import Dashboards
### Proxmox VE dashboard (via pve-exporter)
Dashboard ID: **10347**
1. In Grafana: **Dashboards** > **Import**
2. Enter ID `10347`, click **Load**
3. Select your Prometheus data source
4. Click **Import**
This dashboard shows: node status, VM count, CPU/RAM per VM, storage usage, network I/O.
### Node Exporter Full (host OS metrics)
Dashboard ID: **1860**
Same import process. Shows detailed CPU, memory, disk, and network metrics from node_exporter.
### Proxmox Cluster Dashboard
Dashboard ID: **15356** — useful if you have multiple Proxmox nodes in a cluster.
---
## 9. Alerting with Alertmanager (Optional)
Alertmanager handles routing, grouping, and sending of alerts from Prometheus.
```bash
apt install -y prometheus-alertmanager
systemctl enable --now prometheus-alertmanager
```
Example alert rule — create `/etc/prometheus/alerts/proxmox.yml`:
```yaml
groups:
- name: proxmox
rules:
- alert: ProxmoxNodeDown
expr: up{job="node-pve"} == 0
for: 2m
labels:
severity: critical
annotations:
summary: "Proxmox node unreachable"
description: "node_exporter on {{ $labels.instance }} has been down for 2 minutes."
- alert: ProxmoxHighCPU
expr: 100 - (avg by(instance)(rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 90
for: 5m
labels:
severity: warning
annotations:
summary: "Proxmox high CPU usage"
description: "CPU usage on {{ $labels.instance }} is above 90% for 5 minutes."
- alert: ProxmoxLowDisk
expr: (node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"}) * 100 < 10
for: 1m
labels:
severity: warning
annotations:
summary: "Proxmox root disk almost full"
description: "Less than 10% free disk space on {{ $labels.instance }}."
```
Add the rule file reference to `prometheus.yml`:
```yaml
rule_files:
- /etc/prometheus/alerts/*.yml
```
---
## 10. Useful Queries (PromQL)
```promql
# Proxmox node CPU usage (%)
100 - avg by(instance)(rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100
# Memory used (%)
(1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100
# Disk space used on root (%)
(1 - node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"}) * 100
# Number of running VMs on the cluster
pve_up{type="qemu"}
# VM CPU usage
rate(pve_cpu_usage_ratio{type="qemu"}[5m]) * 100
# Storage usage per pool
pve_disk_usage_bytes / pve_disk_size_bytes * 100
# Network receive rate per interface (bytes/s)
rate(node_network_receive_bytes_total[5m])
```
+175
View File
@@ -0,0 +1,175 @@
# prometheus.yml — Prometheus configuration for Proxmox VE 9.1 homelab monitoring
#
# This config scrapes:
# - Proxmox VE API metrics via prometheus-pve-exporter (port 9221)
# - Host OS metrics via node_exporter on the Proxmox node (port 9100)
# - Prometheus self-monitoring (port 9090)
#
# Prerequisites:
# - prometheus-pve-exporter running on the monitoring VM (192.168.1.15:9221)
# - node_exporter running on each Proxmox node (port 9100)
# - A read-only Proxmox user configured in /etc/prometheus/pve.yml
#
# Documentation:
# - pve-exporter: https://github.com/prometheus-pve/prometheus-pve-exporter
# - node_exporter: https://github.com/prometheus/node_exporter
# - prometheus: https://prometheus.io/docs/prometheus/latest/configuration/configuration/
global:
# How often Prometheus scrapes each target
scrape_interval: 30s
# Timeout for each scrape request
scrape_timeout: 15s
# How often Prometheus evaluates alert rules
evaluation_interval: 30s
# Labels added to all time series scraped by this Prometheus instance
external_labels:
cluster: homelab
region: on-premise
# ---------------------------------------------------------------------------
# Alertmanager configuration (uncomment if Alertmanager is installed)
# ---------------------------------------------------------------------------
# alerting:
# alertmanagers:
# - static_configs:
# - targets:
# - localhost:9093
# ---------------------------------------------------------------------------
# Alert rule files
# ---------------------------------------------------------------------------
rule_files: []
# Uncomment and create the file to enable alerting:
# rule_files:
# - /etc/prometheus/alerts/*.yml
# ---------------------------------------------------------------------------
# Scrape configurations
# ---------------------------------------------------------------------------
scrape_configs:
# --------------------------------------------------------------------------
# Prometheus self-monitoring
# Scrapes Prometheus's own /metrics endpoint.
# --------------------------------------------------------------------------
- job_name: prometheus
static_configs:
- targets:
- localhost:9090
labels:
instance: monitoring
role: prometheus
# --------------------------------------------------------------------------
# Proxmox VE API metrics via prometheus-pve-exporter
#
# The pve-exporter uses a "target" parameter to know which Proxmox node to
# query. The scrape is sent to the exporter (192.168.1.15:9221), which then
# queries the Proxmox API on the specified target.
#
# This configuration uses relabeling to pass the target address to the
# exporter as a URL parameter, following the pve-exporter documentation.
#
# Add one entry per Proxmox node under targets.
# --------------------------------------------------------------------------
- job_name: pve-node
# The actual scrape target is the exporter, not the Proxmox node directly.
# The Proxmox node address is passed as a query parameter.
metrics_path: /pve
params:
# The Proxmox API module defined in /etc/prometheus/pve.yml
module: [default]
# Include per-VM/LXC metrics (set to 1 to enable, 0 for cluster-level only)
# cluster: ["1"]
static_configs:
- targets:
# Replace with your Proxmox node IP(s)
# For multiple nodes, add one entry per node.
- 192.168.1.10 # pve01
# - 192.168.1.11 # pve02 (uncomment for a second node)
labels:
cluster: homelab
relabel_configs:
# Copy the scraped address into the __param_target label
# so pve-exporter knows which Proxmox node to query.
- source_labels: [__address__]
target_label: __param_target
# Keep the Proxmox node address as the "instance" label in metrics.
- source_labels: [__param_target]
target_label: instance
# Redirect the actual HTTP request to the exporter running locally.
- target_label: __address__
replacement: localhost:9221
# --------------------------------------------------------------------------
# Node Exporter — host OS metrics on Proxmox node(s)
#
# node_exporter must be installed and running on each Proxmox node.
# See monitoring/README.md for installation instructions.
# --------------------------------------------------------------------------
- job_name: node-pve
static_configs:
- targets:
# One entry per Proxmox node.
- 192.168.1.10:9100 # pve01
# - 192.168.1.11:9100 # pve02
labels:
role: hypervisor
cluster: homelab
relabel_configs:
# Strip the port from the instance label for cleaner display in Grafana
- source_labels: [__address__]
target_label: instance
regex: '([^:]+)(:[0-9]+)?'
replacement: '$1'
# --------------------------------------------------------------------------
# Node Exporter — monitoring VM itself
# Scrapes the monitoring LXC/VM's own node_exporter.
# Install node_exporter on the monitoring LXC the same way as on PVE nodes.
# --------------------------------------------------------------------------
- job_name: node-monitoring
static_configs:
- targets:
- localhost:9100
labels:
role: monitoring
instance: monitoring
# --------------------------------------------------------------------------
# Optional: Grafana self-monitoring
# Uncomment if you want to track Grafana's own health metrics.
# --------------------------------------------------------------------------
# - job_name: grafana
# static_configs:
# - targets:
# - localhost:3000
# labels:
# instance: monitoring
# --------------------------------------------------------------------------
# Optional: Proxmox Backup Server (PBS) metrics
#
# PBS exposes Prometheus metrics natively since PBS 2.0.
# No exporter needed — just scrape the PBS node directly.
# Enable in PBS: Administration > Options > Metric Server > Add > Prometheus
# Or scrape the built-in endpoint:
# --------------------------------------------------------------------------
# - job_name: pbs
# metrics_path: /metrics
# scheme: https
# tls_config:
# insecure_skip_verify: true
# basic_auth:
# username: pbs-metrics@pbs
# password: your-token-here
# static_configs:
# - targets:
# - 192.168.1.20:8007
# labels:
# instance: pbs-01
# role: backup-server
+301
View File
@@ -0,0 +1,301 @@
# Network Guide — Proxmox VE 9.1
This guide covers network configuration for a homelab Proxmox setup:
bridges, VLANs, trunk ports, and an optional pfSense VM as the network gateway.
---
## Table of Contents
1. [How Proxmox Networking Works](#1-how-proxmox-networking-works)
2. [Linux Bridges](#2-linux-bridges)
3. [VLAN Overview](#3-vlan-overview)
4. [VLAN-Aware Bridge Setup](#4-vlan-aware-bridge-setup)
5. [Trunk Port to a Managed Switch](#5-trunk-port-to-a-managed-switch)
6. [Assigning VMs to VLANs](#6-assigning-vms-to-vlans)
7. [pfSense as a VM Gateway](#7-pfsense-as-a-vm-gateway)
8. [Bonding and Link Aggregation](#8-bonding-and-link-aggregation)
9. [Useful Network Commands](#9-useful-network-commands)
---
## 1. How Proxmox Networking Works
Proxmox uses standard Linux networking. There is no proprietary abstraction. The main file is:
```
/etc/network/interfaces
```
Changes made in the web UI (Node > Network) write to this file. Changes are applied with `ifreload -a` or on reboot.
Key concepts:
- **Physical NIC** (`eno1`, `enp3s0`, etc.): the real hardware interface. Set to `manual` — Proxmox bridges own the IP.
- **Linux bridge** (`vmbr0`, `vmbr1`, etc.): virtual switch. VMs plug into bridges, not directly into NICs.
- **VLAN interface** (`vmbr0.10`, `eno1.10`): tagged sub-interface for a specific VLAN.
- **Bond** (`bond0`): aggregates 2+ physical NICs into one logical link.
---
## 2. Linux Bridges
### Default bridge (vmbr0)
Created during installation. Carries the Proxmox management IP.
```
auto vmbr0
iface vmbr0 inet static
address 192.168.1.10/24
gateway 192.168.1.1
bridge-ports eno1
bridge-stp off
bridge-fd 0
```
`bridge-stp off` and `bridge-fd 0` are important: they prevent the 15-second STP forwarding delay on startup.
### Adding a second bridge (isolated internal network)
```
auto vmbr1
iface vmbr1 inet manual
bridge-ports none
bridge-stp off
bridge-fd 0
# No physical port: internal-only traffic between VMs
```
Apply:
```bash
ifreload -a
```
---
## 3. VLAN Overview
VLANs let you segment traffic on a single physical link. Each VLAN has an ID (1–4094). Traffic is tagged with an 802.1Q header.
Common homelab VLAN layout:
| VLAN ID | Name | Purpose |
|---------|--------------|--------------------------------|
| 1 | Native/Mgmt | Proxmox management, hypervisor |
| 10 | LAN | Trusted devices, desktops |
| 20 | IoT | Smart home devices |
| 30 | DMZ | Public-facing services |
| 40 | Lab | Experimental VMs |
---
## 4. VLAN-Aware Bridge Setup
Proxmox supports VLAN-aware bridges natively since PVE 4.x. This is the recommended approach.
### Manual edit of /etc/network/interfaces
```
auto vmbr0
iface vmbr0 inet static
address 192.168.1.10/24
gateway 192.168.1.1
bridge-ports eno1
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
```
`bridge-vlan-aware yes` enables 802.1Q tagging on this bridge.
`bridge-vids 2-4094` declares all VLANs as allowed. You can restrict to specific IDs if needed.
Apply:
```bash
ifreload -a
```
Verify:
```bash
bridge vlan show
```
For automated setup, use the provided script:
```bash
bash network/vlan-setup.sh --bridge vmbr0 --vlans 10,20,30,40 --dry-run
```
Remove `--dry-run` to apply.
---
## 5. Trunk Port to a Managed Switch
For VLANs to reach other devices on your network, your managed switch must have the Proxmox port configured as a **trunk** (carries all tagged VLANs).
### Cisco-style configuration (example)
```
interface GigabitEthernet0/1
description Proxmox-PVE01
switchport mode trunk
switchport trunk allowed vlan 1,10,20,30,40
spanning-tree portfast trunk
```
### TP-Link / UniFi / Netgear
Set the port connecting to Proxmox as **Tagged** for all VLANs you want to pass through.
The Proxmox management VLAN (VLAN 1 by default) should be the **native** VLAN (untagged) on the trunk port so the Proxmox host itself remains accessible.
---
## 6. Assigning VMs to VLANs
With a VLAN-aware bridge, you tag VMs individually. No extra bridge per VLAN needed.
### In the web UI
When creating or editing a VM: **Hardware** > **Network Device** > set **VLAN Tag** to the desired VLAN ID.
### Via CLI (qm set)
```bash
# Assign VM 201 to VLAN 10
qm set 201 --net0 virtio,bridge=vmbr0,tag=10
# Assign LXC 200 to VLAN 20
pct set 200 --net0 name=eth0,bridge=vmbr0,tag=20,ip=dhcp
```
The VM sees an untagged interface (the hypervisor handles the tagging). No changes needed inside the VM.
---
## 7. pfSense as a VM Gateway
pfSense or OPNsense running as a VM provides routing and firewall between VLANs.
### Architecture
```
Internet (WAN)
|
[Physical NIC eno2] -- vmbr-wan (bridge, no IP)
|
[pfSense VM]
|
[vmbr0, VLAN-aware] -- internal VLANs (LAN, IoT, DMZ, Lab)
```
pfSense gets:
- A WAN interface: plugged into `vmbr-wan` (connected to your router or modem)
- A LAN interface: plugged into `vmbr0` with a VLAN trunk
pfSense then handles inter-VLAN routing and firewall rules.
### Create the WAN bridge
```
auto vmbr-wan
iface vmbr-wan inet manual
bridge-ports eno2
bridge-stp off
bridge-fd 0
```
### pfSense VM configuration
```bash
qm create 300 \
--name pfsense \
--memory 2048 \
--cores 2 \
--net0 virtio,bridge=vmbr-wan \
--net1 virtio,bridge=vmbr0 \
--ostype other \
--scsihw virtio-scsi-pci \
--scsi0 local-lvm:16
```
Inside pfSense, configure net1 as a VLAN trunk and create VLAN interfaces for each VLAN ID.
---
## 8. Bonding and Link Aggregation
If your server has 2+ NICs, you can bond them for redundancy or increased throughput.
### Active-backup bond (redundancy)
```
auto bond0
iface bond0 inet manual
bond-slaves eno1 eno2
bond-miimon 100
bond-mode active-backup
bond-primary eno1
auto vmbr0
iface vmbr0 inet static
address 192.168.1.10/24
gateway 192.168.1.1
bridge-ports bond0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094
```
`bond-miimon 100` checks link state every 100ms. If `eno1` fails, traffic switches to `eno2` automatically.
### LACP / 802.3ad (requires managed switch support)
```
auto bond0
iface bond0 inet manual
bond-slaves eno1 eno2
bond-miimon 100
bond-mode 802.3ad
bond-lacp-rate fast
bond-xmit-hash-policy layer2+3
```
Configure the corresponding port-channel on your switch.
---
## 9. Useful Network Commands
```bash
# Show all interfaces and their state
ip link show
# Show IP addresses
ip addr show
# Show bridge details (ports, VLANs)
bridge link show
bridge vlan show
# Reload network config without full reboot
ifreload -a
# Show the current routing table
ip route show
# Test connectivity between VMs on different VLANs
ping -c 3 -I vmbr0.10 192.168.10.1
# Show bond status (if bonding is configured)
cat /proc/net/bonding/bond0
# Capture traffic on a bridge port (tcpdump)
tcpdump -i vmbr0 -n -v 'vlan 10'
```
+323
View File
@@ -0,0 +1,323 @@
#!/usr/bin/env bash
# vlan-setup.sh — Configure VLAN-aware bridge on Proxmox VE 9.1
#
# Modifies /etc/network/interfaces to:
# 1. Enable bridge-vlan-aware on the target bridge
# 2. Set the allowed VLAN range (bridge-vids)
# 3. Create VLAN sub-interfaces on the bridge for host-level VLAN access
#
# Usage:
# bash vlan-setup.sh [OPTIONS]
#
# Options:
# --bridge <name> Bridge to configure (default: vmbr0)
# --vlans <list> Comma-separated VLAN IDs to configure (e.g. 10,20,30)
# --iface-file <path> Path to interfaces file (default: /etc/network/interfaces)
# --dry-run Print what would be done, do not modify any file
# --help Show this help message
#
# Examples:
# bash vlan-setup.sh --bridge vmbr0 --vlans 10,20,30 --dry-run
# bash vlan-setup.sh --bridge vmbr0 --vlans 10,20,30,40
set -euo pipefail
# ---------------------------------------------------------------------------
# Defaults
# ---------------------------------------------------------------------------
BRIDGE="vmbr0"
VLANS=""
IFACE_FILE="/etc/network/interfaces"
DRY_RUN=false
BACKUP_SUFFIX=".bak.$(date +%Y%m%d_%H%M%S)"
# ---------------------------------------------------------------------------
# Color output (only when stdout is a terminal)
# ---------------------------------------------------------------------------
if [ -t 1 ]; then
RED='\033[0;31m'
YELLOW='\033[1;33m'
GREEN='\033[0;32m'
CYAN='\033[0;36m'
RESET='\033[0m'
else
RED='' YELLOW='' GREEN='' CYAN='' RESET=''
fi
log_info() { echo -e "${CYAN}[INFO]${RESET} $*"; }
log_ok() { echo -e "${GREEN}[OK]${RESET} $*"; }
log_warn() { echo -e "${YELLOW}[WARN]${RESET} $*"; }
log_error() { echo -e "${RED}[ERROR]${RESET} $*" >&2; }
log_dryrun() { echo -e "${YELLOW}[DRY-RUN]${RESET} $*"; }
# ---------------------------------------------------------------------------
# Usage
# ---------------------------------------------------------------------------
usage() {
sed -n '/^# Usage:/,/^$/p' "$0" | head -n 30 | sed 's/^# //'
exit 0
}
# ---------------------------------------------------------------------------
# Argument parsing
# ---------------------------------------------------------------------------
parse_args() {
while [[ $# -gt 0 ]]; do
case "$1" in
--bridge)
BRIDGE="${2:?--bridge requires a value}"
shift 2
;;
--vlans)
VLANS="${2:?--vlans requires a value}"
shift 2
;;
--iface-file)
IFACE_FILE="${2:?--iface-file requires a value}"
shift 2
;;
--dry-run)
DRY_RUN=true
shift
;;
--help|-h)
usage
;;
*)
log_error "Unknown option: $1"
usage
;;
esac
done
}
# ---------------------------------------------------------------------------
# Validation
# ---------------------------------------------------------------------------
validate_args() {
if [[ -z "$VLANS" ]]; then
log_error "--vlans is required. Example: --vlans 10,20,30"
exit 1
fi
if [[ ! -f "$IFACE_FILE" ]]; then
log_error "Interfaces file not found: $IFACE_FILE"
exit 1
fi
# Validate VLAN IDs
IFS=',' read -ra VLAN_ARRAY <<< "$VLANS"
for vlan in "${VLAN_ARRAY[@]}"; do
if ! [[ "$vlan" =~ ^[0-9]+$ ]] || (( vlan < 1 || vlan > 4094 )); then
log_error "Invalid VLAN ID: $vlan (must be 1–4094)"
exit 1
fi
done
# Check we are running on Linux (Proxmox is Linux only)
if [[ "$(uname -s)" != "Linux" ]]; then
log_error "This script is designed for Linux / Proxmox VE only."
exit 1
fi
# Warn if not root (changes to /etc/network/interfaces require root)
if [[ $EUID -ne 0 ]] && [[ "$DRY_RUN" == false ]]; then
log_warn "Not running as root. Changes to $IFACE_FILE will likely fail."
log_warn "Run with sudo or as root, or use --dry-run to preview changes."
fi
log_ok "Arguments validated."
}
# ---------------------------------------------------------------------------
# Check if the bridge exists in the interfaces file
# ---------------------------------------------------------------------------
bridge_exists_in_file() {
grep -q "^auto ${BRIDGE}$" "$IFACE_FILE" || grep -q "^iface ${BRIDGE} " "$IFACE_FILE"
}
# ---------------------------------------------------------------------------
# Check if a specific VLAN sub-interface stanza already exists
# ---------------------------------------------------------------------------
vlan_iface_exists() {
local vlan_id="$1"
grep -q "^auto ${BRIDGE}.${vlan_id}$" "$IFACE_FILE"
}
# ---------------------------------------------------------------------------
# Build the patch for bridge-vlan-aware and bridge-vids
# This function prints the modified block for the target bridge.
# We use awk to insert the options after the last existing bridge-* line
# inside the bridge stanza.
# ---------------------------------------------------------------------------
patch_bridge_stanza() {
local tmpfile
tmpfile=$(mktemp)
awk -v bridge="$BRIDGE" '
BEGIN {
in_bridge = 0
vlan_aware_added = 0
bridge_vids_added = 0
}
/^(auto|iface|allow-hotplug)/ {
# Entering a new stanza: flush pending additions for previous bridge
if (in_bridge && !vlan_aware_added) {
print " bridge-vlan-aware yes"
print " bridge-vids 2-4094"
vlan_aware_added = 1
bridge_vids_added = 1
}
in_bridge = 0
vlan_aware_added = 0
bridge_vids_added = 0
}
/^iface[[:space:]]/ && $2 == bridge {
in_bridge = 1
}
in_bridge && /bridge-vlan-aware/ {
# Already present: normalise the value
sub(/bridge-vlan-aware[[:space:]]+.*/, "bridge-vlan-aware yes")
vlan_aware_added = 1
print
next
}
in_bridge && /bridge-vids/ {
# Already present: normalise the value
sub(/bridge-vids[[:space:]]+.*/, "bridge-vids 2-4094")
bridge_vids_added = 1
print
next
}
in_bridge && /bridge-fd/ && !vlan_aware_added {
# Insert after bridge-fd
print
print " bridge-vlan-aware yes"
print " bridge-vids 2-4094"
vlan_aware_added = 1
bridge_vids_added = 1
next
}
{ print }
END {
if (in_bridge && !vlan_aware_added) {
print " bridge-vlan-aware yes"
print " bridge-vids 2-4094"
}
}
' "$IFACE_FILE" > "$tmpfile"
echo "$tmpfile"
}
# ---------------------------------------------------------------------------
# Build stanzas for VLAN sub-interfaces (host-level VLAN access)
# These are manual interfaces — no IP assigned here.
# Assign IPs at the VM/LXC level or add them manually afterward.
# ---------------------------------------------------------------------------
build_vlan_stanzas() {
local output=""
IFS=',' read -ra VLAN_ARRAY <<< "$VLANS"
for vlan in "${VLAN_ARRAY[@]}"; do
if ! vlan_iface_exists "$vlan"; then
output+="
auto ${BRIDGE}.${vlan}
iface ${BRIDGE}.${vlan} inet manual
# VLAN ${vlan} sub-interface on ${BRIDGE}
# Assign an IP here if the Proxmox host needs to communicate on VLAN ${vlan}
# Example: address 192.168.${vlan}.1/24
"
else
log_warn "VLAN sub-interface ${BRIDGE}.${vlan} already exists in $IFACE_FILE — skipped."
fi
done
echo "$output"
}
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
main() {
parse_args "$@"
log_info "Proxmox VE VLAN setup script"
log_info "Bridge : $BRIDGE"
log_info "VLANs : $VLANS"
log_info "File : $IFACE_FILE"
log_info "Dry-run : $DRY_RUN"
echo
validate_args
# Check bridge exists in file
if ! bridge_exists_in_file; then
log_error "Bridge '$BRIDGE' not found in $IFACE_FILE."
log_error "Create the bridge first via the Proxmox web UI or manually."
exit 1
fi
log_info "Bridge '$BRIDGE' found in $IFACE_FILE."
# Build patched bridge stanza
local patched_file
patched_file=$(patch_bridge_stanza)
# Build VLAN stanzas
local vlan_stanzas
vlan_stanzas=$(build_vlan_stanzas)
if [[ "$DRY_RUN" == true ]]; then
echo
log_dryrun "--- Patched $IFACE_FILE (bridge section) ---"
cat "$patched_file"
echo
if [[ -n "$vlan_stanzas" ]]; then
log_dryrun "--- VLAN sub-interfaces to append ---"
echo "$vlan_stanzas"
fi
log_dryrun "No changes written (--dry-run mode)."
rm -f "$patched_file"
exit 0
fi
# Backup original file
cp "$IFACE_FILE" "${IFACE_FILE}${BACKUP_SUFFIX}"
log_ok "Backup created: ${IFACE_FILE}${BACKUP_SUFFIX}"
# Write patched bridge stanza
cp "$patched_file" "$IFACE_FILE"
rm -f "$patched_file"
log_ok "Bridge stanza patched (bridge-vlan-aware yes, bridge-vids 2-4094)."
# Append VLAN stanzas
if [[ -n "$vlan_stanzas" ]]; then
echo "$vlan_stanzas" >> "$IFACE_FILE"
log_ok "VLAN sub-interfaces appended."
fi
# Apply changes
log_info "Applying network configuration with ifreload..."
if command -v ifreload &>/dev/null; then
ifreload -a && log_ok "Network configuration applied."
else
# WARNING: never run this on vmbr0 (management bridge) — it will cut your SSH session
log_warn "ifreload not found. Reboot to apply changes, or run: ifdown $BRIDGE && ifup $BRIDGE"
fi
echo
log_ok "Done. Verify with: bridge vlan show"
echo
log_info "To assign a VM to a VLAN, run:"
log_info " qm set <vmid> --net0 virtio,bridge=${BRIDGE},tag=<vlan_id>"
log_info "To assign an LXC to a VLAN, run:"
log_info " pct set <ctid> --net0 name=eth0,bridge=${BRIDGE},tag=<vlan_id>,ip=dhcp"
}
main "$@"
+403
View File
@@ -0,0 +1,403 @@
# Security Hardening Guide — Proxmox VE 9.1
This guide covers baseline hardening for a Proxmox homelab.
Not an enterprise checklist — a practical starting point that covers the most impactful actions.
---
## Table of Contents
1. [Web UI and API Hardening](#1-web-ui-and-api-hardening)
2. [SSH Hardening](#2-ssh-hardening)
3. [Two-Factor Authentication (2FA)](#3-two-factor-authentication-2fa)
4. [User and Permission Management](#4-user-and-permission-management)
5. [Proxmox Firewall](#5-proxmox-firewall)
6. [Remove the Subscription Nag](#6-remove-the-subscription-nag)
7. [Kernel and System Hardening](#7-kernel-and-system-hardening)
8. [Audit and Logging](#8-audit-and-logging)
9. [VM Isolation](#9-vm-isolation)
10. [Checklist Summary](#10-checklist-summary)
---
## 1. Web UI and API Hardening
### Change the default port (optional, security by obscurity)
The web UI runs on port 8006 by default. Changing it does not add real security but reduces automated scanning noise.
```bash
# Edit the pveproxy config
cat >> /etc/default/pveproxy << 'EOF'
LISTEN_PORT=8443
EOF
systemctl restart pveproxy
```
If you change the port, update your firewall rules accordingly.
### Restrict web UI access to a specific subnet
The best approach is to allow the Proxmox web UI port only from your management machine or VLAN, using the Proxmox firewall (see section 5).
### Use a valid TLS certificate
The default certificate is self-signed. For a homelab, you can use Let's Encrypt with a DNS challenge if your domain is publicly resolvable — even for internal use.
```bash
# Install acme.sh or use the built-in ACME client in Proxmox
# Web UI: Datacenter > ACME > Add Account, then Node > Certificates > Add ACME Domain
```
Alternatively, generate a certificate from your own internal CA (e.g. step-ca, Easy-RSA).
---
## 2. SSH Hardening
### Disable root SSH login
Root login over SSH should be disabled. Use a non-root user or key-only root if strictly necessary.
```bash
sed -i 's/^#*PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
```
If you need root access via SSH temporarily, prefer: `ssh thomas@pve01 sudo -i`
### Disable password authentication (keys only)
```bash
sed -i 's/^#*PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
sed -i 's/^#*KbdInteractiveAuthentication.*/KbdInteractiveAuthentication no/' /etc/ssh/sshd_config
```
Make sure your SSH public key is in `/root/.ssh/authorized_keys` before applying this, or you will lock yourself out.
### Additional SSH hardening options
Create `/etc/ssh/sshd_config.d/99-proxmox-hardening.conf`:
```
MaxAuthTries 3
LoginGraceTime 30
ClientAliveInterval 300
ClientAliveCountMax 2
AllowUsers thomas root
X11Forwarding no
AllowTcpForwarding no
```
Apply:
```bash
sshd -t # test config before reloading
systemctl restart sshd
```
### Change SSH port (optional)
```bash
# Uncomment and set in /etc/ssh/sshd_config
Port 2222
```
Update firewall rules if you change the port.
---
## 3. Two-Factor Authentication (2FA)
2FA prevents account compromise if credentials are stolen. Proxmox supports TOTP natively.
### Enable 2FA for a user
1. In the web UI: **Datacenter** > **Permissions** > **Two Factor**
2. Click **Add** > **TOTP**
3. Enter the user (e.g. `thomas@pve`)
4. Scan the QR code with an authenticator app: Aegis (Android), Bitwarden Authenticator, or FreeOTP
5. Enter the current OTP to confirm
6. Click **Add**
### Enforce 2FA at the realm level
This requires all users in the realm to have 2FA configured before they can log in.
**Datacenter** > **Permissions** > **Realms** > click **pve** > **Edit** > enable **Two Factor**.
Note: set up 2FA for all users before enforcing, or you will lock them out.
### WebAuthn / hardware keys (alternative)
Proxmox 7.2+ supports WebAuthn (FIDO2/U2F). You can use a YubiKey or another hardware token.
**Datacenter** > **Permissions** > **Two Factor** > **WebAuthn Settings**
Configure the Relying Party ID (your Proxmox FQDN) and origin URL, then register your key.
---
## 4. User and Permission Management
### Never use root for daily operations
Create a dedicated admin user with only the privileges needed:
```bash
# Create a local PVE user
pveum user add thomas@pve --password 'ChangeThisPassword!'
# Create a custom role for homelab administration
pveum role add HomeLabAdmin --privs \
"VM.Allocate VM.Clone VM.Config.CDROM VM.Config.CPU VM.Config.Disk \
VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options \
VM.Console VM.Monitor VM.PowerMgmt VM.Snapshot VM.Backup \
Datastore.AllocateSpace Datastore.AllocateTemplate Datastore.Audit \
Pool.Allocate Sys.Audit Sys.Console Sys.Modify"
# Grant the role to the user at the root path
pveum aclmod / -user thomas@pve -role HomeLabAdmin
```
### Create read-only users for monitoring
```bash
pveum user add monitor@pve --password 'ReadOnlyPassword!'
pveum aclmod / -user monitor@pve -role PVEAuditor
```
### List current users and ACLs
```bash
pveum user list
pveum acl list
pveum role list
```
---
## 5. Proxmox Firewall
Proxmox has a built-in firewall at three levels: datacenter, node, and VM/LXC.
### Enable the datacenter firewall
**Web UI**: **Datacenter** > **Firewall** > **Options** > set **Firewall** to enabled.
Set the default policy:
| Direction | Policy |
|-----------|--------|
| Input | DROP |
| Output | ACCEPT |
| Forward | DROP |
### Add explicit rules
Allow only necessary traffic to the node:
| Type | Action | Source | Dest port | Protocol | Comment |
|-------|--------|----------------|-----------|----------|--------------------------|
| in | ACCEPT | 192.168.1.0/24 | 8006 | TCP | Proxmox web UI |
| in | ACCEPT | 192.168.1.0/24 | 22 | TCP | SSH |
| in | ACCEPT | 192.168.1.0/24 | 3128 | TCP | SPICE proxy (optional) |
| in | ACCEPT | any | icmp | ICMP | Ping (optional) |
Add rules in: **Datacenter** > **Firewall** > **Rules** > **Add**
Or via CLI:
```bash
# Allow SSH from management subnet
pvesh create /nodes/pve01/firewall/rules \
--type in \
--action ACCEPT \
--source 192.168.1.0/24 \
--dport 22 \
--proto tcp \
--comment "SSH from LAN"
# Allow web UI from management subnet
pvesh create /nodes/pve01/firewall/rules \
--type in \
--action ACCEPT \
--source 192.168.1.0/24 \
--dport 8006 \
--proto tcp \
--comment "Proxmox UI from LAN"
```
### Enable the node firewall
After adding rules: **Node** > **Firewall** > **Options** > set **Firewall** to enabled.
Do this AFTER adding your rules, or you will lock yourself out.
---
## 6. Remove the Subscription Nag
The subscription nag dialog appears on every login without a paid subscription. It is cosmetic only.
```bash
# Patch the proxmox widget toolkit JS
sed -Ezi.bak \
"s/(Ext.Msg.show\(\{\s+title: gettext\('No valid sub)/void\(\{ \/\/\1/g" \
/usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
systemctl restart pveproxy
```
This patch is reset on each Proxmox update. Reapply after `apt dist-upgrade`.
---
## 7. Kernel and System Hardening
### Apply security-relevant sysctl settings
```bash
cat > /etc/sysctl.d/99-proxmox-hardening.conf << 'EOF'
# Prevent IP spoofing
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
# Disable IP source routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
# Disable ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
# Enable TCP SYN cookies (protection against SYN flood)
net.ipv4.tcp_syncookies = 1
# Log martian packets (packets with impossible source addresses)
net.ipv4.conf.all.log_martians = 1
EOF
sysctl -p /etc/sysctl.d/99-proxmox-hardening.conf
```
### Keep the system updated
```bash
# Unattended security updates
apt install -y unattended-upgrades
dpkg-reconfigure -f noninteractive unattended-upgrades
```
### Disable unnecessary services
Check which services are running:
```bash
systemctl list-units --type=service --state=running
```
Disable services you do not use:
```bash
# Example: disable rpcbind if no NFS is used
systemctl disable --now rpcbind rpcbind.socket 2>/dev/null || true
```
---
## 8. Audit and Logging
### Proxmox task log
Every action in the web UI is logged to the task log. Access via:
- **Datacenter** > **Tasks**
- CLI: `pvesh get /nodes/$(hostname)/tasks --limit 50`
### System log
```bash
# SSH logins
journalctl -u sshd --since "24h ago"
# Authentication log
grep -E 'sshd|sudo|pam' /var/log/auth.log | tail -50
# Proxmox cluster events
journalctl -u pve-cluster -u pvedaemon -u pveproxy --since "24h ago"
```
### Install auditd (optional, for compliance)
```bash
apt install -y auditd
# Basic rules: monitor sensitive files
cat > /etc/audit/rules.d/proxmox.rules << 'EOF'
# Monitor changes to network interfaces
-w /etc/network/interfaces -p wa -k network-config
# Monitor changes to Proxmox config files
-w /etc/pve/ -p wa -k pve-config
# Monitor SSH config changes
-w /etc/ssh/sshd_config -p wa -k sshd-config
# Monitor authentication events
-w /var/log/auth.log -p wa -k auth-log
EOF
systemctl enable --now auditd
augenrules --load
```
---
## 9. VM Isolation
### Use separate VLANs per trust level
Segment your VMs so that a compromised VM cannot reach the hypervisor management interface.
- Proxmox management: VLAN 1 or a dedicated management VLAN — no VM traffic here
- Production VMs: VLAN 10
- Lab/test VMs: VLAN 40
- IoT: VLAN 20
See [../network/README.md](../network/README.md) for VLAN setup.
### Do not use privileged LXC containers unless required
Privileged LXC containers can theoretically escape to the host if a kernel vulnerability is exploited. Use unprivileged containers:
```bash
pct create ... --unprivileged 1
```
### Enable SecureBoot for VMs handling sensitive data
When creating a VM: **System** > **BIOS** > select **OVMF (UEFI)** instead of SeaBIOS. Enable SecureBoot.
---
## 10. Checklist Summary
| Action | Priority | Done |
|----------------------------------------------------------|----------|------|
| Disable subscription nag | Low | |
| Disable enterprise apt repo, enable no-subscription | High | |
| Run `apt dist-upgrade` after install | High | |
| Disable root SSH login | High | |
| Disable SSH password authentication | High | |
| Enable 2FA for all admin accounts | High | |
| Create a non-root user for daily use | High | |
| Enable Proxmox firewall (datacenter + node level) | High | |
| Restrict web UI access to management subnet only | High | |
| Apply sysctl hardening | Medium | |
| Set up VLAN segmentation (management vs VM traffic) | Medium | |
| Use unprivileged LXC containers | Medium | |
| Enable unattended security updates | Medium | |
| Set up auditd | Low | |
| Use a valid TLS certificate (internal CA or Let's Encrypt)| Low | |
+327
View File
@@ -0,0 +1,327 @@
# VMs and Templates Guide — Proxmox VE 9.1
This guide covers VM creation, cloud-init configuration, template creation, and fast cloning.
---
## Table of Contents
1. [VM Storage Options](#1-vm-storage-options)
2. [Create a VM via the Web UI](#2-create-a-vm-via-the-web-ui)
3. [Create a VM via CLI (qm)](#3-create-a-vm-via-cli-qm)
4. [Cloud-Init Overview](#4-cloud-init-overview)
5. [Prepare a Cloud-Init Ready VM](#5-prepare-a-cloud-init-ready-vm)
6. [Convert to Template](#6-convert-to-template)
7. [Clone a Template](#7-clone-a-template)
8. [Import a Cloud Image (Faster Method)](#8-import-a-cloud-image-faster-method)
9. [VM Configuration Reference](#9-vm-configuration-reference)
10. [Snapshots](#10-snapshots)
---
## 1. VM Storage Options
Proxmox supports multiple storage backends. The most common:
| Backend | Format | Snapshots | Notes |
|---------------|--------------|-----------|----------------------------------|
| LVM-Thin | raw | Yes | Default after installation |
| ZFS | zvol / qcow2 | Yes | Best option if you have NVMe |
| Directory | qcow2, raw | Yes (qcow2)| Simple, works on any filesystem |
| NFS/CIFS | qcow2, raw | Yes (qcow2)| For shared storage |
Use LVM-Thin or ZFS for the best snapshot performance.
---
## 2. Create a VM via the Web UI
1. Click **Create VM** in the top-right of the web UI.
2. **General**: set Node, VM ID (e.g. 100), Name.
3. **OS**: select ISO image from local storage. Set OS type to Linux, version 6.x kernel.
4. **System**: leave defaults (BIOS: SeaBIOS or OVMF for UEFI). Enable QEMU Agent.
5. **Disks**: select storage (local-lvm or ZFS pool). Set size. Disk bus: SCSI, controller: VirtIO SCSI Single.
6. **CPU**: set cores. Type: `host` for best performance.
7. **Memory**: set RAM. Enable ballooning for dynamic memory.
8. **Network**: model `VirtIO (paravirt)`, bridge `vmbr0`. Set VLAN tag if needed.
9. **Confirm**: review and click **Finish**.
---
## 3. Create a VM via CLI (qm)
The `qm` command manages virtual machines from the shell.
### Minimal Ubuntu 24.04 VM
```bash
qm create 101 \
--name ubuntu-2404-base \
--memory 2048 \
--balloon 1024 \
--cores 2 \
--cpu host \
--net0 virtio,bridge=vmbr0 \
--ostype l26 \
--scsihw virtio-scsi-single \
--scsi0 local-lvm:32,discard=on,ssd=1 \
--ide2 local:iso/ubuntu-24.04.4-live-server-amd64.iso,media=cdrom \
--boot order=ide2 \
--agent enabled=1 \
--onboot 0
```
Key flags:
- `--cpu host`: exposes full CPU feature set to the VM.
- `--balloon 1024`: minimum memory for ballooning (requires balloon driver inside VM).
- `--scsihw virtio-scsi-single`: modern, high-performance SCSI controller.
- `--discard=on,ssd=1`: enables TRIM/discard passthrough (important for SSDs and LVM-Thin).
- `--agent enabled=1`: enables QEMU guest agent socket.
- `--onboot 0`: do not start automatically (set to 1 for production VMs).
### Start the VM and open the console
```bash
qm start 101
# Then open the web console or use:
pvesh get /nodes/$(hostname)/vms/101/termproxy --shell 1
```
---
## 4. Cloud-Init Overview
Cloud-init is the standard for initialising cloud instances. On first boot it:
- Sets the hostname
- Creates users and injects SSH keys
- Configures networking (DHCP or static IP)
- Runs custom scripts
Proxmox supports cloud-init natively via a special drive (virtio, IDE, or SCSI). The VM reads this drive on boot like a data CD.
Supported cloud-init parameters in Proxmox:
| `qm set` flag | Effect |
|---------------------|---------------------------------------------|
| `--ciuser` | Default user name |
| `--cipassword` | Password for the default user |
| `--sshkeys` | Authorised SSH public keys |
| `--ipconfig0` | Network config for net0 (ip=dhcp or static) |
| `--nameserver` | DNS server |
| `--searchdomain` | DNS search domain |
| `--cicustom` | Path to a custom cloud-init YAML file |
---
## 5. Prepare a Cloud-Init Ready VM
### Download the Ubuntu 24.04 cloud image
```bash
wget -P /var/lib/vz/template/iso/ \
https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
```
This is a pre-built cloud image with cloud-init already installed. It is much faster than installing from the server ISO.
### Create the base VM
```bash
qm create 9000 \
--name ubuntu-2404-cloud \
--memory 2048 \
--cores 2 \
--cpu host \
--net0 virtio,bridge=vmbr0 \
--ostype l26 \
--scsihw virtio-scsi-single \
--agent enabled=1
```
### Import the cloud image as the primary disk
```bash
qm importdisk 9000 \
/var/lib/vz/template/iso/noble-server-cloudimg-amd64.img \
local-lvm
# Attach it as scsi0
qm set 9000 --scsi0 local-lvm:vm-9000-disk-0,discard=on,ssd=1
```
### Add cloud-init drive and configure boot
```bash
# Cloud-init drive on IDE channel 1
qm set 9000 --ide1 local-lvm:cloudinit
# Boot order: disk first
qm set 9000 --boot order=scsi0
# Serial console (needed for cloud-init to work properly)
qm set 9000 --serial0 socket --vga serial0
```
### Set cloud-init defaults
```bash
qm set 9000 \
--ciuser thomas \
--sshkeys /root/.ssh/authorized_keys \
--ipconfig0 ip=dhcp \
--nameserver 1.1.1.1 \
--searchdomain lab.local
```
See [cloud-init.yaml](cloud-init.yaml) for a full custom cloud-init configuration.
To use a custom file:
```bash
# Place the file in a Proxmox snippets storage (a directory storage with 'snippets' content enabled)
cp cloud-init.yaml /var/lib/vz/snippets/ubuntu-base.yaml
qm set 9000 --cicustom "user=local:snippets/ubuntu-base.yaml"
```
### Resize the disk (cloud images start at ~3 GB)
```bash
qm resize 9000 scsi0 +29G
# Result: 32 GB total
```
---
## 6. Convert to Template
Once the base VM is configured and the disk is clean:
```bash
qm template 9000
```
After this, the VM is read-only. You can no longer start it directly — only clone it.
To update the template in the future: clone it, make changes, re-convert to template (delete the old one first).
---
## 7. Clone a Template
### Linked clone (fast, shares base disk — less storage)
```bash
qm clone 9000 201 --name webserver-01
```
A linked clone shares the base disk blocks and only stores differences. Faster to create, but tied to the template — you cannot delete the template while linked clones exist.
### Full clone (independent — recommended for production)
```bash
qm clone 9000 202 --name db-01 --full
```
Full clones are completely independent. Delete or update the template freely.
### Override cloud-init settings per clone
```bash
qm set 202 \
--ipconfig0 ip=192.168.1.50/24,gw=192.168.1.1 \
--ciuser admin
# Regenerate the cloud-init drive
qm cloudinit dump 202 user
```
Start the clone:
```bash
qm start 202
```
---
## 8. Import a Cloud Image (Faster Method)
If you already have the image locally:
```bash
# Download directly to the right location
wget -q --show-progress \
https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img \
-O /tmp/noble-server-cloudimg-amd64.img
# Create VM + import in one shot
VMID=9001
qm create $VMID --name ubuntu-noble-template --memory 2048 --cores 2 --cpu host \
--net0 virtio,bridge=vmbr0 --ostype l26 --scsihw virtio-scsi-single --agent enabled=1
qm importdisk $VMID /tmp/noble-server-cloudimg-amd64.img local-lvm
qm set $VMID \
--scsi0 local-lvm:vm-${VMID}-disk-0,discard=on,ssd=1 \
--ide1 local-lvm:cloudinit \
--boot order=scsi0 \
--serial0 socket --vga serial0 \
--ciuser thomas \
--sshkeys /root/.ssh/authorized_keys \
--ipconfig0 ip=dhcp
qm resize $VMID scsi0 +29G
qm template $VMID
```
---
## 9. VM Configuration Reference
```bash
# Show full config
qm config <vmid>
# Modify CPU and RAM without stopping
qm set <vmid> --memory 4096
qm set <vmid> --cores 4
# Add a second disk
qm set <vmid> --scsi1 local-lvm:20
# Hotplug a USB device
qm set <vmid> --usb0 host=<vendorid>:<productid>
# Enable start on boot
qm set <vmid> --onboot 1
# Set startup order (lower = earlier)
qm set <vmid> --startup order=1,up=30,down=60
# Protect from accidental deletion
qm set <vmid> --protection 1
```
---
## 10. Snapshots
Snapshots capture the VM state (disk + optionally memory) at a point in time.
```bash
# Create a snapshot (disk only — VM can be running with QEMU agent)
qm snapshot <vmid> snap-$(date +%Y%m%d) --description "Before update"
# Create snapshot including RAM state (VM must be running)
qm snapshot <vmid> snap-with-ram --vmstate 1
# List snapshots
qm listsnapshot <vmid>
# Rollback
qm rollback <vmid> snap-20260401
# Delete snapshot
qm delsnapshot <vmid> snap-20260401
```
Note: snapshots on LVM-Thin are fast and efficient. On raw LVM without thin provisioning, snapshots are not supported.
+185
View File
@@ -0,0 +1,185 @@
#cloud-config
# cloud-init.yaml — Example cloud-init user-data for Ubuntu 24.04 on Proxmox VE 9.1
#
# Usage with Proxmox:
# 1. Place this file in a snippets-enabled storage directory:
# cp cloud-init.yaml /var/lib/vz/snippets/ubuntu-base.yaml
# 2. Apply to a VM:
# qm set <vmid> --cicustom "user=local:snippets/ubuntu-base.yaml"
# 3. Regenerate the cloud-init ISO:
# qm cloudinit update <vmid>
# 4. Start or reboot the VM.
#
# This file is the "user-data" section only.
# Proxmox handles "meta-data" (instance ID, hostname) automatically.
# Proxmox handles "network-config" via --ipconfig0 flags.
# ---------------------------------------------------------------------------
# System locale and timezone
# ---------------------------------------------------------------------------
locale: en_US.UTF-8
timezone: Europe/Brussels
# ---------------------------------------------------------------------------
# Hostname
# Set by Proxmox via --name / meta-data. Defined here as a fallback.
# ---------------------------------------------------------------------------
hostname: proxmox-vm
manage_etc_hosts: true
fqdn: proxmox-vm.lab.local
# ---------------------------------------------------------------------------
# Default user
# Proxmox sets the user via --ciuser. This section extends that user.
# If you set --ciuser thomas in Proxmox, the username below must match.
# ---------------------------------------------------------------------------
users:
- name: thomas
gecos: Homelab User
groups:
- sudo
shell: /bin/bash
sudo: ALL=(ALL) NOPASSWD:ALL
lock_passwd: true
# SSH keys are injected by Proxmox via --sshkeys
# You can also add them here as a fallback:
# ssh_authorized_keys:
# - ssh-ed25519 AAAA... thomas@workstation
# ---------------------------------------------------------------------------
# SSH configuration
# ---------------------------------------------------------------------------
ssh_pwauth: false
disable_root: true
# ---------------------------------------------------------------------------
# Package management
# ---------------------------------------------------------------------------
package_update: true
package_upgrade: true
package_reboot_if_required: true
packages:
# System essentials
- curl
- wget
- git
- vim
- htop
- unzip
- gnupg
- ca-certificates
# QEMU guest agent (required for Proxmox integration)
- qemu-guest-agent
# Network tools
- net-tools
- iputils-ping
- dnsutils
# System monitoring
- sysstat
- iotop
# Optional: remove if not needed
- fail2ban
- ufw
# ---------------------------------------------------------------------------
# Write files
# Creates config files on first boot.
# ---------------------------------------------------------------------------
write_files:
# SSH hardening
- path: /etc/ssh/sshd_config.d/99-hardening.conf
owner: root:root
permissions: '0600'
content: |
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
UsePAM yes
X11Forwarding no
PrintMotd no
AcceptEnv LANG LC_*
Subsystem sftp /usr/lib/openssh/sftp-server
MaxAuthTries 3
ClientAliveInterval 300
ClientAliveCountMax 2
# UFW default rules
- path: /etc/ufw/before.rules.lab
owner: root:root
permissions: '0644'
content: |
# Placeholder for custom UFW rules
# Applied by runcmd section below
# System limits
- path: /etc/sysctl.d/99-lab.conf
owner: root:root
permissions: '0644'
content: |
# Disable IPv6 if not needed in your lab
# net.ipv6.conf.all.disable_ipv6 = 1
# Reduce swappiness for VM workloads
vm.swappiness = 10
# Increase file descriptor limits
fs.file-max = 100000
# Network tuning
net.core.somaxconn = 1024
net.ipv4.tcp_syncookies = 1
# Bash aliases for convenience
- path: /etc/profile.d/lab-aliases.sh
owner: root:root
permissions: '0644'
content: |
alias ll='ls -lah --color=auto'
alias gs='git status'
alias df='df -h'
alias free='free -h'
# ---------------------------------------------------------------------------
# Run commands on first boot (as root, in order)
# ---------------------------------------------------------------------------
runcmd:
# Enable and start QEMU guest agent
- systemctl enable --now qemu-guest-agent
# Apply sysctl settings
- sysctl -p /etc/sysctl.d/99-lab.conf
# Configure UFW
- ufw default deny incoming
- ufw default allow outgoing
- ufw allow ssh
- ufw --force enable
# Configure fail2ban for SSH
- |
cat > /etc/fail2ban/jail.d/sshd.conf << 'EOF'
[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 5
bantime = 3600
findtime = 600
EOF
- systemctl enable --now fail2ban
# Set correct locale
- locale-gen en_US.UTF-8
- update-locale LANG=en_US.UTF-8
# Log cloud-init completion
- echo "cloud-init provisioning completed at $(date)" >> /var/log/cloud-init-lab.log
# ---------------------------------------------------------------------------
# Final message — visible in the cloud-init log (/var/log/cloud-init-output.log)
# ---------------------------------------------------------------------------
final_message: |
Cloud-init setup complete.
System: $HOSTNAME
Python version: $VERSION
Uptime: $UPTIME
Run 'sudo cloud-init status' to verify.