Update Homelab Todo List; add Kopia Backup Setup

02_Projects/Homelab Todo List.md

@@ -31,7 +31,14 @@ Curated list of open homelab work across task history, memory, and second-brain
   - Blocker: hardware purchase decision
 - [ ] Define regular backup flow to parents' NAS
   - Depends on: backup matrix + parents' NAS target design
-- [ ] Set up Kopia/Time Machine backup for Claudio's and Alena's machines
+- [x] Set up Kopia/Time Machine backup for Claudio's and Alena's machines
+  - Kopia server running on Debian VM (Proxmox), Synology NFS backend
+  - MacBook connected as `claudio@macbook-main`
+  - launchd job running `kopia snapshot create --all` every 6h
+  - Documented in `05_Resources/Kopia Backup Setup.md`
+- [ ] Set up weekly `kopia snapshot verify --verify-files-percent=100` cron with failure alert
+- [ ] Set up weekly full restore test cron job
+- [ ] Add Alena's machine as second Kopia user
   - Next action: choose destination and retention policy
 
 ### 2. Simplify the homelab and hosting architecture

05_Resources/Kopia Backup Setup.md

@@ -0,0 +1,146 @@
+# Kopia Backup Setup
+
+> Last updated: 2026-04-20
+
+## Architecture
+
+- **Hypervisor:** Proxmox (MacBook Pro 2017 Intel)
+- **Backup server runtime:** Debian VM on Proxmox
+- **Repository storage:** Synology NAS via NFS (`192.168.1.34:/volume1/kopia-repository`)
+- **Service:** Kopia Repository Server
+- **Access:** Tailscale / LAN
+- **TLS:** self-signed cert at `/etc/kopia/server.cert` + `/etc/kopia/server.key`
+- **First client:** MacBook (claudio@macbook-main)
+
+## Repository model
+
+The repository blobs live on Synology NFS. The Kopia Repository Server acts as the HTTP/S layer in front of it. Clients connect to the server, not directly to the NAS share.
+
+## Important paths (VM)
+
+| Purpose | Path |
+|---------|------|
+| NFS mountpoint | `/srv/kopia-repo` |
+| Repository blobs | `/srv/kopia-repo/repository` |
+| TLS cert | `/etc/kopia/server.cert` |
+| TLS key | `/etc/kopia/server.key` |
+| Env vars | `/etc/kopia-server.env` |
+| Systemd service | `/etc/systemd/system/kopia-server.service` |
+
+## Users
+
+| User | Identity | Machine |
+|------|----------|---------|
+| `claudio` | `claudio@macbook-main` | MacBook |
+
+## Passwords (stored in pass)
+
+All three passwords are stored in `pass`:
+- `KOPIA_REPO_PW` — repository encryption password
+- `KOPIA_SRV_CTRL_PW` — server control plane password
+- `KOPIA_SRV_PW` — web UI login password
+
+## How to get the cert fingerprint
+
+```bash
+openssl x509 -in /etc/kopia/server.cert -noout -fingerprint -sha256 | sed 's/://g' | cut -f 2 -d =
+```
+
+## Server commands
+
+```bash
+# Status
+sudo systemctl status kopia-server --no-pager
+
+# Logs
+sudo journalctl -u kopia-server -n 100 --no-pager
+
+# Check listening
+ss -ltnp | grep 51515
+
+# Refresh credentials (after adding users, etc.)
+kopia server refresh \
+ --address=https://127.0.0.1:51515 \
+ --server-control-username=control \
+ --server-control-password="$KOPIA_SRV_CTRL_PW" \
+ --server-cert-fingerprint=YOUR_FINGERPRINT
+```
+
+## MacBook client commands
+
+```bash
+# Repository status
+kopia repository status
+
+# List snapshots
+kopia snapshot list
+
+# Manual snapshot
+kopia snapshot create --all
+
+# Test restore
+mkdir -p ~/kopia-restore-test
+kopia restore latest ~/kopia-restore-test
+```
+
+## Automatic backups (launchd)
+
+A launchd job at `~/Library/LaunchAgents/com.claudio.kopia-backup.plist` runs `kopia snapshot create --all` every 6 hours.
+
+To reload after editing:
+```bash
+launchctl unload ~/Library/LaunchAgents/com.claudio.kopia-backup.plist 2>/dev/null || true
+launchctl load ~/Library/LaunchAgents/com.claudio.kopia-backup.plist
+launchctl list | grep kopia
+```
+
+## Known failure modes
+
+1. **"not connected to a direct repository"** on Mac — server is running but not connected to repo. Fix: make sure systemd service runs as the same user (`cef`) that created the repository.
+
+2. **`400 Bad Request: not connected`** on refresh — same root cause as above. Check `kopia repository status` as the service user.
+
+3. **Browser works but Mac client fails** — usually cert fingerprint mismatch or HTTPS listener issue. Check fingerprint on client matches server.
+
+4. **Shell quoting bug** — always use `"$VAR"` not `'$VAR'` in kopia commands.
+
+## Maintenance
+
+- `kopia repository status` — verify repo integrity
+- `kopia snapshot verify --verify-files-percent=100 --file-parallelism=10 --parallel=10` — full consistency check (runs weekly via cron)
+- Weekly restore test via cron job (see HEARTBEAT/backup task)
+- Certificate fingerprint must be re-entered after cert rotation
+
+## Adding a new user
+
+On the VM:
+```bash
+kopia server user add partner@hostname
+# set password when prompted
+kopia server refresh \
+ --address=https://127.0.0.1:51515 \
+ --server-control-username=control \
+ --server-control-password="$KOPIA_SRV_CTRL_PW" \
+ --server-cert-fingerprint=YOUR_FINGERPRINT
+```
+
+From the new client machine:
+```bash
+kopia repository connect server \
+ --url=https://YOUR_VM_IP:51515 \
+ --server-cert-fingerprint=YOUR_FINGERPRINT \
+ --override-username=partner \
+ --override-hostname=hostname
+```
+
+## Next steps
+
+- [ ] Finalize Mac backup roots (Documents, Desktop, project folders — avoid full ~/Library initially)
+- [ ] Set retention policies on real backup roots
+- [ ] Test automatic backups from the Mac
+- [ ] Add spouse's machine as second user
+- [ ] Test restore from spouse's machine
+- [ ] Weekly `kopia snapshot verify --verify-files-percent=100` cron job with failure alert
+- [ ] Weekly full restore test cron job
+- [ ] Off-site replication of the Kopia repository (parents' NAS?)
+- [ ] Keep Time Machine in parallel for full-machine restore