Main/2 Personal/Home Lab/Backup System - Kopia Server Setup.md

---
title: Backup System - Kopia Server Setup
created_date: 2026-04-20
updated_date: 2026-04-20
aliases:
tags:
---
# Backup System - Kopia Server Setup

## Overview

This document describes the setup we built for a self-hosted Kopia backup server in the homelab.

### Final architecture

- **Hypervisor:** Proxmox
- **Backup server runtime:** Debian VM on Proxmox (IP:192.168.1.54 - IP managed by pi-hole)
- **Backup repository storage:** Synology NAS via NFS
- **Backup service:** Kopia Repository Server
- **Remote/private access:** intended via Tailscale or LAN
- **First client:** MacBook
- **TLS:** self-generated Kopia TLS certificate

### Why this architecture was chosen

We first tried to run Kopia in an LXC container. That led to multiple issues:

- NFS mount permission issues with an **unprivileged LXC**
- AppArmor / systemd problems with newer Debian / systemd in LXC
- black Proxmox console / awkward LXC behavior

Because of that, we switched to a **VM**, which is the cleaner and more robust setup for this use case.

---

## Final design

### Components

- **Debian VM** runs Kopia Repository Server
- **Synology NAS** stores the actual backup repository blobs
- **Kopia clients** connect to the Repository Server over HTTPS
- **MacBook** connects as `claudio@macbook-main`

### Repository model

Important distinction:

- The **repository** itself is stored on the NAS filesystem
- The **Repository Server** is the HTTP/HTTPS layer in front of it
- Users connect to the **server**, not directly to the NAS share

This is better than mounting the share directly on each laptop because:

- clients do not depend on a mounted NAS path
- you get per-user accounts on the server
- easier multi-user setup
- easier remote use over VPN/Tailscale

---

## Synology setup

### Shared folder

A dedicated shared folder was used on Synology:

- `kopia-repository`

### NFS export

The NFS export path used was:

```bash
192.168.1.34:/volume1/kopia-repository
```

### Synology NFS settings

Recommended / used settings:

- **Privilege:** Read/Write
- **Squash:** No mapping
- **Security:** sys
- **Allow connections from non-privileged ports:** enabled if needed

Notes:

- `Squash: No mapping` is fine for a dedicated Kopia repository share.
- In the LXC attempt, permissions still failed because of UID/GID mapping issues with unprivileged containers.

---

## Failed LXC attempt and why we abandoned it

We first tried to run Kopia in a Proxmox LXC.

### What we tried

- Created an LXC
- Tried mounting NFS inside the LXC
- Then switched to mounting NFS on the Proxmox host and bind-mounting it into the LXC
- Added `/dev/net/tun` for Tailscale / ZeroTier
- Enabled `nesting=1` because of Debian/systemd issues

### Problems encountered

#### 1. Wrong mount point created via Proxmox UI

We accidentally created an extra LXC disk instead of a bind mount.

We had this wrong line:

```ini
mp0: local-lvm:vm-102-disk-1,mp=/mnt/pve/kopia-repo,size=8G
```

This was **not** a bind mount from the host.

The correct bind mount syntax was:

```ini
mp0: /mnt/pve/kopia-repo,mp=/srv/kopia-repo
```

#### 2. Console issues / black screen

The LXC booted, but the Proxmox console was black.

`pct enter` worked, but `pct console` did not work reliably.

#### 3. systemd / AppArmor issues

We saw warnings such as:

- `Systemd 257 detected. You may need to enable nesting.`
- AppArmor denials related to `userns_create`, `mount`, `journald`, `networkd`, etc.

This was partially fixed with:

```bash
pct set 102 --features nesting=1
pct restart 102
```

#### 4. NFS permission issues in unprivileged LXC

Even when the share was mounted correctly, writes failed inside the container:

```bash
touch /srv/kopia-repo/testfile
# Permission denied
```

This happened because unprivileged container root is UID-mapped and Synology/NFS did not allow writes the way we wanted.

### Conclusion

The LXC route was abandoned because it caused unnecessary complexity for a simple service.

We switched to a **Debian VM**, which is simpler and more maintainable.

---

## VM setup

### Assumptions

- Debian VM on Proxmox
- VM has network access to the Synology NAS
- NFS share exists on Synology
- Kopia repository will live on the NAS

### Base packages installed

On the VM, we installed the following:

```bash
apt update
apt install -y nfs-common curl ca-certificates gnupg nano
```

---

## NFS mount inside the VM

### Mountpoint creation

```bash
mkdir -p /srv/kopia-repo
mkdir -p /var/lib/kopia
chmod 700 /var/lib/kopia
```

### Manual test mount

```bash
mount -t nfs 192.168.1.34:/volume1/kopia-repository /srv/kopia-repo
```

### Validation commands

```bash
df -h /srv/kopia-repo
touch /srv/kopia-repo/testfile
ls -l /srv/kopia-repo/testfile
rm /srv/kopia-repo/testfile
```

At this stage, the VM approach worked properly.

### Persistent mount in `/etc/fstab`

We added:

```fstab
192.168.1.34:/volume1/kopia-repository /srv/kopia-repo nfs defaults,_netdev 0 0
```

Then tested with:

```bash
umount /srv/kopia-repo
mount -a
df -h /srv/kopia-repo
```

---

## Kopia installation on the VM

### APT repo setup

We used the official Kopia apt repository.

Commands used:

```bash
install -d -m 0755 /etc/apt/keyrings
curl -s https://kopia.io/signing-key | gpg --dearmor -o /etc/apt/keyrings/kopia-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/kopia-keyring.gpg] http://packages.kopia.io/apt/ stable main" > /etc/apt/sources.list.d/kopia.list
apt update
apt install -y kopia
```

### Verify installation

```bash
kopia --version
```

---

## Kopia repository creation

### Repository directory

```bash
mkdir -p /srv/kopia-repo/repository
```

### Create repository

We created a filesystem repository on the NFS-backed path:

```bash
kopia repository create filesystem --path=/srv/kopia-repo/repository
```

During this step, a **repository password** was chosen.

This password is critical. It encrypts the repository contents.

---

## Environment variables

We used an environment file with exported variables for passwords.

Important shell note:

- `"$VAR"` expands the variable
- `'$VAR'` does **not** expand the variable

So this is **wrong**:

```bash
--server-password='$KOPIA_SRV_PW'
```

And this is **correct**:

```bash
--server-password="$KOPIA_SRV_PW"
```

### Variables used

Example variables:

```bash
export KOPIA_REPO_PW="..."
export KOPIA_SRV_CTRL_PW="..."
export KOPIA_SRV_PW="..."
```

And for manual startup we exported:

```bash
export KOPIA_PASSWORD="$KOPIA_REPO_PW"
```

---

## First manual Kopia server start attempts

### Initial attempt without TLS

We first tried something like:

```bash
kopia server start \
  --address=0.0.0.0:51515 \
  --server-control-username=server-control \
  --server-control-password="$KOPIA_SRV_CTRL_PW" \
  --server-username=kopia \
  --server-password="$KOPIA_SRV_PW"
```

This failed with the message:

- `TLS not configured. To start server without encryption pass --insecure`

So the server did **not** actually listen on the port.

### Working manual TLS start

The working startup command was:

```bash
kopia server start \
    --tls-generate-cert \
    --tls-cert-file ~/my.cert \
    --tls-key-file ~/my.key \
    --address 0.0.0.0:51515 \
    --server-control-username control
```

Notes:

- This generated a self-signed TLS cert and key.
- After generation, future starts must **not** use `--tls-generate-cert` again.

---

## Adding the first Kopia user

We created the first Mac user with:

```bash
kopia server user add claudio@macbook-main
```

This prompts for a password for that user.

This user/password is what the Mac client uses to connect.

---

## Server refresh issue and root cause

We saw this error:

```bash
kopia server refresh ...
400 Bad Request: not connected
```

And from the Mac / KopiaUI we saw:

- `not connected to a direct repository`

### Root cause

The Repository Server was starting, but the process running under systemd was **not connected to the repository**.

This happened because:

- the repository had been created and connected as user **`cef`**
- but the systemd service was running as **root** by default
- root did not have the Kopia repository config/session

### Fix

We changed the systemd service to run as **user `cef`**.

That solved the issue.

---

## TLS certificate handling

### Move cert and key to stable location

We moved the generated files out of the home directory:

```bash
sudo mkdir -p /etc/kopia
sudo mv ~/my.cert /etc/kopia/server.cert
sudo mv ~/my.key /etc/kopia/server.key
sudo chmod 600 /etc/kopia/server.key
sudo chmod 644 /etc/kopia/server.cert
```

### Important permission fix

Because the service runs as user `cef`, that user needed access to the cert and key:

```bash
sudo chown cef:cef /etc/kopia/server.cert /etc/kopia/server.key
sudo chmod 600 /etc/kopia/server.key
sudo chmod 644 /etc/kopia/server.cert
```

---

## Environment file for systemd

We created:

```bash
sudo nano /etc/kopia-server.env
```

Contents:

```bash
KOPIA_PASSWORD=YOUR_REPOSITORY_PASSWORD
KOPIA_SRV_CTRL_PW=YOUR_SERVER_CONTROL_PASSWORD
KOPIA_SRV_PW=YOUR_WEB_UI_PASSWORD
```

Permissions:

```bash
sudo chown root:cef /etc/kopia-server.env
sudo chmod 640 /etc/kopia-server.env
```

---

## Final systemd service

We created:

```bash
sudo nano /etc/systemd/system/kopia-server.service
```

Final service:

```ini
[Unit]
Description=Kopia Repository Server
After=network-online.target remote-fs.target
Wants=network-online.target
Requires=remote-fs.target

[Service]
Type=simple
User=cef
Group=cef
EnvironmentFile=/etc/kopia-server.env
ExecStart=/usr/bin/kopia server start \
  --tls-cert-file=/etc/kopia/server.cert \
  --tls-key-file=/etc/kopia/server.key \
  --address=0.0.0.0:51515 \
  --server-control-username=control \
  --server-control-password=${KOPIA_SRV_CTRL_PW} \
  --server-username=kopia \
  --server-password=${KOPIA_SRV_PW}
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
```

### Enable and start

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now kopia-server
sudo systemctl status kopia-server --no-pager
```

### Check if listening

```bash
ss -ltnp | grep 51515
```

---

## Certificate fingerprint

Because the TLS certificate is self-generated, clients must trust it using the SHA256 fingerprint.

### Get fingerprint

On the VM:

```bash
openssl x509 -in /etc/kopia/server.cert -noout -fingerprint -sha256 | sed 's/://g' | cut -f 2 -d =
```

Save the resulting fingerprint.

---

## Refreshing server credentials

Once the service was working, refresh needed to use:

- HTTPS
- control username/password
- server certificate fingerprint

Command:

```bash
kopia server refresh \
  --address=https://127.0.0.1:51515 \
  --server-control-username=control \
  --server-control-password="$KOPIA_SRV_CTRL_PW" \
  --server-cert-fingerprint=YOUR_FINGERPRINT
```

If you get `not connected`, the server process is not connected to the repository context. Check that the service runs as the same user that has a valid Kopia repository config.

---

## MacBook setup

### Install Kopia on macOS

We used Homebrew:

```bash
brew install kopia
brew install kopiaui
```

### Connect the Mac to the Repository Server

We used the CLI first, because it is more reliable for initial connection than KopiaUI.

Command:

```bash
kopia repository connect server \
  --url=https://YOUR_VM_IP:51515 \
  --server-cert-fingerprint=YOUR_CERT_FINGERPRINT \
  --override-username=claudio \
  --override-hostname=macbook-main
```

The login password here is the password for the Kopia server user:

- `claudio@macbook-main`

### Verify connection

```bash
kopia repository status
```

---

## MacBook first test backup

### Create a test folder

```bash
mkdir -p ~/kopia-test
echo "hello kopia" > ~/kopia-test/file1.txt
date > ~/kopia-test/file2.txt
```

### Create first snapshot

```bash
kopia snapshot create ~/kopia-test
```

### List snapshots

```bash
kopia snapshot list
```

### Test restore

```bash
mkdir -p ~/kopia-restore-test
kopia restore latest ~/kopia-restore-test
ls -la ~/kopia-restore-test
```

This validates the full chain:

- VM
- NFS mount
- repository
- Kopia Repository Server
- Mac client connection
- backup
- restore

---

## Automatic backup on the Mac

### Basic idea

The recommended model is:

1. connect the Mac to the repository once
2. define backup roots
3. use `kopia snapshot create --all` on a schedule

### Suggested first backup roots

Start simple. For example:

- `~/Documents`
- `~/Desktop`
- local project folders

Do **not** immediately back up all of `~/Library`.

### Initial snapshots for real backup roots

Example:

```bash
kopia snapshot create ~/Documents
kopia snapshot create ~/Desktop
```

### Set retention / scheduling policy

Example:

```bash
kopia policy set ~/Documents \
  --snapshot-interval=12h \
  --keep-latest=14 \
  --keep-daily=14 \
  --keep-weekly=8 \
  --keep-monthly=12

kopia policy set ~/Desktop \
  --snapshot-interval=12h \
  --keep-latest=14 \
  --keep-daily=14 \
  --keep-weekly=8 \
  --keep-monthly=12
```

### Robust automatic execution with launchd

Create:

```text
~/Library/LaunchAgents/com.claudio.kopia-backup.plist
```

Contents:

```xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>com.claudio.kopia-backup</string>

    <key>ProgramArguments</key>
    <array>
      <string>/opt/homebrew/bin/kopia</string>
      <string>snapshot</string>
      <string>create</string>
      <string>--all</string>
      <string>--no-progress</string>
    </array>

    <key>StartInterval</key>
    <integer>21600</integer>

    <key>RunAtLoad</key>
    <true/>

    <key>StandardOutPath</key>
    <string>/tmp/kopia-backup.out</string>

    <key>StandardErrorPath</key>
    <string>/tmp/kopia-backup.err</string>
  </dict>
</plist>
```

Load it:

```bash
launchctl unload ~/Library/LaunchAgents/com.claudio.kopia-backup.plist 2>/dev/null || true
launchctl load ~/Library/LaunchAgents/com.claudio.kopia-backup.plist
launchctl list | grep kopia
```

This runs every 6 hours (`21600` seconds).

### Verify automatic backup

```bash
kopia snapshot list
```

And occasionally:

```bash
kopia restore latest ~/kopia-restore-test-2
```

---

## How to add another user, for example your spouse

The model is:

- create a new user on the Kopia server
- connect that user’s machine with a fixed username/hostname identity
- create backup sources from that machine

### Example: spouse on Windows

Suppose you want:

- username: `partner`
- hostname: `windows-laptop`

### 1. Create the user on the Kopia server VM

On the VM:

```bash
kopia server user add partner@windows-laptop
```

Set a password when prompted.

### 2. Refresh credentials if needed

```bash
kopia server refresh \
  --address=https://127.0.0.1:51515 \
  --server-control-username=control \
  --server-control-password="$KOPIA_SRV_CTRL_PW" \
  --server-cert-fingerprint=YOUR_FINGERPRINT
```

### 3. Install Kopia on the spouse’s machine

For Windows, install Kopia / KopiaUI from the official installer.

### 4. Connect that machine to the server

From CLI, the equivalent pattern is:

```bash
kopia repository connect server \
  --url=https://YOUR_VM_IP:51515 \
  --server-cert-fingerprint=YOUR_CERT_FINGERPRINT \
  --override-username=partner \
  --override-hostname=windows-laptop
```

Then enter the password for:

- `partner@windows-laptop`

### 5. Create first backup sources on that machine

For example on Windows:

- Documents
- Desktop
- Pictures

### 6. Run first backup and test restore

Do exactly the same test pattern as on the Mac:

- create first snapshot
- list snapshots
- restore a test folder

### Naming convention recommendation

Use stable names so future maintenance is simple.

Examples:

- `claudio@macbook-main`
- `claudio@windows-main`
- `partner@windows-laptop`
- `partner@android-photos` if you ever use a separate flow later

---

## Maintenance and troubleshooting

### Check service status

```bash
sudo systemctl status kopia-server --no-pager
```

### View Kopia server logs

```bash
sudo journalctl -u kopia-server -n 100 --no-pager
```

### Check if port is listening

```bash
ss -ltnp | grep 51515
```

### Check repository status as service user

```bash
kopia repository status
```

If this fails under the service user context, the Repository Server will not work correctly.

### Test local HTTPS endpoint

```bash
curl -k https://127.0.0.1:51515/
```

### Get certificate fingerprint again

```bash
openssl x509 -in /etc/kopia/server.cert -noout -fingerprint -sha256 | sed 's/://g' | cut -f 2 -d =
```

### Common failure: wrong shell quoting

Wrong:

```bash
--server-password='$KOPIA_SRV_PW'
```

Correct:

```bash
--server-password="$KOPIA_SRV_PW"
```

### Common failure: server not connected

Symptoms:

- `400 Bad Request: not connected`
- Mac / UI error: `not connected to a direct repository`

Fix:

- make sure systemd service runs as the same user that created / connected the repository
- in this setup that was `cef`

### Common failure: browser works but Mac client fails

This usually means:

- HTTPS listener is fine
- web auth is fine
- but the server has no repository connection or client trust settings are wrong

Check:

- `kopia repository status`
- systemd service user
- cert fingerprint on client

---

## Important files in this setup

### On the VM

- Repository storage:
  - `/srv/kopia-repo/repository`
- NFS mountpoint:
  - `/srv/kopia-repo`
- TLS cert:
  - `/etc/kopia/server.cert`
- TLS key:
  - `/etc/kopia/server.key`
- systemd env file:
  - `/etc/kopia-server.env`
- systemd service:
  - `/etc/systemd/system/kopia-server.service`

### On the Mac

- launchd job:
  - `~/Library/LaunchAgents/com.claudio.kopia-backup.plist`

---

## What not to forget later

- Do not rerun `--tls-generate-cert` on every start.
- Keep the repository password safe.
- Keep the server control password safe.
- Keep the Web UI password safe.
- Keep the certificate fingerprint documented.
- Test restores periodically, not just backups.
- Do not assume browser access means repository connectivity is correct.

---

## Suggested next steps

1. Finalize the Mac backup roots
2. Test automatic backups from the Mac
3. Add spouse’s machine as a second user
4. Test restore from spouse’s machine too
5. Later consider offsite replication of the Kopia repository
6. Keep Time Machine in parallel on the Mac if you want easier full-machine restore