# Backup Review — 2026-07-14

## Current State: What's Backed Up Daily

### Z840 → TrueNAS (Hetzner backup script at 02:00) - ✅ WordPress sites (`/home/*/public_html/` — 31GB) - ✅ MySQL databases (all databases, compressed) - ✅ LiteSpeed/OLS vhost configs (`/usr/local/lsws/conf/` — 1.7MB, 56 domains) - ✅ SSH keys (`/root/.ssh/`) - ✅ Secrets (`/root/.secrets/`) - ✅ SSL certificates (`/root/.acme.sh/`). - ✅ Custom scripts (`/usr/local/bin/`) - ✅ Fail2ban config (`/etc/fail2ban/`) - ✅ Postfix config (`/etc/postfix/`) - ✅ Email/mailboxes (`/home/vmail/` — 2GB, rsync with –ignore-errors) - ✅ Monthly full snapshot (1st of month to separate dataset)

### Other Backups (already in place) - ✅ Z840 → TrueNAS (Docker volumes, Ollama configs, wiki) - ✅ Ubuntu-svr → TrueNAS (Frigate, Omada, configs) - ✅ OPNsense → TrueNAS (firewall configs, daily at 05:00)

## Gaps: What's NOT Backed Up

### 1. CyberPanel Application (246MB) Location: `/usr/local/CyberPanel/` Status: Not in daily backup Risk: If Hetzner dies, we lose the CyberPanel application code. Reinstalling takes ~10min but we'd lose custom configs and database relationships. Fix: Add to backup script: ```bash rsync -az –delete /usr/local/CyberPanel/ root@$TRUENAS_IP:$TRUENAS_PATH/cyberpanel-app/ ```

### 2. CyberPanel Configs Location: `/etc/cyberpanel/` Status: Not in daily backup Contains: adminPass, mysqlPassword, webmail.conf, apikeys Risk: Without these, we can't access CyberPanel after restore without manual password reset. Fix: Add to backup script: ```bash rsync -az –delete /etc/cyberpanel/ root@$TRUENAS_IP:$TRUENAS_PATH/cyberpanel-config/ ```

### 3. Docker Compose Files & Volumes Location: `/opt/searxng/`, `/opt/rustdesk/`, `/opt/ollama/` (if exists) Status: Not backed up Containers running: - searxng + valkey (search engine) - rustdesk-hbbs/hbbr (remote desktop relay) - uptime-kuma-hetzner (monitoring)

Risk: If Hetzner dies, we lose: - SearXNG search engine configs and Valkey cache - RustDesk relay keys (clients would need reconfiguration) - Uptime Kuma monitoring history

Fix: Add Docker directory to backup: ```bash rsync -az –delete /opt/ root@$TRUENAS_IP:$TRUENAS_PATH/opt-docker/ ```

Or more granular: ```bash for dir in /opt/searxng /opt/rustdesk /opt/ollama; do

[ -d "$dir" ] && rsync -az --delete "$dir/" root@$TRUENAS_IP:$TRUENAS_PATH/docker/$(basename $dir)/

done ```

### 4. System Packages (apt list) Location: `dpkg –get-selections` Status: Not backed up Risk: After OS reinstall, we'd need to manually reinstall all packages (Postfix, Dovecot, Fail2ban, LiteSpeed, etc.) Fix: Add package list to backup: ```bash dpkg –get-selections > /root/package-selections.txt rsync -az /root/package-selections.txt root@$TRUENAS_IP:$TRUENAS_PATH/system-packages/ ```

## Recovery Time Estimates

### Scenario: Complete Z840 loss, rebuild from scratch

Current backup coverage (without gaps): 1. Install Ubuntu 24.04 (10min) 2. Restore system packages (5min) 3. Restore SSH keys, secrets (2min) 4. Restore CyberPanel app + configs (15min with manual setup) 5. Restore WordPress sites from `/home/` backup (30min for 56 domains) 6. Restore MySQL databases (10min) 7. Restore OLS vhost configs (5min) 8. Restore email/mailboxes (20min for 2GB) 9. Restore Docker containers from compose files (30min) 10. Update DNS to point to new server IP (TTL propagation: 1hr-24hr)

Total: 2-4 hours manual work + DNS propagation

With current gaps filled: - Steps 4, 9, 10 become automated or faster - Total: 1-2 hours manual work + DNS propagation

## Recommendations

### Priority 1: Add CyberPanel to Backup (15min task) Edit `/root/backup.sh` on Z840, add after line 105 (after OLS config sync): ```bash # Sync CyberPanel application log_msg “Syncing CyberPanel application…” rsync -az –delete /usr/local/CyberPanel/ root@$TRUENAS_IP:$TRUENAS_PATH/cyberpanel-app/ » $LOG_FILE 2>&1 CYBERPANEL_APP_SYNC=$?

# Sync CyberPanel configs log_msg “Syncing CyberPanel configs…” rsync -az –delete /etc/cyberpanel/ root@$TRUENAS_IP:$TRUENAS_PATH/cyberpanel-config/ » $LOG_FILE 2>&1 CYBERPANEL_CONFIG_SYNC=$? ```

Update the verification gate (line 204) to include these: ```bash if [ $HOME_SYNC -eq 0 ] && [ $DB_SYNC -eq 0 ] && [ $OLS_SYNC -eq 0 ] &&

 [ $CYBERPANEL_APP_SYNC -eq 0 ] && [ $CYBERPANEL_CONFIG_SYNC -eq 0 ]; then

```

### Priority 2: Add Docker/Opt Directory (15min task) Add after CyberPanel sync: ```bash # Sync Docker configs and data log_msg “Syncing Docker configs…” for dir in /opt/searxng /opt/rustdesk; do

if [ -d "$dir" ]; then
  rsync -az --delete "$dir/" root@$TRUENAS_IP:$TRUENAS_PATH/docker/$(basename $dir)/ >> $LOG_FILE 2>&1
fi

done ```

### Priority 3: Add Package List (5min task) Add at start of backup script: ```bash # Save package selection list log_msg “Saving package list…” dpkg –get-selections > /root/package-selections.txt rsync -az /root/package-selections.txt root@$TRUENAS_IP:$TRUENAS_PATH/system-packages/ ```

### Priority 4: Test Recovery Procedure (2hr task) Document step-by-step recovery in `/home/kaburu/wiki/wiki/procedures/server-recovery.md`: 1. Fresh Ubuntu install 2. Package restore 3. CyberPanel restore 4. Site restore 5. Email restore 6. Docker restore 7. DNS cutover

Action: Schedule a quarterly recovery test (restore to temporary VM, verify everything works, document issues).

## Summary

Current coverage: 90% of critical data is backed up. Biggest gap: CyberPanel application + Docker state. Recovery time with gaps: 2-4 hours + DNS. Recovery time after fixes: 1-2 hours + DNS.

Recommendation: Implement Priority 1 and 2 today (30min total). Test recovery next week.