Backup & restore

The recommended path (v0.6.3+): use the in-app portable backup

Settings → Backup & restore → Download backup. Produces a single AES-256-GCM encrypted .csso-backup file (PBKDF2-SHA256, 150k iters, salt per bundle) containing everything you need to bring a fresh manager to a full clone of this one: master.key, manager.uuid, msmtprc, all admin users, all paired PBXes with their encrypted SSH + HMAC secrets, notes, hardware nodes, custom commands, notification config.

To restore on a new manager: install customsso-manager on the target host, log in as the initial admin, Settings → Backup & restore → Restore, upload the bundle, enter the passphrase. Auto-reclaim runs automatically — for every paired PBX, the destination manager SSHes in with the shipped keys and installs fresh keypair + HMAC secret pointing at itself. Source manager loses access as each PBX flips over. See "Auto-reclaim" section below for what happens when a PBX is unreachable during restore.

The mysqldump-based procedures documented later on this page are still valid and useful for cron-scheduled backups where the passphrase requirement (interactive) doesn't fit.

Portable backup — the .csso-backup bundle

Passphrase policy

Minimum 20 characters (raised from 12 in v0.6.4). The bundle contains the manager master.key plus every paired PBX's ssh_privkey + hmac_secret; a leaked bundle with a weak passphrase is GPU-crackable in hours → full-fleet takeover. Use a diceware phrase (e.g. correct-horse-battery-staple-vault) rather than trying to satisfy a complexity policy.

What's inside

• files/ — master.key, manager.uuid, msmtprc
• data/ — users.json, pbxes.json (secret _ct columns base64-encoded), pbx_notes.json, custom_commands.json, manager_config.json, nodes.json, trusted_devices.json
• metadata.json — bundle format version, source manager UUID + hostname, creation timestamp, description

Restore is whole-manager replace

Restore empties every restore-target table before inserting the bundle's contents. There is no merge. Applied to: users, pbxes, pbx_notes, custom_commands, manager_config, nodes, trusted_devices. Any pbx or user that exists on the destination but NOT in the bundle is gone after restore. If you want to preserve local state, take a fresh backup of the destination FIRST — that's your rollback path.

The restore UI shows an explicit red danger-zone panel listing exactly what gets wiped before requiring you to type RESTORE to confirm.

Auto-reclaim during restore

After DB apply, Backup::apply() calls Reclaim::takeoverAll(). For every pbxes row:

1. Decrypt ssh_privkey_ct using the restored master.key.
2. SSH into the PBX using that key (host-key pinning: pinned from the ssh_host_key column shipped in the bundle; TOFU-captured on first connect if the row predates that column).
3. Generate a fresh ed25519 keypair + fresh HMAC secret on the destination.
4. Atomically rewrite the PBX's /root/.ssh/authorized_keys (new pubkey, permissive — no from= restriction, per the C2 audit fix), rewrite /etc/customsso/pbx.manifest.json + /etc/customsso/agent.conf to point at the destination manager, with the new HMAC secret.
5. Verify with a fresh SSH session using the new key.
6. Update the destination's pbxes row with new material + rotation timestamps.

Source manager loses access as each PBX flips over. Any PBX that can't be reached (network down, PBX off, firewall) enters the soft-pair state (described next). Any PBX that CAN be reached but auth fails (legacy restrictive from= from a pre-v0.6.4 pair still blocking a new manager IP) also enters soft-pair state.

Soft-pair state

When auto-reclaim fails for a PBX, the pbxes row is preserved (label, base_url, ssh_host, hardware node, notes) but the cryptographic material is wiped: ssh_privkey_ct, hmac_secret_ct, ssh_pubkey, ssh_pubkey_fp are all set to NULL. The pbxes.status column goes to offline.

The manager UI surfaces this state via: - Settings tab bar — a red "Unpaired ⚠ (N)" chip appears when at least one row is in soft-pair state. - Global warning banner on every page — count of unpaired PBXes with a link to the recovery wizard. - Settings → Reclaim wizard — lists every unpaired row with a reachability check (TCP connect to ssh_host:ssh_port) + a Re-pair button when reachable.

Re-pair is a 1-field form (root password). Everything else pre-populated from the row. On submit, the sidecar's SSHProvisioner runs the standard pair flow with force_repair=true + the existing pbx_uuid, so the DB row is UPDATEd in place (uuid preserved, notes preserved). See PAIRING.md.

Rationale for wiping instead of keeping the failed keys: the bundle's keys have been sitting in a workstation-side .csso-backup file since backup time — they're potentially compromised. Keeping them for indefinite retry defeats the point of key rotation. Failed reclaims require an explicit Re-pair with root password before the PBX is trusted again.

Tar hardening (H6 audit fix)

Bundle upload is inspected before extraction — any entry that's a symlink, hardlink, or non-regular file, or contains .. or an absolute path, rejects the entire bundle. Extraction uses --no-same-owner --no-same-permissions --no-overwrite-dir. A malicious bundle can't symlink files/master.key → /etc/shadow to trick the follow-up atomicReplace into overwriting or exfiltrating protected paths.

Legacy mysqldump-based procedures

The .csso-backup bundle path above is the recommended one. Below are the older mysqldump-based procedures — still valid, useful when you want a non-interactive cron backup.

⚠ The single most catastrophic loss vector

/etc/customsso-manager/master.key decrypts every secret this manager holds — every paired PBX's HMAC secret and SSH private key. Lose the master key and the entire fleet needs to be unpaired + re-paired from scratch, and any active manager→PBX operation dies until then.

Any backup that captures the MariaDB database WITHOUT also capturing this file is worse than useless — you'll have encrypted junk plus a false sense of security. Every backup procedure below captures both together.

The customsso-manager state lives in three places: a MariaDB database, a master encryption key file, and DB credentials file. Lose any one and you can recover. Lose them together and the encrypted secrets are gone forever — every paired PBX needs to be revoked + re-paired from scratch.

Also to consider — additional install-time state that survives on the box:

• /etc/customsso-manager/msmtprc — SMTP relay credentials (plaintext password inside). Back up if you don't want to re-enter after restore.
• /etc/customsso-manager/first-run.token — DELETE-safely; only useful pre-setup. Post-setup its presence is a hygiene issue, not a state loss.
• /etc/customsso-manager/VERSION — one line, easy to regenerate.
• /etc/php.d/99-customsso-timezone.ini + /etc/my.cnf.d/customsso.cnf — required for timestamp arithmetic to behave. Regenerated by install.sh --update if missing.

What to back up

Backup procedures

Manual one-shot

ssh root@your-manager-host

# 1. Dump the manager database (encrypted secrets included, but useless without master.key)
DEST=/var/backups/customsso-manager-$(date +%Y%m%d-%H%M%S).tar.gz
. /etc/customsso-manager/db.creds
mysqldump -u "$DB_USER" -p"$DB_PASS" "$DB_NAME" > /tmp/csm-dump.sql

# 2. Bundle the secrets + config + dump
tar -czf "$DEST" \
    -C / etc/customsso-manager \
    -C /tmp csm-dump.sql

rm /tmp/csm-dump.sql
chmod 600 "$DEST"

echo "Backup written to $DEST — copy off-host"

The resulting .tar.gz contains the master key in plaintext. Copy it off-host immediately to encrypted storage (Restic to an encrypted target, BorgBackup with a passphrase, USB key in a safe — anything where leaving it on the manager defeats the purpose).

Scheduled (cron)

# /etc/cron.d/customsso-manager-backup
MAILTO=""
17 4 * * * root /usr/local/sbin/customsso-backup.sh >> /var/log/customsso-manager/backup.log 2>&1

With /usr/local/sbin/customsso-backup.sh:

#!/usr/bin/env bash
set -euo pipefail

BACKUP_DIR=/var/backups/customsso-manager
mkdir -p "$BACKUP_DIR"
DEST="$BACKUP_DIR/csm-$(date +%Y%m%d-%H%M%S).tar.gz"

. /etc/customsso-manager/db.creds
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT

mysqldump -u "$DB_USER" -p"$DB_PASS" "$DB_NAME" > "$TMP/db.sql"
tar -czf "$DEST" -C / etc/customsso-manager -C "$TMP" db.sql
chmod 600 "$DEST"

# Retain 30 days of local backups; off-host copy is your responsibility
find "$BACKUP_DIR" -type f -name 'csm-*.tar.gz' -mtime +30 -delete

# Off-host copy — pick one:
# rclone copy "$DEST" remote:csm-backups/
# rsync -av --remove-source-files "$DEST" user@backup-host:/srv/customsso/
# aws s3 cp "$DEST" s3://your-bucket/customsso/

Master-key offline escrow

For disaster recovery, keep the master key in a place that survives a full manager loss:

The key is exactly 32 bytes. To print as hex:

xxd -p -c 32 /etc/customsso-manager/master.key
# 65f3a8d0e7b1c9520f4e8d6b3c1a7f2d8e4b0c9a5f3e2d1c8b7a6f5e4d3c2b1a

Restore procedures

Restore the master key after disaster

If you have the master key file (from any backup) and lost the rest:

# Fresh OS install on the new manager host
./install.sh --listen-host sso.yourdomain.tld --allow-unencrypted    # whatever flags fit
# At first-admin prompt: pick anything (you'll re-pair anyway)

# Stop services
systemctl stop customsso-sidecar httpd

# Restore the master key
mv /etc/customsso-manager/master.key /etc/customsso-manager/master.key.fresh-install
cp /your/backup/master.key /etc/customsso-manager/master.key
chown apache:apache /etc/customsso-manager/master.key
chmod 600 /etc/customsso-manager/master.key

# Restore the DB
. /etc/customsso-manager/db.creds
mysql -u "$DB_USER" -p"$DB_PASS" "$DB_NAME" < /your/backup/db.sql

# Bring services back
systemctl start httpd customsso-sidecar

Log in with whatever admin user is in the restored DB. Every paired PBX should still work because the per-PBX secrets stored in the encrypted columns now decrypt with the restored master key.

Restore from a full bundle (.tar.gz)

# Fresh install
./install.sh --listen-host sso.yourdomain.tld --admin-user admin --admin-pass temp123456789

systemctl stop customsso-sidecar httpd

# Extract the backup over the install
tar -xzf /your/backup/csm-20260626-041700.tar.gz -C /tmp
cp /tmp/etc/customsso-manager/* /etc/customsso-manager/
chown apache:apache /etc/customsso-manager/master.key /etc/customsso-manager/db.creds
chmod 600 /etc/customsso-manager/{master.key,db.creds}

# Re-create DB user with the restored password
. /etc/customsso-manager/db.creds
mysql -uroot -e "DROP USER IF EXISTS '$DB_USER'@'localhost'; CREATE USER '$DB_USER'@'localhost' IDENTIFIED BY '$DB_PASS'; GRANT ALL PRIVILEGES ON \`$DB_NAME\`.* TO '$DB_USER'@'localhost'; FLUSH PRIVILEGES;"
mysql -u "$DB_USER" -p"$DB_PASS" "$DB_NAME" < /tmp/csm-dump.sql

rm -rf /tmp/etc /tmp/csm-dump.sql

systemctl start httpd customsso-sidecar

Restore only the DB (master key intact, DB corrupted)

systemctl stop customsso-sidecar
. /etc/customsso-manager/db.creds
mysql -u "$DB_USER" -p"$DB_PASS" "$DB_NAME" < /your/backup/db.sql
systemctl start customsso-sidecar

What can NOT be recovered

Recommended cadence