Backup & restore

⚠ The single most catastrophic loss vector

/etc/customsso-manager/master.key decrypts every secret this manager holds — every paired PBX's HMAC secret and SSH private key. Lose the master key and the entire fleet needs to be unpaired + re-paired from scratch, and any active manager→PBX operation dies until then.

Any backup that captures the MariaDB database WITHOUT also capturing this file is worse than useless — you'll have encrypted junk plus a false sense of security. Every backup procedure below captures both together.

The customsso-manager state lives in three places: a MariaDB database, a master encryption key file, and DB credentials file. Lose any one and you can recover. Lose them together and the encrypted secrets are gone forever — every paired PBX needs to be revoked + re-paired from scratch.

Also to consider — additional install-time state that survives on the box:

/etc/customsso-manager/msmtprc — SMTP relay credentials (plaintext password inside). Back up if you don't want to re-enter after restore.
/etc/customsso-manager/first-run.token — DELETE-safely; only useful pre-setup. Post-setup its presence is a hygiene issue, not a state loss.
/etc/customsso-manager/VERSION — one line, easy to regenerate.
/etc/php.d/99-customsso-timezone.ini + /etc/my.cnf.d/customsso.cnf — required for timestamp arithmetic to behave. Regenerated by install.sh --update if missing.

What to back up

Item	Path	Without it…
Master AES-256-GCM key	`/etc/customsso-manager/master.key` (32 raw bytes)	DB ciphertext columns are unreadable — every PBX needs re-pairing
DB credentials	`/etc/customsso-manager/db.creds` (text: `DB_NAME=… / DB_USER=… / DB_PASS=…`)	Can be regenerated from MariaDB privileges, but easier to back up
MariaDB database	`customsso_manager` schema (default name; check `db.creds`)	All pairings, audit log, sessions, jobs, users lost
Self-signed TLS cert (optional)	`/etc/httpd/customsso-manager-tls/manager.{crt,key}`	Browser TLS warnings; regenerate via install.sh
PBX-side agent state	`customsso_*` MySQL tables on each PBX	Per-PBX agent state — but FreePBX's own backup catches the `asterisk` DB

Backup procedures

Manual one-shot

ssh root@your-manager-host

# 1. Dump the manager database (encrypted secrets included, but useless without master.key)
DEST=/var/backups/customsso-manager-$(date +%Y%m%d-%H%M%S).tar.gz
. /etc/customsso-manager/db.creds
mysqldump -u "$DB_USER" -p"$DB_PASS" "$DB_NAME" > /tmp/csm-dump.sql

# 2. Bundle the secrets + config + dump
tar -czf "$DEST" \
    -C / etc/customsso-manager \
    -C /tmp csm-dump.sql

rm /tmp/csm-dump.sql
chmod 600 "$DEST"

echo "Backup written to $DEST — copy off-host"

The resulting .tar.gz contains the master key in plaintext. Copy it off-host immediately to encrypted storage (Restic to an encrypted target, BorgBackup with a passphrase, USB key in a safe — anything where leaving it on the manager defeats the purpose).

Scheduled (cron)

# /etc/cron.d/customsso-manager-backup
MAILTO=""
17 4 * * * root /usr/local/sbin/customsso-backup.sh >> /var/log/customsso-manager/backup.log 2>&1

With /usr/local/sbin/customsso-backup.sh:

#!/usr/bin/env bash
set -euo pipefail

BACKUP_DIR=/var/backups/customsso-manager
mkdir -p "$BACKUP_DIR"
DEST="$BACKUP_DIR/csm-$(date +%Y%m%d-%H%M%S).tar.gz"

. /etc/customsso-manager/db.creds
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT

mysqldump -u "$DB_USER" -p"$DB_PASS" "$DB_NAME" > "$TMP/db.sql"
tar -czf "$DEST" -C / etc/customsso-manager -C "$TMP" db.sql
chmod 600 "$DEST"

# Retain 30 days of local backups; off-host copy is your responsibility
find "$BACKUP_DIR" -type f -name 'csm-*.tar.gz' -mtime +30 -delete

# Off-host copy — pick one:
# rclone copy "$DEST" remote:csm-backups/
# rsync -av --remove-source-files "$DEST" user@backup-host:/srv/customsso/
# aws s3 cp "$DEST" s3://your-bucket/customsso/

Master-key offline escrow

For disaster recovery, keep the master key in a place that survives a full manager loss:

Method	Why it works
Printed QR code in a safe	Air-gapped, eyeball-verifiable, indefinite shelf life
Encrypted password manager (1Password / Bitwarden / KeePass)	Multi-device sync; the manager's own encryption protects it
Two trusted humans, each holding half (Shamir Secret Sharing)	M-of-N threshold; resists single-person compromise
USB key in a bank safe deposit box	Physical security; consider M-of-N with multiple keys

The key is exactly 32 bytes. To print as hex:

xxd -p -c 32 /etc/customsso-manager/master.key
# 65f3a8d0e7b1c9520f4e8d6b3c1a7f2d8e4b0c9a5f3e2d1c8b7a6f5e4d3c2b1a

Restore procedures

Restore the master key after disaster

If you have the master key file (from any backup) and lost the rest:

# Fresh OS install on the new manager host
./install.sh --listen-host sso.yourdomain.tld --allow-unencrypted    # whatever flags fit
# At first-admin prompt: pick anything (you'll re-pair anyway)

# Stop services
systemctl stop customsso-sidecar httpd

# Restore the master key
mv /etc/customsso-manager/master.key /etc/customsso-manager/master.key.fresh-install
cp /your/backup/master.key /etc/customsso-manager/master.key
chown apache:apache /etc/customsso-manager/master.key
chmod 600 /etc/customsso-manager/master.key

# Restore the DB
. /etc/customsso-manager/db.creds
mysql -u "$DB_USER" -p"$DB_PASS" "$DB_NAME" < /your/backup/db.sql

# Bring services back
systemctl start httpd customsso-sidecar

Log in with whatever admin user is in the restored DB. Every paired PBX should still work because the per-PBX secrets stored in the encrypted columns now decrypt with the restored master key.

Restore from a full bundle (.tar.gz)

# Fresh install
./install.sh --listen-host sso.yourdomain.tld --admin-user admin --admin-pass temp123456789

systemctl stop customsso-sidecar httpd

# Extract the backup over the install
tar -xzf /your/backup/csm-20260626-041700.tar.gz -C /tmp
cp /tmp/etc/customsso-manager/* /etc/customsso-manager/
chown apache:apache /etc/customsso-manager/master.key /etc/customsso-manager/db.creds
chmod 600 /etc/customsso-manager/{master.key,db.creds}

# Re-create DB user with the restored password
. /etc/customsso-manager/db.creds
mysql -uroot -e "DROP USER IF EXISTS '$DB_USER'@'localhost'; CREATE USER '$DB_USER'@'localhost' IDENTIFIED BY '$DB_PASS'; GRANT ALL PRIVILEGES ON \`$DB_NAME\`.* TO '$DB_USER'@'localhost'; FLUSH PRIVILEGES;"
mysql -u "$DB_USER" -p"$DB_PASS" "$DB_NAME" < /tmp/csm-dump.sql

rm -rf /tmp/etc /tmp/csm-dump.sql

systemctl start httpd customsso-sidecar

Restore only the DB (master key intact, DB corrupted)

systemctl stop customsso-sidecar
. /etc/customsso-manager/db.creds
mysql -u "$DB_USER" -p"$DB_PASS" "$DB_NAME" < /your/backup/db.sql
systemctl start customsso-sidecar

What can NOT be recovered

Scenario	Status
Master key lost + DB intact	Unrecoverable — encrypted columns are random bytes. Reinstall the manager, revoke + re-pair every PBX from each PBX's agent UI.
Master key intact + DB lost (no backup)	Recoverable manually — re-pair every PBX. The agent module on each PBX is independent.
Both lost	Fully unrecoverable — fresh manager, every PBX needs revoke + re-pair via each PBX's admin UI.
Audit log lost	No recovery path — log is append-only state, not regenerable. Off-host audit shipping (syslog, etc.) is the way to survive this.

Recommended cadence

What	Frequency	Storage
Full bundle (DB + secrets)	Daily	30 days local, 90 days off-host
Master key (separately, immutable)	After any change (install, rotation)	Indefinite, multiple secure locations
Test restore	Quarterly	Spin up a fresh OL VM, restore latest backup, log in, verify PBX list matches