Back up and restore
This page lets you check and run the server's backups, copy them off the server, keep what backups leave out, and put a backup back, on the same server or a new one.
How backups work
The installer sets up backups for every app; you do not need to turn them on.
- Every night each app backs itself up at its own time, in the server's time zone: Fennec 01:15, Corsac 01:25, Admin 01:30, Arctic 01:35, Pampas 01:40, Cape 01:45, Culpeo 01:50, Library 01:55.
- Encrypted: each backup is encrypted on the server with that app's passphrase file, which the installer makes in
/etc/foxden(for Fennec,/etc/foxden/fennec-backup.pass) and lists on the hand-over screen. The backup refuses to run without it. That file is the only key to the app's backups. - Kept 14 days in the app's
backupsfolder (for Fennec,/opt/codefox-crm/backups). TheBACKUP_KEEP_DAYSsetting changes it. - Tested every week: every Sunday after 03:00, at the same minute past the hour as its nightly backup (Fennec at 03:15), each app checks that its newest backup opens and is whole.
- Logged to
/var/log/<stack>-backup.log, for example/var/log/codefox-crm-backup.log.
Until you set up an off-site copy, the backups are on the same server as the data, and losing the server loses both.
What a backup holds
| App | In its nightly backup |
|---|---|
| Foxden Admin | Its database. |
| Fennec | Its database, and the uploaded files (proposals, partner agreements and documents). |
| Arctic | Its database: meetings, transcripts and calendar connections. |
| Corsac | Its database, and its data folder: every sealed document, signature image and certificate. |
| Pampas | Its database, and the files people upload with their answers. |
| Culpeo | Its database, and the attachments in conversations. |
| Cape | Its database. Call recordings are not in it (see below). |
| Library | Its database, and a copy of every file the database refers to. The backup fails if one is missing. |
On a server with Library, Library is the file store for the other apps too: once Fennec, Corsac, Pampas, Culpeo and Admin are linked to it, the files they store from then on are kept in Library, and Library's backup holds them. Their own backups still hold the files from before. Restore Library together with them. See Running Library.
What a backup does not hold
None of these are in any backup. Keep each one some other way.
| Not backed up | Why it matters | How to keep it |
|---|---|---|
Each app's .env file | It holds the secrets the installer made. Some must be the same on a new server, or data cannot be opened. | Copy each app's .env (in its folder under /opt, for Fennec /opt/codefox-crm/.env) to your password manager or another safe place off the server, and again after you change a secret. |
The files in /etc/foxden | The backup passphrase files, the box key, and the shared settings in suite.env. Without the passphrase file, the backups cannot be opened. | Copy the folder off the server, as above. Never keep the passphrase files in the same place as the off-site backups. |
Root's rclone settings, /root/.config/rclone/rclone.conf | The key to your off-site bucket. | Keep the bucket's key in your password manager. |
| The certificates | None needed: Let's Encrypt issues new ones on a new server. | Nothing to do. |
| Cape's call recordings | Calls lose their audio. | Copy them to a storage bucket of their own (below). |
| Library's file copy, off the server | The off-site copy holds Library's database, not its files, which on a server with Library include the other apps' files. | Copy the files to a bucket of their own (below). |
| Arctic's notetaker sign-ins and video recordings | The notetaker accounts must be connected again. Video is kept only when video recording is on. | Connect the accounts again after a restore; see Running Arctic. |
| Files assistants hand through Admin's assistant gateway | They expire after seven days anyway. On a server with Library, Library holds them. | Nothing to do. |
| The updater's database snapshots | They exist only to roll back an update. | Nothing to do. |
The hand-over screen at the end of the install lists the secrets and passphrase files to keep, and foxden doctor reminds you of the secrets on every run.
Copy backups off the server
In Admin:
- Create a bucket with your storage provider (Amazon S3, an S3-compatible service such as Cloudflare R2 or Hetzner, or Backblaze B2), and a key that may write to it.
- Open Server › Backups and select Set up on the Off-site copy card.
- Choose the Storage, type the Bucket, and optionally a Folder. For S3-compatible storage that is not Amazon's, type its Endpoint.
- Paste the Key ID and Key secret, then select Save destination.
With the CLI:
foxden backup offsite set --provider s3 --bucket company-backups --endpoint https://fsn1.your-objectstorage.comfoxden asks for the key pair, checks that it can list the bucket, and points every app at it. Use --provider b2 for Backblaze B2. foxden backup offsite check tests the destination again; foxden backup offsite clear stops the copy.
Each app's copies go to <bucket>/<folder>/<app>. Only encrypted backups leave the server. The backups already on the server are not copied; the next one is the first. Server › Backups shows, for each app, whether its newest backup arrived.
Cape's recordings and Library's files
These are copied as plain files, not encrypted archives, so each needs an rclone crypt remote that you set up with rclone config on the server, and whose key you keep.
- Cape: set
CAPE_DATA_DIRandCAPE_RECORDINGS_REMOTE, then schedule/opt/cape/deploy/backup.sh --recordingsin root's crontab (for example every 15 minutes). Until you do,foxden backupsays Cape's recordings are not backed up. - Library: set
LIBRARY_FILES_REMOTEto the crypt remote. The nightly backup then copies the files there too.
Set both with the CLI on the server, for example printf '%s' 'cape-recordings:' | foxden config set CAPE_RECORDINGS_REMOTE --stdin --product cape. Admin does not set where backups are sent.
Back up now and check a backup
- In Admin: on Server › Backups, select Back up now for every app, or Back up on one app's row.
- With the CLI:
foxden backup --now --product fennecruns the backup and shows its output.foxden backup --verify --product fennecproves the newest backup restores.foxden backup --scheduleshows when each app is backed up.
To be told when a backup fails, set BACKUP_ALERT_CMD to a command the server runs on failure, and BACKUP_HEARTBEAT_CMD to one it runs on success (for example a ping to an uptime monitor). Set them with the CLI; Admin never sets a setting whose value runs as a command.
Restore a backup on the same server
Restoring replaces the app's data, so it is done on the server with the CLI, never from Admin.
- Find the backup. Each app's backups are in its
backupsfolder, named by the time they were taken, for examplecodefox_crm-20261118-011500.dump.gpgfor Fennec's database. Its files, when the app has them, sit beside it with the same time. - Run
foxden backup restore /opt/codefox-crm/backups/codefox_crm-20261118-011500.dump.gpg --product fennec. To restore from the off-site copy, add--from-remotewith the remote path instead of a local file.--dry-runshows what would happen and touches nothing. - Read the summary and confirm.
foxden checks that the backup opens and is whole before it touches anything, keeps a copy of the live data from before the restore, restores the database and files, makes new values for the secrets that only pair the app with its server, and lists anything you must change elsewhere (for example a redirect address at Google). It then runs foxden doctor and fails if a check is red.
Recover onto a new server
- Get a new server and point your DNS at it. See Plan your server.
- Run
foxden initwith the same apps and the same domain. - Put each old passphrase file back in
/etc/foxden, replacing the new one of the same name. A new passphrase does not open old backups. - As root, edit each of these apps'
.envand put back the oldAPP_SECRETfrom your copy, then restart the app.foxden configrefuses to set it, because the installer makes it.- Admin: it protects Admin's signing keys. With a new one, no app can trust Admin again.
- Fennec: Google connections, assistant keys and unsubscribe links already sent depend on it.
- Corsac: it protects the key that seals documents.
- Cape: it protects stored documents and the phone account saved in Cape.
- Library: a new one signs everybody out.
- Arctic has
AUTH_SECRETinstead: put it back too, or calendar connections must be made again (unless you setCALENDAR_TOKEN_KEY, which then needs putting back).
- Restore each app's newest backup with
foxden backup restore --from-remote <path> --product <app>. - Do the steps each restore lists, and run
foxden checkup.
Pampas and Culpeo get a new APP_SECRET: links already sent to respondents and visitors stop working. Culpeo also needs its Slack and WhatsApp tokens pasted again in its settings.
Troubleshooting
BACKUP_PASSPHRASE_FILE does not exist
foxden backup restore: BACKUP_PASSPHRASE_FILE (...) does not exist. Put the passphrase file the backup was made with in place first. A new passphrase will not open the old backup.
The backup is local only
The backup log says BACKUP_RCLONE_REMOTE is unset and this backup is LOCAL ONLY and does not survive box loss. Set up an off-site copy, above.
A backup is scheduled twice
foxden doctor says ... is scheduled 2 times. Two copies of the same backup running at once damage each other. Delete all but one of the lines it names.