Tech Digest

Calculator

Backup planner

Most backup jobs fail at restore time because they copied a database file while it was being written. This one puts the export first.

Last reviewed Runs entirely in your browser

Select what you run and this sorts it into the export each service actually needs, then writes a nightly backup script in the order a restore requires: consistent exports first, one snapshot second, prune and verify third, stopped services restarted at the end whether or not the job succeeded.

The grouping is the point. A stack of ten services is rarely ten identical backup problems. Two need pg_dump, one needs mariadb-dump, three are SQLite and need the backup API rather than a file copy, three are plain files that copy safely while running, and one has no safe online export and has to be stopped for a few seconds. A single restic backup /srv/docker treats all ten the same way and quietly gets three of them wrong.

Pick restic, Borg or Kopia, tell it where your service data lives and where the repository is, and the script comes out ready to read. Nothing is uploaded: the app dataset is embedded in this page and the script is assembled in your browser.

1. Pick what you want to run

2. Where does it go

How it works#

Each service is grouped by the backup shape derived from its declared datastore: Postgres, MySQL or MariaDB, MongoDB, SQLite, plain files, a rebuildable cache, a rebuildable search index, or mixed, which means no documented online export. That derivation is a rule, not a claim by the project, and the rules are on the our methodology page.

The generated script then follows one order:

  1. mktemp -d for a dump directory, with a trap that removes it and restarts any stopped service on exit.
  2. One export command per database service, compressed with zstd into that directory.
  3. stop for anything in the mixed group.
  4. A single snapshot covering both the dump directory and your data root, so the dumps and the files that go with them land in the same restore point.
  5. Prune to 7 daily, 5 weekly and 12 monthly, then a verification pass.
  6. Start the stopped services again.

The rough restore time shown above the table is a planning prompt, not a measurement: eight minutes per service plus twelve for each one needing an export, with a floor of twenty. Treat it as an argument for rehearsing, not as an estimate you can quote.

What it deliberately does not model#

  • Application-specific exporters. Several projects ship something better than a generic dump. Paperless-ngx has document_exporter, Home Assistant has its own backup integration, and Nextcloud wants maintenance mode on. Where one exists, use it and keep the generic dump as a second copy.
  • Off-site copies and encryption keys. The script writes to one repository. Getting a second copy somewhere your server cannot delete from, and storing the repository password outside the machine, are both on you.
  • Scheduling and alerting. No timer, no notification on failure. A backup that has been failing silently for four months is the normal failure mode, not a rare one.
  • Restore. It generates the easy half. The order to bring services back, and which of them need the database restored before first start, belongs in your notes.

What to do with the answer#

Read it, fix the paths, run it once by hand and watch the output. Then wire it to a systemd timer with Persistent=true, send the exit status somewhere you will see it, and put the first restore rehearsal in the calendar for a date you will actually keep. Backups that actually restore covers the rehearsal and the numbers worth writing down, Backing up a running database explains why the export step exists, and the Resilience scorecard scores the parts this script cannot do for you.

Questions#

Can I back up a running Postgres container with rsync?

No. rsync copies the data directory file by file over seconds or minutes while Postgres keeps writing, so the copy contains pages from several different points in time. It will often restore, start, and then fail on a query that touches an index written after the table it points into. Use docker compose exec -T db pg_dump -U postgres appdb into a file, then back up the file. If you genuinely want the data directory, stop the container first or take an atomic filesystem snapshot.

Is copying a SQLite file safe?

Not while the service is running in WAL mode, which most of them are. The database is three files: app.db, app.db-wal and app.db-shm, and copying only the first gives you a database missing every transaction still in the write-ahead log. Use sqlite3 app.db ".backup /tmp/app.db", which the planner generates, or stop the container and copy all three. This is the single most common way a self-hosted backup turns out to be worthless.

restic, Borg or Kopia?

restic if you have no strong opinion: one static binary, backends for almost everything including S3-compatible object storage, and restic check --read-data-subset to verify without a full download. BorgBackup if your destination is a box you have SSH to, because its server-side append-only mode is the best protection against a compromised client deleting history. Kopia if somebody in the house needs a GUI. All three deduplicate and encrypt; the script the planner writes is nearly identical for each.

Do I need to stop containers to back them up?

Most of them, no. Anything storing plain files, YAML or a rebuildable index copies safely while running. The planner puts only the services with no safe online export into the stop group and wraps the restart in a shell trap, so they come back even if the snapshot fails. If you do stop things, stop them for the seconds it takes to snapshot a filesystem, not for the minutes it takes to copy to a remote.

How often should the job run?

Nightly is the default and it sets your recovery point objective at 24 hours: a failure at 8pm loses a day of work. If that is unacceptable for one service, back up that service more often rather than everything. Run it from a systemd timer with Persistent=true instead of cron, so a machine that was asleep or rebooting at 03:00 still runs the job when it comes back.

Where should the repository live?

Not on the pool you are backing up. A second folder on the same disks survives a mistake and nothing else. The workable home setup is one local repository on a separate disk for fast restores and one remote that your server can write to but not delete from, using restic's rest-server in append-only mode, Borg's append-only SSH restriction, or object storage with a bucket policy. Keep the repository password somewhere you can reach when the server is dead, on paper or in a password manager that syncs elsewhere.

How do I know the backup works?

Restore from it. Once a quarter, take the newest snapshot onto a spare machine or a VM, bring one service up against restored data, and write down how long it took. That number is your recovery time objective and it is usually two to three times what people guess. Weekly, do the cheap version: restore one file and diff it. restic check --read-data-subset=5% verifies repository integrity but proves nothing about whether your dump was consistent.

Published . Last reviewed . Found something out of date? Tell us and we will fix it and log the change.