Docker Compose conventions that survive year two

Year one of a Compose stack is easy because you remember everything. Year two is when you upgrade a host, or a disk dies, or you finally read your own backup and discover that half your state was in named volumes you never mapped. These are the conventions that make the second year uneventful, in rough order of how much they save you.

One directory per service, and the data tree beside it#

/srv/
├── stacks/                     # in git, no secrets
│   ├── caddy/
│   │   ├── compose.yaml
│   │   └── Caddyfile
│   └── paperless/
│       ├── compose.yaml
│       └── .env                # gitignored
└── data/                       # every bind mount target
    ├── caddy/
    └── paperless/
        ├── db/
        ├── data/
        ├── media/
        └── consume/

Two roots, one for declarations and one for state. The declaration tree is text, it goes in git, and it contains no passwords. The state tree is what your backup job points at. restic backup /srv covers both, and you never have to remember which of the two mattered.

The .env beside each compose.yaml is loaded automatically by Compose from the file's own directory. Be clear about what it does: variables in .env are used for interpolation into the compose file, which is not the same as passing them to the container. If a container needs the variable itself, either reference it explicitly (PAPERLESS_DBPASS: ${POSTGRES_PASSWORD}) or use env_file:. And any literal dollar sign in a value must be doubled to $$, which is the single most common reason a Vaultwarden ADMIN_TOKEN argon2 hash silently fails to work.

The annotated compose.yaml#

This is a real stack, not a skeleton. Every line that is a convention is commented.

# /srv/stacks/paperless/compose.yaml
name: paperless                       # explicit project name, not the directory

services:
  db:
    image: docker.io/library/postgres:17   # databases: pin the MAJOR, never float across it
    restart: unless-stopped
    environment:
      POSTGRES_DB: paperless
      POSTGRES_USER: paperless
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    volumes:
      - /srv/data/paperless/db:/var/lib/postgresql/data   # absolute host path
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U paperless -d paperless"]
      interval: 10s
      timeout: 5s
      retries: 10
      start_period: 30s

  broker:
    image: docker.io/library/redis:7
    restart: unless-stopped
    volumes:
      - /srv/data/paperless/redis:/data

  webserver:
    image: ghcr.io/paperless-ngx/paperless-ngx:3.0.0     # exact tag, changed on purpose
    restart: unless-stopped
    depends_on:
      db:
        condition: service_healthy    # "started" is not "ready"
      broker:
        condition: service_started
    environment:
      PAPERLESS_REDIS: redis://broker:6379
      PAPERLESS_DBHOST: db
      PAPERLESS_DBPASS: ${POSTGRES_PASSWORD}
      PAPERLESS_SECRET_KEY: ${PAPERLESS_SECRET_KEY}
      PAPERLESS_URL: https://paperless.example.com
      USERMAP_UID: "1001"             # this image's spelling of PUID
      USERMAP_GID: "1001"
    volumes:
      - /srv/data/paperless/data:/usr/src/paperless/data
      - /srv/data/paperless/media:/usr/src/paperless/media
      - /srv/data/paperless/export:/usr/src/paperless/export
      - /srv/data/paperless/consume:/usr/src/paperless/consume
    networks: [default, proxy]        # default for db/broker, proxy for Caddy
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

networks:
  proxy:
    external: true                    # docker network create proxy, once, by hand

Note what is not there. No published ports on the web service: the reverse proxy reaches it over the proxy network by container name, so nothing is exposed on the host at all. No version: key, which is obsolete. No latest.

The shared proxy network, created once#

docker network create proxy

Every stack that needs to be reachable from a browser joins proxy as an external network; everything else (databases, brokers, workers) stays on the stack's own default network where nothing outside the stack can reach it. Caddy or Traefik sits on proxy and nothing else, and its reverse_proxy webserver:8000 resolves by Docker's internal DNS.

This is also the fix for a genuinely dangerous Docker default: published ports bypass ufw. A container started with -p 8080:80 is reachable from your LAN and, behind a port-forward, from the internet, even when the host firewall says that port is denied, because Docker writes its own iptables chains ahead of yours. If nothing publishes a port, that whole class of accident disappears. Where you do need to publish, bind to an interface: 127.0.0.1:8080:80. Reverse proxy and TLS covers the proxy side, and the Port conflict checker will tell you which of your services collide before you find out the hard way.

Why latest is a data-loss risk, not a convenience#

latest is not a channel. It is a tag the maintainer points at whatever they last built, and it moves across major versions without asking. The failure mode is not "my container is on a newer version". It is this sequence: your host reboots, Docker pulls a newer image because your tag floated, the new version runs a forward-only schema migration on startup, and now your database is on a schema the old version cannot read. You did not choose to upgrade, and you cannot go back.

That is not hypothetical. Immich states plainly that downgrading is unsupported even within a minor, because migrations run forward on startup, and the recovery path is a database dump you took beforehand. Jellyfin 10.11's database migration is one way. Uptime Kuma's v1 to v2 upgrade rewrites heartbeat history into aggregate tables with no downgrade path. Nextcloud cannot skip a major version at all, so a floating tag that jumps you two majors leaves you with an instance that will not start and no supported path forward.

Pin exact tags for applications. Pin the major for database images, where minor updates are safe and major upgrades require a dump and reload. Then handle upgrades deliberately, which is what An update strategy that does not lose data is for.

Restart policy, and what unless-stopped actually promises#

restart: unless-stopped restarts the container on any exit code, but stops doing so once you have explicitly stopped it. always will restart a container you deliberately stopped when the daemon next starts, which is how a service you took down for maintenance comes back at 3am. Use unless-stopped everywhere and reserve always for nothing.

Understand what this is not. A restart policy is not process supervision with dependency ordering: Compose will not wait for your database on a host reboot, it will restart everything and let the losers crash-loop until they win. If you want real ordering, systemd is the tool, which is the argument for Podman and Quadlet in Docker vs Podman.

Healthchecks are for other containers, not for you#

A healthcheck marks a container healthy or unhealthy; it does not restart anything by itself. Its value is that depends_on: condition: service_healthy can wait on it, and that docker compose ps tells you the truth instead of "Up 4 minutes" for a process that has been returning 500s the whole time. Set start_period generously on anything with migrations: a 30 second grace period on Postgres is normal, and databases that restore large datasets on first boot need more.

For applications, the cheapest useful check is the one the project already exposes:

healthcheck:
  test: ["CMD", "curl", "-fsS", "http://localhost:8000/api/health"]
  interval: 30s
  timeout: 5s
  retries: 3
  start_period: 60s

If curl is not in the image, the check fails forever and marks a working service unhealthy. Check before you copy.

File ownership, the thing that breaks first boots#

A container writing into a bind mount writes as whatever uid the process inside uses. Three patterns, all common:

• LinuxServer.io images take PUID and PGID and drop to that user. Set them to a dedicated host account, not to your login.
• Project-specific spellings. Paperless-ngx uses USERMAP_UID and USERMAP_GID.
• Hardcoded uids. The Grafana image runs as uid 472, so a fresh bind mount produces permission denied on first boot and an immediate exit. Fix it with chown -R 472:472 on the host directory or a user: override.

Create one unauthenticated system user per service, or at minimum a shared services group, and make the data tree owned by it. It costs five minutes and it is the difference between "restore the files" and "restore the files and then spend an hour working out why nothing starts".

What a bind mount actually buys you at restore time#

Here is the concrete difference. With named volumes, a restore means recreating volumes with the exact names Compose expects (which are prefixed by the project name, so they change if you renamed the directory), unpacking your archive into each one as root, and hoping you captured all of them. With bind mounts, a restore is:

restic restore latest --target / --include /srv/data/paperless
cd /srv/stacks/paperless && docker compose up -d

The paths in the compose file are absolute and they already match. Nothing has to be reconstructed from memory.

The uncomfortable part: bind mounts do not make the backup correct. Of the 105 tools profiled here, 81.9% cannot be safely copied as live files. Copying a running SQLite database in WAL mode gives you a vault that restores corrupt, which is documented behavior for Vaultwarden and linkding alike, and copying a Postgres data directory under load gives you a database that will not start. A bind mount makes the files easy to reach; it is Backing up a running database that makes them restorable.

Small things that pay for themselves#

• Log rotation on the daemon, in /etc/docker/daemon.json, before you need it. Existing containers keep old settings until recreated.
• docker system df monthly. /var/lib/docker accumulates dangling images and build cache quietly, and a busy host adds tens of gigabytes a year.
• Dozzle for logs (15 MB, retains nothing) rather than sshing in to tail. Just know that recreating a container destroys the old container's logs with it, including whatever explained the crash.
• Keep the UI honest. Portainer and Komodo are both useful, but if stacks are authored in a browser they drift from the files in git. Author on disk, view in the UI.

What to do next#

Convert one existing stack: give it a name:, replace latest with the tag you are actually running, move its named volumes to /srv/data/<service>, and attach it to the shared proxy network. Then run a restore of that one service into a scratch directory to prove the layout works. When it does, apply the same shape everywhere and read An update strategy that does not lose data before your next upgrade window.