Skip to content

Upgrades

How to move a running Trimus from one release to the next — including migrations, the deploy model, rollback, and the things that go wrong.

The deploy model in one sentence

trimus-server runs all pending database migrations itself on every boot (db.RunMigrations at startup, via goose), so an upgrade is usually just swap the binary and restart — the schema catches up automatically.

trimus-seed is for dev seeding, not migrations

You do not need a separate trimus-seed step to apply migrations in production — the server applies them on boot. trimus-seed is the idempotent dev seeder (default user + sample data) and also runs migrations as a side effect; it's handy locally but unnecessary for a server upgrade.

Versioning

Releases are SemVer tags vMAJOR.MINOR.PATCH, built by GoReleaser. The canonical line is v3.0.0 onward. Read the tag annotation (git tag -l -n99 <tag>) and CHANGELOG.md for what's in a release and any required action items — look for BREAKING markers.

The running version is baked in at build time via -ldflags (-X main.version, -X main.commit, -X main.date). A plain go build from source leaves version=dev.

The standard upgrade

Almost every release follows this shape:

  1. Read the tag annotation + CHANGELOG for BREAKING markers.
  2. Take a full backup before applying migrations — Postgres plus TRIMUS_HOME, TRIMUS_STORAGE_DIR, and a copy of the TRIMUS_ENCRYPTION_KEY. Postgres alone is not enough. See backup-and-restore.md.
  3. Roll the new binary forward.
  4. Restart trimus-server — migrations apply on boot.
  5. Verify /readyz reports the new commit and all checks pass.

Total downtime is usually under a minute; most migrations are additive and complete in milliseconds.

Step by step

In‑place on a host (the make deploy path)

When the repo is checked out on the host running trimus-server under systemd — the typical self‑hosted pattern:

git fetch --tags
git checkout v3.0.0
make deploy

make deploy chains three steps (Linux‑only — it needs systemctl):

  1. build-release — compiles trimus-{server,worker,seed,cli} with the GoReleaser -ldflags and -s -w; CGO_ENABLED=0 for static binaries. Version comes from git describe --tags --exact-match HEAD.
  2. install — atomically swaps each trimus-* binary in $(INSTALL_PREFIX) (default /usr/local/bin) using cp → mv on the same filesystem, so the running daemon keeps its open inode while new exec()s pick up the new file (a plain cp over a running binary fails with "Text file busy").
  3. systemctl restart $(SERVICE_NAME) (default trimus), waits 3 s, then tails journalctl for the "server starting" line and prints it; exits non‑zero if it doesn't appear.

Overrides for non‑default layouts:

# User-level unit under ~/.local/bin (no sudo, claude_local-friendly)
make deploy INSTALL_PREFIX=$HOME/.local/bin SERVICE_NAME=trimus SUDO=

# A differently-named unit
make deploy SERVICE_NAME=trimus-prod

On macOS there is no systemctl — run make install (steps 1–2) and restart the LaunchAgent by hand.

Boot waits for the database

The shipped systemd unit (scripts/service/trimus.service) has an ExecStartPre that waits up to 90 s for the database to accept connections on 127.0.0.1:5432 before starting trimus-server, and StartLimitIntervalSec/StartLimitBurst live in [Unit] with generous headroom. After a host reboot where Postgres (often a container) is still coming up, the service no longer crash‑loops on "connection refused" — it comes up cleanly in a single attempt. Override the DB host:port via a drop‑in if it isn't local.

Release‑archive install (install.sh)

For new‑host installs the GoReleaser archive ships an install.sh; to upgrade an archive‑managed host, fetch the new archive and re‑run the installer (it's idempotent — it reuses existing secrets and re‑renders the unit). See installing.md.

Docker / Compose

# 1. Backup
docker compose exec postgres pg_dump -Fc -U trimus trimus > /backups/trimus-pre-$(date +%F).dump

# 2. Pull / build the new image
docker compose pull trimus   # or: docker compose build trimus

# 3. Restart — migrations run on boot
docker compose up -d trimus

# 4. Verify
curl -sS http://localhost:3100/readyz

From source (development)

git pull
make run    # builds + runs; migrations apply on boot

There is no reset.sh in this repo

Older docs referenced a host‑specific reset.sh deploy script. The supported in‑place upgrade is make deploy (above). If your host still carries a bespoke reset.sh, treat it as deploy‑host‑specific and unmanaged — make deploy is the maintained path.

Migrations: what to expect

Migrations live at internal/db/migrations/<NNN>_<name>.sql (176 files as of v5.0.2; highest is 176_worker_shared_models.sql). On boot, goose applies every file past the database's current version up to the binary's latest, one transaction per file. Typical shapes:

  • Additive columns / new tables / indexes — fast (ms to seconds).
  • Backfills via UPDATE — proportional to table size; run during a quiet window if the table is huge.
  • DROP COLUMN — rare; fast, but ensure no agents are mid‑run with queries referencing the old column.

Check the schema version

SELECT * FROM goose_db_version ORDER BY id DESC LIMIT 1;

The version_id is the highest applied migration number; compare against the highest file in internal/db/migrations/ for the binary you're running. A mismatch means the server hasn't booted the new binary yet.

Zero(ish)‑downtime options

Migrations are additive, so old and new binaries can run against the new schema during the migration window.

  • Blue‑green at the load balancer — stand up a second trimus-server (new binary) against the same Postgres, let it apply migrations on boot, cut the LB over, drain the old node.
  • Rolling restart in k8s — a standard rolling Deployment update; optionally run migrations as a pre‑deploy Job (or just let the new pods apply them on boot).

Caveat: in‑flight heartbeat runs whose worker is killed mid‑run get marked failed when a worker next reads the stale running row; agents recover on their next heartbeat. There's no graceful drain — worst case is one wasted heartbeat per active agent. (The systemd unit does send SIGTERM with a TimeoutStopSec=15s window for a clean HTTP/worker shutdown.)

Rollback

There is no first‑class downgrade. goose down blocks exist but are not exercised in CI and may not preserve data. Practical rollback:

  1. Restore Postgres from your pre‑upgrade backup (trimus-cli restore <file> --yes).
  2. Roll the binary / container back to the previous tag.
  3. Restart.

Anything written since the upgrade is lost in the restore, so a clean rollback requires the backup to be a clean pre‑upgrade snapshot. For data‑only fixes after a bad upgrade (no schema rollback), hand‑fix the rows and ship the next release — most "bad upgrade" cases are config, not data corruption.

Specific upgrade traps

  • New encrypted column — a migration that adds an encrypted column wraps only data written after the upgrade; pre‑existing rows lazy‑migrate on edit. Use trimus-cli rewrap-all to backfill where it applies (see key-rotation.md).
  • New default‑value columns — companies whose rows predate the migration get DB defaults; visit Settings → Hired Agent Defaults once after upgrading.
  • Route changes — a moved endpoint stays as a redirect for a release or two, then is removed; the HTTP API reference is always current for the latest release.
  • Worker‑protocol changes — keep remote trimus-worker processes in lockstep with trimus-server; a mismatched worker fails to claim tasks. See remote-workers.md.

Disk hygiene

Each release build and binary swap leaves artefacts behind. On long‑lived deploy hosts, watch root‑filesystem pressure: the Go build cache, old container images (docker image prune), accumulated backups under TRIMUS_BACKUP_DIR, and bin/ artefacts add up. /readyz escalates to not‑ready at TRIMUS_READYZ_DISK_PCT (default 90%) so the orchestrator drains the node before backups and systemd start failing — treat a climbing disk check as the signal to prune. See monitoring.md.

Checking a running server's version

curl -sS http://localhost:3100/healthz
# {"status":"alive","version":"v3.0.0","commit":"abc1234"}

/readyz carries the same version/commit alongside its per‑check rows. Useful for "is the deploy actually live yet?".

CI / automation

# pseudocode "deploy on tag"
on: { push: { tags: ['v*'] } }
jobs:
  deploy:
    steps:
      - back up postgres (+ TRIMUS_HOME / TRIMUS_STORAGE_DIR / key)
      - roll the new binary / image
      - restart trimus-server   # migrations apply on boot
      - smoke-test /readyz

Don't auto‑deploy MAJOR or MINOR bumps without a human reading the changelog.

Where to next