# Runbook

Operational reference for famapp in production. See `deploy/README.md` for the one-time host setup and `docs/tasks/09-pre-deploy-checklist.md` for the pre-deploy checklist.

## Cutting a release

From `main` on the dev machine, with a clean working tree:

```bash
pnpm release:patch   # or :minor / :major
# — bumps package.json version
# — prepends to CHANGELOG.md
# — creates and pushes a signed git tag (v0.x.y)
# — creates a GitHub Release with generated notes
# Requires GITHUB_TOKEN in .env
```

Gitea Actions (`release.yml`) then builds and pushes the Docker image to the self-hosted registry:

- `registry.ginnoir.com/ginnoir/famapp:v0.x.y`
- `registry.ginnoir.com/ginnoir/famapp:0.x` (minor alias)
- `registry.ginnoir.com/ginnoir/famapp:latest`

## Deploying a release

On the home server, in `/srv/famapp/deploy/`:

```bash
# Pin the new tag
sed -i 's|FAMAPP_IMAGE=.*|FAMAPP_IMAGE=registry.ginnoir.com/ginnoir/famapp:v0.x.y|' .env

# Pull and restart only the app container
docker compose pull famapp
docker compose up -d famapp

# Watch the boot log — migrations run before the server starts
docker compose logs -f famapp
```

The container entrypoint runs `node scripts/migrate.mjs` automatically. A healthy boot ends with a log line like `ready on http://0.0.0.0:3000`.

## Health check

```bash
# Container status
docker compose ps

# App response (200 = healthy)
curl -sf https://fam.yourdomain.com/ -o /dev/null -w "%{http_code}\n"

# Recent app logs
docker compose logs --tail=100 famapp

# Database connectivity
docker compose exec famapp-db pg_isready -U famapp -d famapp
```

## Rollback

1. Find the previous working tag in `CHANGELOG.md` or `docker images`.
2. Pin it in `deploy/.env`:
   ```bash
   sed -i 's|FAMAPP_IMAGE=.*|FAMAPP_IMAGE=registry.ginnoir.com/ginnoir/famapp:v0.x.y|' .env
   ```
3. Restart the container:
   ```bash
   docker compose up -d famapp
   ```

If the rollback target predates a migration that has already been applied to the database, restore from backup first — see **Backups** below. To skip auto-migration on a given start (rarely needed):

```bash
RUN_MIGRATIONS=false docker compose up -d famapp
```

## Backups

The `famapp-backup` container runs nightly `pg_dump` for both `famapp` and `authentik` databases into the `backups` volume.

```bash
# Check last backup run
docker compose logs famapp-backup | tail -20

# List backup files
docker compose exec famapp-backup ls -lh /backups

# Manual backup now
docker compose exec famapp-backup sh /scripts/backup.sh

# Restore from a backup file
docker compose exec famapp-backup sh /scripts/restore.sh famapp_2026-06-01.sql.gz
```

See `deploy/backups/README.md` for retention policy and full restore details.

## Common issues

### App container exits immediately

```bash
docker compose logs famapp
```

Likely causes:

- **Missing required env var** — the app throws on startup if `AUTH_SECRET`, `DATABASE_URL`, or `AUTH_OIDC_*` are absent.
- **`ENABLE_DEV_LOGIN=true` in production** — `src/lib/dev-login-config.ts` throws an assertion error on import. Remove the variable from the production env.
- **Database not ready** — Postgres healthcheck should prevent this, but if the DB is slow to start, increase `start_period` in `compose.yaml`.

### Migrations fail on boot

```bash
docker compose logs famapp | grep -i migration
```

- Schema is ahead of code: roll back the image or forward-migrate manually.
- Database unreachable: confirm `famapp-db` is healthy (`docker compose ps`).

### OIDC login fails

1. Confirm `AUTH_OIDC_ISSUER` matches the Authentik provider URL exactly (trailing slash matters).
2. Confirm the OIDC client redirect URI in Authentik includes `https://fam.yourdomain.com/api/auth/callback/oidc`.
3. Check Authentik logs: `docker compose logs authentik-server | tail -50`.

### Push notifications not arriving

1. Confirm `VAPID_PUBLIC_KEY`, `VAPID_PRIVATE_KEY`, and `VAPID_SUBJECT` are all set in the production env.
2. Verify the browser's push subscription is still valid (Settings → notifications → re-enable).
3. If using ntfy as a secondary channel, confirm `NTFY_URL` and `NTFY_TOPIC` are set.

### MinIO / garden image uploads failing

```bash
docker compose logs famapp-minio | tail -30
```

- Confirm `MINIO_ENDPOINT` is `http://famapp-minio:9000` in the compose env (not `localhost`).
- Confirm the `garden` bucket exists — create it manually via the MinIO console at port `9001` if it is missing.

### Out of disk space

```bash
df -h /var/lib/docker
docker system prune --volumes   # removes stopped containers, dangling images, unused volumes
```

Old backup files accumulate in the `backups` volume. The retention script (`deploy/backups/retain.sh`) runs nightly — check its logs if the volume keeps growing.

## Authentik admin access

The Authentik admin UI is at `https://auth.yourdomain.com`. Log in with the superuser credentials set during bootstrap — see `deploy/authentik/README.md`.

To reset the Authentik admin password from the CLI:

```bash
docker compose exec authentik-server ak create_recovery_key 1 akadmin
# prints a one-time recovery URL
```