docs+ops: rewrite CLAUDE.md, skills, runner for post-split topology
Post-monolith documentation and ops cleanup: - CLAUDE.md rewritten end-to-end. Documents the 11 Portainer-managed stacks + raw-compose management plane, the single shared `edge` network, the SSD-vs-ZFS bind-mount tiering, the four deployment channels (git push for app stacks, runner for Caddyfile, apply-compose.ps1 for mgmt plane / Caddyfile / vault), and the fact that the repo is now canonical. - homelab-apply skill rewritten for the new channels — no more `dc up -d`, no more monolith. - homelab-ssh skill rewritten — no `dc` alias, plain `docker` against container names; per-stack compose ops via /data/compose/<id>/... - homelab-sync skill + sync-prod.ps1 retired. The repo is canonical now; pulling from prod is the wrong direction. - .github/workflows/deploy.yml: drop the dc up -d steps, gate on paths:[Caddyfile, .github/workflows/deploy.yml], reload caddy via `docker exec` (no longer through compose). - apply-compose.ps1: drop -Compose and -DevStack flags; -Caddy now reloads via `docker exec caddy` (Caddy is in its own Portainer stack now). No live container is touched by this commit. The runner workflow is currently disabled at the repo level; re-enabling it makes Caddyfile pushes auto-deploy again.
This commit is contained in:
ginnoir
@@ -1,37 +1,67 @@
|
||||
---
|
||||
name: homelab-apply
|
||||
description: Push local config changes (docker-compose.yml, .env, Caddyfile) to the valhalla server and apply them to the live Docker stack. Use after editing homelab config to deploy it. This mutates the running production stack.
|
||||
description: Deploy changes to the valhalla homelab. For application stacks (foundry, media, owncloud, etc.), commit and push — Portainer git-polls every 5 min. For the management plane (portainer, vault, watchtower, runner), Caddyfile, or .env on the server, use the apply-compose.ps1 helper. Use after editing any homelab config to get it deployed.
|
||||
---
|
||||
|
||||
# homelab-apply
|
||||
|
||||
Deploys local repo changes to the live stack on valhalla. **This mutates production** — sync and review your diff first (see the `homelab-sync` skill).
|
||||
The repo is canonical. Different config layers deploy through different channels:
|
||||
|
||||
## Run it
|
||||
| Edit | Deployment channel | Lag |
|
||||
|---|---|---|
|
||||
| `stacks/<domain>/docker-compose.yml` or `stacks/<domain>/.env` | **git push** → Portainer polls every 5 min and redeploys that stack | up to 5 min |
|
||||
| `Caddyfile` | git push triggers `.github/workflows/deploy.yml` → runner pushes file + `docker exec caddy caddy reload` | seconds (if workflow is enabled) |
|
||||
| `portainer-compose.yml` (mgmt plane: portainer, vault, watchtower, runner) | `apply-compose.ps1 -Portainer` — Portainer can't manage itself | seconds |
|
||||
| `vault.hcl` | `apply-compose.ps1 -Portainer` (bundled with mgmt plane push) | seconds |
|
||||
| `.env` at `~/valhalla-lab/.env` (only used by mgmt plane + vault unseal) | `apply-compose.ps1 -EnvFile` | seconds |
|
||||
|
||||
## For application stack edits — just push
|
||||
|
||||
```powershell
|
||||
powershell -File apply-compose.ps1 # push .env + Caddyfile + compose, then apply
|
||||
powershell -File apply-compose.ps1 -EnvFile # .env only
|
||||
powershell -File apply-compose.ps1 -Caddy # Caddyfile only + hot-reload Caddy
|
||||
powershell -File apply-compose.ps1 -Compose # docker-compose.yml only + dc pull + dc up -d
|
||||
git add stacks/media/.env
|
||||
git commit -m "..."
|
||||
git push
|
||||
```
|
||||
|
||||
Flags combine (e.g. `-Caddy -Compose`); no flags pushes & applies all three.
|
||||
|
||||
## What each step does
|
||||
|
||||
- **`.env` / compose** → `scp` to `~/htpc-download-box/`, then `dc pull` (latest images) + `dc up -d` (recreates only containers whose resolved config changed).
|
||||
- **Caddyfile** → `scp` to `/config/caddy/Caddyfile`, then `dc exec caddy caddy reload …` — a hot reload with no downtime. If the new Caddyfile is invalid the reload fails and the old config keeps running; read the output.
|
||||
|
||||
## After applying, verify
|
||||
Portainer's per-stack auto-update will pick up the new commit within 5 minutes and redeploy. Watch progress:
|
||||
|
||||
```powershell
|
||||
ssh ginnoir@valhalla "bash -ic 'dc ps'"
|
||||
ssh ginnoir@valhalla "bash -ic 'dc logs --tail=50 <service>'"
|
||||
ssh -o BatchMode=yes ginnoir@valhalla "docker compose -f /data/compose/<id>/stacks/<domain>/docker-compose.yml ps"
|
||||
```
|
||||
|
||||
Or just `docker ps` and filter by name. To force an immediate redeploy without waiting, trigger it from the Portainer UI ("Update the stack").
|
||||
|
||||
## For Caddyfile edits — push and wait for the runner
|
||||
|
||||
Workflow is gated by `paths: [Caddyfile, .github/workflows/deploy.yml]` so only Caddyfile commits trigger it.
|
||||
|
||||
If you don't want to wait (or the workflow is disabled), the manual fallback is:
|
||||
|
||||
```powershell
|
||||
powershell -File apply-compose.ps1 -Caddy
|
||||
```
|
||||
|
||||
This `scp`s the local Caddyfile to `/config/caddy/Caddyfile` and runs `docker exec caddy caddy reload --config /etc/caddy/Caddyfile`. Invalid Caddyfile → reload fails, old config keeps running; read the output.
|
||||
|
||||
## For management-plane edits
|
||||
|
||||
```powershell
|
||||
powershell -File apply-compose.ps1 -Portainer # portainer-compose.yml + vault.hcl + up -d
|
||||
powershell -File apply-compose.ps1 -EnvFile # ~/valhalla-lab/.env only
|
||||
powershell -File apply-compose.ps1 -VaultUnseal # unseal vault after restart (keys from .env)
|
||||
```
|
||||
|
||||
No flags = `-EnvFile` + `-Caddy` (the two ad-hoc pushes you'd want for a quick manual deploy).
|
||||
|
||||
## Verify after applying
|
||||
|
||||
```powershell
|
||||
ssh -o BatchMode=yes ginnoir@valhalla "docker ps --format 'table {{.Names}}\t{{.Status}}'"
|
||||
ssh -o BatchMode=yes ginnoir@valhalla "docker logs --tail 50 <container>"
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- Compose is **v1.27.4** (`docker-compose`, hyphenated); `dc` runs over `bash -ic` to load the alias.
|
||||
- Editing config without applying does nothing — the server only sees pushed files.
|
||||
- Cosmetic/whitespace-only compose edits are no-ops to running containers; `dc up -d` won't recreate them.
|
||||
- Compose is **v2** (`docker compose`, space).
|
||||
- There is no monolithic `dc` alias anymore — each stack has its own compose dir under `/data/compose/<id>/stacks/<domain>/`. Address services by container name with `docker logs`, `docker exec`, etc. — `docker compose ...` only makes sense when you explicitly point at a per-stack compose file.
|
||||
- A **GateGuard hook** blocks the first `Bash` and every `Write`/`Edit` until you state the user request + what the operation does/affects. State them, retry.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: homelab-ssh
|
||||
description: Connect to and run commands on the valhalla homelab server (ginnoir@valhalla) over SSH, including docker-compose operations through the `dc` alias. Use when inspecting the live stack, tailing logs, restarting a service, reloading Caddy, or checking container/file state on the server.
|
||||
description: Connect to and run commands on the valhalla homelab server (ginnoir@valhalla) over SSH. Use when inspecting the live per-domain stacks, tailing logs, restarting a container, reloading Caddy, or checking container/network/volume state. There is no `dc` alias anymore — services are addressed by container name with plain `docker` commands.
|
||||
---
|
||||
|
||||
# homelab-ssh
|
||||
@@ -15,37 +15,73 @@ Always pass `-o BatchMode=yes` so a missing key or password prompt fails fast in
|
||||
ssh -o BatchMode=yes -o ConnectTimeout=8 ginnoir@valhalla "<command>"
|
||||
```
|
||||
|
||||
The first `Bash` command of a session is gated by a GateGuard hook — state the user request + what the command does, then retry.
|
||||
The first `Bash` of a session is gated by a GateGuard hook — state the user request + what the command does, then retry.
|
||||
|
||||
## docker-compose via the `dc` alias
|
||||
## The estate is now 11 Portainer-managed stacks + the management plane
|
||||
|
||||
The stack uses Docker Compose v2 (`docker compose` plugin, installed via apt). On the server, `dc` is aliased in `~/.bashrc` to:
|
||||
| Stack (Portainer project name) | Containers |
|
||||
|---|---|
|
||||
| `proxy` | `caddy` |
|
||||
| `media` | `deluge`, `nzbget`, `sonarr`, `radarr`, `bazarr`, `prowlarr`, `overseerr`, `tautulli`, `qbittorrent`, `whisparr`, `stash` |
|
||||
| `foundry` | `foundry`, `foundry2`, `5etools` |
|
||||
| `owncloud` | `owncloud_server`, `owncloud_mariadb`, `owncloud_redis` |
|
||||
| `resume` | `resume`, `postgres_resume`, `minio_resume`, `chrome` |
|
||||
| `famapp` | `famapp`, `postgres_famapp`, `minio_famapp` |
|
||||
| `authentik` | `authentik_server`, `authentik_worker`, `postgres_authentik`, `redis_authentik` |
|
||||
| `notify` | `ntfy`, `freshrss`, `vigilant` |
|
||||
| `monitoring` | `uptime_kuma`, `homarr` |
|
||||
| `remote` | `hbbr`, `hbbs` |
|
||||
| `dev` | `gitea`, `code_server`, `bookstack`, `mariadb_bookstack`, `dbx`, `registry`, `postgres_gitea`, `postgres_plane`, `redis_plane`, `plane_minio`, `plane_api`, `plane_web`, `plane_worker`, `plane_beat`, `plane_create_bucket` |
|
||||
| management plane (raw compose at `~/valhalla-lab/portainer-compose.yml`, project name `valhalla-lab`) | `portainer`, `vault`, `github_runner`, `watchtower` |
|
||||
|
||||
```
|
||||
docker compose -f ~/htpc-download-box/docker-compose.yml --env-file ~/htpc-download-box/.env
|
||||
```
|
||||
## Common ops by container name
|
||||
|
||||
It's a shell alias, so it only exists in an interactive shell. Invoke it with `bash -ic`:
|
||||
There's no umbrella `dc` alias. Use plain `docker`:
|
||||
|
||||
```powershell
|
||||
ssh ginnoir@valhalla "bash -ic 'dc ps'"
|
||||
ssh ginnoir@valhalla "bash -ic 'dc logs -f --tail=200 <service>'"
|
||||
ssh ginnoir@valhalla "bash -ic 'dc restart <service>'"
|
||||
ssh ginnoir@valhalla "bash -ic 'dc up -d'"
|
||||
ssh ginnoir@valhalla "bash -ic 'dc exec caddy caddy reload --config /etc/caddy/Caddyfile'"
|
||||
ssh -o BatchMode=yes ginnoir@valhalla "docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Label \"com.docker.compose.project\"}}'"
|
||||
ssh -o BatchMode=yes ginnoir@valhalla "docker logs --tail 200 -f <container>"
|
||||
ssh -o BatchMode=yes ginnoir@valhalla "docker restart <container>"
|
||||
ssh -o BatchMode=yes ginnoir@valhalla "docker exec <container> <command>"
|
||||
ssh -o BatchMode=yes ginnoir@valhalla "docker exec caddy caddy reload --config /etc/caddy/Caddyfile"
|
||||
```
|
||||
|
||||
Use the compose **service name** (`sonarr`, `caddy`, `app`, …). Without `bash -ic`, `dc` is "command not found".
|
||||
## Per-stack `docker compose` operations
|
||||
|
||||
Each Portainer git stack lives at `/data/compose/<id>/stacks/<domain>/docker-compose.yml` (Portainer chooses the `<id>` when the stack is created). To inspect a stack's resolved config or run a one-off compose op:
|
||||
|
||||
```powershell
|
||||
# Pick the stack file from its label
|
||||
ssh ginnoir@valhalla "docker inspect <container> --format '{{ index .Config.Labels \"com.docker.compose.project.config_files\"}}'"
|
||||
# Then point compose at it
|
||||
ssh ginnoir@valhalla "docker compose -f /data/compose/14/stacks/owncloud/docker-compose.yml ps"
|
||||
```
|
||||
|
||||
For management plane only:
|
||||
|
||||
```powershell
|
||||
ssh ginnoir@valhalla "docker compose -f ~/valhalla-lab/portainer-compose.yml --env-file ~/valhalla-lab/.env ps"
|
||||
```
|
||||
|
||||
## Key paths on the server
|
||||
|
||||
- `~/htpc-download-box/docker-compose.yml`, `~/htpc-download-box/.env` — stack definition
|
||||
- `/config/caddy/Caddyfile` — Caddy config (mounted into the `caddy` container at `/etc/caddy/Caddyfile`)
|
||||
- `/config/<service>/` — per-service persisted config; `/storage1/` — media & data; plus named Docker volumes
|
||||
- `~/htpc-download-box/` contains only `docker-compose.yml` and `.env` — the legacy cruft has been cleaned out
|
||||
- `~/valhalla-lab/portainer-compose.yml` + `~/valhalla-lab/.env` — management plane (raw compose, applied via `apply-compose.ps1 -Portainer`)
|
||||
- `/config/caddy/Caddyfile` — Caddy config (mounted into `caddy` at `/etc/caddy/Caddyfile`)
|
||||
- `/config/<service>/` — per-service persisted config (SSD tier: configs + Postgres + Redis)
|
||||
- `/storage1/labdata/<service>/` — high-density data on the ZFS pool (MinIO buckets, registry layers, gitea repos)
|
||||
- `/storage1/` — media library (Books, Tabletop, etc.)
|
||||
- `/data/compose/<id>/stacks/<domain>/` — Portainer's working copy of each stack (Portainer auto-pulls these from git every 5 min)
|
||||
|
||||
## Networks
|
||||
|
||||
- `edge` — shared reverse-proxy network. `caddy` + every service Caddy proxies. Anything new that needs to be public must join this.
|
||||
- `<domain>` (e.g. `media`, `authentik`, `dev`) — per-stack private network for intra-stack traffic (app ↔ its db/cache/minio).
|
||||
- `portainer_proxy`, `valhalla-lab_default` — management-plane networks (legacy names, kept).
|
||||
- `bridge`, `host`, `none` — Docker defaults.
|
||||
|
||||
## Notes
|
||||
|
||||
- Read-only exploration of `/config`, `/storage1`, and container state is fine for answering questions.
|
||||
- Avoid destructive commands against the live stack without explicit confirmation.
|
||||
- To push config changes and apply them, use the `homelab-apply` skill; to pull prod config into the repo, use `homelab-sync`.
|
||||
- For destructive ops (`docker rm -f`, network/volume pruning), get explicit confirmation first.
|
||||
- To deploy config changes, use the `homelab-apply` skill (per-domain stacks → git push; mgmt plane / Caddyfile → `apply-compose.ps1`).
|
||||
- `htpc-download-box_*` and `valhalla-lab_*` named volumes still exist as a data safety net from the pre-split monolith — every per-domain stack uses bind mounts now, so those orphans can be reaped once you're confident in the new persistence.
|
||||
|
||||
@@ -1,38 +0,0 @@
|
||||
---
|
||||
name: homelab-sync
|
||||
description: Pull the live production configs (docker-compose.yml, .env, Caddyfile) from the valhalla server into this repo so local matches what's deployed. Use at the start of any session that will edit homelab deployment config, or to check for drift between repo and production.
|
||||
---
|
||||
|
||||
# homelab-sync
|
||||
|
||||
Production (valhalla) is the source of truth. Pull before editing so you never edit a stale copy.
|
||||
|
||||
## Run it
|
||||
|
||||
```powershell
|
||||
powershell -File sync-prod.ps1
|
||||
```
|
||||
|
||||
This `scp`s three files from valhalla, **overwriting** the local copies:
|
||||
|
||||
| From valhalla | Into repo |
|
||||
|---------------|-----------|
|
||||
| `~/htpc-download-box/docker-compose.yml` | `docker-compose.yml` |
|
||||
| `~/htpc-download-box/.env` | `.env` |
|
||||
| `/config/caddy/Caddyfile` | `Caddyfile` |
|
||||
|
||||
It overwrites local working copies — commit or stash anything you want to keep first.
|
||||
|
||||
## Check drift without overwriting
|
||||
|
||||
To see how local differs from prod before deciding direction (git is installed):
|
||||
|
||||
```powershell
|
||||
$tmp = Join-Path $env:TEMP 'valhalla-compose.yml'
|
||||
scp -o BatchMode=yes "ginnoir@valhalla:~/htpc-download-box/docker-compose.yml" $tmp
|
||||
git --no-pager diff --no-index -- $tmp .\docker-compose.yml
|
||||
```
|
||||
|
||||
(`.env` and `Caddyfile` can be compared the same way.)
|
||||
|
||||
After syncing, review `git status` / `git diff` to see what changed on the server since your last sync. To push local changes the other direction, use the `homelab-apply` skill.
|
||||
Reference in New Issue
Block a user