Commit Graph
158 Commits
Author SHA1 Message Date
ginnoir 49bda4234a fix(caddy): make roms.ginnoir.com internal-only
Deploy to valhalla / deploy (push) Has been cancelled
External access for a trusted friend now goes through the share stack's
Tailscale serve node (roms-share.<tailnet>.ts.net), so roms.ginnoir.com
no longer needs public exposure. Add import internal_only (LAN + tailnet).
2026-06-06 23:18:22 -05:00
ginnoir bf68f5a3da fix(share): absolute host bind for serve config
Portainer's git-stack checkout does not materialize sibling repo files
at the relative bind path, so ./serve-roms.json mounted as an empty
auto-created directory and tailscale serve failed ('is a directory').
Switch to an absolute /config/share/serve-roms.json bind (matching every
other stack's /config convention); the file is mirrored on the host.
2026-06-06 23:06:30 -05:00
ginnoir 2aa8122cd3 feat(share): Tailscale serve stack to share internal services with external tailnet users
Adds stacks/share/ — a dedicated stack of userspace Tailscale 'serve'
nodes that expose individual internal services to external tailnet
users (a friend on his own tailnet) over each node's 100.x identity.
No public exposure, no LAN access, immune to the friend's home-subnet
addressing. First node ts-roms serves RomM at roms-share.<tailnet>.ts.net
-> romm:8080. One reusable tag:share auth key + one ACL rule cover every
node; adding a service is a serve-<svc>.json + a copied service block.

roms stack reverted to a pointer comment.
2026-06-06 22:52:15 -05:00
ginnoir dc0a4eb58f revert(caddy): drop RomM empty-Bearer workaround
Deploy to valhalla / deploy (push) Has been cancelled
decky-romm-sync has completed its first-run token mint, so the
Authorization: Bearer (empty token) probe that crashed RomM 4.8.x no
longer occurs. Collapse roms.ginnoir.com back to a plain reverse_proxy.
2026-06-06 20:53:19 -05:00
ginnoir cb18047973 feat: add RomM save staging and import utility scripts 2026-06-06 20:51:52 -05:00
ginnoir ca4cdf3d8a fix(caddy): allow Tailscale CGNAT range in internal_only snippet
Deploy to valhalla / deploy (push) Has been cancelled
Tailnet clients that reach Caddy without subnet-route SNAT arrive from
100.64.0.0/10 and were getting 403 from LAN-only site blocks. Add the
range to the internal_only matcher so split-DNS tailnet access works
regardless of the --snat-subnet-routes setting.
2026-06-06 20:31:39 -05:00
ginnoir 04202a97b7 feat: add RomM database path mismatch resolver script 2026-06-06 19:50:21 -05:00
ginnoir 3bcfe26db9 chore: track Codex/Antigravity agent config (.agents skills + .codex) 2026-06-06 19:44:51 -05:00
ginnoir cba0ae7f4b docs: add AGENTS.md pointer so Codex/Antigravity load project context 2026-06-06 18:24:22 -05:00
ginnoir a96ff619a8 fix(caddy): strip empty Bearer header on roms so RomM stops 500ing decky-romm-sync
Deploy to valhalla / deploy (push) Has been cancelled
RomM 4.8.x crashes with HTTP 500 on a malformed Authorization header:
hybrid_auth.py unpacks `scheme, token = header.split()` with no guard, so
any value without exactly two space-separated parts raises ValueError.

decky-romm-sync >=0.20.0 (Client API Token auth, added 2026-06-05) sends
`Authorization: Bearer ` with an empty token on its first-run heartbeat/
platforms/devices probes, before it has minted a token, which trips that
crash and deadlocks setup. The Basic-auth mint (POST /api/client-tokens)
is never reached.

Strip the empty-token header for the roms host so the probe gets a clean
200 and the mint can run; thereafter the plugin sends a real Bearer rmm_
token. Well-formed Bearer tokens and browser session-cookie auth are
unaffected. Remove once the plugin omits the empty header or RomM guards
the split.
2026-06-06 16:00:25 -05:00
ginnoir 7ec4e2ca57 chore: spin off Pokémon ROM-hack material to standalone repo
The Pokémon catalog + pipeline are moved out of this infra repo. The catalog
notes now live in the self-hosted Obsidian vault (Pokémon ROM Hacks/); the
acquisition/patching scripts live in their own repo at Documents\pokemon.
homelabstack stays focused on the homelab.

- rm pokemon-romhack-vault/ (75 hack notes + Index/Types/README/Platforms)
- rm scripts/{build-romhack-vault,romhack-import,romhack-fetch,romhack-apply}.py
- rm pokemon-romhacks-wanted.md
- .gitignore: drop the now-dead pokemon/ drop-folder rule
- .claude/skills/vault/SKILL.md: condense the Pokemon-vault subtree to a one-line pointer

RomM stack (stacks/roms) and the general igir library scripts stay.
2026-06-06 15:49:26 -05:00
ginnoir b99a0a18f0 docs(vault): document Pokémon ROM Hacks vault section in skill
Adds the ROM Hacks folder tree to the vault structure reference so
future sessions know where _Claude.md, Index.md, Play Queue, Bases/,
and Platforms/ live without re-deriving it.
2026-06-06 14:57:42 -05:00
ginnoir 061147caea docs(vault): update skill with correct livesync two-level chunk format 2026-06-06 03:50:29 -05:00
ginnoir 3a2a4b4dc9 feat(vault): add vault skill for Claude read/write access to Obsidian notes 2026-06-06 03:26:10 -05:00
ginnoir f871d1cabf chore(env): store Portainer stacks PAT (Contents:read) separately from runner PAT 2026-06-06 03:19:45 -05:00
ginnoir eea602d100 feat(roms): add Obsidian vault knowledge base for 75 library romhacks
build-romhack-vault.py generates pokemon-romhack-vault/: one note per hack
(YAML properties + tags + summary + links), an Index MOC (Dataview + static
table), a Types taxonomy, and per-platform MOCs. Dataset is the single source
of truth — edit + re-run to extend.
2026-06-06 03:15:36 -05:00
ginnoir 5f53df0070 feat(notes): add Obsidian livesync stack (CouchDB) + Caddy entry
stacks/notes/: new stack with CouchDB 3 for obsidian-livesync sync backend.
Vault data at /config/couchdb/data; CORS must be configured post-deploy
via Fauxton (/_utils). Claude integration via CouchDB REST API documented
in compose comments.

Caddyfile: obsidian.ginnoir.com → couchdb:5984 (public for mobile sync).
2026-06-06 03:15:17 -05:00
ginnoir e997172c53 docs(roms): Adventure Red Chapter (Beta 15) identified + placed 2026-06-06 03:03:41 -05:00
ginnoir d0e097e2d4 feat(roms): add romhack drop-folder importer; import 53 hacks
romhack-import.py: classifies a mixed drop folder (ROMs/patches/docs/archives),
clean-names ROMs to 'Pokemon - <Hack> (Hack)', auto-detects patch base via
BPS/UPS CRC, recurses zips/rar, archives patches+docs. Imported the pokemon/
drop (now gitignored) into the valhalla library + Following Renegade Platinum.
Updated wanted-list for newly acquired hacks.
2026-06-06 03:00:17 -05:00
ginnoir 601150dcac feat(roms): make roms.ginnoir.com publicly accessible 2026-06-06 00:26:28 -05:00
ginnoir c744ff5a25 docs(roms): add manual-acquisition list for gated Pokemon romhacks 2026-06-05 22:54:39 -05:00
ginnoir f23b197324 feat(roms): add Pokemon romhack patch + fetch pipeline
romhack-apply.py: pure-Python IPS/BPS/UPS applier (BPS/UPS self-verify via
embedded CRC32). romhack-fetch.py: data-driven driver that downloads, patches
against owned base ROMs, validates (header/logo), and places worthwhile hacks
into roms/<platform>/Hacks/ with 'Pokemon - <Hack> (Hack).<ext>' names.
Supports --batch, --place, --retain (store source artifacts).
2026-06-05 22:52:57 -05:00
ginnoir 797fe425b1 feat(roms): add igir per-platform dry-run + rename sweep helpers
igir-dryrun.sh: read-only igir report for one platform vs given DAT(s),
prints an identified%% summary + sample of unmatched files (the scoring
step of the platform sweep).

igir-dorename.sh: igir move to canonical names in place for one platform,
with before/after manifests for reversibility and --dir-game-subdir never
to keep folders flat. Cart-safe; not for disc systems with .m3u.
2026-06-05 22:37:35 -05:00
ginnoir 57aba346f9 feat(roms): add igir report + SNES header-strip ops scripts
igir-report.ps1: read-only igir report over the EmuDeck tree on valhalla,
matching ROMs by checksum against libretro's No-Intro + Redump DAT sets
(sparse-cloned + cached). Scoped (-Platforms) or whole-library (-All);
CSV per run under /storage1/igir/reports. Uses node:lts (alpine breaks
igir on musl) and metadat/no-intro (not the curated dat/ folder).

igir-strip-snes-headers.sh: reversible backup-then-strip of 512-byte SNES
copier headers (size%1024==512) that igir can't auto-detect, renaming to
.sfc so headered ROMs match headerless No-Intro.
2026-06-05 20:54:17 -05:00
ginnoir f5031e297b feat(roms): wire IGDB, SteamGridDB, ScreenScraper, RetroAchievements metadata keys 2026-06-05 00:35:12 -05:00
ginnoir 3a9a81ee35 feat(roms): add RomM stack (romm + mariadb) over /storage1/Emulation
New per-domain stack centralizing the emulation/ROM library:

- romm (rommapp/romm:4) + dedicated MariaDB; RomM bundles its own Redis

- library mounts existing EmuDeck tree /storage1/Emulation (RomM Structure A) read-write; tiered binds (config->/config, blobs->/storage1/labdata)

- env_file-only config (no ${VAR}) so the git stack needs no Portainer UI env vars

- Hasheous metadata enabled; IGDB/SteamGridDB/ScreenScraper scaffolded empty

- Caddy: roms.ginnoir.com internal_only -> romm:8080; regenerated bookmarks
2026-06-04 23:58:24 -05:00
ginnoir 1e2970d220 chore: add env_file: stack.env to all compose services
Portainer git-stack deployments require an explicit env_file reference
in the compose file to load variables from the repo's stack.env.
Adds env_file: [stack.env] before every service-level environment:
block across all 10 stacks.
2026-06-04 22:11:51 -05:00
ginnoir 441e1ee272 chore: rename stack .env files to stack.env for Portainer git stack compatibility
Portainer git-stack deploys require stack.env (not .env) in the repo.
Renames all stacks/*/. env → stacks/*/stack.env so Portainer reads
secrets directly from git on each redeploy, making the repo the single
source of truth instead of requiring manual UI sync.

Also fixes the dev stack BOOKSTACK_APP_KEY gap — the key was already
present in the file but missing from Portainer's stored envVars; it
will now be picked up automatically from stack.env on next redeploy.

Updates CLAUDE.md to reflect the new filename convention.
2026-06-04 21:41:56 -05:00
ginnoir 1cb0af1b0d feat(dev): add joxit registry-ui container + registry-ui.ginnoir.com Caddy block 2026-06-04 20:29:32 -05:00
ginnoir 49fefd57f4 fix(plane): bind plane_web to 0.0.0.0 so Caddy can reach it over edge network 2026-06-04 20:19:28 -05:00
ginnoir d73008dd9c fix(plane): add CELERY_BROKER_URL so api/worker/beat use Redis not AMQP 2026-06-04 20:06:20 -05:00
ginnoir c5f8e41e8a fix(bookstack): set APP_KEY so init no longer halts (docs.ginnoir.com 502)
linuxserver/bookstack refuses to start without APP_KEY:
  'The application key is missing, halting init!'
nginx never bound :80, so caddy -> bookstack:80 -> connection refused -> 502.

Key generated via 'docker run --rm --entrypoint /bin/bash
lscr.io/linuxserver/bookstack:latest appkey' on valhalla. Stored in
stacks/dev/.env per per-stack secrets policy; do NOT rotate without
invalidating existing sessions.
2026-06-04 19:25:01 -05:00
ginnoir f96efd9601 docs(claude): add bookmark-regen step to 'Adding a service' workflow 2026-06-04 19:22:01 -05:00
ginnoir 1934ace8d1 feat(scripts): add gen-bookmarks.ps1 + checked-in bookmark lists
scripts/gen-bookmarks.ps1 parses Caddyfile (site blocks grouped by
banner comments) and stacks/*/docker-compose.yml (services with
published ports) into two Netscape-format HTML bookmark files importable
by any browser. Run after editing Caddyfile or stack compose; commit the
regenerated HTMLs alongside the source change.
2026-06-04 19:20:43 -05:00
ginnoir 98b3fe4fd8 fix(caddy): drop tls_insecure_skip_verify on code.ginnoir.com (502)
tls_insecure_skip_verify inside transport http implicitly opts the
upstream connection into TLS. code-server (lscr.io/linuxserver) serves
plain HTTP on 8443, so Caddy's TLS handshake failed with
'tls: first record does not look like a TLS handshake' -> 502.
2026-06-04 18:50:24 -05:00
ginnoir f995014d48 docs+ops: rewrite CLAUDE.md, skills, runner for post-split topology
Post-monolith documentation and ops cleanup:

- CLAUDE.md rewritten end-to-end. Documents the 11 Portainer-managed stacks
  + raw-compose management plane, the single shared `edge` network, the
  SSD-vs-ZFS bind-mount tiering, the four deployment channels (git push
  for app stacks, runner for Caddyfile, apply-compose.ps1 for mgmt plane /
  Caddyfile / vault), and the fact that the repo is now canonical.
- homelab-apply skill rewritten for the new channels — no more `dc up -d`,
  no more monolith.
- homelab-ssh skill rewritten — no `dc` alias, plain `docker` against
  container names; per-stack compose ops via /data/compose/<id>/...
- homelab-sync skill + sync-prod.ps1 retired. The repo is canonical now;
  pulling from prod is the wrong direction.
- .github/workflows/deploy.yml: drop the dc up -d steps, gate on
  paths:[Caddyfile, .github/workflows/deploy.yml], reload caddy via
  `docker exec` (no longer through compose).
- apply-compose.ps1: drop -Compose and -DevStack flags; -Caddy now reloads
  via `docker exec caddy` (Caddy is in its own Portainer stack now).

No live container is touched by this commit. The runner workflow is
currently disabled at the repo level; re-enabling it makes Caddyfile pushes
auto-deploy again.
2026-06-04 18:17:16 -05:00
ginnoir cee43aa733 refactor(cutover): retire monolith — runner + apply script post-split
Application stacks now live in stacks/<domain>/ and are deployed by
Portainer's git poll. The runner workflow no longer pushes or applies the
monolithic root docker-compose.yml — it just pushes the Caddyfile and reloads
Caddy (now via docker exec, since `dc exec` no longer makes sense). apply-
compose.ps1 drops its -Compose and -DevStack flags for the same reason.

Local docker-compose.yml and dev-compose.yml are deleted; their content has
been split into stacks/* and is already running in production. The server
copies at /valhalla-lab/docker-compose.yml and /valhalla-lab/dev-compose.yml
are now orphan files (no workflow or script writes to them) and will be
removed after Caddy is cut over to its own proxy Portainer stack.

No live container is touched by this commit. The runner's next invocation
runs the new strict-subset workflow (Caddyfile push + reload only).
2026-06-04 17:59:23 -05:00
ginnoir b7f51524d9 fix(foundry): disable dead plutonium patch gist (404 -> crash/backoff loop) 2026-06-04 17:04:12 -05:00
ginnoir 10f30b4331 refactor(cutover): attach caddy to shared edge network 2026-06-04 16:51:54 -05:00
ginnoir 57cd6ed270 refactor: split monolith into per-domain stacks (Phase A: build)
Add stacks/<domain>/ compose + env for the 11 target stacks (proxy, media,
foundry, owncloud, resume, famapp, authentik, notify, monitoring, remote, dev).
Each app stack joins a shared external `edge` network for Caddy and keeps its
DB/cache co-located (no shared backing services). All named volumes convert to
tiered bind mounts: DBs/configs -> /config (SSD), blobs/repos/registry ->
/storage1/labdata (ZFS). Gitea repos+LFS split to ZFS.

Move Vault into the management plane (portainer-compose.yml) and add the shared
`edge` network there. apply-compose.ps1 -Portainer now also pushes vault.hcl.

Additive only: root docker-compose.yml/.env/Caddyfile untouched, so the live
monolith is unchanged. Live cutover (Phase B) is next.
2026-06-04 16:11:31 -05:00
ginnoir e174807481 feat: add full dev stack (Gitea, code-server, registry, DBX, Vault, BookStack, Plane)
- dev-compose.yml: 16-service dev stack on dedicated devstack network
- vault.hcl: Vault file-backend config; Vault initialized and unsealed
- docker-compose.yml: add devstack as external network on caddy
- Caddyfile: LAN-only site blocks for all 7 dev services; Plane routed
  via Caddy path handles (no bundled proxy container)
- apply-compose.ps1: add -DevStack and -VaultUnseal flags
- .env: dev stack secrets + all 5 Vault unseal keys + root token

Dev URLs (LAN-only): gitea, code, registry, dbx, vault, docs, plane
Stack managed by Portainer as dev-stack (id 7)
2026-06-04 15:14:49 -05:00
ginnoir 353263ddf8 refactor: move runner+watchtower to portainer-compose (management plane)
- portainer-compose.yml now owns all three management-plane services:
  portainer, github-runner, watchtower. All mount the docker socket;
  none belong to the application stack.
- Watchtower carries over its existing env (DOCKER_API_VERSION, ntfy URL).
  portainer, runner, and watchtower itself are labeled to opt out of
  watchtower auto-updates.
- docker-compose.yml: remove github-runner (was causing self-kill during
  dc up -d; exclusion workaround no longer needed)
- deploy.yml: simplified back to plain dc up -d with no service filtering
2026-06-04 14:31:33 -05:00
ginnoir 421cde626f chore: add GitHub runner access token to .env 2026-06-04 14:27:18 -05:00
ginnoir c741310a1e fix: exclude github-runner from dc up -d to avoid self-kill during deploy 2026-06-04 14:21:08 -05:00
ginnoir 76487a28c8 feat: move GitHub Actions runner to Docker container
- Replace host systemd runner with myoung34/github-runner container
  in the main compose stack
- Mounts docker.sock (GID 1001), /valhalla-lab, and /config/caddy
  so the workflow can deploy compose and reload Caddy without SSH
- Update deploy.yml to write files via mounted paths and use a shared
  COMPOSE env var for cleaner step commands
- Add GITHUB_RUNNER_ACCESS_TOKEN placeholder to .env (fine-grained PAT
  with Administration:write needed for runner self-registration)
2026-06-04 14:15:44 -05:00
ginnoir 8560bf0f34 chore: opt into Node.js 24 for actions runner before June 16 forced migration 2026-06-04 14:09:03 -05:00
ginnoir 1ed7e93298 feat: add GitHub Actions runner workflow and fix portainer network race
- portainer-compose.yml: moves portainer from standalone container to
  compose-managed, giving the portainer_proxy network a compose owner.
  This ensures the network is reliably created before the main stack
  on fresh installs or after Docker state is wiped.
- .github/workflows/deploy.yml: self-hosted runner on valhalla runs
  dc pull + up on push to main, and hot-reloads Caddyfile.
- apply-compose.ps1: adds -Portainer flag to manage portainer-compose.yml.
2026-06-04 14:06:02 -05:00
ginnoir bb46b1c10e feat: migrate stack to Portainer git-managed, rename project to valhalla-lab
- Rename project from htpc-download-box to valhalla-lab (directory + dc alias)
- Remove portainer and watchtower from compose; run standalone outside managed stack
  to prevent self-termination during Portainer-triggered redeployments
- Add portainer_proxy external network shared by caddy and standalone portainer
  so Caddy can route to Portainer via Docker DNS without them being in the same stack
- Carve out POST /api/stacks/webhooks/* in Caddyfile so Portainer webhooks can
  reach through the internal_only guard without exposing the full UI
- Update apply-compose.ps1 and sync-prod.ps1 paths from htpc-download-box to valhalla-lab
2026-06-03 23:09:53 -05:00
ginnoir 8558872362 chore: consistency pass — restart policies, backdoor ports, remove dead services
docker-compose.yml:
- Remove jackett and nzbhydra2 services
- Fix restart: "no" → unless-stopped on deluge, nzbget, sonarr, radarr, bazarr, tautulli
- Fix restart: always → unless-stopped on owncloud, mariadb (owncloud), redis (owncloud)
- Add backdoor host ports: portainer 9100:9000, uptime-kuma 3001:3001, homarr 7575:7575

Caddyfile:
- Remove jackett.ginnoir.com and hydra.ginnoir.com blocks
- Drop stray @httpget redirect blocks from freshrss, resume, storage.j-costa.com
- Simplify resume.ginnoir.com (drop redundant https:// host alias)
- Fix tab indentation in j-costa.com and storage.j-costa.com tls blocks
- Fix space indent in dev.ginnoir.com

.env:
- Remove unused famapp_MINIO_ENDPOINT (hardcoded in compose)
- Remove unused NEXT_PUBLIC_APP_URL
2026-06-03 22:22:35 -05:00
ginnoir ea669e815c refactor: move homarr SECRET_ENCRYPTION_KEY to .env
Define HOMARR_SECRET_ENCRYPTION_KEY in .env and reference it from docker-compose.yml instead of hardcoding it inline, keeping all secrets in .env. Resolved value is unchanged (no container recreate); deployed to valhalla and validated with 'dc config homarr'.
2026-06-03 04:21:33 -05:00