Commit Graph
176 Commits
Author SHA1 Message Date
ginnoir 9878f17673 Add internal-only Caddy proxy for Kopia backup UI.
backup.ginnoir.com reverse-proxies kopia:51515; regenerate bookmark pages.
2026-06-10 21:32:06 -05:00
ginnoir bb18e0e338 Mount backup scripts from /config/backup on host.
Portainer CE git stacks cannot bind-mount relative repo paths; sync auxiliary files to the host path documented in README.
2026-06-10 21:28:56 -05:00
ginnoir f660bc9c4b Fix backup stack env for Portainer git deploy.
Remove compose ${VAR} overrides that blanked stack.env values and map KOPIA_REPOSITORY_PASSWORD in init-kopia; use hostname-only B2 endpoint.
2026-06-10 21:26:02 -05:00
ginnoir 7c7f6dfebc Add Kopia backup stack with Backblaze B2 offsite target.
Protects /config, selective labdata, and nightly DB dumps; documents homelab improvement plan briefs.
2026-06-10 21:21:34 -05:00
ginnoir 225078afbe chore(roms): pin romm to 4.9.0-beta.2 for save-sync endpoints + content_hash fix 2026-06-09 18:36:14 -05:00
ginnoir 3cc86d866f fix(vault): raise memlock ulimit to unlimited
Newer hashicorp/vault:latest mlocks more memory than docker's default
64KB memlock ulimit allows; vault crash-looped for ~5900 restarts with
'Failed to lock memory: cannot allocate memory' despite cap_add IPC_LOCK.
Deployed via apply-compose.ps1 -Portainer (mgmt plane, not a git stack).
2026-06-09 18:06:38 -05:00
ginnoir 7aa95fa00a refactor: pure env_file stack.env convention across all stacks
Every service now reads stack.env via env_file with container-exact var
names — no ${VAR} parse-time interpolation, no Portainer UI env vars
(except stacks/dev, kept on UI-env interpolation as documented exception).

- media: drop PUID/PGID/TZ env lines (env_file covers); resolve ${ROOT}
- foundry: secrets via env_file; FOUNDRY_PATCH_URLS -> CONTAINER_PATCH_URLS
- monitoring: HOMARR_SECRET_ENCRYPTION_KEY -> SECRET_ENCRYPTION_KEY
- notify: FRESHRSS_USER composed in stack.env; TZ via env_file
- owncloud: container-exact OWNCLOUD_*/MYSQL_* keys; runtime $$ healthcheck
- resume/famapp: composites (DATABASE_URL etc.) written out in stack.env
- authentik: container-exact keys; image tag pinned in compose
- notes: couchdb creds via env_file
- remote: gains stack.env (no secrets; convention completeness)
- CLAUDE.md: env convention, valhalla-lab registration-only note, correct
  on-host Portainer working-copy path (/config/portainer/compose/<id>)

Validated with docker compose config on valhalla for all 15 stacks.
2026-06-09 16:03:47 -05:00
ginnoir bb2bc6a20e feat: serve romhack archive site
Deploy to valhalla / deploy (push) Has been cancelled
2026-06-08 21:17:22 -05:00
ginnoir 27aa1bf2fe fix(romhacks): only link library files that exist on disk
handoff.json lists files captured at routing time, but the
extraction-cruft cleanup later removes files extracted out of a kept
archive. The Caddy file_server can only serve real files (not archive
interiors), so existence-filter the art + guide rel-paths against
library/<slug>/ before embedding/linking them — dropping phantom
entries that would 404.
2026-06-08 05:02:34 -05:00
ginnoir 9210cd23f8 feat(romhacks): serve library over internal Caddy file_server; notes embed art + guide links
Deploy to valhalla / deploy (push) Has been cancelled
- proxy: read-only bind /storage1/labdata/romhacks/library -> /srv/romhacks
- Caddyfile: romhacks-files.ginnoir.com (internal_only) file_server + browse
- render_catalog_notes.py + handoff.py: embed box art and link guides/
  spreadsheets via https://romhacks-files.ginnoir.com/<slug>/<file>
  (handoff art/guides; guide filter drops buried game-data txt, caps at 30)
- regenerate bookmarks
2026-06-08 04:44:38 -05:00
ginnoir 8224a8fddb chore(romhacks): catalog publisher tooling (analyze + render Obsidian notes) 2026-06-08 04:25:58 -05:00
ginnoir ae57c7f661 chore(romhacks): add extraction-cruft cleanup script (keeps archives, audits for loss) 2026-06-08 03:38:37 -05:00
ginnoir 1360a8fa65 feat(romhacks): automated hand-off — route downloads to RomM + library, render notes
handoff.py classifies completed downloads (console rom vs RPG-Maker fangame vs
patch, via bsdtar archive peek incl. RAR5), copies console roms into
Emulation/roms/<platform> for RomM, moves the full folder (rom + guides) into
the library archive, and renders note.md + handoff.json per hack for the
Obsidian publish step. Wired into the cycle; single /data mount keeps moves as
fast renames. Validated on 4 real games.
2026-06-07 20:25:30 -05:00
ginnoir 328b5df776 feat(romhacks): arm downloads (DRY_RUN=false) + fix dry-run state burn
process_file now marks messages processed only after a real crawljob write, so
DRY_RUN no longer poisons the state and link-less posts get re-checked after the
host filter widens. State file reset alongside this so the arm cycle re-enqueues
all threads.
2026-06-07 19:55:00 -05:00
ginnoir 4fda9a3b3e feat(romhacks): widen host filter (pokeharbor, raw.githubusercontent, pokehostel) 2026-06-07 19:40:01 -05:00
ginnoir d9724f39e6 fix(romhacks): build on Alpine DCE base (apk + self-contained CLI)
The tyrrrz/discordchatexporter image is Alpine, not Debian, and ships DCE as a
self-contained executable (no dotnet on PATH). Use apk for python3 and invoke
/opt/app/DiscordChatExporter.Cli directly.
2026-06-07 18:57:42 -05:00
ginnoir ba207a0fcd feat(romhacks): add Discord rom-hack ingestion stack (DCE -> JDownloader -> RomM)
Inert by default: empty token keeps the orchestrator idle and DRY_RUN=true
suppresses downloads. Orchestrator exports per-generation forum threads via
DiscordChatExporter, extracts download links + art, stages metadata, and writes
JDownloader crawljobs. Armed here with an alt account token and all 11 romhack
forums, still in DRY_RUN pending first-cycle validation.
2026-06-07 18:53:23 -05:00
ginnoir 91d6724175 feat(deck): add SteamGridDB batch art scripts and skill
scripts/deck-sgdb-art.py  — runs on Deck; fetches hero, wide capsule,
and portrait/poster art from SGDB for all RomM-synced shortcuts.
scripts/deck-sgdb-art.ps1 — Windows wrapper; handles SSH ASKPASS
upload and execution.
.claude/skills/deck-sgdb/ — Claude skill documenting invocation,
file naming conventions, appid formula, and known gotchas.
2026-06-07 18:17:45 -05:00
ginnoir 49bda4234a fix(caddy): make roms.ginnoir.com internal-only
Deploy to valhalla / deploy (push) Has been cancelled
External access for a trusted friend now goes through the share stack's
Tailscale serve node (roms-share.<tailnet>.ts.net), so roms.ginnoir.com
no longer needs public exposure. Add import internal_only (LAN + tailnet).
2026-06-06 23:18:22 -05:00
ginnoir bf68f5a3da fix(share): absolute host bind for serve config
Portainer's git-stack checkout does not materialize sibling repo files
at the relative bind path, so ./serve-roms.json mounted as an empty
auto-created directory and tailscale serve failed ('is a directory').
Switch to an absolute /config/share/serve-roms.json bind (matching every
other stack's /config convention); the file is mirrored on the host.
2026-06-06 23:06:30 -05:00
ginnoir 2aa8122cd3 feat(share): Tailscale serve stack to share internal services with external tailnet users
Adds stacks/share/ — a dedicated stack of userspace Tailscale 'serve'
nodes that expose individual internal services to external tailnet
users (a friend on his own tailnet) over each node's 100.x identity.
No public exposure, no LAN access, immune to the friend's home-subnet
addressing. First node ts-roms serves RomM at roms-share.<tailnet>.ts.net
-> romm:8080. One reusable tag:share auth key + one ACL rule cover every
node; adding a service is a serve-<svc>.json + a copied service block.

roms stack reverted to a pointer comment.
2026-06-06 22:52:15 -05:00
ginnoir dc0a4eb58f revert(caddy): drop RomM empty-Bearer workaround
Deploy to valhalla / deploy (push) Has been cancelled
decky-romm-sync has completed its first-run token mint, so the
Authorization: Bearer (empty token) probe that crashed RomM 4.8.x no
longer occurs. Collapse roms.ginnoir.com back to a plain reverse_proxy.
2026-06-06 20:53:19 -05:00
ginnoir cb18047973 feat: add RomM save staging and import utility scripts 2026-06-06 20:51:52 -05:00
ginnoir ca4cdf3d8a fix(caddy): allow Tailscale CGNAT range in internal_only snippet
Deploy to valhalla / deploy (push) Has been cancelled
Tailnet clients that reach Caddy without subnet-route SNAT arrive from
100.64.0.0/10 and were getting 403 from LAN-only site blocks. Add the
range to the internal_only matcher so split-DNS tailnet access works
regardless of the --snat-subnet-routes setting.
2026-06-06 20:31:39 -05:00
ginnoir 04202a97b7 feat: add RomM database path mismatch resolver script 2026-06-06 19:50:21 -05:00
ginnoir 3bcfe26db9 chore: track Codex/Antigravity agent config (.agents skills + .codex) 2026-06-06 19:44:51 -05:00
ginnoir cba0ae7f4b docs: add AGENTS.md pointer so Codex/Antigravity load project context 2026-06-06 18:24:22 -05:00
ginnoir a96ff619a8 fix(caddy): strip empty Bearer header on roms so RomM stops 500ing decky-romm-sync
Deploy to valhalla / deploy (push) Has been cancelled
RomM 4.8.x crashes with HTTP 500 on a malformed Authorization header:
hybrid_auth.py unpacks `scheme, token = header.split()` with no guard, so
any value without exactly two space-separated parts raises ValueError.

decky-romm-sync >=0.20.0 (Client API Token auth, added 2026-06-05) sends
`Authorization: Bearer ` with an empty token on its first-run heartbeat/
platforms/devices probes, before it has minted a token, which trips that
crash and deadlocks setup. The Basic-auth mint (POST /api/client-tokens)
is never reached.

Strip the empty-token header for the roms host so the probe gets a clean
200 and the mint can run; thereafter the plugin sends a real Bearer rmm_
token. Well-formed Bearer tokens and browser session-cookie auth are
unaffected. Remove once the plugin omits the empty header or RomM guards
the split.
2026-06-06 16:00:25 -05:00
ginnoir 7ec4e2ca57 chore: spin off Pokémon ROM-hack material to standalone repo
The Pokémon catalog + pipeline are moved out of this infra repo. The catalog
notes now live in the self-hosted Obsidian vault (Pokémon ROM Hacks/); the
acquisition/patching scripts live in their own repo at Documents\pokemon.
homelabstack stays focused on the homelab.

- rm pokemon-romhack-vault/ (75 hack notes + Index/Types/README/Platforms)
- rm scripts/{build-romhack-vault,romhack-import,romhack-fetch,romhack-apply}.py
- rm pokemon-romhacks-wanted.md
- .gitignore: drop the now-dead pokemon/ drop-folder rule
- .claude/skills/vault/SKILL.md: condense the Pokemon-vault subtree to a one-line pointer

RomM stack (stacks/roms) and the general igir library scripts stay.
2026-06-06 15:49:26 -05:00
ginnoir b99a0a18f0 docs(vault): document Pokémon ROM Hacks vault section in skill
Adds the ROM Hacks folder tree to the vault structure reference so
future sessions know where _Claude.md, Index.md, Play Queue, Bases/,
and Platforms/ live without re-deriving it.
2026-06-06 14:57:42 -05:00
ginnoir 061147caea docs(vault): update skill with correct livesync two-level chunk format 2026-06-06 03:50:29 -05:00
ginnoir 3a2a4b4dc9 feat(vault): add vault skill for Claude read/write access to Obsidian notes 2026-06-06 03:26:10 -05:00
ginnoir f871d1cabf chore(env): store Portainer stacks PAT (Contents:read) separately from runner PAT 2026-06-06 03:19:45 -05:00
ginnoir eea602d100 feat(roms): add Obsidian vault knowledge base for 75 library romhacks
build-romhack-vault.py generates pokemon-romhack-vault/: one note per hack
(YAML properties + tags + summary + links), an Index MOC (Dataview + static
table), a Types taxonomy, and per-platform MOCs. Dataset is the single source
of truth — edit + re-run to extend.
2026-06-06 03:15:36 -05:00
ginnoir 5f53df0070 feat(notes): add Obsidian livesync stack (CouchDB) + Caddy entry
stacks/notes/: new stack with CouchDB 3 for obsidian-livesync sync backend.
Vault data at /config/couchdb/data; CORS must be configured post-deploy
via Fauxton (/_utils). Claude integration via CouchDB REST API documented
in compose comments.

Caddyfile: obsidian.ginnoir.com → couchdb:5984 (public for mobile sync).
2026-06-06 03:15:17 -05:00
ginnoir e997172c53 docs(roms): Adventure Red Chapter (Beta 15) identified + placed 2026-06-06 03:03:41 -05:00
ginnoir d0e097e2d4 feat(roms): add romhack drop-folder importer; import 53 hacks
romhack-import.py: classifies a mixed drop folder (ROMs/patches/docs/archives),
clean-names ROMs to 'Pokemon - <Hack> (Hack)', auto-detects patch base via
BPS/UPS CRC, recurses zips/rar, archives patches+docs. Imported the pokemon/
drop (now gitignored) into the valhalla library + Following Renegade Platinum.
Updated wanted-list for newly acquired hacks.
2026-06-06 03:00:17 -05:00
ginnoir 601150dcac feat(roms): make roms.ginnoir.com publicly accessible 2026-06-06 00:26:28 -05:00
ginnoir c744ff5a25 docs(roms): add manual-acquisition list for gated Pokemon romhacks 2026-06-05 22:54:39 -05:00
ginnoir f23b197324 feat(roms): add Pokemon romhack patch + fetch pipeline
romhack-apply.py: pure-Python IPS/BPS/UPS applier (BPS/UPS self-verify via
embedded CRC32). romhack-fetch.py: data-driven driver that downloads, patches
against owned base ROMs, validates (header/logo), and places worthwhile hacks
into roms/<platform>/Hacks/ with 'Pokemon - <Hack> (Hack).<ext>' names.
Supports --batch, --place, --retain (store source artifacts).
2026-06-05 22:52:57 -05:00
ginnoir 797fe425b1 feat(roms): add igir per-platform dry-run + rename sweep helpers
igir-dryrun.sh: read-only igir report for one platform vs given DAT(s),
prints an identified%% summary + sample of unmatched files (the scoring
step of the platform sweep).

igir-dorename.sh: igir move to canonical names in place for one platform,
with before/after manifests for reversibility and --dir-game-subdir never
to keep folders flat. Cart-safe; not for disc systems with .m3u.
2026-06-05 22:37:35 -05:00
ginnoir 57aba346f9 feat(roms): add igir report + SNES header-strip ops scripts
igir-report.ps1: read-only igir report over the EmuDeck tree on valhalla,
matching ROMs by checksum against libretro's No-Intro + Redump DAT sets
(sparse-cloned + cached). Scoped (-Platforms) or whole-library (-All);
CSV per run under /storage1/igir/reports. Uses node:lts (alpine breaks
igir on musl) and metadat/no-intro (not the curated dat/ folder).

igir-strip-snes-headers.sh: reversible backup-then-strip of 512-byte SNES
copier headers (size%1024==512) that igir can't auto-detect, renaming to
.sfc so headered ROMs match headerless No-Intro.
2026-06-05 20:54:17 -05:00
ginnoir f5031e297b feat(roms): wire IGDB, SteamGridDB, ScreenScraper, RetroAchievements metadata keys 2026-06-05 00:35:12 -05:00
ginnoir 3a9a81ee35 feat(roms): add RomM stack (romm + mariadb) over /storage1/Emulation
New per-domain stack centralizing the emulation/ROM library:

- romm (rommapp/romm:4) + dedicated MariaDB; RomM bundles its own Redis

- library mounts existing EmuDeck tree /storage1/Emulation (RomM Structure A) read-write; tiered binds (config->/config, blobs->/storage1/labdata)

- env_file-only config (no ${VAR}) so the git stack needs no Portainer UI env vars

- Hasheous metadata enabled; IGDB/SteamGridDB/ScreenScraper scaffolded empty

- Caddy: roms.ginnoir.com internal_only -> romm:8080; regenerated bookmarks
2026-06-04 23:58:24 -05:00
ginnoir 1e2970d220 chore: add env_file: stack.env to all compose services
Portainer git-stack deployments require an explicit env_file reference
in the compose file to load variables from the repo's stack.env.
Adds env_file: [stack.env] before every service-level environment:
block across all 10 stacks.
2026-06-04 22:11:51 -05:00
ginnoir 441e1ee272 chore: rename stack .env files to stack.env for Portainer git stack compatibility
Portainer git-stack deploys require stack.env (not .env) in the repo.
Renames all stacks/*/. env → stacks/*/stack.env so Portainer reads
secrets directly from git on each redeploy, making the repo the single
source of truth instead of requiring manual UI sync.

Also fixes the dev stack BOOKSTACK_APP_KEY gap — the key was already
present in the file but missing from Portainer's stored envVars; it
will now be picked up automatically from stack.env on next redeploy.

Updates CLAUDE.md to reflect the new filename convention.
2026-06-04 21:41:56 -05:00
ginnoir 1cb0af1b0d feat(dev): add joxit registry-ui container + registry-ui.ginnoir.com Caddy block 2026-06-04 20:29:32 -05:00
ginnoir 49fefd57f4 fix(plane): bind plane_web to 0.0.0.0 so Caddy can reach it over edge network 2026-06-04 20:19:28 -05:00
ginnoir d73008dd9c fix(plane): add CELERY_BROKER_URL so api/worker/beat use Redis not AMQP 2026-06-04 20:06:20 -05:00
ginnoir c5f8e41e8a fix(bookstack): set APP_KEY so init no longer halts (docs.ginnoir.com 502)
linuxserver/bookstack refuses to start without APP_KEY:
  'The application key is missing, halting init!'
nginx never bound :80, so caddy -> bookstack:80 -> connection refused -> 502.

Key generated via 'docker run --rm --entrypoint /bin/bash
lscr.io/linuxserver/bookstack:latest appkey' on valhalla. Stored in
stacks/dev/.env per per-stack secrets policy; do NOT rotate without
invalidating existing sessions.
2026-06-04 19:25:01 -05:00