fix(streaming): reach Plex HTTP via host :32402 proxy

Dry-run connected to both servers with matching ginnoir usernames.
Library counts already at/above Plex baseline while the rescan finishes.

.agents/skills/homelab-ssh/SKILL.md

@@ -5,7 +5,7 @@ description: Connect to and run commands on the valhalla homelab server (ginnoir
 
 # homelab-ssh
 
-The production homelab runs on a headless Ubuntu host reachable at `ssh ginnoir@valhalla` (key-based auth; resolves and works from this Windows host directly).
+The production homelab runs on a headless **EndeavourOS (Arch)** host reachable at `ssh ginnoir@valhalla` (key-based auth; resolves and works from this Windows host directly). Package manager is `pacman` — there is no `apt`/`apt-get`. `ginnoir` has passwordless `sudo`.
 
 ## Connect (non-interactive / safe for automation)
 

.claude/skills/homelab-ssh/SKILL.md

@@ -5,7 +5,7 @@ description: Connect to and run commands on the valhalla homelab server (ginnoir
 
 # homelab-ssh
 
-The production homelab runs on a headless Ubuntu host reachable at `ssh ginnoir@valhalla` (key-based auth; resolves and works from this Windows host directly).
+The production homelab runs on a headless **EndeavourOS (Arch)** host reachable at `ssh ginnoir@valhalla` (key-based auth; resolves and works from this Windows host directly). Package manager is `pacman` — there is no `apt`/`apt-get`. `ginnoir` has passwordless `sudo`.
 
 ## Connect (non-interactive / safe for automation)
 

.env

@@ -111,7 +111,8 @@ GITHUB_RUNNER_ACCESS_TOKEN=github_pat_11ACRHQAI0q3n1svrl9jmT_ai2bQlXWSxY8HJjEI6q
 # Used by StackCreateDockerStandaloneRepository and StackGitRedeploy.
 # Pass as RepositoryAuthorizationType:0 (Basic), RepositoryUsername=ginnoir.
 # Runner PAT above does NOT work for this — different scopes.
-GITHUB_STACKS_PAT=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
+# Gitea PAT for Portainer git stacks (env var name kept from GitHub era)
+GITHUB_STACKS_PAT=54ea93904439c2919f1d8dab1ba545e54a2a3e86
 
 # =============================================================
 # DEV STACK

.gitea/workflows/deploy-caddy.yml

@@ -17,6 +17,15 @@ jobs:
 
       - name: Push Caddyfile and reload Caddy
         run: |
+          # Job containers do NOT inherit the runner's /config/caddy bind.
+          # Docker-from-Docker with a host bind is the reliable path: the
+          # volume source is resolved on valhalla, not inside the job container.
+          # (Direct `cp /config/caddy/...` only works on the runner container
+          # itself, which is not where this step runs.)
           apt-get update -qq && apt-get install -y -qq docker.io
-          cp Caddyfile /config/caddy/Caddyfile
+          docker run --rm \
+            -v /config/caddy:/dest \
+            -v "$PWD/Caddyfile:/src/Caddyfile:ro" \
+            alpine:3.20 \
+            cp /src/Caddyfile /dest/Caddyfile
           docker exec caddy caddy reload --config /etc/caddy/Caddyfile

AGENTS.md

@@ -9,9 +9,9 @@ from your tool's user-level instructions (`~/.codex/AGENTS.md` / `~/.gemini/GEMI
 
 ## Essentials (full detail in `CLAUDE.md`)
 
-- **What this is:** deployment config for a live, single-host Docker homelab on a headless Ubuntu
+- **What this is:** deployment config for a live, single-host Docker homelab on a headless
-  server reachable at `ssh ginnoir@valhalla`. ~50 containers in per-domain Portainer-managed stacks.
+  **EndeavourOS (Arch)** server reachable at `ssh ginnoir@valhalla` — `pacman`, not `apt`.
-  This is **not** application code.
+  ~50 containers in per-domain Portainer-managed stacks. This is **not** application code.
 - **The repo is canonical.** Portainer polls `main` every 5 min and redeploys any app stack whose
   `stacks/<domain>/*` files changed. Editing here changes nothing until you `git push`.
 - **Deployment channels:**

CLAUDE.md

@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## What this repo is
 
-Deployment configuration for a **live, single-host Docker homelab** running on a headless Ubuntu server reachable at `ssh ginnoir@valhalla`. This repo is **not application code** — it is the canonical source for ~50 containers organized into per-domain Portainer-managed stacks.
+Deployment configuration for a **live, single-host Docker homelab** running on a headless **EndeavourOS (Arch)** server reachable at `ssh ginnoir@valhalla` — package manager is `pacman`, there is no `apt`. This repo is **not application code** — it is the canonical source for ~50 containers organized into per-domain Portainer-managed stacks.
 
 **The repo is canonical.** Portainer polls the **Gitea** repo `https://gitea.ginnoir.com/ginnoir/homelabstack` on `main` every 5 min and redeploys any application stack whose source files have changed. Editing here changes nothing until you push (or, for the management plane, run `apply-compose.ps1 -Portainer`). GitHub is a temporary push mirror only.
 
@@ -173,12 +173,15 @@ These run on valhalla but are managed outside of Portainer/Docker.
 
 ### Plex Media Server
 
-- **Install:** deb package (`plexmediaserver`), managed by systemd. Not containerised.
+- **Install:** AUR/pacman package (`plex-media-server`), managed by systemd. Not containerised.
-- **Version:** 1.43.2 (Watchtower doesn't touch it — update via `apt upgrade plexmediaserver` or the Plex UI).
+- **Version:** 1.43.2 (Watchtower doesn't touch it — update via `pacman -Syu plex-media-server` or the Plex UI).
 - **Service:** `systemctl {start,stop,restart,status} plexmediaserver`
 - **Binary:** `/usr/lib/plexmediaserver/`
-- **App data / metadata / DB:** `/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/`
+- **App data / metadata / DB:** `/var/lib/plex/Plex Media Server/` — note `/var/lib/plex` is a
-- **Media libraries:** `/storage1/Media/{Movies,TV,Anime}` — same ZFS pool bind-mounted into the *arr containers.
+  **symlink to `/storage1/labdata/plex`**, so Plex's 383G of app data lives on the ZFS array, not
+  the root disk. (`/var/lib/plexmediaserver/` does not exist.)
+- **Media libraries:** `/storage1/{Movies,TV,Anime}` — `/storage1/Media/*` are symlinks to these.
+  Same ZFS pool bind-mounted into the *arr containers.
 - **Tautulli** (`tautulli` container in the media stack) reads PMS logs from `/storage1/Media` and connects to Plex over the LAN. It is **not** behind Authentik — Plex login is its own auth.
 - **Caddy:** Plex is not reverse-proxied through Caddy (Plex handles its own relay/direct connections). No `plex.ginnoir.com` block exists.
 

Caddyfile

@@ -17,6 +17,18 @@
 	respond @blocked "Access denied" 403
 }
 
+# Same IP gate as (internal_only), but for use INSIDE a route{} block.
+# Caddy sorts the `route` directive ahead of `respond`, so an `import
+# internal_only` sitting above a catch-all `route {}` is dead code — the route
+# matches everything and handles the request before the 403 is ever reached.
+# Inside a route, directives run in written order, so importing this as the
+# first line of the route makes the gate fire. Use this, not internal_only,
+# on any site that wraps its handlers in route{} (i.e. the Authentik sites).
+(internal_gate) {
+	@blocked_ip not remote_ip 192.168.1.0/24 172.16.0.0/12 100.64.0.0/10 127.0.0.1
+	respond @blocked_ip "Access denied" 403
+}
+
 # TB-006 — Authentik forward auth (embedded outpost on authentik-server:9000).
 # Wrap protected sites in `route { ... }` so bypass handles run before forward_auth.
 (authentik_outpost) {
@@ -36,6 +48,32 @@
 	remote_ip 192.168.1.0/24 172.16.0.0/12 100.64.0.0/10 127.0.0.1
 }
 
+# Shared-link gate — external users visit ?k=<SHARE_KEY> once to get a 30-day cookie.
+# Internal IPs and tailnet clients pass through unconditionally.
+# Usage: import share_gate  (replaces import internal_only on shared services)
+# Uses {$SHARE_KEY} (parse-time substitution) — baked in at caddy reload/start.
+# SameSite=Lax (not Strict): Strict causes Safari/Firefox to drop the cookie on the
+# redirect when the top-level navigation originated from an external app (Discord).
+# Rotate: update SHARE_KEY in stack.env, push → Portainer redeploys → new key.
+(share_gate) {
+	@external not remote_ip 192.168.1.0/24 172.16.0.0/12 100.64.0.0/10 127.0.0.1
+	handle @external {
+		route {
+			@has_key query k={$SHARE_KEY}
+			handle @has_key {
+				header Set-Cookie "share_access={$SHARE_KEY}; Path=/; Max-Age=2592000; HttpOnly; Secure; SameSite=Lax"
+				redir {http.request.uri.path} 302
+			}
+			@has_cookie expression `{http.request.cookie.share_access} == "{$SHARE_KEY}"`
+			handle @has_cookie {
+			}
+			handle {
+				respond "Access denied" 403
+			}
+		}
+	}
+}
+
 # =============================================================
 # FOUNDRY VTT — public
 # =============================================================
@@ -54,16 +92,24 @@ foundry2.ginnoir.com {
 	reverse_proxy 5etools:80
 }
 
-pf2e.ginnoir.com {
-	root * /srv/aon
-	file_server
-}
-
 # =============================================================
 # FILE STORAGE — internal only (Nextcloud)
 # =============================================================
 files.ginnoir.com {
-	import internal_only
+	# Public share links must resolve for external users; the rest of Nextcloud
+	# stays LAN/tailnet-only. Can't use `import internal_only` here — this needs
+	# the same IP gate with a path-based exemption, so the matcher is inlined.
+	# Blocked = external client AND not a public-share path. The /core, /dist,
+	# /css, /js and theming paths are the assets the share page itself loads;
+	# without them an external visitor gets an unstyled, non-functional page.
+	@blocked {
+		not remote_ip 192.168.1.0/24 172.16.0.0/12 100.64.0.0/10 127.0.0.1
+		not path /s/* /index.php/s/* /public.php /public.php/*
+		not path /apps/files_sharing/* /index.php/apps/files_sharing/* /ocs/v2.php/apps/files_sharing/*
+		not path /core/* /dist/* /css/* /js/* /themes/* /apps/theming/* /index.php/apps/theming/*
+	}
+	respond @blocked "Access denied" 403
+
 	redir /.well-known/carddav /remote.php/dav 301
 	redir /.well-known/caldav /remote.php/dav 301
 	reverse_proxy nextcloud:80 {
@@ -75,11 +121,6 @@ files.ginnoir.com {
 # =============================================================
 # STATIC SITES — public
 # =============================================================
-ffttsystems.ginnoir.com {
-	root * /srv/ffttsystems
-	file_server
-}
-
 wa4.ginnoir.com {
 	root * /srv/wa4
 	file_server
@@ -92,13 +133,24 @@ requests.ginnoir.com {
 	reverse_proxy seerr:5055
 }
 
+# =============================================================
+# STREAMING — public (replaces Plex's own remote access)
+# =============================================================
+jellyfin.ginnoir.com {
+	reverse_proxy jellyfin:8096
+}
+
+wizarr.ginnoir.com {
+	reverse_proxy wizarr:5690
+}
+
 # =============================================================
 # MEDIA MANAGEMENT — internal only
 # =============================================================
 
 sonarr.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle /api/* {
 			reverse_proxy sonarr:8989
@@ -111,8 +163,8 @@ sonarr.ginnoir.com {
 }
 
 radarr.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle /api/* {
 			reverse_proxy radarr:7878
@@ -125,8 +177,8 @@ radarr.ginnoir.com {
 }
 
 bazarr.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle /api/* {
 			reverse_proxy bazarr:6767
@@ -139,8 +191,8 @@ bazarr.ginnoir.com {
 }
 
 prowlarr.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle /api/* {
 			reverse_proxy prowlarr:9696
@@ -153,8 +205,8 @@ prowlarr.ginnoir.com {
 }
 
 tautulli.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle {
 			import authentik_forward_auth
@@ -163,12 +215,23 @@ tautulli.ginnoir.com {
 	}
 }
 
+jellystat.ginnoir.com {
+	route {
+		import internal_gate
+		import authentik_outpost
+		handle {
+			import authentik_forward_auth
+			reverse_proxy jellystat:3000
+		}
+	}
+}
+
 # =============================================================
 # DOWNLOAD CLIENTS — internal only
 # =============================================================
 qbittorrent.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle {
 			import authentik_forward_auth
@@ -178,8 +241,8 @@ qbittorrent.ginnoir.com {
 }
 
 nzbget.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle {
 			import authentik_forward_auth
@@ -189,8 +252,8 @@ nzbget.ginnoir.com {
 }
 
 whisparr.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle /api/* {
 			reverse_proxy whisparr:6969
@@ -203,8 +266,8 @@ whisparr.ginnoir.com {
 }
 
 stash.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle {
 			import authentik_forward_auth
@@ -219,12 +282,12 @@ stash.ginnoir.com {
 # serve node (roms-share.<tailnet>.ts.net), NOT public exposure here.
 # =============================================================
 roms.ginnoir.com {
-	import internal_only
+	import share_gate
 	reverse_proxy romm:8080
 }
 
 romhacks.ginnoir.com {
-	import internal_only
+	import share_gate
 	root * /srv/romhacks-wiki
 	file_server
 }
@@ -360,6 +423,47 @@ homarr.ginnoir.com {
 	reverse_proxy homarr:7575
 }
 
+# Hermes Desktop remote backend. Only API/WS is exposed for native
+# clients; the browser dashboard is blocked so its injected session token
+# is not leaked to the public web.
+hermes.ginnoir.com {
+	route {
+		handle /api/* {
+			reverse_proxy 172.20.0.1:9119 {
+				header_up Host 172.20.0.1:9119
+			}
+		}
+		handle {
+			respond "Hermes dashboard browser UI is disabled on this public hostname." 403
+		}
+	}
+}
+
+# Hermes WebUI for browser/phone access to the valhalla Hermes runtime.
+# Internal-only; WebUI also enforces its own password auth.
+webui.ginnoir.com {
+	import internal_only
+	reverse_proxy 172.20.0.1:8787
+}
+
+# ComfyUI node editor — inference runs on the Mac at 192.168.1.121 (Metal/MPS).
+# LAN/tailnet only; no Authentik (WebSocket queue/progress breaks under forward_auth).
+#
+# The explicit `resolvers` below is what made DNS-01 work here, not the imgstudio
+# hostname (an earlier comment blamed a "false .com" in comfyui.ginnoir.com —
+# that was wrong). The LAN resolver returns NODATA for `SOA ginnoir.com`, so
+# certmagic's zone lookup climbs to `com.` and Cloudflare rejects it. The caddy
+# container now pins public resolvers stack-wide (stacks/proxy/docker-compose.yml),
+# making this block redundant; kept as belt-and-braces.
+imgstudio.ginnoir.com {
+	import internal_only
+	tls {
+		dns cloudflare {env.CF_API_TOKEN}
+		resolvers 1.1.1.1
+	}
+	reverse_proxy 192.168.1.121:8188
+}
+
 backup.ginnoir.com {
 	route {
 		import authentik_outpost

apply-compose.ps1

@@ -41,8 +41,11 @@ if ($EnvFile) {
 }
 
 if ($Caddy) {
+    # /config/caddy is root-owned on valhalla — scp directly fails with
+    # "Permission denied". Stage via /tmp and sudo-install.
     Write-Host "Pushing Caddyfile ..."
-    scp $caddyLocal "${server}:/config/caddy/Caddyfile"
+    scp $caddyLocal "${server}:/tmp/Caddyfile.new"
+    ssh $server "sudo cp /tmp/Caddyfile.new /config/caddy/Caddyfile && sudo chown root:root /config/caddy/Caddyfile && rm -f /tmp/Caddyfile.new"
     Write-Host "Reloading Caddy ..."
     ssh $server "docker exec caddy caddy reload --config /etc/caddy/Caddyfile"
 }

bookmarks-domains.html

@@ -15,17 +15,16 @@
         <DT><H3>TABLETOP TOOLS (public)</H3>
         <DL><p>
             <DT><A HREF="https://5etools.ginnoir.com">5etools</A>
-            <DT><A HREF="https://pf2e.ginnoir.com">pf2e</A>
         </DL><p>
 
-        <DT><H3>FILE STORAGE — public (Authentik on admin UI; shares + WebDAV bypass) (public)</H3>
+        <DT><H3>FILE STORAGE — internal only (Nextcloud) (internal)</H3>
         <DL><p>
             <DT><A HREF="https://files.ginnoir.com">files</A>
         </DL><p>
 
         <DT><H3>STATIC SITES (public)</H3>
         <DL><p>
-            <DT><A HREF="https://ffttsystems.ginnoir.com">ffttsystems</A>
+            <DT><A HREF="https://wa4.ginnoir.com">wa4</A>
         </DL><p>
 
         <DT><H3>MEDIA REQUESTS (public)</H3>
@@ -33,26 +32,33 @@
             <DT><A HREF="https://requests.ginnoir.com">requests</A>
         </DL><p>
 
-        <DT><H3>MEDIA MANAGEMENT (internal)</H3>
+        <DT><H3>STREAMING — public (replaces Plex's own remote access) (public)</H3>
+        <DL><p>
+            <DT><A HREF="https://jellyfin.ginnoir.com">jellyfin</A>
+            <DT><A HREF="https://wizarr.ginnoir.com">wizarr</A>
+        </DL><p>
+
+        <DT><H3>MEDIA MANAGEMENT (public)</H3>
         <DL><p>
             <DT><A HREF="https://sonarr.ginnoir.com">sonarr</A>
             <DT><A HREF="https://radarr.ginnoir.com">radarr</A>
             <DT><A HREF="https://bazarr.ginnoir.com">bazarr</A>
             <DT><A HREF="https://prowlarr.ginnoir.com">prowlarr</A>
             <DT><A HREF="https://tautulli.ginnoir.com">tautulli</A>
+            <DT><A HREF="https://jellystat.ginnoir.com">jellystat</A>
         </DL><p>
 
-        <DT><H3>DOWNLOAD CLIENTS (internal)</H3>
+        <DT><H3>DOWNLOAD CLIENTS</H3>
         <DL><p>
             <DT><A HREF="https://qbittorrent.ginnoir.com">qbittorrent</A>
-            <DT><A HREF="https://deluge.ginnoir.com">deluge</A>
             <DT><A HREF="https://nzbget.ginnoir.com">nzbget</A>
             <DT><A HREF="https://whisparr.ginnoir.com">whisparr</A>
             <DT><A HREF="https://stash.ginnoir.com">stash</A>
             <DT><A HREF="https://roms.ginnoir.com">roms</A>
             <DT><A HREF="https://romhacks.ginnoir.com">romhacks</A>
-            <DT><A HREF="https://jd.ginnoir.com">jd</A>
+            <DT><A HREF="https://jd.ginnoir.com">jd [internal]</A>
-            <DT><A HREF="https://romhacks-files.ginnoir.com">romhacks-files</A>
+            <DT><A HREF="https://idlegame.ginnoir.com">idlegame [internal]</A>
+            <DT><A HREF="https://romhacks-files.ginnoir.com">romhacks-files [internal]</A>
         </DL><p>
 
         <DT><H3>NOTIFICATIONS &amp; RSS — public (ntfy native auth; FreshRSS HTTP auth via Authentik) (public)</H3>
@@ -85,6 +91,9 @@
             <DT><A HREF="https://portainer.ginnoir.com">portainer</A>
             <DT><A HREF="https://uptime.ginnoir.com">uptime</A>
             <DT><A HREF="https://homarr.ginnoir.com">homarr [internal]</A>
+            <DT><A HREF="https://hermes.ginnoir.com">hermes</A>
+            <DT><A HREF="https://webui.ginnoir.com">webui [internal]</A>
+            <DT><A HREF="https://imgstudio.ginnoir.com">imgstudio [internal]</A>
             <DT><A HREF="https://backup.ginnoir.com">backup</A>
             <DT><A HREF="https://router.ginnoir.com">router [internal]</A>
         </DL><p>
@@ -99,6 +108,13 @@
             <DT><A HREF="https://vault.ginnoir.com">vault</A>
             <DT><A HREF="https://docs.ginnoir.com">docs [internal]</A>
             <DT><A HREF="https://plane.ginnoir.com">plane</A>
+            <DT><A HREF="https://nextcloud.ginnoir.com">nextcloud [internal]</A>
+            <DT><A HREF="https://office.ginnoir.com">office [internal]</A>
+        </DL><p>
+
+        <DT><H3>PHOTOS (internal)</H3>
+        <DL><p>
+            <DT><A HREF="https://photos.ginnoir.com">photos</A>
         </DL><p>
 
         <DT><H3>NOTES / PKM — internal only (LiveSync over LAN or tailnet) (internal)</H3>

bookmarks-ports.html

@@ -18,10 +18,7 @@
 
         <DT><H3>Dev</H3>
         <DL><p>
-            <DT><A HREF="ssh://git@valhalla:2222">gitea :2222</A>
-            <DT><A HREF="http://valhalla:3030">gitea :3030</A>
             <DT><A HREF="http://valhalla:8443">code_server</A>
-            <DT><A HREF="http://valhalla:5000">registry</A>
             <DT><A HREF="http://valhalla:4224">dbx</A>
             <DT><A HREF="http://valhalla:6875">bookstack :6875</A>
         </DL><p>
@@ -42,13 +39,12 @@
 
         <DT><H3>Media</H3>
         <DL><p>
-            <DT><A HREF="http://valhalla:8112">deluge</A>
             <DT><A HREF="http://valhalla:6789">nzbget</A>
             <DT><A HREF="http://valhalla:8989">sonarr</A>
             <DT><A HREF="http://valhalla:7878">radarr</A>
             <DT><A HREF="http://valhalla:6767">bazarr</A>
             <DT><A HREF="http://valhalla:9696">prowlarr</A>
-            <DT><A HREF="http://valhalla:5055">overseerr</A>
+            <DT><A HREF="http://valhalla:5055">seerr</A>
             <DT><A HREF="http://valhalla:8181">tautulli</A>
             <DT><A HREF="http://valhalla:3232">qbittorrent :3232</A>
             <DT><A HREF="http://valhalla:6881">qbittorrent :6881 (TCP/UDP)</A>
@@ -78,6 +74,11 @@
             <DT><A HREF="http://valhalla:8080">owncloud_server</A>
         </DL><p>
 
+        <DT><H3>Photos</H3>
+        <DL><p>
+            <DT><A HREF="http://valhalla:2283">immich-server</A>
+        </DL><p>
+
         <DT><H3>Proxy</H3>
         <DL><p>
             <DT><A HREF="http://valhalla:80">caddy :80</A>
@@ -108,5 +109,12 @@
             <DT><A HREF="http://valhalla:8997">romm :8997</A>
         </DL><p>
 
+        <DT><H3>Streaming</H3>
+        <DL><p>
+            <DT><A HREF="http://valhalla:8096">jellyfin</A>
+            <DT><A HREF="http://valhalla:3009">jellystat :3009</A>
+            <DT><A HREF="http://valhalla:5690">wizarr</A>
+        </DL><p>
+
     </DL><p>
 </DL><p>

docs/2026-06-27-ornith-9b-benchmark.md

@@ -0,0 +1,61 @@
+# Ornith-1.0-9B vs gpt-oss-20b — P100 benchmark (2026-06-27)
+
+**TL;DR:** Ornith-1.0-9B works cleanly as a Hermes backend (tool-calls, `<think>`,
+content all correct) and matches gpt-oss-20b on **code quality** — but it is a **dense
+9B**, so on the compute-bound P100 it generates **~2–3× slower** than the MoE gpt-oss-20b
+(~6.3 vs ~13–23 tok/s). **Keep gpt-oss-20b as the default daily driver.** Ornith earns a
+spot in the menu as a coding *specialist* to A/B on real tasks, but doesn't displace it.
+
+## What was done
+- Downloaded `ornith-1.0-9b-Q5_K_M.gguf` (6.47 GB, exact byte match) → `/storage1/labdata/llm/models/`.
+- Added `ornith-1.0-9b` to `stacks/llm/llama-swap-config.yaml` (same Pascal macro: q8/q8 KV,
+  `--parallel 1`, `--flash-attn on`, `--jinja`, 64k ctx; native 256k so no YaRN). Deployed to
+  `/config/llm/` + restarted llama-swap. **Live in the menu now.**
+- Quant choice: **Q5_K_M** (safe overnight, no OOM). Loads using only **7.7 GB VRAM** at 64k —
+  huge headroom, so Q6_K or Q8_0 (9.5 GB, near-lossless) would also fit easily if you want more quality.
+
+## Results (identical prompts, temp 0.6 / top_p 0.95 / top_k 20)
+
+| Test | Metric | **ornith-1.0-9b** | **gpt-oss-20b** | Winner |
+|---|---|---|---|---|
+| Tool call | emits valid `tool_calls`? | ✅ `get_weather({"city":"Tokyo"})` | ✅ identical | tie |
+| Codegen (merge_intervals) | correct? | ✅ correct, non-mutating, tests pass | ✅ correct (mutates input list) | ornith (slightly cleaner) |
+| Debug (second_largest) | correct? | ✅ correct + edge-case guard | ✅ correct + O(n) alt | tie (gpt-oss more thorough) |
+| **Gen speed** | tok/s (codegen) | **6.4** | **21.7** | **gpt-oss 3.4×** |
+| **Gen speed** | tok/s (debug) | **6.4** | **18.9** | **gpt-oss 2.9×** |
+| Prefill (small) | tok/s | 49–104 | 133–188 | gpt-oss |
+| Prefill (deep ~25–30k ctx) | tok/s | 101.9 (n=30295) | 154.9 (n=23422) | gpt-oss 1.5× |
+| Reasoning verbosity | think chars (codegen/debug) | 433 / 452 (concise) | 1671 / 2118 (verbose) | — |
+| VRAM @ 64k | MiB | **7684** | ~12600 | ornith (more headroom) |
+
+## Why the speed gap (the important takeaway)
+gpt-oss-20b is **MoE with ~3.6B active params/token**; Ornith-9B is **dense (9B active/token)**.
+The P100 (GP100, cc 6.0) is compute-bound, so per-token cost scales with *active* params —
+2.5× more active params ≈ the ~2.5–3× slower generation we measured. This is architectural,
+not a config problem; no amount of tuning closes it. Your memory's "~5 tok/s ceiling" held for
+the MoE models; a dense 9B genuinely sits below that line on this GPU.
+
+## Verdict for your Hermes backend
+- **Default stays gpt-oss-20b.** Gen speed is already the UX pain point (~6 vs ~20 tok/s is the
+  difference between usable and painful for interactive agent loops), and Ornith doesn't win on
+  quality to justify being 3× slower.
+- **Keep Ornith as a menu specialist.** It's MIT, purpose-built for agentic coding, refreshingly
+  concise (less token waste on overthinking), and tool-calls cleanly. Good for one-shot coding
+  asks where you'll wait for quality.
+- **The real coding upgrade you already have on disk:** `Qwen3-Coder-30B-A3B-Instruct-UD-Q2_K_XL.gguf`
+  is a **coder-tuned MoE (3B active)** — it would be both *faster* than Ornith (MoE) *and*
+  code-specialized. If you want a faster coding model than gpt-oss, that's the one to benchmark
+  next, not Ornith.
+
+## Caveats
+- Benchmarks are a small hand-written suite (4 tasks), not SWE-bench. Ornith's headline 69.4
+  SWE-Bench Verified is **self-reported/unverified** by DeepReinforce.
+- The `deep_prefill` row measures prefill speed only — both models hit the 256-token cap mid-think
+  (`finish=length`, empty `content`), which is expected for that probe.
+- Raw responses saved on valhalla at `/tmp/ornith-bench/` (one `.json` per model+test).
+
+## Open decision for you
+The repo config change (`stacks/llm/llama-swap-config.yaml`) is **deployed to the host but NOT
+committed/pushed** — I don't push without your say-so, and pushing would trigger a Portainer
+redeploy of stack 34. Tell me to commit+push if you want the repo (canonical source) to match the
+live host, or to revert the menu entry if you'd rather not keep Ornith around.

docs/2026-06-28-gpt-oss-20b-ollama-benchmark.md

@@ -0,0 +1,84 @@
+# gpt-oss-20b — `.73` Ollama box vs valhalla P100 (2026-06-28)
+
+**TL;DR:** The same model (`gpt-oss-20b`) runs **~1.3–1.5× faster on generation** and
+**~1.6–2× faster on prefill** on ginnoir's `192.168.1.73` Ollama box than on valhalla's
+Tesla P100 llama-swap backend — *despite* the `.73` box partially offloading to CPU at 64K
+context. Tool-calls and code outputs are correct on both. For interactive Hermes use the
+`.73` box is the better backend whenever it's powered on; the P100 remains the always-on
+fallback. **Caveat:** at very deep context (~43K tokens) `.73` generation drops to ~8 tok/s
+(KV no longer fully GPU-resident).
+
+## Setup
+
+- **Endpoint:** `http://192.168.1.73:11434` — Ollama **0.30.11**, model `gpt-oss:20b`,
+  quant **MXFP4** (OpenAI's native gpt-oss 4-bit format), reported 20.9B params.
+- **Serving config:** `num_ctx 65536` (matches the P100's 64K and Hermes' hard ≥64K
+  requirement), sampling `temperature 0.6 / top_p 0.95 / top_k 20` (identical to the
+  2026-06-27 Ornith/P100 run).
+- **Measured via** Ollama's native `/api/chat` (exact `prompt_eval`/`eval` token counts +
+  durations). Cold model load at 64K ctx took **~65 s** (one-time).
+- **P100 baseline** = the gpt-oss-20b column from `docs/2026-06-27-ornith-9b-benchmark.md`
+  (llama-swap, q8/q8 KV, `--parallel 1`, `--jinja`, 64K).
+- GPU on `.73` **could not be identified** — no SSH (port 22 filtered). Inferred from VRAM
+  behavior (below): a fast but VRAM-limited card (or unified-memory host).
+
+## Results
+
+| Test | Metric | **.73 Ollama (MXFP4)** | **valhalla P100 (GGUF)** | `.73` advantage |
+|---|---|---|---|---|
+| Tool call | valid `tool_calls`? | ✅ `get_weather({"city":"Tokyo"})` | ✅ identical | tie (both correct) |
+| Codegen (`merge_intervals`) | correct? | ✅ correct, non-mutating, type-hinted | ✅ correct (mutates input) | `.73` slightly cleaner |
+| Debug (`second_largest`) | correct? | ✅ correct single-pass, handles dup max | ✅ correct | tie |
+| Deep-prefill | answered? | ✅ "quick brown fox" | ✅ (capped probe) | tie |
+| **Gen speed** | tok/s — tool_call | **33.1** | 23.0 | **1.44×** |
+| **Gen speed** | tok/s — codegen | **27.4** | 21.7 | **1.26×** |
+| **Gen speed** | tok/s — debug | **27.6** | 18.9 | **1.46×** |
+| **Prefill** | tok/s — shallow (~100–135 tok) | **240–287** | 133–188 | ~1.5–1.8× |
+| **Prefill** | tok/s — deep | **2542** (n=43.8k) | 155 (n=23.4k) | far faster (diff depths) |
+| Gen @ deep ctx | tok/s | 8.4 (n=43.8k ctx) | 12.7 (n=23.4k ctx) | **P100 wins at depth** |
+| Reasoning verbosity | think chars (codegen/debug) | 1760 / 1317 | 1671 / 2118 | comparable |
+| VRAM @ 64K | on-GPU / total | **8.82 / 14.16 GB** (partial CPU offload) | ~12.6 GB (100% GPU) | — |
+
+## Reading the numbers
+
+- **Shallow/typical depth is where `.73` wins decisively.** All three real tasks (tool-call,
+  codegen, debug) run at shallow context, and `.73` generates at **27–33 tok/s vs the P100's
+  ~19–23** — roughly the difference between "comfortable" and "sluggish" for an interactive
+  agent loop. Prefill is also ~1.5–1.8× faster, so first-token latency improves too.
+- **The `.73` box is VRAM-limited, not compute-limited.** At 64K ctx only **8.82 GB of the
+  14.16 GB** working set sits in VRAM — the rest (weights tail + deep KV) spills to system
+  RAM. It still beats the full-GPU P100, which means the card itself is much faster than the
+  P100; with more VRAM (or a smaller `num_ctx`) it would pull further ahead.
+- **The one place the P100 wins: very deep context.** At ~43K resident tokens, `.73`
+  generation falls to **8.4 tok/s** because the KV cache is partly in CPU RAM (memory-
+  bandwidth-bound attention). The P100 holds its whole 64K KV in VRAM and degrades more
+  gracefully (12.7 tok/s at 23K). In practice Hermes' steady-state prompt is ~16K, so this
+  rarely bites — but long sessions on `.73` will slow down more than on the P100.
+- **Quant differs**, so this isn't a pure hardware A/B: `.73` runs MXFP4 (gpt-oss's native,
+  near-lossless 4-bit) while the P100 GGUF quant is whatever llama-swap pulled. Both are
+  genuine gpt-oss-20b and both produced correct outputs; no quality regression observed.
+
+## Verdict for the Hermes backend
+
+- **Prefer `.73` when it's up.** It's the faster daily driver for gpt-oss-20b at the depths
+  Hermes actually runs at. Switch in-session with `/model --provider ollama --model gpt-oss:20b`.
+- **Keep the P100 (`valhalla-p100`) as the always-on default.** It's a container on the
+  24/7 server; the `.73` box may be a desktop/workstation that isn't always powered. The
+  P100 also degrades more gracefully at very deep context.
+- **If you want `.73` to be strictly better,** drop its `num_ctx` toward what Hermes needs
+  (it hard-requires ≥64K, so you can't go below that for Hermes) **or** put gpt-oss on a
+  bigger-VRAM card there — eliminating the CPU spill would lift both prefill and deep-context
+  generation.
+
+## Caveats
+
+- Small hand-written suite (4 tasks), not SWE-bench — measures latency/throughput and basic
+  correctness, not deep code quality.
+- The codegen prompt lost its back-ticked tokens to shell quoting during the run (prompt_n 95
+  vs the P100's 113); the model still produced a correct `merge_intervals`, and gen tok/s is
+  prompt-content-independent, so the speed comparison stands.
+- Deep-prefill rows use different context depths (43.8K on `.73` vs 23.4K on P100), so the
+  prefill-tok/s cells aren't directly comparable — read them as "each box's deep-prefill rate
+  at that depth," not a head-to-head ratio.
+- Raw responses saved on valhalla at `/tmp/ollama-bench/` (one `.json` per task); P100
+  baselines at `/tmp/ornith-bench/`.

docs/2026-06-28-ollama-73-fleet-benchmark.md

@@ -0,0 +1,87 @@
+# `.73` Ollama fleet benchmark — all 9 models (2026-06-28)
+
+**TL;DR:** Throughput across every model on `192.168.1.73`. Generation speed spans an
+**~5× range** — from `gemma4:e4b` at **~93 tok/s** down to the big `qwen3:30b-a3b` at
+**~18 tok/s**. **All 9 models emit valid tool-calls.** For an interactive agent backend the
+sweet spot is **`gpt-oss:20b` (~29 tok/s)** or **`gemma4:12b` (~48 tok/s)** if 12B quality
+suffices; the 30B-class Qwen MoEs are the slowest here (heavy CPU offload at 64K on this
+VRAM-limited box). **Caveat:** code-correctness for the heavy *thinking* models is
+indeterminate — they used the whole 768-token gen cap reasoning and never emitted code (see
+Caveats); re-run with a bigger budget to judge quality.
+
+## Setup
+- Endpoint `http://192.168.1.73:11434`, Ollama 0.30.11. Each model served at **`num_ctx
+  65536`** (Hermes' ≥64K requirement), sampling `temp 0.6 / top_p 0.95 / top_k 20`.
+- Native `/api/chat` timings. 4 tasks: tool-call, codegen (`merge_intervals`), debug
+  (`second_largest`), deep-prefill (~16K-token filler). Gen capped: 256 / 768 / 768 / 128.
+- Same `.73` box as the gpt-oss head-to-head in
+  `docs/2026-06-28-gpt-oss-20b-ollama-benchmark.md` (GPU still unidentified — no SSH).
+
+## Generation speed (tok/s) — the headline
+
+Average of the three real tasks (tool-call / codegen / debug), sorted fastest first:
+
+| Model | avg gen t/s | tool | codegen | debug | deep-ctx gen | cold load s | tool-call? |
+|---|--:|--:|--:|--:|--:|--:|:--:|
+| **gemma4:e4b** | **92.7** | 91.1 | 93.3 | 93.6 | 82.6 | 24.8 | ✅ |
+| **gemma4:12b** | **48.2** | 46.8 | 49.0 | 48.8 | 45.7 | 10.6 | ✅ |
+| **gpt-oss:20b** | **29.4** | 32.7 | 27.9 | 27.7 | 29.9 | 0.4¹ | ✅ |
+| **qwen3.6:35b-a3b** | **27.3** | 29.1 | 26.4 | 26.4 | 28.2 | 35.9 | ✅ |
+| **gemma4:26b** | **25.7** | 27.6 | 25.0 | 24.5 | 26.5 | 53.8 | ✅ |
+| **glm-4.7-flash** | **21.2** | 24.2 | 19.7 | 19.7 | 21.4 | 34.8 | ✅ |
+| **qwen3-vl:30b-a3b** | **19.1** | 22.7 | 16.5 | 18.2 | 20.6 | 29.8 | ✅ |
+| **qwen3-coder:30b** | **19.0** | 22.8 | 17.1 | 17.2 | 20.4 | 25.3 | ✅ |
+| **qwen3:30b-a3b** | **17.6** | 19.5 | 16.6 | 16.6 | 18.5 | 25.0 | ✅ |
+
+¹ gpt-oss was already resident from the prior run; real cold load is ~65 s.
+
+## Prefill speed (tok/s)
+
+| Model | shallow (~100 tok) | deep (~16K tok) |
+|---|--:|--:|
+| gemma4:e4b | 1193–1799 | 7283 |
+| gemma4:12b | 667–1078 | 3318 |
+| gpt-oss:20b | 281–379 | 2655 |
+| qwen3:30b-a3b | 63–149 | 1030 |
+| qwen3-coder:30b | 76–270 | 979 |
+| gemma4:26b | 110–127² | 977 |
+| qwen3.6:35b-a3b | 78–209 | 620 |
+| glm-4.7-flash | 70–163 | 649 |
+| qwen3-vl:30b-a3b | 68–127 | 588 |
+
+² gemma4:26b's first request after load measured 11.7 t/s (cold-cache artifact); ignore.
+
+## What stands out
+- **The two small gemmas are in a different league.** `gemma4:e4b` (~93 t/s) and
+  `gemma4:12b` (~48 t/s) are dense but small, so they sit fully on GPU and fly. If a 4B/12B
+  is smart enough for the job, they're the most responsive options by far.
+- **gpt-oss:20b is the best "big-brain, still-fast" pick** (~29 t/s) — MoE ~3.6B active keeps
+  it quick despite 20B total. `qwen3.6:35b-a3b` nearly matches it (~27 t/s) and may be
+  stronger; worth A/B-ing on real tasks.
+- **The 30B-a3b Qwen trio is the slowest** (~17–19 t/s). Same "3B-active" MoE label, but
+  larger total weights → more spills to CPU RAM at 64K on this VRAM-limited box, dragging
+  generation below gpt-oss. `qwen3-coder` being this slow undercuts it as a *fast* coding
+  model here.
+- **Every model tool-calls.** All 9 emitted a valid `get_weather({"city":"Tokyo"})`, so any
+  of them can drive Hermes' tool loop.
+
+## Correctness (partial — see caveat)
+- **Confirmed correct** code on the non-/light-thinking models that finished within the cap:
+  `gpt-oss:20b`, `gemma4:12b`, `qwen3-vl:30b-a3b`, `qwen3-coder:30b` (clean `def`,
+  `reason=stop` or code present), plus `gemma4:26b` & `qwen3:30b-a3b` on the task each
+  finished.
+- **Indeterminate** (truncated mid-reasoning, `content=0`, `reason=length`): `glm-4.7-flash`
+  (both), `gemma4:e4b` (both), `qwen3.6:35b-a3b` (both), `gemma4:26b` (codegen),
+  `qwen3:30b-a3b` (debug). These spent all 768 gen tokens in the `thinking` channel — **not
+  wrong, just unfinished.** A re-run at `num_predict ~3072` is needed to grade their output.
+
+## Caveats
+- The 768-token gen cap was too low for heavy chain-of-thought models — it bounds runtime but
+  truncates their answers. Speed (tok/s) is unaffected and valid; code *quality* for the
+  truncated set is not measured here.
+- Per-model VRAM split not captured (models unload after 2 min `keep_alive`); only
+  gpt-oss-20b is known (8.82 GB on-GPU / 14.16 GB total at 64K → partial CPU offload). The
+  slow 30B-class numbers are consistent with heavier offload.
+- Small hand-written suite, not SWE-bench. Quants are each model's Ollama default.
+- Raw per-task responses on valhalla at `/tmp/ollama-bench-all/` (`summary.json` + one JSON
+  per model/task); progress log `/tmp/ollama-bench-all/progress.txt`.

docs/superpowers/plans/2026-06-26-llm-backend-hermes.md

@@ -0,0 +1,385 @@
+# LLM inference backend for Hermes — Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Stand up a llama.cpp OpenAI-compatible inference server in a new `stacks/llm/` Portainer stack, serving Qwen2.5-14B-Instruct on valhalla's Tesla P100, and add it as a provider in the Hermes agent.
+
+**Architecture:** Single `llama-server` container (image `ghcr.io/ggml-org/llama.cpp:server-cuda`, verified to run on the P100's sm_60) gets the GPU via CDI (`nvidia.com/gpu=0`), loads a GGUF from `/storage1/labdata/llm/models`, and publishes its OpenAI `/v1` API on the host at `172.20.0.1:8090`. Host-side Hermes (systemd-managed) reaches it directly — no Caddy. Config follows repo conventions: pure `env_file`, no `${VAR}` interpolation, infra image pinned out of Watchtower.
+
+**Tech Stack:** Docker Compose (Portainer git stack), llama.cpp server, CUDA/CDI, Gitea-polled deploy, Hermes (Nous Research agent) YAML config.
+
+**Reference spec:** `docs/superpowers/specs/2026-06-26-llm-backend-hermes-design.md`
+
+---
+
+## Pre-verified facts (do not re-derive)
+
+- GPU: Tesla P100-PCIE-16GB, cc 6.0; CDI device `nvidia.com/gpu=0` valid. Prebuilt image runs with full GPU offload (tested live 2026-06-26).
+- `/storage1/labdata` is root-owned; `ginnoir` has **passwordless sudo**.
+- `edge` network gateway = `172.20.0.1` (host IP on `br-b5aa55c3fedf`). Hermes binds here.
+- Image contains `curl` and `bash`.
+- Hermes services: `hermes-dashboard.service`, `hermes-gateway.service`, `hermes-webui.service` (system systemd). Config: `~/.hermes/config.yaml` with a `providers:` list (existing `ollama` entry as a template).
+- Repo deploy: app stacks deploy via git push → Portainer polls Gitea every 5 min. **New** stacks must be registered once in Portainer (see memory `portainer-new-stack-registration`).
+- `.gitattributes` forces LF — ensure `stack.env` / compose are LF on commit.
+
+## File structure
+
+- **Create** `stacks/llm/docker-compose.yml` — the llama-server service (one responsibility: serve the model on the GPU).
+- **Create** `stacks/llm/stack.env` — `LLAMA_API_KEY` only (committed per repo policy).
+- **Host-side (not in repo):** `/storage1/labdata/llm/models/Qwen2.5-14B-Instruct-Q4_K_M.gguf`; one new `providers:` entry in `~/.hermes/config.yaml`.
+
+---
+
+### Task 1: Pre-stage the model on the host (must precede deploy)
+
+The container crash-loops if the GGUF is absent, so download it before Portainer deploys the stack.
+
+**Files:** none in repo (host filesystem only).
+
+- [ ] **Step 1: Create the model directory**
+
+Run:
+```bash
+ssh ginnoir@valhalla "sudo mkdir -p /storage1/labdata/llm/models && sudo ls -ld /storage1/labdata/llm/models"
+```
+Expected: directory exists.
+
+- [ ] **Step 2: Download Qwen2.5-14B-Instruct Q4_K_M (~9 GB)**
+
+Run:
+```bash
+ssh ginnoir@valhalla "cd /storage1/labdata/llm/models && sudo curl -fL -o Qwen2.5-14B-Instruct-Q4_K_M.gguf https://huggingface.co/bartowski/Qwen2.5-14B-Instruct-GGUF/resolve/main/Qwen2.5-14B-Instruct-Q4_K_M.gguf"
+```
+(Run in background if it's slow; it's a single ~9 GB file.)
+
+- [ ] **Step 3: Verify the download**
+
+Run:
+```bash
+ssh ginnoir@valhalla "sudo ls -lh /storage1/labdata/llm/models/Qwen2.5-14B-Instruct-Q4_K_M.gguf"
+```
+Expected: file ~8.9–9.0 GB. If the size is wildly off (e.g. a few KB), it's an HTML error page — re-download.
+
+---
+
+### Task 2: Create `stacks/llm/stack.env`
+
+**Files:**
+- Create: `stacks/llm/stack.env`
+
+- [ ] **Step 1: Generate an API key**
+
+Run:
+```bash
+openssl rand -hex 32
+```
+Copy the output for the next step.
+
+- [ ] **Step 2: Write the file** (replace `<HEX>` with the generated key)
+
+`stacks/llm/stack.env`:
+```dotenv
+# llm stack secrets — read directly by the container via env_file.
+# llama.cpp's server reads LLAMA_API_KEY from the environment (no --api-key flag,
+# no ${VAR} interpolation), matching the repo's pure-env_file convention.
+LLAMA_API_KEY=<HEX>
+```
+
+- [ ] **Step 3: Confirm LF line endings**
+
+Run:
+```bash
+git check-attr text eol -- stacks/llm/stack.env
+```
+Expected: `eol: lf` (enforced by `.gitattributes`).
+
+---
+
+### Task 3: Create `stacks/llm/docker-compose.yml`
+
+**Files:**
+- Create: `stacks/llm/docker-compose.yml`
+
+- [ ] **Step 1: Write the compose file**
+
+`stacks/llm/docker-compose.yml`:
+```yaml
+# llm stack — local LLM inference backend for the Hermes agent.
+#
+# Single service: llama.cpp's OpenAI-compatible server (llama-server) serving
+# Qwen2.5-14B-Instruct (Q4_K_M GGUF) on the host's Tesla P100-16GB via CDI.
+# Chosen over vLLM because the P100 (GP100, compute capability 6.0) lacks the
+# DP4A INT8 instructions vLLM's AWQ/GPTQ kernels require — see
+# docs/superpowers/specs/2026-06-26-llm-backend-hermes-design.md.
+#
+# Pure env_file (LLAMA_API_KEY) — no Portainer UI env, no ${VAR} interpolation.
+# Image is infra-pinned out of Watchtower (manual tag bumps only).
+#
+# The server's OpenAI API is published on the host at 172.20.0.1:8090 (the edge
+# bridge gateway, a local host IP). Host-side Hermes reaches it there directly;
+# no Caddy block this round. Model weights live on the ZFS tier; the
+# /storage1/labdata/llm/models dir is pre-created with the GGUF before deploy.
+services:
+  llama-server:
+    image: ghcr.io/ggml-org/llama.cpp:server-cuda
+    container_name: llama-server
+    restart: unless-stopped
+    labels:
+      - "com.centurylabs.watchtower.enable=false"
+    networks: [llm]
+    env_file:
+      - stack.env
+    devices:
+      - "nvidia.com/gpu=0"
+    volumes:
+      - /storage1/labdata/llm/models:/models
+    command:
+      - "-m"
+      - "/models/Qwen2.5-14B-Instruct-Q4_K_M.gguf"
+      - "--alias"
+      - "qwen2.5-14b-instruct"
+      - "-ngl"
+      - "99"
+      - "--ctx-size"
+      - "32768"
+      - "-fa"
+      - "--cache-type-k"
+      - "q8_0"
+      - "--cache-type-v"
+      - "q8_0"
+      - "--host"
+      - "0.0.0.0"
+      - "--port"
+      - "8080"
+    ports:
+      - "172.20.0.1:8090:8080"
+    healthcheck:
+      test: ["CMD", "curl", "-fsS", "http://localhost:8080/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 5
+      start_period: 180s
+
+networks:
+  llm:
+    name: llm
+    driver: bridge
+```
+
+- [ ] **Step 2: Validate compose syntax locally**
+
+Run:
+```bash
+docker compose -f stacks/llm/docker-compose.yml config -q
+```
+Expected: no output (valid). If it errors on the CDI `devices` entry, that's a local-CLI version quirk, not a deploy blocker — the daemon on valhalla (Docker 29.5.2) supports it; proceed.
+
+---
+
+### Task 4: Commit and push the stack to Gitea
+
+**Files:** none new (commits Tasks 2–3).
+
+- [ ] **Step 1: Stage and commit**
+
+```bash
+git add stacks/llm/docker-compose.yml stacks/llm/stack.env docs/superpowers/specs/2026-06-26-llm-backend-hermes-design.md docs/superpowers/plans/2026-06-26-llm-backend-hermes.md
+git commit -m "feat(llm): add llama.cpp inference stack for Hermes (Qwen2.5-14B on P100)"
+```
+
+- [ ] **Step 2: Push (deploys nothing yet — stack isn't registered)**
+
+Use the **homelab-apply** skill's push path (push to Gitea; GitHub is the mirror). A `stacks/llm/*` change only redeploys once the stack is registered (Task 5).
+
+Run (per repo convention — token via one-off http.extraheader, never in git config):
+```bash
+git push   # to the configured remote(s); Gitea is primary
+```
+Expected: push succeeds; Portainer cannot yet act on `stacks/llm` because no stack references it.
+
+---
+
+### Task 5: Register the new Portainer git stack (one-time)
+
+New stacks aren't auto-created by polling — register once, then future pushes redeploy. See memory `portainer-new-stack-registration`.
+
+**Files:** none (Portainer state).
+
+- [ ] **Step 1: Read an existing git stack's config to copy repo URL + credential reference**
+
+Use the portainer MCP (invoke `get_guidance` first per the portainer-mcp-hygiene skill). Inspect a working app stack (e.g. `roms`) to copy the exact Gitea repo URL, ref (`refs/heads/main`), and the working fine-grained PAT/credential the other stacks use:
+```
+mcp__portainer__StackList  (select: name, GitConfig)
+mcp__portainer__StackInspect  on the roms stack id
+```
+
+- [ ] **Step 2: Create the stack from the git repository**
+
+Create a Docker standalone stack from the Gitea repo with:
+- compose path: `stacks/llm/docker-compose.yml`
+- ref: `refs/heads/main`
+- auto-update / git polling: **on** (match other app stacks)
+- env: **empty** (pure env_file)
+- credentials: the same working fine-grained PAT the other stacks use (the runner PAT cannot clone)
+
+Use `mcp__portainer__StackCreateDockerStandaloneRepository`. Per the memory note, the MCP call may time out but still succeed.
+
+- [ ] **Step 3: Verify the stack registered and deployed**
+
+Poll:
+```
+mcp__portainer__StackList  (select: [].{name:Name,status:Status})
+```
+Expected: a `llm` stack appears. Then confirm the container is running:
+```bash
+ssh ginnoir@valhalla "docker ps --filter name=llama-server --format '{{.Names}} {{.Status}}'"
+```
+Expected: `llama-server  Up … (health: starting|healthy)`.
+
+---
+
+### Task 6: Verify deploy — health, GPU offload, OpenAI API
+
+**Files:** none.
+
+- [ ] **Step 1: Confirm the model loaded on the GPU**
+
+Run:
+```bash
+ssh ginnoir@valhalla "docker logs --tail 60 llama-server 2>&1 | grep -iE 'P100|model loaded|listening|error|assert|cache_type|n_ctx'"
+```
+Expected: `Tesla P100`, `model loaded`, `server is listening`, no asserts. If logs show a `-fa` parse error, edit the compose to replace `-fa` with `--flash-attn` + `on` (two list items), re-commit/push, and let it redeploy.
+
+- [ ] **Step 2: Confirm VRAM is held (real offload, not CPU)**
+
+Run:
+```bash
+ssh ginnoir@valhalla "nvidia-smi --query-compute-apps=pid,used_memory --format=csv,noheader; nvidia-smi --query-gpu=memory.used,memory.free --format=csv,noheader"
+```
+Expected: a `llama-server`-owned process holding ~14–16 GB; free memory small. If `memory.used` is near 0 and the model is in RAM, GPU offload failed — recheck the CDI `devices` entry deployed correctly (`docker inspect llama-server --format '{{json .HostConfig.Devices}}{{json .HostConfig.DeviceRequests}}'`).
+
+- [ ] **Step 3: Confirm healthcheck is green**
+
+Run:
+```bash
+ssh ginnoir@valhalla "docker inspect llama-server --format '{{.State.Health.Status}}'"
+```
+Expected: `healthy` (allow up to `start_period` = 3 min).
+
+- [ ] **Step 4: Exercise the OpenAI endpoint from the host (as Hermes will)**
+
+Run (substitute the real key from `stacks/llm/stack.env`):
+```bash
+ssh ginnoir@valhalla "curl -fsS http://172.20.0.1:8090/v1/chat/completions -H 'Authorization: Bearer <LLAMA_API_KEY>' -H 'Content-Type: application/json' -d '{\"model\":\"qwen2.5-14b-instruct\",\"messages\":[{\"role\":\"user\",\"content\":\"Reply with exactly: OK\"}],\"max_tokens\":8}'"
+```
+Expected: a JSON chat completion containing `OK`. A 401 means the key is wrong; a connection refused means the port publish/bind is wrong.
+
+---
+
+### Task 7: Stretch to 64k context (live tuning)
+
+Attempt the larger context now that the baseline works; keep it only if VRAM holds under load.
+
+**Files:**
+- Modify: `stacks/llm/docker-compose.yml` (only if 64k holds)
+
+- [ ] **Step 1: Try 64k with YaRN + lighter V cache, ephemerally**
+
+Run a throwaway container (doesn't touch the deployed stack), driving a long context:
+```bash
+ssh ginnoir@valhalla "docker run --rm --device nvidia.com/gpu=0 -v /storage1/labdata/llm/models:/models -p 172.20.0.1:8091:8080 ghcr.io/ggml-org/llama.cpp:server-cuda -m /models/Qwen2.5-14B-Instruct-Q4_K_M.gguf --alias q -ngl 99 --ctx-size 65536 --rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 32768 -fa --cache-type-k q8_0 --cache-type-v q4_0 --host 0.0.0.0 --port 8080 > /tmp/llm_64k.log 2>&1 & sleep 60; nvidia-smi --query-gpu=memory.used,memory.free --format=csv,noheader; grep -iE 'model loaded|error|assert|out of memory|failed to allocate' /tmp/llm_64k.log | head; docker ps --filter publish=8091 -q | xargs -r docker rm -f"
+```
+Expected to decide: if `model loaded` with `memory.free` > ~300 MiB and no allocation failures, 64k is viable. If it OOMs / fails to allocate, 64k at this quant doesn't fit — keep 32k (stop here, leave the committed config as-is).
+
+- [ ] **Step 2 (only if 64k held): promote the 64k args into the stack**
+
+Edit `stacks/llm/docker-compose.yml` `command:` to: `--ctx-size 65536`, add `--rope-scaling yarn`, `--rope-scale 2`, `--yarn-orig-ctx 32768`, and set `--cache-type-v q4_0` (keep `--cache-type-k q8_0`). Then:
+```bash
+git add stacks/llm/docker-compose.yml
+git commit -m "feat(llm): raise llama-server context to 64k (YaRN + q4 V-cache)"
+git push
+```
+Let Portainer redeploy, then re-run Task 6 Steps 2–4. If the live 14B OOMs under a real long prompt, revert this commit.
+
+---
+
+### Task 8: Wire the provider into Hermes and verify end-to-end
+
+**Files:** host-side `~/.hermes/config.yaml` (not in repo).
+
+- [ ] **Step 1: Back up the Hermes config**
+
+Run:
+```bash
+ssh ginnoir@valhalla "cp ~/.hermes/config.yaml ~/.hermes/config.yaml.bak.$(date +%s) && ls -l ~/.hermes/config.yaml.bak.*"
+```
+
+- [ ] **Step 2: Add the provider entry under `providers:`**
+
+Append this entry to the `providers:` list in `~/.hermes/config.yaml` (same shape as the existing `ollama` entry; substitute the real key):
+```yaml
+- name: valhalla-p100
+  type: openai
+  base_url: http://172.20.0.1:8090/v1
+  api_key: <LLAMA_API_KEY>
+  models:
+  - qwen2.5-14b-instruct
+```
+Edit by reading the file, inserting the entry, and writing it back (preserve indentation exactly). Do **not** change the `model:` default block — we add the provider alongside the current default rather than silently replacing it (per spec).
+
+- [ ] **Step 3: Restart Hermes**
+
+Run:
+```bash
+ssh ginnoir@valhalla "sudo systemctl restart hermes-dashboard hermes-gateway hermes-webui && sleep 5 && systemctl is-active hermes-dashboard hermes-gateway hermes-webui"
+```
+Expected: three `active` lines. If any failed, check `journalctl -u hermes-gateway -n 50` — a YAML error means the edit broke indentation; restore the backup and retry.
+
+- [ ] **Step 4: Confirm Hermes sees the model and routes to the P100**
+
+Run a one-shot prompt forcing the new provider/model:
+```bash
+ssh ginnoir@valhalla "~/.hermes/hermes-agent/venv/bin/hermes -z 'Reply with exactly: HELLO FROM P100' -m qwen2.5-14b-instruct --provider valhalla-p100 2>&1 | tail -20"
+```
+Expected: a completion containing the phrase. Simultaneously, `nvidia-smi` (separate shell) should show llama-server utilization spike during generation.
+
+- [ ] **Step 5: (Optional) make it the default**
+
+If ginnoir wants the P100 model as Hermes' default rather than a per-call choice, run interactively:
+```bash
+ssh -t ginnoir@valhalla "~/.hermes/hermes-agent/venv/bin/hermes model"
+```
+and select `valhalla-p100` / `qwen2.5-14b-instruct`. Leave the default unchanged otherwise.
+
+---
+
+### Task 9: Cleanup and documentation
+
+**Files:** possibly `CLAUDE.md` (known-quirks note).
+
+- [ ] **Step 1: Remove the tiny test model**
+
+Run:
+```bash
+ssh ginnoir@valhalla "sudo rm -f /storage1/labdata/llm/models/qwen2.5-0.5b-instruct-q4_k_m.gguf && sudo ls /storage1/labdata/llm/models"
+```
+Expected: only the 14B GGUF remains.
+
+- [ ] **Step 2: Remove the config backup once verified (optional)**
+
+```bash
+ssh ginnoir@valhalla "ls ~/.hermes/config.yaml.bak.*"
+```
+Keep the most recent backup until the setup is confirmed stable, then remove.
+
+- [ ] **Step 3: Add a known-quirks note (optional, if desired)**
+
+Add a short bullet to `CLAUDE.md` under "External services" / "Known quirks": the `llm` stack serves Qwen2.5-14B on the P100 via llama.cpp; Hermes points at it via the `valhalla-p100` provider in `~/.hermes/config.yaml`; vLLM was rejected due to the P100's cc 6.0. Commit if added.
+
+---
+
+## Self-review notes
+
+- **Spec coverage:** engine (Task 3), model + storage (Tasks 1, 3), 32k baseline + q8 KV (Task 3), 64k stretch (Task 7), CDI GPU (Task 3, verified Task 6), `172.20.0.1:8090` publish (Task 3, verified Task 6), Hermes provider entry (Task 8), new-stack registration (Task 5), pure env_file / no `${VAR}` (Tasks 2–3), Watchtower pin (Task 3), no Caddy/no SSO (by omission), tiny-model cleanup (Task 9). All covered.
+- **No placeholders:** the only `<...>` tokens are the generated API key and (in Task 5) the repo URL/credential copied from an existing stack — both are runtime secrets/values, not undefined behavior.
+- **Consistency:** the alias `qwen2.5-14b-instruct` is the single contract used by the compose `--alias`, the curl test, and the Hermes provider `models:` / `-m` flag throughout.

docs/superpowers/plans/2026-06-26-r730xd-post-migration-tasks.md

@@ -0,0 +1,380 @@
+# R730XD Post-Migration Host Tasks — Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Close out the five remaining post-migration items on the Proxmox host `valhalla-dev`: a permanent fix for virtiofsd FD exhaustion, the hookscript bug, GPU thermal safety, container GPU access, and retiring the last failing pool disk.
+
+**Architecture:** These are five **independent** host-ops tasks against a live single-host Proxmox box (no application code, no test suite). Each is self-contained and can be executed in any order, with two dependencies noted below. "Tests" here are verification commands with expected output; "rollback" replaces "revert commit." Most changes are on the Proxmox host (`.68`) or the `valhalla` VM (`.69`), neither of which is tracked in this git repo — persistence is via on-host files + the vault migration note `[[Server Migration (Proxmox R730XD)]]`.
+
+**Tech Stack:** Proxmox VE (Debian) host, `virtiofsd` 1.13.2, ZFS (`storage1` stripe), EndeavourOS guest VM (id 100), NVIDIA Tesla P100 + driver 580.159.04 + nvidia-ctk 1.19.1, systemd, ipmitool fan control.
+
+## Access & conventions (read first)
+
+- **Host (`valhalla-dev`, Proxmox):** `wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "<cmd>"'` — the **WSL** `id_ed25519` (`ginnoir@TELLUS`) key is authorized for root.
+- **Guest (`valhalla` VM):** `wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "<cmd>"'` (sudo is NOPASSWD).
+- For multi-line remote scripts use the heredoc form: `wsl.exe -- bash -lc 'ssh ... root@192.168.1.68 bash -s' <<'EOF' … EOF` (avoids shell-variable mangling through the WSL→ssh layers).
+- **GateGuard:** the first `Bash` and every `Write`/`Edit` requires stating the user request + what the op does/affects before it runs.
+
+## Recommended sequence & dependencies
+
+1. **Task 1** (virtiofsd `--inode-file-handles`) and **Task 2** (hookscript fix) are both the FD-exhaustion fix — do them together. Task 1 is the real fix; Task 2 repairs the fallback. Both require a VM restart, so batch them into one VM bounce.
+2. **Task 3** (GPU fan safety) is a **hard prerequisite** for **Task 4** (container GPU) being *used* under load — never run a sustained CUDA job until Task 3 is live (passive P100 + Dell-auto fans that can't see GPU temp = overheat risk).
+3. **Task 5** (replace `sdb`) is fully independent and starts a ~1–1.5 day resilver — run it when power is stable.
+
+---
+
+## Task 1: Permanent virtiofsd FD fix — `--inode-file-handles=prefer`
+
+**Problem:** virtiofsd holds one `O_PATH` file descriptor per inode the guest touches on `/storage1`. With ~78 TB of media scanned by *arr/Plex it climbed to ~1,000,000 open FDs and the guest got `Too many open files` on new `/storage1` access. `--inode-file-handles=prefer` makes virtiofsd use `name_to_handle_at`/`open_by_handle_at` instead, keeping near-zero FDs. Proxmox 9 has **no config option** to pass this flag, so we inject it via a `dpkg-divert` wrapper around the virtiofsd binary (survives package updates).
+
+**Targets (host `.68`):**
+- Divert: `/usr/libexec/virtiofsd` → `/usr/libexec/virtiofsd.real`
+- Create: `/usr/libexec/virtiofsd` (wrapper script)
+- Restart: VM 100 (cold)
+
+- [ ] **Step 1: Baseline — capture the current launch args and FD count**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
+echo "=== current virtiofsd cmdline ==="
+for p in $(pgrep virtiofsd); do tr '\0' ' ' </proc/$p/cmdline; echo; done
+echo "=== worker FD count (the one that climbs) ==="
+for p in $(pgrep virtiofsd); do echo "pid $p fds=$(ls /proc/$p/fd 2>/dev/null | wc -l)"; done
+EOF
+```
+Expected: two virtiofsd processes, args include `--shared-dir=/storage1 --xattr`; worker FD count is whatever it has grown to (could be tens of thousands+). Record it.
+
+- [ ] **Step 2: Confirm the binary supports the flag**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "/usr/libexec/virtiofsd --help 2>&1 | grep -- --inode-file-handles"'
+```
+Expected: a line containing `--inode-file-handles=<INODE_FILE_HANDLES>`. (Confirmed present on 1.13.2.)
+
+- [ ] **Step 3: Create the diversion and the wrapper**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
+set -e
+# Move the real binary aside, persistently across apt upgrades
+dpkg-divert --add --rename --divert /usr/libexec/virtiofsd.real /usr/libexec/virtiofsd
+# Write a wrapper that injects the flag, then execs the real binary with all original args
+cat > /usr/libexec/virtiofsd <<'WRAP'
+#!/bin/bash
+# Proxmox-launched virtiofsd wrapper: force inode-file-handles to stop FD growth.
+# See docs/superpowers/plans/2026-06-26-r730xd-post-migration-tasks.md Task 1.
+exec /usr/libexec/virtiofsd.real --inode-file-handles=prefer "$@"
+WRAP
+chmod 0755 /usr/libexec/virtiofsd
+echo "=== verify ==="
+dpkg-divert --list /usr/libexec/virtiofsd
+ls -l /usr/libexec/virtiofsd /usr/libexec/virtiofsd.real
+EOF
+```
+Expected: diversion listed (`diversion of /usr/libexec/virtiofsd to /usr/libexec/virtiofsd.real`), wrapper is a 0755 regular file, `.real` is the original ELF binary.
+
+- [ ] **Step 4: Cold-restart VM 100 so a new virtiofsd launches via the wrapper**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
+qm shutdown 100 --timeout 180 && qm status 100
+qm start 100 && qm status 100
+EOF
+```
+Expected: `status: stopped` then `status: running`.
+
+- [ ] **Step 5: VERIFY — new virtiofsd carries the flag and FDs stay low**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
+echo "=== flag present in launch args? ==="
+for p in $(pgrep virtiofsd); do tr '\0' ' ' </proc/$p/cmdline | grep -o -- '--inode-file-handles=prefer' && echo " (pid $p OK)"; done
+echo "=== /storage1 still works in guest ==="
+ssh -o BatchMode=yes -o ConnectTimeout=8 -i /root/.ssh/id_ed25519 ginnoir@192.168.1.69 'ls /storage1 >/dev/null 2>&1 && echo storage1-OK || echo storage1-FAIL' 2>/dev/null || echo "(set up host->guest key per Task 3, or check from the admin box)"
+EOF
+```
+Expected: `--inode-file-handles=prefer (pid … OK)` for the worker. If the host→guest key isn't set yet, verify `/storage1` from the admin box instead: `wsl.exe -- bash -lc 'ssh ... ginnoir@192.168.1.69 "ls /storage1 >/dev/null && echo OK"'`.
+
+- [ ] **Step 6: Soak check — FD count after the library gets re-scanned**
+
+Re-run Step 1's FD-count command after a Plex/*arr scan cycle (or ~an hour of normal use). Expected: worker FD count stays in the **hundreds/low-thousands**, not climbing toward 1M. This confirms the fix.
+
+**Rollback:**
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
+rm -f /usr/libexec/virtiofsd
+dpkg-divert --remove --rename /usr/libexec/virtiofsd
+qm shutdown 100 --timeout 180 && qm start 100
+EOF
+```
+
+**Persist:** copy the wrapper to this repo under `docs/host/usr-libexec-virtiofsd.wrapper` (reference only) and note the diversion in the vault migration page.
+
+---
+
+## Task 2: Fix the `virtiofsd-limits.sh` hookscript bug
+
+**Problem:** `qm start 100` logs `hookscript error for 100 on post-start: /var/lib/vz/snippets/virtiofsd-limits.sh: line 19: exit: 0: numeric argument required`. Line 19 is `exit 0` — the "0 not numeric" error means a stray carriage return (`\r`) is attached (`exit $'0\r'`). The script is the FD-limit fallback (raises virtiofsd nofile to 10M); after Task 1 it's a belt-and-suspenders safety net, but it should run cleanly.
+
+**Targets (host `.68`):**
+- Rewrite: `/var/lib/vz/snippets/virtiofsd-limits.sh` (clean LF, correct content)
+
+- [ ] **Step 1: Confirm the CRLF/stray-char hypothesis**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "file /var/lib/vz/snippets/virtiofsd-limits.sh; grep -c $'"'"'\r'"'"' /var/lib/vz/snippets/virtiofsd-limits.sh"'
+```
+Expected: `file` reports `... with CRLF line terminators` and/or a non-zero `\r` count. (If zero, the bug is a different stray char — proceed to Step 2 anyway; the rewrite fixes it regardless.)
+
+- [ ] **Step 2: Rewrite the script cleanly (LF only)**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
+cp /var/lib/vz/snippets/virtiofsd-limits.sh /var/lib/vz/snippets/virtiofsd-limits.sh.bak
+cat > /var/lib/vz/snippets/virtiofsd-limits.sh <<'SCRIPT'
+#!/bin/bash
+# Fallback: raise virtiofsd open-file limit after VM 100 starts.
+# Primary fix is the --inode-file-handles=prefer wrapper (Task 1); this is a safety net.
+VMID="$1"
+PHASE="$2"
+[ "$VMID" = "100" ] && [ "$PHASE" = "post-start" ] || exit 0
+sleep 3
+pgrep virtiofsd | while read -r pid; do
+    prlimit --pid "$pid" --nofile=10000000:10000000 \
+      && logger -t virtiofsd-limits "raised nofile for PID $pid to 10M"
+done
+exit 0
+SCRIPT
+chmod 0755 /var/lib/vz/snippets/virtiofsd-limits.sh
+# guarantee no CRLF crept back in
+sed -i 's/\r$//' /var/lib/vz/snippets/virtiofsd-limits.sh
+echo "=== syntax check ==="
+bash -n /var/lib/vz/snippets/virtiofsd-limits.sh && echo "syntax OK"
+file /var/lib/vz/snippets/virtiofsd-limits.sh
+EOF
+```
+Expected: `syntax OK` and `file` reports a plain `Bourne-Again shell script, ASCII text executable` (no CRLF).
+
+- [ ] **Step 3: VERIFY — dry-run the hook and confirm no error**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "/var/lib/vz/snippets/virtiofsd-limits.sh 100 post-start; echo exit=$?"'
+```
+Expected: `exit=0`, no `numeric argument required` error. (It will also bump the live virtiofsd nofile to 10M — harmless.)
+
+- [ ] **Step 4: VERIFY at next real start (do this opportunistically with Task 1's restart)**
+
+After any `qm start 100`, check the journal:
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "journalctl -u qemu-server@100 -n 30 --no-pager | grep -iE \"hookscript|virtiofsd-limits\" | tail"'
+```
+Expected: no `hookscript error`; a `virtiofsd-limits: raised nofile …` logger line.
+
+**Rollback:** `cp /var/lib/vz/snippets/virtiofsd-limits.sh.bak /var/lib/vz/snippets/virtiofsd-limits.sh`
+
+---
+
+## Task 3: GPU thermal safety — wire P100 temp into the fan daemon
+
+**Problem:** The P100 is passively cooled. Fans are on **Dell-auto, which cannot read the GPU's temperature** (non-Dell card), and the custom `valhalla-thermal-monitor.service` (the CPU/GPU-reactive controller with a watchdog→Dell-auto failsafe) is currently **disabled** (shut off during the June panics). The daemon already supports GPU temp via a `VM_SSH` env var (`gpu_max()` runs `$VM_SSH 'nvidia-smi --query-gpu=temperature.gpu …'`), but `VM_SSH` is unset → `gpu=off`. We set up host→guest SSH, point `VM_SSH` at it, and re-enable the daemon so GPU temp drives the fans. **This must be live before any sustained GPU workload.**
+
+**Targets:**
+- Create (host): `/root/.ssh/id_ed25519` keypair (if absent) for host→guest SSH
+- Modify (guest `.69`): `~ginnoir/.ssh/authorized_keys` (append host pubkey)
+- Modify (host): `valhalla-thermal-monitor.service` drop-in with `Environment=VM_SSH=…`
+- Enable + start: `valhalla-thermal-monitor.service`
+
+- [ ] **Step 1: Generate a host→guest SSH key (host side)**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
+[ -f /root/.ssh/id_ed25519 ] || ssh-keygen -t ed25519 -N "" -C "valhalla-dev-host->vm" -f /root/.ssh/id_ed25519
+echo "=== host pubkey ==="
+cat /root/.ssh/id_ed25519.pub
+EOF
+```
+Expected: prints an `ssh-ed25519 … valhalla-dev-host->vm` pubkey. Copy this line.
+
+- [ ] **Step 2: Authorize that key on the guest (bridge via the admin/WSL key)**
+
+Substitute `<HOST_PUBKEY>` with the line from Step 1:
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "mkdir -p ~/.ssh && chmod 700 ~/.ssh && grep -qxF \"<HOST_PUBKEY>\" ~/.ssh/authorized_keys 2>/dev/null || echo \"<HOST_PUBKEY>\" >> ~/.ssh/authorized_keys; chmod 600 ~/.ssh/authorized_keys; echo done"'
+```
+Expected: `done`.
+
+- [ ] **Step 3: VERIFY host→guest SSH + nvidia-smi works non-interactively**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "ssh -o BatchMode=yes -o ConnectTimeout=5 -i /root/.ssh/id_ed25519 ginnoir@192.168.1.69 \"nvidia-smi --query-gpu=temperature.gpu --format=csv,noheader,nounits\""'
+```
+Expected: a bare integer like `56`. If it errors, fix the key before continuing — do not enable the daemon without a working GPU temp source.
+
+- [ ] **Step 4: Set `VM_SSH` on the service via a drop-in**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
+mkdir -p /etc/systemd/system/valhalla-thermal-monitor.service.d
+cat > /etc/systemd/system/valhalla-thermal-monitor.service.d/gpu.conf <<'DROP'
+[Service]
+Environment=VM_SSH=ssh -o BatchMode=yes -o ConnectTimeout=5 -i /root/.ssh/id_ed25519 ginnoir@192.168.1.69
+DROP
+systemctl daemon-reload
+echo "=== drop-in ==="; systemctl cat valhalla-thermal-monitor.service | grep -i VM_SSH
+EOF
+```
+Expected: the `Environment=VM_SSH=…` line echoed back.
+
+- [ ] **Step 5: Enable + start the daemon**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "systemctl enable --now valhalla-thermal-monitor.service; systemctl is-active valhalla-thermal-monitor.service"'
+```
+Expected: `active`.
+
+- [ ] **Step 6: VERIFY — GPU temp is in the curve, and the failsafe is intact**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "journalctl -t thermal-monitor -n 8 --no-pager"'
+```
+Expected: lines like `cpu=NNC gpu=NNC -> fan NN%` with a **real number** for `gpu=` (not `na`), and the startup line shows `gpu=on`.
+
+- [ ] **Step 7: VERIFY the stop/crash failsafe returns fans to Dell-auto**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "systemctl cat valhalla-thermal-monitor.service | grep -iE \"ExecStopPost|trap\"; echo --- ; grep -nE \"0x30 0x30 0x01 0x01|trap\" /usr/local/sbin/valhalla-thermal-monitor.sh"'
+```
+Expected: confirms an `ExecStopPost`/`trap` that issues `ipmitool raw 0x30 0x30 0x01 0x01` (return to Dell auto). This was previously proven; just re-confirm it's present.
+
+**Rollback:**
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "systemctl disable --now valhalla-thermal-monitor.service; rm -f /etc/systemd/system/valhalla-thermal-monitor.service.d/gpu.conf; systemctl daemon-reload; ipmitool raw 0x30 0x30 0x01 0x01"'
+```
+(The final `ipmitool` call forces fans back to Dell auto.)
+
+---
+
+## Task 4: Container GPU access (nvidia-ctk + CDI) in the guest
+
+**Problem:** The guest has `nvidia-ctk` 1.19.1 and a working driver, but Docker has only the `runc` runtime and no CDI spec, so containers can't use the P100. Generate a CDI spec and enable CDI in Docker so workloads request the GPU with `--device nvidia.com/gpu=all`. **Prerequisite: Task 3 must be live before running any non-trivial GPU container.** Restarting Docker bounces all ~73 containers — do it in a maintenance window.
+
+**Targets (guest `.69`):**
+- Create: `/etc/cdi/nvidia.yaml` (CDI spec)
+- Modify: `/etc/docker/daemon.json` (enable CDI feature)
+- Restart: `docker` service
+
+- [ ] **Step 1: Baseline — Docker version (CDI needs Engine ≥ 25) and current runtimes**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "docker version --format \"{{.Server.Version}}\"; docker info 2>/dev/null | grep -iE \"Runtimes|Default Runtime\""'
+```
+Expected: a server version. If **≥ 25.0**, use CDI (Steps 2–5). If older, use the classic nvidia runtime instead (see Alternative below).
+
+- [ ] **Step 2: Generate the CDI spec**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "sudo mkdir -p /etc/cdi && sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml && sudo nvidia-ctk cdi list"'
+```
+Expected: `nvidia-ctk cdi list` shows `nvidia.com/gpu=all` (and `nvidia.com/gpu=0`).
+
+- [ ] **Step 3: Enable the CDI feature in Docker**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 bash -s' <<'EOF'
+sudo install -d /etc/docker
+# merge "features.cdi=true" into daemon.json (create if absent)
+sudo python3 - <<'PY'
+import json,os
+p="/etc/docker/daemon.json"
+d=json.load(open(p)) if os.path.exists(p) and os.path.getsize(p) else {}
+d.setdefault("features",{})["cdi"]=True
+json.dump(d,open(p,"w"),indent=2)
+print(open(p).read())
+PY
+EOF
+```
+Expected: prints a `daemon.json` containing `"features": { "cdi": true }`.
+
+- [ ] **Step 4: Restart Docker (MAINTENANCE WINDOW — bounces all containers)**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "sudo systemctl restart docker && sleep 20 && echo running:$(docker ps -q|wc -l)"'
+```
+Expected: Docker restarts; running container count climbs back toward ~72 over the next minute.
+
+- [ ] **Step 5: VERIFY — a CUDA container sees the P100**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "docker run --rm --device nvidia.com/gpu=all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi --query-gpu=name --format=csv,noheader"'
+```
+Expected: `Tesla P100-PCIE-16GB`. Watch the host thermal log (Task 3) during any longer test to confirm fans respond to GPU temp.
+
+**Alternative (Docker < 25, classic runtime):**
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker && docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi -L"'
+```
+Expected: `GPU 0: Tesla P100-PCIE-16GB (UUID: …)`.
+
+**Rollback:** remove `/etc/cdi/nvidia.yaml`, revert `daemon.json` (drop `features.cdi`), `sudo systemctl restart docker`.
+
+---
+
+## Task 5: Replace the last failing disk `sdb` with the staged 4 TB
+
+**Problem:** `storage1` is still `DEGRADED` because `sdb` (4 TB WD Red, serial `WD-WCC4E4TSK1S2`, "too many errors") is failing. A new 4 TB disk (serial `V6HXW23W`) is already installed in the old Slot 2 (currently `sdc`, not in the pool). `zpool replace` swaps it in. This starts a **~1–1.5 day resilver** over a no-redundancy stripe — run only when power is stable, and keep `sdb` seated during the resilver (pulling it first loses its data).
+
+**Targets (host `.68`):** `storage1` pool — no files.
+
+- [ ] **Step 1: Resolve the new disk's stable by-id (do NOT use bare `sdc` — letters shuffle on reboot)**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "ls -l /dev/disk/by-id/ | grep -iE \"V6HXW23W\" | grep -vE \"part\""'
+```
+Expected: one or more by-id symlinks for serial `V6HXW23W` (e.g. `wwn-0x…` / `ata-…V6HXW23W`). Pick the `wwn-…` or `ata-…` whole-disk link → call it `$NEWDISK` (full path `/dev/disk/by-id/<that>`).
+
+- [ ] **Step 2: Pre-flight — confirm target is `sdb` and new disk is unused**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
+echo "=== sdb serial (must be WD-WCC4E4TSK1S2) ==="; lsblk -dno NAME,SERIAL | awk '$1=="sdb"'
+echo "=== new disk NOT in pool ==="; zpool status -P storage1 | grep -q "V6HXW23W" && echo "ALREADY IN POOL?!" || echo "new disk not in pool (good)"
+echo "=== pool baseline ==="; zpool status storage1 | grep -E "state:|scan:"
+EOF
+```
+Expected: `sdb` serial is `WD-WCC4E4TSK1S2`; new disk not in pool; no active scan.
+
+- [ ] **Step 3: Kick off the replace** (substitute `$NEWDISK` from Step 1)
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "zpool replace storage1 sdb /dev/disk/by-id/<NEWDISK>; echo exit=$?; zpool status storage1 | grep -A4 replacing"'
+```
+Expected: `exit=0`; status now shows a `replacing-N` vdev with `sdb` (old) and the new disk `(resilvering)`.
+
+- [ ] **Step 4: VERIFY at completion** (poll periodically — reuse the existing resilver-watch cadence)
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "zpool status storage1"'
+```
+Expected at completion: `scan: resilvered … with N errors`, the `replacing` vdev is gone, `sdb` is **evicted**, and the new disk is a normal member. Pool state should move toward `ONLINE` (it may still report the 7 pre-existing data errors — see Step 5).
+
+- [ ] **Step 5: After resilver — clear stale errors if no live counters remain**
+
+```bash
+wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "zpool status -v storage1"'
+```
+If only the 3 known media files remain and all device READ/WRITE/CKSUM are 0, optionally delete + re-grab those files via the *arr stack, then `zpool clear storage1` to reset the error state. Expected after clear + a scrub: `state: ONLINE`, `errors: No known data errors`.
+
+- [ ] **Step 6: Physically remove the evicted `sdb`** (optional, later)
+
+Use the SES locate-LED method to confirm the bay before pulling (host must be up): match `sdb`'s SAS address to its `sg_ses --dev-slot-num` and `sg_ses --dev-slot-num=<N> --set=ident /dev/sg12`. Verify the drive label reads `WD-WCC4E4TSK1S2` before full removal.
+
+**Rollback (only before/early in resilver):** `zpool detach storage1 <NEWDISK>` cancels the replace and keeps the original `sdb` in service.
+
+---
+
+## Self-review notes
+
+- **Coverage:** all five requested items have a task (fan safety = Task 3, container GPU = Task 4, hookscript = Task 2, `sdb` = Task 5, virtiofs permanent fix = Task 1). ✅
+- **Dependencies flagged:** Task 3 before Task 4 under load; Tasks 1+2 share a VM restart.
+- **Stable identifiers:** disk ops use serial/by-id, never bare `sdX` (letters shuffled across the week's reboots).
+- **Each task has explicit verification + rollback.** No `zpool`/Docker/systemd change lacks a confirmation command.

docs/superpowers/plans/2026-06-27-hermes-ecosystem-integration.md

@@ -0,0 +1,780 @@
+# Hermes Ecosystem Integration Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Integrate selected Hermes-ecosystem tools into ginnoir's live valhalla deployment — a host-side delegation skill and self-improvement plugin (Phase 1), a stealth-browser homelab stack and a skill pre-filter trial (Phase 2), and an optional UI upgrade (Phase 3) — each reversible and sized for the single-P100 / weak-local-model constraints.
+
+**Architecture:** Two integration classes. **Class A** (host-side Hermes plugins/skills) install into `~/.hermes/` on the valhalla host and are applied by SSH + `hermes` CLI + `sudo systemctl restart hermes-gateway.service`; they are **host-managed, NOT committed to this repo** (tracked in project memory + the Obsidian vault, like the rustdesk/obsidian/xvfb units). **Class B** (Docker services) become `stacks/<name>/` entries deployed via the normal Gitea-poll path, fronted by Caddy `internal_only` + Authentik. The companion spec is `docs/superpowers/specs/2026-06-27-hermes-ecosystem-integration-design.md`.
+
+**Tech Stack:** Hermes Agent v0.17.0 (host systemd), llama-swap/Tesla P100 backend (`gpt-oss-20b`, `--parallel 1`), Python 3.11 (`~/.hermes/hermes-agent/venv`), `uv`, SQLite, Docker Compose + Portainer (Gitea-polled), Caddy, Authentik, Codex + Claude Code CLIs.
+
+---
+
+## How to read this plan (operational, not codebase-TDD)
+
+These are operational integrations against a live host and third-party services, so the TDD rhythm
+is adapted: each task is **back up → change → verify with a smoke test → document/commit**. The
+"test" is a real verification command with expected output. **Class A (host) changes are not git
+commits** — their checkpoint is a backup + smoke test + a memory/vault note. **Class B (repo)
+changes do commit** (and push triggers Portainer). Run every step from the Windows workstation;
+host steps use `ssh -o BatchMode=yes ginnoir@valhalla "..."`.
+
+**Global guardrails (apply to every task):**
+- `hermes` is only on the **login-shell** PATH → over SSH call it by full path: `~/.local/bin/hermes`.
+- Gateway restart needs root: `sudo systemctl restart hermes-gateway.service`.
+- Always back up `~/.hermes/config.yaml` before editing (`cp ...bak.$(date +%s)`).
+- **Never** load a second model onto the P100. Keep curator/eagle-eye semantic layers on CPU or off.
+- Read third-party code before running it (curator writes skills; acp-skill spawns external agents; camofox automates a browser).
+
+**Decisions (RESOLVED 2026-06-27 — all phases actionable):** trial **both** UIs and keep the winner
+(Phase 3); curator stays **report-only** (Task 3); **trial eagle-eye** — it's the only direct
+skill-router in the ecosystem (Task 6); camofox wired as a **minimal 2-tool skill** (Task 5 Step 8).
+
+---
+
+# PHASE 1 — Host-side, reversible, high-leverage (actionable now)
+
+## Task 1: Pre-flight — capture current Hermes state
+
+**Files:**
+- Host only (no repo files).
+
+- [ ] **Step 1: Verify host reachability and Hermes services are up**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "systemctl is-active hermes-gateway.service hermes-dashboard.service hermes-webui.service"
+```
+Expected: three lines, each `active`.
+
+- [ ] **Step 2: Snapshot config + inventory skills/plugins/sessions dirs**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "cp ~/.hermes/config.yaml ~/.hermes/config.yaml.bak.preflight.$(date +%s) && ls -la ~/.hermes/skills ~/.hermes/plugins ~/.hermes/sessions 2>&1 | head -60 && ~/.local/bin/hermes --version"
+```
+Expected: a backup is created; directory listings print (note whether `~/.hermes/plugins` exists yet); `hermes` prints a version (≈ v0.17.0). Record the skills-dir path — confirms `~/.hermes/skills` is correct for later tasks.
+
+- [x] **Step 3: Confirm Codex and Claude Code are invocable — DONE 2026-06-27 (installed this session)**
+
+Both delegation CLIs were installed on valhalla this session:
+- `claude` → `~/.local/bin/claude` **v2.1.195** (login pending)
+- `codex` → `/usr/bin/codex` **v0.142.3** (login pending; harmless PATH-alias warning at install)
+
+**Gotcha recorded:** `/home/ginnoir/.claude` existed as an empty **root-owned** dir (created Jun 17,
+likely a prior sudo op) and blocked the installer until `sudo chown ginnoir:ginnoir ~/.claude`. Codex
+global install needs `sudo` (npm global prefix is `/usr`). Re-verify any time with:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "bash -lc 'claude --version; codex --version'"
+```
+**ginnoir must log in** to each (`claude`, then `codex login`) before Task 2's external delegation
+smoke-tests will succeed. Cursor + Antigravity targets are added later in **Task 10**.
+
+- [ ] **Step 4: Checkpoint**
+
+No commit (host inventory only). Record findings (skills-dir path, whether `plugins/` exists, Codex/Claude availability) in the session notes for use in Tasks 2–3.
+
+---
+
+## Task 2: Install `hermes-agent-acp-skill` (multi-agent delegation)
+
+**Files:**
+- Host: `~/.hermes/skills/hermes-acp-orchestrator/` (skill files), `~/.hermes/config.yaml` (delegation block).
+- Scratch: clone under `/storage1/hermes/workspace/clones/` (never root; see the disk gotcha).
+
+- [ ] **Step 1: Clone and read the skill before installing**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "mkdir -p /storage1/hermes/workspace/clones && git -C /storage1/hermes/workspace/clones clone https://github.com/Rainhoole/hermes-agent-acp-skill && sed -n '1,200p' /storage1/hermes/workspace/clones/hermes-agent-acp-skill/SKILL.md"
+```
+Expected: repo clones; `SKILL.md` prints. **Read it** to confirm: the skill folder/name, how `delegate_task()` is wired, and whether it expects a specific install path or a config key. The README omits install steps, so the SKILL.md is authoritative — follow whatever placement it documents. If SKILL.md specifies a different mechanism than the manual copy below, use SKILL.md's.
+
+- [ ] **Step 2: Place the skill into the Hermes skills directory**
+
+Run (adjust the destination name to match SKILL.md's declared skill name):
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "mkdir -p ~/.hermes/skills/hermes-acp-orchestrator && cp -r /storage1/hermes/workspace/clones/hermes-agent-acp-skill/SKILL.md /storage1/hermes/workspace/clones/hermes-agent-acp-skill/README.md ~/.hermes/skills/hermes-acp-orchestrator/ && ls -la ~/.hermes/skills/hermes-acp-orchestrator/"
+```
+Expected: `SKILL.md` and `README.md` present in the new skill dir.
+
+- [ ] **Step 3: Add the delegation config block (with a safe backup)**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "cp ~/.hermes/config.yaml ~/.hermes/config.yaml.bak.acp.$(date +%s) && printf '\ndelegation:\n  external_timeout_seconds: 900\n  external_max_output_chars: 24000\n' >> ~/.hermes/config.yaml && tail -8 ~/.hermes/config.yaml"
+```
+Expected: a `.bak.acp.*` backup exists; the `delegation:` block is appended and printed. (If SKILL.md says the block belongs under a different key or nesting, edit accordingly instead of this append.)
+
+- [ ] **Step 4: Restart the gateway and confirm the skill registers**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "sudo systemctl restart hermes-gateway.service && sleep 5 && systemctl is-active hermes-gateway.service && ~/.local/bin/hermes skills list 2>&1 | grep -i acp"
+```
+Expected: gateway `active`; the ACP/orchestrator skill appears in `hermes skills list`. (If the subcommand differs, use `~/.local/bin/hermes skills --help` to find the list command — verify on host.)
+
+- [ ] **Step 5: Smoke-test a trivial delegation to the local hermes subagent first**
+
+Run (a no-external-dependency delegation — routes to `hermes`, not Codex/Claude, to isolate the skill from CLI availability):
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "~/.local/bin/hermes run 'Use delegate_task to ask a hermes subagent to reply with exactly the word PONG, then report its output.' 2>&1 | tail -30"
+```
+Expected: the delegated subagent returns `PONG` and the parent reports it. **This proves the skill mechanics.** (Exact `hermes` one-shot invocation may differ — confirm the non-interactive run command via `~/.local/bin/hermes --help` in Step 1's read-through.)
+
+- [ ] **Step 6: Smoke-test an external delegation (only if Codex/Claude were found in Task 1)**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "~/.local/bin/hermes run 'Use delegate_task with agent=claude-code to print the current working directory and nothing else, then report it.' 2>&1 | tail -40"
+```
+Expected: Claude Code is spawned within the 900 s timeout, returns the cwd, output is captured under the 24,000-char cap. **If it hangs or auths interactively**, the external CLI needs non-interactive credentials on the service env — note for ginnoir; the `hermes`-target path (Step 5) still works.
+
+- [ ] **Step 7: Checkpoint (host note + reversibility recorded)**
+
+No git commit. Record in session notes: skill installed at `~/.hermes/skills/hermes-acp-orchestrator/`, config backup at `~/.hermes/config.yaml.bak.acp.*`. **Rollback** = `rm -rf ~/.hermes/skills/hermes-acp-orchestrator`, restore the `.bak.acp.*`, restart gateway.
+
+---
+
+## Task 3: Install `hermes-curator-evolver` (self-improvement, report-only)
+
+**Files:**
+- Host: `~/.hermes/plugins/curator-evolver/` (plugin + `data/evidence.sqlite`), systemd **user** timer.
+
+- [ ] **Step 1: Read the plugin source before installing (it can write to skills)**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "git -C /storage1/hermes/workspace/clones clone https://github.com/pingchesu/hermes-curator-evolver && sed -n '1,160p' /storage1/hermes/workspace/clones/hermes-curator-evolver/README.md"
+```
+Expected: repo clones; README prints. Confirm the apply path requires `--approve` (it does per the README) and that `auto-run` without `--apply-low-risk --approve-auto-apply` is **dry-run only**.
+
+- [ ] **Step 2: Install the plugin (no semantic/embedding extras — keep it off the P100)**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "bash -lc '~/.local/bin/hermes plugins install pingchesu/hermes-curator-evolver --enable && uv pip install --python ~/.hermes/hermes-agent/venv/bin/python -e ~/.hermes/plugins/curator-evolver && ~/.hermes/hermes-agent/venv/bin/hermes-curator-evolver bootstrap'"
+```
+Expected: plugin installs to `~/.hermes/plugins/curator-evolver`; editable pip install succeeds; `bootstrap` configures and installs a **systemd user timer**. **Do NOT pass `--semantic`** (that pulls Qwen/BGE models — CPU/VRAM cost we're avoiding for now; BM25/FTS ranking is the v1 default).
+
+- [ ] **Step 3: Backfill recent sessions and generate the first dry-run report**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "~/.hermes/hermes-agent/venv/bin/hermes-curator-evolver backfill-sessions --sessions-dir ~/.hermes/sessions --days 30 --format json 2>&1 | tail -20 && ~/.hermes/hermes-agent/venv/bin/hermes-curator-evolver report --days 7 --format json 2>&1 | tail -40"
+```
+Expected: evidence is mined into `~/.hermes/plugins/curator-evolver/data/evidence.sqlite`; `report` prints a JSON summary of candidate skill improvements. **No skill files are modified** (report is read-only).
+
+- [ ] **Step 4: Generate a dry-run proposal for one skill and inspect it**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "~/.hermes/hermes-agent/venv/bin/hermes-curator-evolver auto-run --skills-dir ~/.hermes/skills --format json 2>&1 | tail -60"
+```
+Expected: a JSON set of **proposed** (not applied) edits. Confirm no files under `~/.hermes/skills` changed:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "find ~/.hermes/skills -newermt '-10 minutes' -type f 2>/dev/null"
+```
+Expected: empty output (nothing modified) — proves dry-run safety.
+
+- [ ] **Step 5: Confirm the scheduled timer is report-only**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "bash -lc 'systemctl --user list-timers \"hermes-curator-evolver*\" --all --no-pager' && ssh -o BatchMode=yes ginnoir@valhalla \"systemctl --user cat 'hermes-curator-evolver*' 2>&1 | grep -iE 'ExecStart|approve|apply'\""
+```
+Expected: a timer is listed; its `ExecStart` runs `auto-run` **without** `--apply-low-risk`/`--approve-auto-apply`. **If the bootstrap-installed unit includes those flags, override it** to remove them (the morning decision in spec §6.2 defaults to report-only). If user-lingering isn't enabled the timer won't fire across logout — enable with `sudo loginctl enable-linger ginnoir` (note for ginnoir).
+
+- [ ] **Step 6: Checkpoint (host note + reversibility recorded)**
+
+No git commit. Record: plugin at `~/.hermes/plugins/curator-evolver`, DB at `.../data/evidence.sqlite`, timer name, report-only confirmed. **Rollback** = `systemctl --user disable --now <timer>`, `~/.local/bin/hermes plugins uninstall curator-evolver` (verify exact uninstall verb), `rm -rf ~/.hermes/plugins/curator-evolver`.
+
+---
+
+## Task 3b: Install `hermes-motif` (skill DISCOVERY, proposal-only)
+
+> Complements curator-evolver — does **not** compete with it (spec §7.1). motif *discovers new*
+> skills by mining repeated tool sequences; curator *refines existing* ones. **Zero P100 cost**
+> (makes no LLM calls). Together with eagle-eye (routing) they form a skill factory:
+> motif creates → curator refines → eagle-eye routes.
+
+**Files:**
+- Host: `~/.hermes/plugins/` (motif plugin), `~/.hermes/plugins/<motif>/plugin/plugin.yaml`.
+
+- [ ] **Step 1: Clone and read; confirm proposal-only config**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "git -C /storage1/hermes/workspace/clones clone https://github.com/Saurav0989/hermes-motif && sed -n '1,160p' /storage1/hermes/workspace/clones/hermes-motif/README.md && cat /storage1/hermes/workspace/clones/hermes-motif/plugin/plugin.yaml 2>&1"
+```
+Expected: README + `plugin.yaml` print. **Confirm `auto_install: false`** (proposal-only) and note
+`min_occurrences` / sequence-length thresholds. Note the referenced Hermes trace bug (#12922) that
+can affect trace completeness — acceptable for a proposal-only trial.
+
+- [ ] **Step 2: Install the plugin**
+
+Run (per its README — clone + pip + `scripts/install_plugin.sh`):
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "cd /storage1/hermes/workspace/clones/hermes-motif && bash scripts/install_plugin.sh 2>&1 | tail -20"
+```
+Expected: the plugin lands under `~/.hermes/plugins/` and registers. (If the script expects a
+different layout, follow the README's exact steps.)
+
+- [ ] **Step 3: Verify it mines and PROPOSES without modifying skills**
+
+Restart the gateway, run the agent through a couple of repeated multi-tool workflows, then check for
+proposals (drafts), confirming nothing under `~/.hermes/skills` was auto-written:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "sudo systemctl restart hermes-gateway.service && sleep 5 && find ~/.hermes/plugins -iname '*propos*' -o -iname '*draft*' 2>/dev/null | head && find ~/.hermes/skills -newermt '-10 minutes' -type f 2>/dev/null"
+```
+Expected: proposal/draft artifacts may appear under the plugin dir; the second `find` is **empty**
+(no skill files auto-modified) — proves `auto_install: false` safety.
+
+- [ ] **Step 4: Checkpoint**
+
+No repo commit (host-side). **Rollback** = remove the motif plugin dir + restart gateway. Record in
+`memory/hermes-extensions.md` alongside curator (skill factory: motif=create, curator=refine).
+
+---
+
+## Task 4: Document Phase 1 in memory + vault (durable knowledge)
+
+**Files:**
+- Memory: `C:\Users\MattC\.claude\projects\C--Users-MattC-Documents-homelabstack\memory\hermes-extensions.md` + `MEMORY.md` pointer.
+- Vault: append to the Hermes project note via Obsidian MCP (`mcp__obsidian__*`).
+
+- [ ] **Step 1: Write the memory file**
+
+Create `memory/hermes-extensions.md` (frontmatter `type: project`) recording: acp-skill installed (delegation to hermes/codex/claude-code/cursor/antigravity, 900s/24k caps); curator-evolver installed **report-only** (CPU ranking, no `--semantic`, no auto-apply flags); **motif** installed proposal-only (skill factory: motif creates → curator refines → eagle-eye routes); **claude v2.1.195 + codex v0.142.3 installed on valhalla 2026-06-27** (login pending; `~/.claude` was root-owned → chowned); exact paths and rollback commands; the host-vs-repo boundary. Link `[[llm-stack-hermes]]`, `[[multi-agent-tool-configs]]`, `[[obsidian-app-on-valhalla]]`.
+
+- [ ] **Step 2: Add the MEMORY.md index pointer**
+
+Append one line to `MEMORY.md`:
+`- [Hermes host extensions](hermes-extensions.md) — acp delegation skill + curator-evolver (report-only) on valhalla; host-managed in ~/.hermes, not in the repo`
+
+- [ ] **Step 3: Write back to the Obsidian vault**
+
+Per the global rule, use the Obsidian MCP (never write CouchDB directly) to append a session note to the Hermes project folder summarizing Phase 1 (what, why report-only, rollback). If the MCP is unreachable, tell ginnoir and skip — do not hand-edit.
+
+- [ ] **Step 4: Checkpoint**
+
+No code commit required (memory files live outside the repo). Phase 1 complete and documented.
+
+---
+
+# PHASE 2 — New capability + experiment
+
+> Decisions resolved (spec §6.3 eagle-eye trial; §6.4 camofox minimal). Actionable.
+
+## Task 5: `camofox-browser` as a homelab stack (Class B)
+
+**Files:**
+- Create: `stacks/camofox/docker-compose.yml`, `stacks/camofox/stack.env`.
+- Modify: `Caddyfile` (new site block), `bookmarks-domains.html` + `bookmarks-ports.html` (regenerated).
+- Host (image): build under `/storage1/hermes/workspace/clones/camofox-browser`.
+
+- [ ] **Step 1: Decide image provenance and build it**
+
+camofox publishes **no registry image** (`make up` builds locally). Recommended default: build on
+valhalla and tag `camofox-browser:local`, reference that tag from compose (Watchtower already
+disabled for pinned infra). Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "git -C /storage1/hermes/workspace/clones clone https://github.com/jo-inc/camofox-browser && cd /storage1/hermes/workspace/clones/camofox-browser && docker build -t camofox-browser:local . 2>&1 | tail -20 && docker image ls camofox-browser:local"
+```
+Expected: image builds; `camofox-browser:local` is listed. **Alternative (if a reproducible/Gitea-Actions build is preferred, like famapp):** build + push to `registry.ginnoir.com/ginnoir/camofox-browser` and reference that instead — flag this choice for ginnoir.
+
+- [ ] **Step 2: Write the stack compose**
+
+Create `stacks/camofox/docker-compose.yml`:
+```yaml
+# camofox stack — stealth headless browser REST API for the Hermes agent.
+# No published image: built on-host as camofox-browser:local (see plan Task 5).
+# Internal-only; reachable by Caddy over edge and by host-side Hermes.
+services:
+  camofox:
+    image: camofox-browser:local
+    container_name: camofox
+    restart: unless-stopped
+    labels:
+      - "com.centurylabs.watchtower.enable=false"
+    env_file:
+      - stack.env
+    networks: [edge, camofox]
+    volumes:
+      - /config/camofox/cookies:/home/node/.camofox/cookies
+      - /config/camofox/profiles:/home/node/.camofox/profiles
+    ports:
+      - "172.20.0.1:9377:9377"
+    healthcheck:
+      test: ["CMD", "curl", "-fsS", "http://localhost:9377/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 5
+      start_period: 40s
+
+networks:
+  edge:
+    external: true
+  camofox:
+    name: camofox
+    driver: bridge
+```
+(The `172.20.0.1:9377` host-port mirrors the llm stack's pattern so host-side Hermes can reach it directly; Caddy reaches it over `edge` by container name.)
+
+- [ ] **Step 3: Write `stack.env` (secrets; LF endings)**
+
+Create `stacks/camofox/stack.env` with a generated bearer key (replace the value with a real secret before push):
+```
+CAMOFOX_ACCESS_KEY=GENERATE_A_LONG_RANDOM_KEY
+CAMOFOX_ADMIN_KEY=GENERATE_A_SECOND_RANDOM_KEY
+CAMOFOX_PORT=9377
+```
+Generate the keys: `ssh ... "openssl rand -hex 32"` (run twice). **Ensure LF line endings** (`.gitattributes` enforces this — verify the file isn't CRLF before committing). Leave `CAMOFOX_API_KEY` unset (cookie-import endpoint stays disabled).
+
+- [ ] **Step 4: Add the Caddy site block (internal-only)**
+
+Add to `Caddyfile` (place near other internal admin services). Since camofox enforces its own bearer auth and Hermes calls it machine-to-machine, gate by LAN only (no Authentik forward-auth, which would block the agent's API calls):
+```caddy
+camofox.ginnoir.com {
+	import internal_only
+	reverse_proxy camofox:9377
+}
+```
+
+- [ ] **Step 5: Create host config dirs, regenerate bookmarks, then deploy**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "sudo mkdir -p /config/camofox/cookies /config/camofox/profiles && sudo chown -R ginnoir:ginnoir /config/camofox"
+```
+Then regenerate bookmarks and push (Portainer must have the `stacks/camofox` git stack registered — see Step 6):
+```powershell
+./scripts/gen-bookmarks.ps1
+git add stacks/camofox/ Caddyfile bookmarks-domains.html bookmarks-ports.html
+git commit -m "feat(camofox): stealth browser stack for the Hermes agent"
+git push
+```
+Expected: commit + push; Gitea Actions reloads Caddy (Caddyfile changed); Portainer redeploys the camofox stack within 5 min.
+
+- [ ] **Step 6: Register the stack in Portainer if new, and verify it runs**
+
+New stacks need one-time Portainer registration (see `portainer-new-stack-registration` memory). After deploy, verify:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "docker ps --filter name=camofox --format '{{.Names}} {{.Status}}' && curl -fsS http://172.20.0.1:9377/health"
+```
+Expected: container `Up (healthy)`; `/health` returns OK.
+
+- [ ] **Step 7: Smoke-test the browser API end-to-end**
+
+Run (creates a tab, snapshots a page):
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "K=\$(grep CAMOFOX_ACCESS_KEY /config/portainer/compose/*/stacks/camofox/stack.env | cut -d= -f2); ID=\$(curl -fsS -H \"Authorization: Bearer \$K\" -H 'Content-Type: application/json' -d '{\"userId\":\"smoke\",\"sessionKey\":\"t1\",\"url\":\"https://example.com\"}' http://172.20.0.1:9377/tabs | python3 -c 'import sys,json;print(json.load(sys.stdin)[\"id\"])'); curl -fsS -H \"Authorization: Bearer \$K\" \"http://172.20.0.1:9377/tabs/\$ID/snapshot?userId=smoke\" | head -20"
+```
+Expected: a tab id comes back; the snapshot returns accessibility text containing "Example Domain". (Adjust the JSON id field name to match the real response from Step 1's README read.)
+
+- [ ] **Step 8: Wire camofox into Hermes as a minimal tool surface (per §6.4 decision)**
+
+Default recommendation: a **small Hermes skill** (2 high-level tools — `browse(url)` and `search(query)`) that curls camofox, rather than exposing the full REST surface (respects the tool-budget that keeps gpt-oss-20b functional). Create `~/.hermes/skills/camofox-browse/SKILL.md` documenting the two operations against `http://172.20.0.1:9377` with the bearer key, restart the gateway, and smoke-test:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "sudo systemctl restart hermes-gateway.service && sleep 5 && ~/.local/bin/hermes run 'browse https://example.com and tell me the page heading' 2>&1 | tail -20"
+```
+Expected: Hermes uses the camofox tool and reports "Example Domain". **If §6.4 chose an MCP shim instead**, build/register the MCP server and add it to `mcp_servers:` with a 2-tool `tools.include` allowlist (per the MCP-curation pattern in `llm-stack-hermes`).
+
+- [ ] **Step 9: Checkpoint**
+
+Repo changes are committed (Step 5). Update `memory/hermes-extensions.md` + the vault note with the camofox stack + tool wiring and the bearer-key location.
+
+---
+
+## Task 5b: Install `hermes-web-search-plus` (multi-provider search; pairs with camofox)
+
+> Mature (v2.6.1, MIT, stdlib-only). Complements camofox (spec §7.2): search-plus *finds* via cheap
+> provider APIs, camofox *browses/renders*. Lighter and higher-frequency — good default reach-for.
+
+**Files:**
+- Host: `~/.hermes/plugins/` (plugin), provider key(s) in `~/.hermes/config.yaml` (or the plugin's config).
+
+- [ ] **Step 1: Install the plugin**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "bash -lc '~/.local/bin/hermes plugins install robbyczgw-cla/hermes-web-search-plus --enable'"
+```
+Expected: plugin installs and enables.
+
+- [ ] **Step 2: Configure at least one provider key (free tier)**
+
+All provider keys are optional but ≥1 is needed to function. Pick a free-tier provider (e.g. Tavily,
+Exa, or self-hosted SearXNG; Keenable has a keyless public tier). Add the key per the plugin's README
+(read `/storage1/hermes/workspace/clones/...` or the plugin docs for the exact env/config key), then:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "sudo systemctl restart hermes-gateway.service && sleep 5 && systemctl is-active hermes-gateway.service"
+```
+Expected: gateway `active`. **Mind the tool-budget rule** — if it exposes both `web_search_plus` +
+extract, that's fine (2 tools); don't also enable redundant search MCPs.
+
+- [ ] **Step 3: Smoke-test a search**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "~/.local/bin/hermes run 'search the web for the latest Hermes Agent release version and cite the source' 2>&1 | tail -20"
+```
+Expected: the agent calls the search tool, returns a current result with a source URL.
+
+- [ ] **Step 4: Checkpoint**
+
+No repo commit (host-side). Document the chosen provider + key location in `memory/hermes-extensions.md`.
+
+---
+
+## Task 6: Trial `eagle-eye` skill pre-filter (Class A, behind a switch)
+
+> Confirmed the chosen tool: eagle-eye is the **only** direct skill-router in the Hermes ecosystem
+> (per `awesome-hermes-agent` / Hermes Atlas). It directly serves the goal of "many skills installed,
+> few injected per turn." Complementary (not a substitute) and worth a later look on the *tool* side:
+> `llmtrim` (compresses tool schemas + MCP output before each request). `hermes-motif` overlaps
+> curator-evolver (trace→micro-skill), not this router.
+
+**Files:**
+- Host: `~/.hermes/plugins/eagle-eye/` (or skills dir per its README), config toggle in `~/.hermes/config.yaml`.
+
+- [ ] **Step 1: Clone and read; confirm graceful-degradation and the jieba dependency**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "git -C /storage1/hermes/workspace/clones clone https://github.com/willingning-coder/eagle-eye && sed -n '1,200p' /storage1/hermes/workspace/clones/eagle-eye/README.md"
+```
+Expected: README prints. Confirm the install hook, the on/off switch, and that L2–L5 (incl. dense embeddings) are optional. **Plan to run with the dense layer disabled** (CPU/keep off the P100) — lean on L1 (hard triggers) + L2 (BM25) only for the trial.
+
+- [ ] **Step 2: Install with an easy off-switch and minimal deps**
+
+Install per the README (likely `~/.local/bin/hermes plugins install willingning-coder/eagle-eye --enable`), then restart the gateway:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "bash -lc '~/.local/bin/hermes plugins install willingning-coder/eagle-eye --enable' && ssh -o BatchMode=yes ginnoir@valhalla 'sudo systemctl restart hermes-gateway.service && systemctl is-active hermes-gateway.service'"
+```
+Expected: plugin enabled; gateway `active`. (If install fails on `jieba`, `uv pip install --python ~/.hermes/hermes-agent/venv/bin/python jieba` then retry — note the foreign-language dep for maintenance.)
+
+- [ ] **Step 3: A/B test skill selection on representative prompts**
+
+Pick 5 prompts that should each map to a known skill and 2 that should map to none. Run each with eagle-eye enabled, then disable it (`~/.local/bin/hermes plugins disable eagle-eye` + gateway restart) and run the same 7. Record which skills each surfaced and whether the local model then picked the right one.
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "~/.local/bin/hermes run '<representative prompt>' 2>&1 | tail -25"
+```
+Expected: with eagle-eye on, the prompt's prompt-injected skill candidates are ≤5 and include the right one; the "no skill needed" prompts proceed without forced skill loading.
+
+- [ ] **Step 4: Keep-or-cut decision**
+
+**Keep only if** skill selection measurably improved (right skill surfaced more often AND/OR fewer wrong skills loaded) without regressions. Otherwise disable and uninstall:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "~/.local/bin/hermes plugins uninstall eagle-eye && sudo systemctl restart hermes-gateway.service"
+```
+Record the verdict + evidence in `memory/hermes-extensions.md`.
+
+- [ ] **Step 5: Checkpoint**
+
+No repo commit (host-side). Document the A/B result and final state (kept/cut) in memory + vault.
+
+---
+
+# PHASE 2b — Delegation fabric & context efficiency
+
+> Extends acp-skill (Task 2) from 3 targets to 4 external agents, and adds optional token-trimming.
+
+## Task 10: Wire Cursor + Antigravity into the delegation fabric
+
+**Files:**
+- Host: Cursor + `agy` binaries; acp-skill config or a generic shell-agent skill in `~/.hermes/skills/`.
+
+- [ ] **Step 1: Install the Cursor CLI (official cursor.com)**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "bash -lc 'curl https://cursor.com/install -fsS | bash' && ssh -o BatchMode=yes ginnoir@valhalla 'bash -lc \"command -v cursor-agent && cursor-agent --version\"'"
+```
+Expected: `cursor-agent` installs and prints a version. ginnoir logs in later.
+
+- [ ] **Step 2: Install the Antigravity CLI (`agy`) from the OFFICIAL Google source**
+
+Do **not** use blog-derived URLs. Get the exact installer from the official pages first:
+`https://antigravity.google/download` and `https://antigravity.google/docs/gcli-migration`. Then run
+the official one-line installer they document, e.g.:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "bash -lc '<official agy installer from antigravity.google/docs>' && ssh -o BatchMode=yes ginnoir@valhalla 'bash -lc \"command -v agy && agy --version\"'"
+```
+Expected: `agy` (Go binary, ideal for headless SSH) installs and prints a version. Auth later via
+keyring/Google sign-in or `ANTIGRAVITY_TOKEN`.
+
+- [ ] **Step 3: Confirm each agent answers in headless mode (after ginnoir logs in)**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "bash -lc 'claude -p \"say PONG\"; codex exec \"say PONG\"; cursor-agent -p --output-format json --trust \"say PONG\"; agy -p \"say PONG\"'"
+```
+Expected: each prints PONG-ish output. **Cursor caveat:** `-p` has a known hang bug — always pass
+`--output-format json` and wrap with a timeout (`timeout 120 cursor-agent ...`).
+
+- [ ] **Step 4: Extend acp-skill routing (or add a generic shell-agent skill)**
+
+Read `~/.hermes/skills/hermes-acp-orchestrator/SKILL.md` to see if `agent=` routing is extensible.
+- **If extensible:** add `cursor` and `antigravity` targets mapping to the Step 3 invocations
+  (with the cursor timeout + json flags), honoring the `delegation:` `external_timeout_seconds: 900`
+  / `external_max_output_chars: 24000` caps.
+- **If not:** add `~/.hermes/skills/shell-agent/SKILL.md` exposing **one** tool
+  `delegate(agent, goal)` that shells out to claude/codex/cursor/agy with the caps + cursor guard.
+  One tool keeps the surface within the tool-budget.
+Then restart the gateway:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "sudo systemctl restart hermes-gateway.service && sleep 5 && systemctl is-active hermes-gateway.service"
+```
+
+- [ ] **Step 5: End-to-end smoke-test each delegation target**
+
+Run one delegated task per agent (e.g. `agent=cursor`, `agent=antigravity`) and confirm output is
+captured under the cap. Record any that hang/auth-fail for follow-up.
+
+- [ ] **Step 6: Checkpoint**
+
+No repo commit (host-side). Document the four-target fabric + cursor caveat in `memory/hermes-extensions.md`.
+
+---
+
+## Task 11: (OPTIONAL) Context efficiency — llmtrim on the cloud-delegation path
+
+> Opt-in. Start where the win is unambiguous and local-risk-free: trimming the **cloud** delegation
+> agents' traffic (Claude Code/Codex/Cursor → Anthropic/OpenAI). Defer the llama-swap-fronting idea
+> until validated. `rtk-hermes` (shell-output trimming) is a separate lighter opt-in.
+
+**Files:**
+- Host: llmtrim service + `HTTPS_PROXY` env for the delegation agents.
+
+- [ ] **Step 1: Install llmtrim and run setup**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "bash -lc 'npm install -g @llmtrim/cli@latest && llmtrim setup' 2>&1 | tail -20"
+```
+Expected: installs; `setup` installs the name-constrained CA + background proxy. **Review the MITM-CA
+trust implication first** — it's name-constrained to LLM API domains, but it's still a CA on the host.
+
+- [ ] **Step 2: Point the cloud delegation agents through it; measure**
+
+Ensure the delegation agents inherit `HTTPS_PROXY` (llmtrim sets this). Run a representative delegated
+task via Claude Code/Codex and compare token counts / cost before vs after (llmtrim reports savings).
+Keep only if the reduction is real with no quality loss.
+
+- [ ] **Step 3: (Later, separate) Evaluate llama-swap fronting + rtk-hermes**
+
+Document — do not implement here — the two deferred ideas: (a) llmtrim in front of `172.20.0.1:8090`
+via `LLMTRIM_EXTRA_HOSTS` to shrink prompts and speed Pascal prefill (needs validation; unproven for
+local OpenAI-compatible backends); (b) `rtk-hermes` (`pre_tool_call` shell rewrite) if the `rtk`
+binary is available on Ubuntu.
+
+- [ ] **Step 4: Checkpoint**
+
+No repo commit. Record the decision + measured savings (or rejection) in `memory/hermes-extensions.md`.
+`llmtrim uninstall` fully reverses the proxy + CA if rejected.
+
+---
+
+# PHASE 3 — UI trial: stand up BOTH, keep the winner
+
+> Decision resolved (spec §6.1): trial `hermes-ui` (Task 7) **and** `hermes-workspace` (Task 8) in
+> parallel, compare head-to-head against the bundled webui (Task 9), keep one and tear down the
+> rest. `mission-control` is skipped.
+
+## Task 7: Trial `hermes-ui` (lightweight, no build)
+
+**Files:**
+- Host: clone at `/storage1/hermes/workspace/clones/hermes-ui`; optional `hermes-ui.service` (host unit) or a tiny container; Caddy block if exposed.
+
+- [ ] **Step 1: Clone and run the stdlib proxy against the live gateway**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "git -C /storage1/hermes/workspace/clones clone https://github.com/pyrate-llama/hermes-ui && cd /storage1/hermes/workspace/clones/hermes-ui && (~/.hermes/hermes-agent/venv/bin/python3 serve_lite.py >/tmp/hermes-ui.log 2>&1 &) && sleep 3 && curl -fsS http://127.0.0.1:3333/hermes-ui.html | head -5"
+```
+Expected: the proxy starts on :3333 (defaults to gateway `127.0.0.1:8642`, which matches your deployment), and the HTML serves. If your gateway port differs, edit the `HERMES` variable at the top of `serve_lite.py` (no env var exists).
+
+- [ ] **Step 2: Expose it on the LAN for evaluation (don't finalize yet)**
+
+Bind the proxy to the host IP so Caddy can reach it, add a temporary internal-only Caddy block, and
+keep it running for the Task 9 comparison:
+```caddy
+hermes-ui.ginnoir.com {
+	import internal_only
+	reverse_proxy 172.20.0.1:3333
+}
+```
+Run `serve_lite.py` bound appropriately (edit its bind host if it defaults to `127.0.0.1`), regenerate
+bookmarks, push the Caddyfile change. **Do not** create the persistent `hermes-ui.service` yet — that
+happens in Task 9 only for the winner.
+
+- [ ] **Step 3: Checkpoint**
+
+hermes-ui is reachable at `https://hermes-ui.ginnoir.com` (LAN) for the head-to-head. Leave the
+final keep/revert + boot-persistence to Task 9.
+
+---
+
+## Task 8: Deploy `hermes-workspace` as a stack (Class B) — for evaluation
+
+**Files:**
+- Create: `stacks/hermes-workspace/docker-compose.yml`, `stacks/hermes-workspace/stack.env`.
+- Modify: `Caddyfile` (Authentik-gated site block), `bookmarks-domains.html` + `bookmarks-ports.html`.
+- Host (image): build under `/storage1/hermes/workspace/clones/hermes-workspace`.
+
+- [ ] **Step 1: Clone and read its deployment docs (get exact build, ports, env)**
+
+Run:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "git -C /storage1/hermes/workspace/clones clone https://github.com/outsourc-e/hermes-workspace && sed -n '1,200p' /storage1/hermes/workspace/clones/hermes-workspace/README.md && ls /storage1/hermes/workspace/clones/hermes-workspace/{Dockerfile,docker-compose*.yml,.env*} 2>&1"
+```
+Expected: README + a `Dockerfile`/compose appear. Record the exact image build command, the served
+port, and the env var(s) that point the frontend at the gateway (`:8642`) and dashboard (`:9119`).
+**Note the swarm caveat for Task 9:** Swarm Mode (tmux worker pools) can't parallelize inference on
+one P100 — evaluate the workspace/observability features, not swarm.
+
+- [ ] **Step 2: Resolve container→host-service reachability**
+
+hermes-workspace (a container) must reach the host's gateway `:8642` and dashboard `:9119`. Check
+what interface those bind to:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "ss -ltnp | grep -E ':8642|:9119'"
+```
+Expected: shows the bind address. **If bound to `127.0.0.1`**, the container can't reach them — pick
+one: (a) add `extra_hosts: ["host.docker.internal:host-gateway"]` and target `host.docker.internal`,
+or (b) rebind the Hermes services to the docker-bridge host IP `172.20.0.1` (config change + gateway
+restart, with backup). Default recommendation: **(a)** (no Hermes config change; reversible).
+
+- [ ] **Step 3: Build the image on-host**
+
+Run (use the build command discovered in Step 1; tag locally since there's no published image):
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "cd /storage1/hermes/workspace/clones/hermes-workspace && docker build -t hermes-workspace:local . 2>&1 | tail -20 && docker image ls hermes-workspace:local"
+```
+Expected: `hermes-workspace:local` is built and listed.
+
+- [ ] **Step 4: Write the stack compose**
+
+Create `stacks/hermes-workspace/docker-compose.yml` (adjust the served port and gateway/dashboard env
+keys to Step 1's findings; this uses host.docker.internal per Step 2 option (a)):
+```yaml
+# hermes-workspace stack — full web command center for the Hermes agent (trial).
+# No published image: built on-host as hermes-workspace:local (see plan Task 8).
+# Human-facing UI → Authentik-gated. Reaches host gateway :8642 + dashboard :9119
+# via host.docker.internal.
+services:
+  hermes-workspace:
+    image: hermes-workspace:local
+    container_name: hermes-workspace
+    restart: unless-stopped
+    labels:
+      - "com.centurylabs.watchtower.enable=false"
+    env_file:
+      - stack.env
+    networks: [edge]
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    ports:
+      - "172.20.0.1:8088:8088"
+
+networks:
+  edge:
+    external: true
+```
+
+- [ ] **Step 5: Write `stack.env` (gateway/dashboard targets; LF endings)**
+
+Create `stacks/hermes-workspace/stack.env` using the real env keys from Step 1, e.g.:
+```
+HERMES_GATEWAY_URL=http://host.docker.internal:8642
+HERMES_DASHBOARD_URL=http://host.docker.internal:9119
+PORT=8088
+```
+Verify LF endings before committing.
+
+- [ ] **Step 6: Add an Authentik-gated Caddy block**
+
+Unlike camofox (machine-to-machine), this is a human UI → gate with Authentik forward_auth (Pattern B):
+```caddy
+workspace.ginnoir.com {
+	import internal_only
+	route {
+		import authentik_outpost
+		import authentik_forward_auth
+		reverse_proxy hermes-workspace:8088
+	}
+}
+```
+
+- [ ] **Step 7: Deploy and verify**
+
+Run:
+```powershell
+./scripts/gen-bookmarks.ps1
+git add stacks/hermes-workspace/ Caddyfile bookmarks-domains.html bookmarks-ports.html
+git commit -m "feat(hermes-workspace): trial command-center stack (eval vs hermes-ui)"
+git push
+```
+Register the stack in Portainer if new, then:
+```bash
+ssh -o BatchMode=yes ginnoir@valhalla "docker ps --filter name=hermes-workspace --format '{{.Names}} {{.Status}}' && curl -fsS http://172.20.0.1:8088/ | head -5"
+```
+Expected: container `Up`; the workspace HTML serves; logging into `https://workspace.ginnoir.com`
+via Authentik shows live chat/memory/skills wired to your gateway.
+
+- [ ] **Step 8: Checkpoint**
+
+Repo changes committed (Step 7). Leave the keep/tear-down decision to Task 9.
+
+---
+
+## Task 9: Head-to-head UI decision — keep one, tear down the rest
+
+**Files:**
+- Modify (on tear-down): `Caddyfile`, `stacks/...` (remove the loser), bookmarks; host unit for the winner.
+
+- [ ] **Step 1: Compare bundled webui vs hermes-ui vs hermes-workspace**
+
+Use all three live for representative work (chat/streaming, tasks/kanban, files, terminal, skills,
+MCP browser, cron, memory, health). Score against: does it surface your curated tools cleanly, does
+it stay responsive against the P100's latency, and does it add real value over the bundled webui.
+Record the verdict in the vault.
+
+- [ ] **Step 2: Make the winner permanent**
+
+- If **hermes-ui** wins: create host unit `hermes-ui.service` (host-managed, like `obsidian.service`
+  — NOT in this repo), `After=hermes-gateway.service`, `Restart=on-failure`; keep its Caddy block.
+- If **hermes-workspace** wins: keep its stack + Authentik block as-is.
+- If **bundled webui** wins: keep status quo.
+
+- [ ] **Step 3: Tear down the losers (reversible, clean)**
+
+- Remove the hermes-workspace stack if it lost: delete `stacks/hermes-workspace/`, its Caddy block,
+  regenerate bookmarks, commit + push, then delete the stack in Portainer and
+  `docker rm -f hermes-workspace`, `docker image rm hermes-workspace:local`.
+- Stop/remove hermes-ui if it lost: `pkill -f 'serve_lite[.]py'` (bracket trick), remove its Caddy
+  block + clone, commit the Caddyfile change.
+
+- [ ] **Step 4: Checkpoint**
+
+One UI kept and documented in memory + vault; losers fully removed; repo reflects the final state.
+
+---
+
+## Self-Review (completed)
+
+- **Spec coverage:** Original 7 repos — acp-skill (T2), curator-evolver (T3), camofox (T5),
+  eagle-eye (T6), hermes-ui (T7), hermes-workspace (T8 deploy) + keep-one decision (T9);
+  mission-control (skipped per spec §2.5/§5, intentional). **Ecosystem expansion (spec §7)** —
+  hermes-motif (T3b), hermes-web-search-plus (T5b), delegation fabric for cursor+antigravity (T10),
+  optional llmtrim/rtk context efficiency (T11). Claude Code + Codex install is **done** (T1 Step 3).
+  Phase ordering, single-P100 discipline, host-vs-repo boundary, provenance (official installers
+  only — Antigravity URL verified to `antigravity.google`), reversibility, and the §6 decisions are
+  all reflected.
+- **Placeholders:** None of the prohibited kinds. Where a third-party command form can't be verified
+  remotely (e.g. exact `hermes` subcommand spelling, acp-skill install mechanism, response field
+  names), the plan's **first step is a concrete "clone + read the README/SKILL.md" command** that
+  resolves it before use — a real action with expected output, not a TBD.
+- **Consistency:** Paths and names are consistent throughout (`~/.hermes/skills`, `~/.hermes/plugins/curator-evolver`, `camofox-browser:local`, port 9377, gateway 8642, `172.20.0.1` host-IP publish pattern, `sudo systemctl restart hermes-gateway.service`).
+- **Decision gates:** Phases 2–3 are clearly gated on spec §6 and must not start before ginnoir answers.

docs/superpowers/specs/2026-06-26-llm-backend-hermes-design.md

@@ -0,0 +1,202 @@
+# LLM inference backend for Hermes (valhalla) — design
+
+**Date:** 2026-06-26
+**Status:** Approved (pending spec review)
+**Stack:** new `stacks/llm/`
+
+## Goal
+
+Stand up a local, OpenAI-compatible LLM inference endpoint on valhalla and point the
+existing **Hermes** agent backend at it, so Hermes runs against a self-hosted 14B model
+instead of an external provider.
+
+## Key decision: llama.cpp, not vLLM
+
+The request was "set up vLLM," but valhalla's GPU is a **Tesla P100-PCIE-16GB**, die
+**GP100, compute capability 6.0**. That rules out vLLM for the desired model class:
+
+- The GP100 (cc 6.0) lacks the **DP4A INT8** instructions that vLLM's AWQ/GPTQ kernels
+  require (those need Turing 7.5 / Ampere). So vLLM **cannot run quantized 13B+** here.
+- An unquantized 13B in fp16 is ~26 GB → does not fit 16 GB.
+- Current vLLM refuses cc < 7.0 outright; even a ~7B fp16 would need a pinned old vLLM +
+  `--dtype float16` + `VLLM_ATTENTION_BACKEND=XFORMERS`, and still no 13B.
+
+Since the user wants a **13B+ class model** and only needs an **OpenAI-compatible** API
+(confirmed), the right engine is **llama.cpp's `llama-server`**: rock-solid Pascal (sm_60)
+support, GGUF quantization, and a native OpenAI-compatible `/v1` API that Hermes consumes
+unchanged.
+
+## Hardware / host facts (verified live 2026-06-26)
+
+- GPU: 1× Tesla P100-PCIE-16GB, cc 6.0, driver 580.159.04, CUDA 13. Idle. (2nd staged P100
+  not installed — single-GPU design.)
+- Docker 29.5.2, NVIDIA Container Toolkit 1.19.1.
+- **CDI already configured:** `nvidia.com/gpu=0` is a valid device. No host runtime changes
+  needed — the compose just references the CDI device.
+- `/storage1` (ZFS, virtiofs): 17 TB free. Weights live here; VM root is only 200 GB.
+- **Hermes** runs on the host (not Docker) from `/home/ginnoir/.hermes/`:
+  - `hermes dashboard --host 172.20.0.1 --port 9119` (gateway/agent API+WS, pid 1116)
+  - `hermes_cli.main gateway run` (pid 1119)
+  - `hermes-webui/server.py` on `172.20.0.1:8787` (pid 1128)
+  - Bound to `172.20.0.1` = the Docker bridge gateway, so host↔container reachability is
+    trivial: a container published on `172.20.0.1:<port>` is reachable by Hermes and by
+    other containers. No Caddy hop in the inference path.
+
+## Architecture
+
+```
+Hermes agent/gateway (host, 172.20.0.1)
+        │  OpenAI base_url → http://172.20.0.1:8090/v1   (api_key = LLM_API_KEY)
+        ▼
+llama-server container (stacks/llm)  ──CDI nvidia.com/gpu=0──▶  Tesla P100
+   model: Qwen2.5-14B-Instruct-Q4_K_M.gguf
+   weights bind-mounted from /storage1/labdata/llm/models
+```
+
+No Caddy endpoint (Hermes-only, per decision). Endpoint is unauthenticated-but-API-keyed
+and only reachable on the host/bridge — matching the Pattern-B fallback for internal tools.
+
+## The stack — `stacks/llm/docker-compose.yml`
+
+Single service `llama-server`:
+
+- **Image:** `ghcr.io/ggml-org/llama.cpp:server-cuda` at a pinned tag.
+  - Infra-pinned per repo convention: label `com.centurylabs.watchtower.enable=false` so
+    Watchtower won't drift it.
+  - **Verification gate:** confirm the pinned prebuilt image includes Pascal `sm_60`
+    kernels and is CUDA ≤ 13 compatible. If it errors on the P100, fall back to a locally
+    built image with `-DCMAKE_CUDA_ARCHITECTURES=60`.
+- **GPU:** `devices: ["nvidia.com/gpu=0"]` (CDI).
+- **Volumes:** `/storage1/labdata/llm/models:/models` (bind).
+- **Command / args (AS DEPLOYED — 64k, required by Hermes' 64K minimum):**
+  - `-m /models/Qwen2.5-14B-Instruct-Q4_K_M.gguf`
+  - `--alias qwen2.5-14b-instruct` (stable model name Hermes references)
+  - `--parallel 1` (one slot gets the FULL context; default 4 slots split it to 32k/seq → fails Hermes)
+  - `-ngl 99` (full offload — 14B Q4 fits in VRAM)
+  - `--ctx-size 65536`
+  - `--rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 32768` (YaRN extends Qwen2.5's 32k native → 64k)
+  - `--override-kv qwen2.context_length=int:65536` (raises GGUF training-context metadata so
+    llama-server does NOT cap the slot back to 32768 — without this the slot is capped and Hermes still sees 32k)
+  - `--flash-attn on` (this build needs the explicit `on` value; a bare `-fa` swallows the next arg)
+  - `--cache-type-k q8_0 --cache-type-v q8_0` (**both q8_0** — q4_0 V-cache is pathological on Pascal:
+    1.28 tok/s gen at 5–8% GPU util. q8_0/q8_0 → 9.2 tok/s and still fits 64k.)
+  - `--host 0.0.0.0 --port 8080`
+  - API key via `LLAMA_API_KEY` env (env_file) — NOT a CLI flag (no `${VAR}` interpolation; llama-server reads the env var natively)
+- **Ports:** `"172.20.0.1:8090:8080"` (reachable by Hermes on host + by containers).
+- **Networks:** private `llm` net only (no `edge` — no Caddy endpoint this round).
+- **restart:** `unless-stopped`. **Healthcheck:** GET `/health` on 8080.
+- **`env_file: stack.env`** per repo convention.
+
+### `stacks/llm/stack.env`
+
+- `LLM_API_KEY=<generated>` (committed per repo policy — secrets are versioned here).
+
+### VRAM budget (Qwen2.5-14B, GQA: 48 layers, 8 KV heads, head_dim 128)
+
+- KV cache ≈ 0.375 MiB/token fp16 → **q8_0 halves to ≈ 0.1875 MiB/token**.
+- Weights Q4_K_M ≈ 9.0 GB; reserve ~0.8 GB compute buffers.
+- 32k @ q8_0 KV ≈ 6.0 GB → **~15.7 GB total, fits** (tight but safe at 16 GB).
+
+## Model acquisition (one-time)
+
+Download `Qwen2.5-14B-Instruct-Q4_K_M.gguf` (~9 GB) from
+`bartowski/Qwen2.5-14B-Instruct-GGUF` into `/storage1/labdata/llm/models/` on the host
+(e.g. `huggingface-cli download` or `wget` the single GGUF). Documented host-side step,
+done before first stack deploy.
+
+## Hermes integration (host-side, not in git)
+
+Hermes is the **Nous Research Hermes agent** (`hermes-agent.nousresearch.com`). Its config
+is `~/.hermes/config.yaml`, which already has a `providers:` list whose entries are exactly
+OpenAI-compatible upstreams — there's a working `ollama` provider in it today
+(`type: openai`, `base_url: http://192.168.1.73:11434/v1`). Adding the P100 is one more
+entry of the same shape; no new integration surface.
+
+1. **Add a provider** to `providers:` in `~/.hermes/config.yaml`:
+   ```yaml
+   - name: valhalla-p100
+     type: openai
+     base_url: http://172.20.0.1:8090/v1
+     api_key: <LLM_API_KEY>
+     models:
+     - qwen2.5-14b-instruct
+   ```
+   Use `hermes config` / the `hermes` CLI where possible; a direct YAML edit + restart is
+   the fallback (the CLI is the source of truth for `_config_version`).
+2. **Select the model** as the active one via `hermes model` (interactive) — or set
+   `model.default: qwen2.5-14b-instruct` (+ matching provider) if it should be the gateway
+   default rather than a switchable option. The current default is `gpt-5.5` /
+   `openai-codex`; we add ours alongside and let the user choose, rather than silently
+   replacing the default.
+3. **Restart** the three Hermes processes (gateway dashboard pid-class, `gateway run`,
+   webui) so the new provider/model is live.
+4. **Verify** end-to-end: a Hermes prompt routed to `qwen2.5-14b-instruct` produces a
+   completion served by the P100 (confirm via `nvidia-smi` showing the llama-server process
+   holding VRAM during generation).
+
+These host-side steps are documented in the plan (and worth a note in CLAUDE.md
+known-quirks), not committed as repo changes — Hermes isn't in compose, and the alias
+`qwen2.5-14b-instruct` set via `--alias` is the contract between llama-server and this
+provider entry.
+
+## Deployment
+
+`stacks/llm/` is a new Portainer git stack → must be **registered once** (new stacks aren't
+auto-created by the poller). Per repo precedent (memory: portainer-new-stack-registration):
+create via MCP/Portainer, poll `StackList` to confirm, use the stacks' working fine-grained
+PAT for git creds. Pure `env_file` (empty Portainer UI env). After registration, normal
+git-push → 5-min poll redeploys apply.
+
+## Final deployed state (verified live 2026-06-26)
+
+Portainer stack `llm` (id 34), container `llama-server` healthy. Config: 64k / q8_0 KV /
+YaRN / `--parallel 1` / `--override-kv qwen2.context_length=int:65536`. VRAM 15.3 GB used,
+~0.9 GB free. Hermes `model:` block points at provider `custom` → `http://172.20.0.1:8090/v1`
+(matched to the `valhalla-p100` entry in the `providers:` list); active model
+`qwen2.5-14b-instruct`. Original config backed up at `~/.hermes/config.yaml.bak.*`.
+
+### Measured performance (Qwen2.5-14B-Q4_K_M, 64k q8/q8, P100)
+
+- **Generation: ~9.2 tok/s** (memory-bound; fine for a personal assistant).
+- **Prefill: ~54 tok/s** on a large prompt (the misleading ~10 tok/s figure is small-prompt
+  overhead, not throughput).
+- **Hermes system prompt ≈ 16,400 tokens** → first (cold) turn ≈ **5 min** (all prefill).
+- **Prompt cache makes it usable:** llama-server matches by longest-common-prefix
+  (`sim_best = 0.999`), so subsequent turns — even new conversations sharing the stable
+  system prompt — reuse the prefix and respond in **~20 s**. The 5 min is a one-time
+  post-restart warmup.
+
+### Hard-won config gotchas (all verified the slow/broken way first)
+
+1. **`-fa` needs an explicit value** in this build: use `--flash-attn on`. A bare `-fa`
+   swallows the next arg (`--cache-type-k`) and crash-loops.
+2. **`--parallel 1`** — the default 4 slots split `--ctx-size` to 32k/sequence, which fails
+   Hermes' 64K minimum. One slot serves the full window.
+3. **`--override-kv qwen2.context_length=int:65536`** — without it, llama-server *caps the
+   slot back to the GGUF training context (32768)* even with YaRN set, so per-seq stays 32k.
+4. **q8_0 V-cache, NOT q4_0** — q4_0 V-cache is pathological on the GP100 (cc 6.0, no DP4A):
+   **1.28 tok/s** generation at 5–8% GPU util. q8_0/q8_0 → 9.2 tok/s and *still* fits 64k.
+5. **Hermes requires ≥64K context** and rejects smaller models outright (or set
+   `model.context_length` to override — but then the server must actually serve it).
+6. **Hermes provider wiring:** a `providers:` *list* entry is a "named custom provider",
+   activated only by setting the `model:` block to `provider: custom` + matching `base_url`.
+   It is NOT selectable via `--provider <name>` (that path wants a `providers:` *dict*).
+
+## Non-goals / out of scope
+
+- vLLM (ruled out by hardware — see decision above).
+- SSO/Authentik on the endpoint (LAN/host-only, API-keyed).
+- A public `llm.ginnoir.com` Caddy endpoint (declined; easy to add later via `edge` +
+  `internal_only`).
+- Multi-GPU / 2nd P100 install.
+
+## Open risks / follow-ups
+
+- **VRAM is tight (~0.9 GB free).** A full 64k prefill held under real load (16k-token Hermes
+  prompt succeeded), but watch for OOM if other GPU users appear; fall back to `--ctx-size
+  60000` or a smaller weight quant for margin.
+- **Cold-start latency (~5 min).** Inherent to a 16k system prompt at Pascal prefill speed.
+  Mitigation if it annoys: trim Hermes' prompt (disable `environment_probe`, fewer toolsets)
+  to shrink the cached prefix.
+- **Prompt-cache persistence across restarts** is in-memory; a container restart re-pays the
+  cold prefill once.

docs/superpowers/specs/2026-06-27-hermes-ecosystem-integration-design.md

@@ -0,0 +1,376 @@
+# Hermes Ecosystem Integration — Triage & Design
+
+**Date:** 2026-06-27
+**Author:** Claude (overnight autonomous research for ginnoir; for morning review)
+**Status:** Draft — awaiting ginnoir's decisions on the flagged items below
+
+---
+
+## ⚠️ REVISION 2026-06-27 (during execution — live-CLI findings supersede README research)
+
+This spec was written from the repos' READMEs. Executing Phase 1 against the **live** valhalla host
+revealed **Hermes v0.17 already ships native equivalents** of several planned adds. Net changes:
+
+- **`hermes-curator-evolver` → DROPPED.** Native `hermes curator` is already enabled, weekly, and
+  conservative (`consolidate: off`, prune-only, never deletes, auto-backups), managing **74
+  agent-created skills**. It satisfies the "report-only skill maintenance" intent. Installing the
+  plugin would put a second curator on the same `~/.hermes/skills/`. Use the native one (tune config
+  line 417 if desired).
+- **`camofox-browser` → RE-EVALUATE.** Native bundled plugins `browser-browserbase`
+  (stealth/proxies/keep-alive, cloud) + `browser-browser-use`, plus `hermes acp --setup-browser`
+  (local Playwright), already cover browsing. camofox now only wins the *self-hosted-stealth-on-
+  valhalla, no-cloud-key* niche. Otherwise enable a native browser plugin (far less work).
+- **`hermes-agent-acp-skill` → KEPT & INSTALLED (enabled).** Native `hermes acp` is an *editor-
+  integration server* (Zed/VS Code/JetBrains), distinct from this delegation skill; they coexist.
+- **`eagle-eye` → KEPT / ELEVATED.** 74 agent skills + 59 bundled makes per-turn skill routing more
+  valuable (confirm native doesn't already pre-select).
+- **`hermes-motif` → OPTIONAL / lower priority.** Still additive (native curator does lifecycle, not
+  trace→new-skill discovery), but the library is already large and well-managed.
+- **`hermes-web-search-plus` → KEPT** (no native web search observed). **`llmtrim`** overlaps native
+  `hermes insights` for *measurement*; its trimming value stands.
+
+**As-built so far:** acp-orchestrator skill installed + enabled. Cursor CLI installed; Claude Code +
+Codex installed (login pending). `agy` (Antigravity) install blocked by the sandbox classifier — see
+§7.4. The phase tables below are otherwise unchanged; treat the bullets above as the authoritative
+deltas.
+
+---
+
+## 0. TL;DR for the morning
+
+You asked me to research seven Hermes-ecosystem repos and plan how each fits. I verified all
+seven are real and active, mapped each against your **actual** Hermes deployment (host systemd
+install on valhalla, llama-swap/P100 backend, ~29-tool curated MCP surface, Codex + Claude Code
+also installed), and triaged them into adopt / trial / decide / skip.
+
+**Recommended path:**
+
+| Phase | Repo | Verdict | Why |
+|---|---|---|---|
+| **1** | `hermes-agent-acp-skill` | **ADOPT** | Pure skill drop-in; lets the local 20B offload hard work to Codex/Claude Code, which you already run. Highest leverage, lowest risk. |
+| **1** | `hermes-curator-evolver` | **ADOPT (dry-run first)** | Closes Hermes' self-improvement loop; read-only by default; aligns with your curation habit. |
+| **2** | `camofox-browser` | **ADOPT (new homelab stack)** | Gives the local agent real web browsing; clean Docker/Caddy fit; independent of Hermes internals. |
+| **2** | `eagle-eye` | **TRIAL / DEFER** | Solves the exact tool-overload pain you already hit — but very early (5★, 1 commit) and carries a required Chinese-tokenizer dep. Trial behind a flag; don't depend on it. |
+| **3** | `hermes-ui` | **ADOPT (trial)** | Single-HTML + stdlib proxy; stood up next to the bundled webui to evaluate. |
+| **3** | `hermes-workspace` | **ADOPT (trial)** | Full command center; stood up in parallel to compare head-to-head, then keep one. Swarm Mode won't parallelize on one P100 — value is the workspace/observability features. |
+| **3** | `mission-control` | **SKIP** | Not Hermes-native (built for OpenClaw/CrewAI/LangGraph/AutoGen). Redundant with the two Hermes UIs. |
+
+**Decisions — RESOLVED 2026-06-27 (see Section 6):** (1) trial **both** UIs, keep the winner;
+(2) curator-evolver stays **report-only** for now; (3) **trial eagle-eye** — it's the only direct
+skill-router in the ecosystem, no more-mature equivalent exists; (4) wire camofox as a **minimal
+2-tool skill**.
+
+---
+
+## 1. The anchor: your current Hermes deployment
+
+Everything below is sized against what you actually run (from project memory, verified against the
+repo and host conventions):
+
+- **Hermes Agent v0.17.0 (NousResearch)** runs on the **valhalla host**, not in Docker — systemd
+  units `hermes-gateway.service`, `hermes-dashboard.service`, `hermes-webui.service`. Editable
+  install at `~/.hermes/hermes-agent/`; config at `~/.hermes/config.yaml`; CLI `~/.local/bin/hermes`.
+  Workspace at `/storage1/hermes/workspace/`.
+- **Backend:** llama-swap (`stacks/llm`, Portainer id 34) on a single **Tesla P100 16GB**, serving
+  one GGUF at a time, `--parallel 1`, 64k ctx, default `gpt-oss-20b`. OpenAI API at
+  `http://172.20.0.1:8090/v1`.
+- **MCP surface deliberately curated to ~29 tools** because gpt-oss-20b drowns in tool overload.
+  This is the single most important constraint for everything below: **the local model is weak at
+  selection under load, and the GPU runs one model, one sequence at a time.**
+- **You also run Codex (`~/.codex`) and Claude Code**, kept in sync with your Claude global config.
+  This is what makes the ACP delegation skill genuinely useful rather than theoretical.
+
+### 1.1 Two integration classes (this drives the whole plan)
+
+- **Class A — host-side Hermes extensions** (plugins & skills): live in `~/.hermes/` on valhalla,
+  applied by SSH + `hermes ...` + gateway restart. **These are NOT homelabstack repo content.**
+  They are host-managed exactly like the rustdesk/obsidian/xvfb units — tracked in project memory
+  and the Obsidian vault, not committed here. Fully reversible (config backups + plugin uninstall).
+  → `hermes-agent-acp-skill`, `hermes-curator-evolver`, `eagle-eye`.
+- **Class B — Docker services**: become `stacks/<name>/` entries, deployed via the normal
+  Gitea-poll path, fronted by Caddy (`internal_only` + Authentik), secrets in `stack.env`,
+  infra images pinned out of Watchtower.
+  → `camofox-browser`, and (if adopted) `hermes-ui` / `hermes-workspace` / `mission-control`.
+
+---
+
+## 2. Per-repo analysis
+
+### 2.1 `hermes-agent-acp-skill` (Rainhoole) — **ADOPT, Phase 1**
+
+- **What:** A Hermes *skill* (`SKILL.md` + `README.md`, MIT) implementing `delegate_task()` —
+  multi-agent delegation across Hermes internal subagents, **Codex**, and **Claude Code**. Handles
+  agent routing (`agent=...`), context isolation, and safety controls (recommended 900 s external
+  timeout, 24,000-char output cap).
+- **Fit:** Strong and specific. Your local model is gpt-oss-20b — fine for orchestration and light
+  tasks, weak for real implementation/review. This skill lets Hermes *route the hard parts to the
+  frontier CLIs you already have installed and configured*. It turns the weak local brain into a
+  dispatcher.
+- **Risk:** Low. It's a skill file, not running code you didn't write — but it *invokes external
+  agents*, so the real work is verifying Codex/Claude Code are launchable from the Hermes service
+  environment (PATH, auth, non-interactive flags) and that timeouts/output caps are sane.
+- **Implementation sketch:** Drop into `~/.hermes` skills dir (or `hermes skills install` if the
+  repo packages it that way — verify on the host); set delegation targets to the Codex + Claude
+  Code invocations; restart gateway; smoke-test a trivial delegated task end-to-end.
+
+### 2.2 `hermes-curator-evolver` (pingchesu) — **ADOPT, dry-run first, Phase 1**
+
+- **What:** Local-first Hermes *plugin* (Python 3.11, SQLite, `hermes plugins install`). Mines
+  evidence from sessions/tool-calls, ranks skill-improvement candidates, and generates **dry-run,
+  reviewable** proposals. **Read-only by default; mutations require explicit `--approve`.** Only
+  touches *local agent-created* skills — skips bundled, hub-installed, and external skills. Size
+  guardrails, backup/rollback manifests, optional Qwen-embeddings + BGE-reranker for semantic
+  ranking, native scheduler (systemd) integration.
+- **Fit:** This *is* Hermes' "self-improving" thesis made operational, and it matches how you
+  already work (curate, gate, keep things reversible). The default posture (report-only, dry-run,
+  human-review queue) is exactly the safe on-ramp.
+- **Risk / caveats:**
+  - It writes to skills with `--approve`. **Recommendation: run report-only / dry-run indefinitely
+    at first**; promote to `--approve` only after you've seen a few proposal batches you'd have
+    accepted anyway.
+  - Optional embeddings/reranker want a model. **Do not load them onto the P100** alongside
+    gpt-oss — the GPU is single-model/`--parallel 1`. Run ranking on **CPU**, or skip the dense
+    layer initially (BM25/FTS ranking is enough to start).
+  - Schedule the mining pass **off-peak** via a systemd timer so it never contends with live
+    inference.
+- **Implementation sketch:** `hermes plugins install` on the host; point it at the session store;
+  run one manual `collect`→`rank`→`report` cycle by hand; inspect the review queue; only then wire
+  a nightly systemd timer in report-only mode.
+
+### 2.3 `eagle-eye` (willingning-coder) — **TRIAL / DEFER, Phase 2**
+
+- **What:** Zero-invasive skill *pre-filter* plugin. Before the LLM call, narrows 50+ skills to the
+  top ~5 via a 5-layer cascade (hard triggers → FTS5/BM25 → synonyms → dense embeddings → RRF
+  fusion), degrading gracefully when optional deps are absent. "Not matching is a valid result."
+- **Fit (conceptual):** Excellent — this is the **skill-side analog of the MCP tool-overload fix
+  you already did**. Fewer, better skill candidates in the prompt = better selection by a weak
+  model + less Pascal prefill. If it works, it directly improves gpt-oss-20b's behavior.
+- **Risk / caveats (why it's a trial, not an adopt):**
+  - **Maturity:** 5 stars, single commit. Early. Treat as experimental.
+  - **Dependency smell:** `jieba` (Chinese tokenization) is a *required* dep; your skill library is
+    English. Works, but signals the author's primary use case differs from yours — watch for
+    English-tokenization edge cases and maintenance drift.
+  - Same P100 rule: keep its dense-embedding layer on **CPU** or disabled; lean on L1–L2
+    (deterministic + BM25) first.
+- **Implementation sketch:** Install behind an easy on/off switch on the host; A/B a handful of
+  representative prompts with it on vs off; keep it **only if** measured skill-selection quality
+  improves. Do not let any workflow depend on it.
+
+### 2.4 `camofox-browser` (jo-inc) — **ADOPT as a homelab stack, Phase 2**
+
+- **What:** Stealth headless browser (Camoufox/Firefox) exposing a **REST API for AI agents** —
+  anti-fingerprinting, element-reference interaction (`e1`,`e2`), session isolation, proxy/GeoIP,
+  YouTube transcripts, Docker/Fly/Railway deploy. 7.2k★, actively maintained. **No Hermes coupling**
+  — it's a generic tool backend.
+- **Fit:** Clean Class-B homelab fit and a real capability gap-filler: your agent currently has
+  freshrss + obsidian but no general web browser. As `stacks/camofox`, internal-only behind Caddy,
+  it gives Hermes (and Codex/Claude) reliable browsing.
+- **Risk / caveats:**
+  - It's anti-detection tooling — appropriate for personal research; **keep it strictly LAN/SSO-gated
+    (`import internal_only` + Authentik)** and off the public internet. New attack surface; pin the
+    image and read the Dockerfile before running.
+  - Heavier service (~300 MB Camoufox binary, Firefox). Fine on valhalla, but it's not free.
+  - **Wiring to Hermes:** either a thin MCP shim around its REST API, or a Hermes skill that curls
+    it. Given your tool-overload sensitivity, expose it as **one or two** high-level tools (e.g.
+    `browse(url)`, `search(query)`), not its full surface.
+- **Implementation sketch:** New `stacks/camofox/docker-compose.yml` (bind config to `/config/camofox`,
+  blobs/cache to `/storage1/labdata/camofox`, join `edge`), `stack.env`, a Caddy block
+  (`camofox.ginnoir.com`, `import internal_only`), regenerate bookmarks, push. Then add the minimal
+  Hermes tool/skill wrapper on the host and restart the gateway.
+
+### 2.5 The three dashboards — pick **at most one** (Phase 3, DECISION)
+
+You already run the bundled `hermes-webui.service`, so a UI is an *upgrade*, not a gap. These three
+overlap heavily; adopting more than one is waste.
+
+| | `hermes-ui` (pyrate-llama) | `hermes-workspace` (outsourc-e) | `mission-control` (builderz-labs) |
+|---|---|---|---|
+| Stars | 173 | 5.9k | 5.4k |
+| Stack | 1 HTML file + Python **stdlib** proxy (`serve_lite.py`, :3333 → gateway :8642) | React/TS/**Vite build**, Node 22+, → gateway :8642 + dashboard :9119 | Next.js 16 / React 19, better-sqlite3 |
+| Hermes-native? | **Yes** | **Yes** (best with vanilla hermes-agent) | **No** (OpenClaw/CrewAI/LangGraph/AutoGen/Claude SDK) |
+| Effort to host | **Trivial** (no build) | Medium (build → container/stack) | Medium + **adapter work** |
+| Headline features | chat, tasks/kanban, files, terminal, skills, MCP browser, cron, memory, health | all of the above + ops dashboards, "Conductor" missions, **Swarm Mode (tmux worker pools)** | 32 panels, RBAC, spend/trust scoring, multi-framework |
+| Verdict | **Trial first** | **Consider as a proper stack later** | **Skip / defer** |
+
+- **`hermes-ui` — recommended first move.** Lowest cost to evaluate a richer UI. Can run as a tiny
+  host service or a minimal container next to the bundled webui; no build pipeline. If it's better
+  than the bundled webui, keep it; if not, delete one file.
+- **`hermes-workspace` — the ambitious option.** Genuinely the most capable, best run as a real
+  `stacks/hermes-workspace` Docker build behind Caddy + Authentik. **Caveat to flag:** its
+  marquee **Swarm Mode (parallel tmux worker pools) cannot actually parallelize inference on your
+  setup** — the single P100 runs one model with `--parallel 1`, so "swarm" workers serialize on the
+  GPU (or must use a different/remote backend per worker). Adopt for the workspace/observability
+  features, not for swarm, unless/until you add GPU capacity or point swarm workers at cloud
+  providers.
+- **`mission-control` — skip unless requirements change.** It's framework-agnostic and **not
+  Hermes-native** — using it means writing a Hermes adapter for a dashboard whose job the two
+  Hermes-native UIs already do. Only worth it if you later want a single ops plane across *multiple*
+  agent frameworks (you don't run those today).
+
+---
+
+## 3. Recommended phased rollout
+
+**Phase 1 — host-side, reversible, high-leverage (no repo changes):**
+1. `hermes-agent-acp-skill` — install skill, wire Codex + Claude Code delegation targets, smoke-test.
+2. `hermes-curator-evolver` — install plugin in **report-only/dry-run**, manual cycle, then nightly
+   systemd timer (report-only). CPU ranking; no P100 embeddings.
+
+**Phase 2 — new capability + experiment:**
+3. `camofox-browser` — new `stacks/camofox` (Class B), Caddy internal-only + Authentik, minimal
+   Hermes tool wrapper.
+4. `eagle-eye` — host-side trial behind an on/off switch; A/B skill-selection; keep only if it helps.
+
+**Phase 3 — UI decision (your call):**
+5. Trial `hermes-ui` alongside the bundled webui. Evaluate `hermes-workspace` as a stack if the
+   workspace features justify the build. Skip `mission-control`.
+
+Each phase is independently shippable and independently reversible. Stop after any phase.
+
+---
+
+## 4. Cross-cutting concerns (apply to every adoption)
+
+- **Single-P100 discipline.** One GPU, one model, `--parallel 1`. Anything implying parallel or
+  extra model loads — curator embeddings, eagle-eye dense layer, workspace swarm — must run on
+  **CPU**, **off-peak**, or **not at all**. Never co-resident with gpt-oss on the P100.
+- **Host vs repo boundary.** Class-A extensions are host-managed in `~/.hermes/` (document in memory
+  + vault, like the rustdesk/obsidian units). Only Class-B services touch this repo. Don't
+  accidentally commit host config here.
+- **Tool/skill surface minimalism.** The whole reason your MCP is curated to ~29 tools. Expose new
+  capabilities (camofox, delegation) as a *few high-level* tools, not raw surfaces, or you reintroduce
+  the overload you fixed.
+- **Provenance & safety.** Several repos are small/young (eagle-eye 1 commit; acp-skill 1 commit).
+  **Read the code before running it**, especially anything that writes skills (curator) or executes
+  shell / external agents (acp-skill, camofox). **Pin to a reviewed commit/tag**; keep stack images
+  out of Watchtower auto-update.
+- **Reversibility.** Back up `~/.hermes/config.yaml` before each Class-A change (you already do this);
+  snapshot before promoting curator to `--approve`.
+- **Security posture.** Class-B services: `import internal_only` + Authentik, secrets in `stack.env`
+  (LF endings), never public.
+
+---
+
+## 5. What I am explicitly NOT proposing
+
+- Not running curator-evolver in `--approve` (auto-skill-editing) mode out of the gate.
+- Not loading any second model onto the P100 (no GPU embeddings, no GPU swarm).
+- Not adopting more than one dashboard.
+- Not committing host-side Hermes plugins/skills into the homelabstack repo.
+- Not exposing camofox or any new UI to the public internet.
+
+---
+
+## 6. DECISIONS — RESOLVED 2026-06-27
+
+1. **UI direction → trial BOTH.** Stand up `hermes-ui` *and* `hermes-workspace` in parallel,
+   evaluate head-to-head against the bundled webui, then keep the winner and remove the other.
+   (Phase 3 now builds both.)
+2. **Curator autonomy → report-only for now.** `hermes-curator-evolver` runs dry-run/report only;
+   no `--approve` / `--apply-low-risk` / `--approve-auto-apply`. Revisit after a trust period.
+3. **eagle-eye → trial it.** Rationale (ginnoir): wants a large skill library available without
+   weighting every turn — which is exactly eagle-eye's pre-LLM skill-selection job. **Research
+   result:** eagle-eye is the *only* direct skill-router in the Hermes ecosystem (per
+   `awesome-hermes-agent` / Hermes Atlas); the nominal "alternatives" do different jobs —
+   `llmtrim` (compresses tool schemas + MCP output; **complementary**, worth a later look for the
+   tool side), `rtk-hermes` (shell-output compression), `hermes-snow-search` (faster session
+   search), `hermes-motif` (mines traces into micro-skills; a curator-evolver *rival*, not a
+   router). So eagle-eye stands, with the trial caveat (1-commit maturity, `jieba` dep): keep only
+   if it measurably improves selection; run L1+L2 only (no P100 embeddings).
+4. **camofox wiring → minimal.** A small 2-tool Hermes skill (`browse`, `search`) over the REST
+   API, not the full surface or an MCP shim — protects the curated tool budget.
+
+All four resolved ⇒ **Phases 1, 2, and 3 are all actionable.** The implementation plan that
+follows details every phase.
+
+**Ecosystem expansion (added 2026-06-27, per ginnoir):** while researching eagle-eye alternatives I
+surveyed the wider Hermes ecosystem and the external-agent landscape. The worthwhile additions and
+the motif-vs-curator verdict are in **§7**. Claude Code + Codex are **already installed on valhalla**
+(this session); Cursor + Antigravity are scoped for the delegation fabric.
+
+---
+
+## 7. Ecosystem expansion & the delegation fabric (added 2026-06-27)
+
+### 7.1 `hermes-motif` vs `hermes-curator-evolver` — verdict: **complementary, run both**
+
+I earlier mis-framed these as rivals. They do **different halves of one loop**:
+
+- **`hermes-motif` (Saurav0989) — skill DISCOVERY.** Watches tool calls live, finds repeated
+  sequences (same tools, varying args → `{variable}`), and proposes them as **new** `SKILL.md`
+  drafts. **Makes no LLM calls** — pure pattern-mining, so **zero P100 cost**. `auto_install: false`
+  = proposal-only. Zero-to-one skill creation.
+- **`hermes-curator-evolver` (pingchesu) — skill REFINEMENT.** Improves **existing** agent-created
+  skills from session evidence; bounded, reviewable edits; report-only default; more mature
+  (CI, systemd scheduler, backup/rollback).
+
+**Fit for you:** your stated goal (eagle-eye rationale) is a *large* skill library without per-turn
+weight. That's a **skill factory**: **motif manufactures** new skills from what you already do
+repeatedly → **curator refines** them → **eagle-eye routes** among them at call time. They compose.
+**Recommendation: run both in proposal/report-only mode.** If you insist on only one to start,
+**motif** is the lower-risk, higher-immediate-value pick for *growing* a library (no model cost, no
+edits to existing files) — but there's no reason to drop curator; it's already Phase 1 and free to
+leave in report-only. Plan adds motif as **Task 3b**.
+
+### 7.2 `hermes-web-search-plus` — **ADOPT** (pairs with camofox)
+
+Mature (v2.6.1, MIT, stdlib-only) Hermes plugin: multi-provider web **search** + extraction with
+intent-aware routing (news/shopping/docs/GitHub/academic/security) and a `research` mode. 14
+providers, **all keys optional** — works with a single configured provider (free tiers exist: Tavily,
+Exa, SearXNG self-host, Keenable keyless). **Complements camofox**, doesn't overlap it: search-plus
+*finds* (cheap API calls), camofox *browses/interacts* (heavy headless Firefox). For a P100-limited
+local agent, search-plus is the lighter, higher-frequency tool — arguably the one to reach for first;
+camofox for when a page needs real rendering/interaction. Plan adds it as **Task 5b**, before/with
+camofox. Install: `hermes plugins install robbyczgw-cla/hermes-web-search-plus --enable`.
+
+### 7.3 Context-efficiency layer — **OPTIONAL**, helps the Pascal bottleneck
+
+Your memory notes prefill is the P100 differentiator and context bloat is the enemy. Two opt-in tools:
+
+- **`llmtrim` (fkiene, MPL-2.0)** — HTTPS-proxy / MCP / CLI that compresses **tool schemas + history
+  + tool output** before requests (claims −31% input / −74% output tokens, BM25+ context ranking,
+  log-templating). Works with anything honoring `HTTPS_PROXY` (Claude Code, Cursor, Aider). **Two
+  distinct payoffs:** (a) in front of the **cloud delegation agents** (Claude Code/Codex/Cursor →
+  Anthropic/OpenAI) it cuts real $ cost; (b) potentially in front of **llama-swap** (via
+  `LLMTRIM_EXTRA_HOSTS`) to shrink prompts and speed Pascal prefill — needs validation, it's not
+  documented for local OpenAI-compatible backends. Caveat: it installs a name-constrained MITM CA;
+  acceptable but a real trust decision.
+- **`rtk-hermes` (ogallotti, MIT)** — `pre_tool_call` hook that rewrites shell commands through the
+  `rtk` binary so terminal output arrives pre-filtered (cache-safe; preserves tool schema). Lighter,
+  shell-only. Needs the `rtk` binary on the host (`brew`/Linux build — verify availability).
+
+**Recommendation:** defer both to an **opt-in Task 11**. Start llmtrim on the *cloud-delegation* path
+(clear cost win, no local risk) before considering it in front of llama-swap.
+
+### 7.4 The delegation fabric — claude-code · codex · cursor · antigravity
+
+Your acp-skill (Task 2) natively routes to `codex`, `claude-code`, and `hermes` subagents. You also
+want **Cursor** and **Antigravity** in the mix. All four have real **headless** CLIs usable as
+delegation targets from the valhalla host:
+
+| Agent | CLI | Headless invocation | Install (official) | Status on valhalla |
+|---|---|---|---|---|
+| Claude Code | `claude` | `claude -p` | `curl -fsSL https://claude.ai/install.sh \| bash` → `~/.local/bin/claude` | **Installed** v2.1.195 (login pending) |
+| Codex | `codex` | `codex exec` / non-interactive | `npm i -g @openai/codex` (Node 22+) | **Installed** v0.142.3 (login pending) |
+| Cursor | `cursor-agent` | `cursor-agent -p --output-format json --trust` | `curl https://cursor.com/install -fsS \| bash` | Scoped (Task 10) — note: known `-p` hang bug, set `--output-format json` + timeout |
+| Antigravity | `agy` (ex-Gemini CLI, Go binary, **best for headless SSH**) | `agy -p` / `--headless --approve` | from **official** `antigravity.google/download` + `/docs/gcli-migration` (auth via keyring/Google sign-in or `ANTIGRAVITY_TOKEN`) | Scoped (Task 10) |
+
+**Wiring approach (Task 10):** acp-skill supports `codex`/`claude-code`/`hermes` out of the box —
+verify whether its `agent=` routing is **extensible** (read `SKILL.md`). If yes, add `cursor` and
+`antigravity` targets pointing at the headless invocations above. If not, add a **thin generic
+"shell-agent" skill** that shells out to any of the four with the timeout/output caps from the
+`delegation:` config — keeping the exposed tool surface minimal (your standing tool-budget rule).
+**Alternatives noted, not adopted:** `42-evey/hermes-plugins` ships `evey-bridge` (file inbox/outbox
++ MCP bridge to Claude Code) and `evey-delegate-model` (model routing w/ fallback) — heavier,
+opinionated, unknown maturity; acp-skill is the simpler purpose-built choice. `evey-cost-guard`
+(Langfuse budget enforcement) becomes worth a look **once cloud delegation is live** and spend
+matters.
+
+### 7.5 Updated rollout placement
+
+- **Phase 1:** + **Task 3b `hermes-motif`** (proposal-only, zero P100 cost).
+- **Phase 2:** + **Task 5b `hermes-web-search-plus`** (light search, pairs with camofox).
+- **Phase 2b (new):** **Task 10 delegation fabric** (install cursor + agy, wire all four into
+  acp-skill) and **Task 11 (optional) context efficiency** (llmtrim on the cloud-delegation path;
+  rtk for shell).
+- **Phase 3:** unchanged (both UIs → keep one).

docs/superpowers/specs/2026-08-05-jellyfin-migration-design.md

@@ -0,0 +1,327 @@
+# Jellyfin migration — design
+
+**Date:** 2026-08-05
+**Status:** approved, not yet implemented
+**Supersedes:** `plans/homelab-improvements/tasks/TB-027-jellyfin.md` (stub — resolve it to "Do it" and link here)
+
+---
+
+## 1. Why
+
+Plex now requires a Plex Pass (server owner) or a Remote Watch Pass (viewer) for remote
+playback of personal media. Enforcement began on Roku in late 2025 and reaches Fire TV,
+Android TV, Google TV, Apple TV and third-party clients through 2026. valhalla's Plex
+server has **no Plex Pass**, so every remote viewer is being pushed toward paying for
+access to a server they don't own.
+
+Jellyfin has no such gate. This spec replaces host-native Plex with containerised Jellyfin,
+managed in this repo like everything else.
+
+## 2. Current state (measured 2026-08-05, not assumed)
+
+### Plex
+
+| Fact | Value |
+|---|---|
+| Install | AUR `plex-media-server` 1.43.2, systemd unit `plexmediaserver`, runs as `plex:plex` |
+| Binary | `/usr/lib/plexmediaserver/` |
+| Data dir | `/var/lib/plex` → **symlink to `/storage1/labdata/plex`** (virtiofs, *not* the root disk) |
+| Data size | **383G total** — `Media` 249G, `localhost` 116G, `Metadata` 15G, `Plug-in Support` 2.1G |
+| Reverse proxy | none — Plex handles its own relay/direct connect |
+
+> `CLAUDE.md` documents the data dir as `/var/lib/plexmediaserver/...`. **That path does not
+> exist.** Correcting it is a Phase 0 task.
+
+### Libraries
+
+Served from `/storage1`, exposed to the \*arrs through `/storage1/Media/*` symlinks:
+
+| Library | Real path | Items |
+|---|---|---|
+| Movies | `/storage1/Movies` | 1357 |
+| TV | `/storage1/TV` | 173 |
+| Anime | `/storage1/Anime` | 361 |
+
+`/storage1/Music`, `/storage1/Books` and `/storage1/LinuxISOs` (Stash's) are **out of scope**.
+
+### Usage (Tautulli, trailing 90 days / 600-play sample)
+
+| Metric | Value |
+|---|---|
+| Users | 37 registered, 29 flagged active — but **many are dormant in practice** |
+| Stream decisions | 90 direct play · 3 direct stream · **84 transcode (~48%)** |
+| Peak concurrent streams | **7** |
+| Transcode drivers | audio 23/25 · video 15/25 · **subtitle 0/25** |
+| Source content | h264 18 / hevc 7 · **100% SDR** |
+| Platforms | Android 281 · tvOS 102 · browsers 53 · Roku 39 · iOS 18 · Tizen 2 · PlayStation 1 |
+
+Two workloads that would have sunk a CPU-only plan are **absent**: no subtitle burn-in and
+no HDR tone-mapping. The residual load is 1080p SDR, which 52 Xeon threads can carry.
+
+### Host constraints
+
+| Constraint | Detail |
+|---|---|
+| 🔴 Root disk | `/dev/sda2` 194G, **179G used, 5.7G free (97%)**. `/config` lives here. Docker holds 123.5G of images, 51G reclaimable. |
+| `/storage1` | 94T, 78T used, **17T free**, virtiofs from the Proxmox host |
+| GPU | Tesla P100-PCIE-16GB. **GP100 has no NVENC** — decode-only per NVIDIA's support matrix. Currently held by llama-swap. |
+| CPU / RAM | 52 threads · 94G RAM (69G available) |
+| Docker GPU | CDI configured (`/etc/cdi/nvidia.yaml`) |
+
+## 3. Decisions
+
+| # | Decision | Rationale |
+|---|---|---|
+| D1 | New Portainer stack `stacks/streaming/` | Jellyfin config changes must not redeploy sonarr/radarr/qbittorrent. Plex-era cleanup stays contained. |
+| D2 | Public `jellyfin.ginnoir.com` via Caddy | Replaces Plex's relay. Tailscale-only or `share_gate` would break Roku/Fire TV/tvOS clients. |
+| D3 | **No SSO for Jellyfin** | `jellyfin-plugin-sso` completes OIDC only inside a browser. Roku, Android TV/Fire TV and Swiftfin cannot use it at all — and that is the entire user base. Native Jellyfin accounts. **Deliberate exception to the project auth standard.** |
+| D4 | Parallel run, then retire | Plex untouched and running through Phase 6. No failure before Phase 7 costs anything but time. |
+| D5 | Plexyfin once, then removed | Copies the curated 15G of posters/backdrops and rebuilds collections from PMS. It deletes and replaces artwork wholesale on every run, so leaving it installed would destroy later Jellyfin customisations. |
+| D6 | JellyPlex-Watched, two-way, scheduled | Users can move between servers freely during the overlap. |
+| D7 | Jellystat replaces Tautulli | Closest 1:1 equivalent. Tautulli is Plex-only and dies with Plex. |
+| D8 | CPU-only transcoding, measured | P100 cannot encode. Workload is 1080p SDR, no burn-in, no tone-mapping. Jellystat measures the truth. |
+| D9 | Escape hatch: Intel Arc A310/A380 | ~$100-150, no session limit, AV1. Only if Phase 5 data shows CPU strain. |
+| D10 | Seerr converted in place at cutover | `seerr-team/seerr` already supports Jellyfin natively. In-place conversion is unsupported upstream ([seerr#2522](https://github.com/seerr-team/seerr/issues/2522)) — mitigated by config backup and a fresh-config fallback. |
+| D11 | Transcode dir on disk, **not tmpfs** | [jellyfin#16608](https://github.com/jellyfin/jellyfin/issues/16608): the segment cleaner first checks at 20s and the throttler at 5s, but Remux/DirectStream jobs finish in 2-7s, so cleanup never fires and segments orphan. DirectStream (copy video + transcode audio) is 8 of 25 sampled sessions — routine here, not an edge case. Plex already writes transcodes to virtiofs at a 48% transcode rate without trouble. |
+| D12 | Trickplay at full quality | Plex's 365G of BIF thumbnails is **format-incompatible** with Jellyfin's tiled JPEG trickplay. None of it is reusable; Jellyfin regenerates from scratch. |
+| D13 | Wizarr invites issued **manually and selectively** | Most Plex users are dormant. No bulk invite generation, no scripted account creation, no broadcast announcement. ginnoir sends individual invites as people ask or as he chooses. |
+
+## 4. Architecture
+
+New Portainer stack `stacks/streaming/` — private `streaming` network plus `edge`, all
+config via `env_file: stack.env` per the repo convention, no Portainer UI env vars.
+
+| Container | Image | Purpose |
+|---|---|---|
+| `jellyfin` | `lscr.io/linuxserver/jellyfin` | media server (PUID/PGID from `stack.env`) |
+| `jellystat` | `cyfershepard/jellystat` | Tautulli replacement |
+| `jellystat-db` | `postgres:16-alpine` | pinned + `com.centurylabs.watchtower.enable=false` per infra convention |
+| `wizarr` | `ghcr.io/wizarrrr/wizarr` | invite + guided per-device onboarding, used ad-hoc |
+| `jellyplex-watched` | `luigi311/jellyplex-watched` | scheduled two-way watch-state sync |
+
+### Ordering constraint
+
+Plexyfin and JellyPlex-Watched both read from the **live PMS API**. Everything that pulls
+from Plex must complete before Plex is decommissioned. This is the hard sequencing rule of
+the whole migration.
+
+## 5. Storage layout
+
+The root disk is the primary hazard, so Jellyfin's bulky data is split off deliberately.
+
+| Container path | Host path | Filesystem | Rationale |
+|---|---|---|---|
+| `/config` | `/config/jellyfin` | root SSD | SQLite DBs — better fsync semantics, easy to back up. A few GB. |
+| `/config/metadata` | `/storage1/labdata/jellyfin/metadata` | virtiofs | artwork + NFO, grows unbounded |
+| `/config/data/trickplay` | `/storage1/labdata/jellyfin/trickplay` | virtiofs | **~300-400G.** 10.11 moved trickplay here and it is *not* relocatable from the UI, hence an explicit sub-bind. |
+| `/config/cache` | `/storage1/labdata/jellyfin/cache` | virtiofs | image cache |
+| `/transcode` | `/storage1/labdata/jellyfin/transcodes` | virtiofs | see D11 |
+| `/data/movies` | `/storage1/Movies` | virtiofs, **`:ro`** | Jellyfin never writes to the library |
+| `/data/tv` | `/storage1/TV` | virtiofs, **`:ro`** | |
+| `/data/anime` | `/storage1/Anime` | virtiofs, **`:ro`** | |
+
+Net effect: only a few GB ever lands on the 194G root disk.
+
+### Note on SQLite and virtiofs
+
+Plex has run its SQLite library DB on virtiofs here for months without incident, so this is
+proven workable — the root-SSD placement is a preference, not a hard requirement.
+
+It carries one accepted trade-off. If `/storage1` fails to mount again (as on 2026-07-06),
+Jellyfin with its DB on root will **start up healthy against an empty library** and flag the
+entire catalogue as missing — the RomM failure mode. With the DB on `/storage1` it would
+simply fail to start, which is the louder and safer failure. `storage1-guard.service`
+already blocks guest start when `/storage1` isn't real, so root SSD stands.
+
+## 6. Caddy
+
+```caddy
+# =============================================================
+# STREAMING — public
+# =============================================================
+jellyfin.ginnoir.com {
+	reverse_proxy jellyfin:8096
+}
+
+wizarr.ginnoir.com {
+	reverse_proxy wizarr:5690
+}
+
+# =============================================================
+# STREAMING STATS — internal only
+# =============================================================
+jellystat.ginnoir.com {
+	route {
+		import internal_gate
+		import authentik_outpost
+		handle {
+			import authentik_forward_auth
+			reverse_proxy jellystat:3000
+		}
+	}
+}
+```
+
+`jellyfin` and `wizarr` are public by design — remote users and invite recipients reach them
+from anywhere. `jellystat` follows the tautulli/sonarr pattern: `internal_gate` first inside
+the `route` block (an `import internal_only` above a catch-all route is dead code), then
+Authentik forward auth.
+
+In Jellyfin → Networking, register the Caddy container as a known proxy and enable forwarded
+headers, so real client IPs reach Jellystat and the remote-vs-LAN bitrate rules.
+
+## 7. Metadata strategy
+
+1. Jellyfin scans and fetches fresh from TMDB/TVDB — the same sources Plex used, so baseline
+   quality is identical.
+2. **Plexyfin runs once**, overwriting posters/backdrops with the curated Plex artwork and
+   rebuilding collections. This is where the 15G `Metadata` investment is preserved.
+3. **Plexyfin is then removed.** See D5.
+
+Plex's 365G of `Media` + `localhost` (BIF preview thumbnails) is not reusable — see D12.
+
+## 8. Plugins
+
+**Install**
+
+| Plugin | Purpose | Note |
+|---|---|---|
+| Plexyfin | collections + curated artwork from PMS | 3rd-party repo. Temporary — remove after Phase 2. |
+| Intro Skipper | skip intros/recaps/credits | **Verify 10.11 compatibility at install.** The original is unmaintained; use the active fork. |
+| AniDB / AniList | anime metadata | Anime library only |
+| Trakt | per-user scrobbling | optional |
+
+**Deliberately skipped**
+
+| Plugin | Why |
+|---|---|
+| `jellyfin-plugin-sso` | browser-only OIDC — useless for Roku/Fire TV/tvOS (D3) |
+| Open Subtitles | Bazarr already writes subtitles to disk and does it better |
+| Playback Reporting | Jellystat supersedes it |
+
+## 9. Transcoding
+
+CPU-only on 52 threads: `veryfast` preset, per-session thread cap, throttling enabled,
+**segment deletion on with a ~360s keep window**. Measured worst case is 7 concurrent streams
+at ~48% transcode, 1080p SDR, no burn-in, no tone-mapping.
+
+Jellyfin's Android client (the heaviest platform here at 281 plays) direct-plays audio codecs
+Plex's client refuses, so the transcode rate may **fall** after migration. Jellystat confirms
+or refutes this during Phase 5.
+
+If it does strain: Intel Arc A310/A380 (D9). Needs a PCIe slot, an R730XD shutdown, and an
+IOMMU passthrough to the valhalla VM — check the new card's IOMMU group, as only the P100's
+group 7 is known clean.
+
+## 10. Phases
+
+Plex stays running and untouched through Phase 6.
+
+### Phase 0 — Disk remediation and baseline
+
+The root disk is at 97%. Nothing else may proceed until it isn't.
+
+- `docker image prune` / `builder prune` — ~51G reclaimable
+- Audit the remaining 179G on `/dev/sda2`; identify anything else structurally growing
+- Uptime Kuma monitor on root-disk free space + ntfy alert (this is the second root fill)
+- Correct the Plex paths in `CLAUDE.md` (`/var/lib/plex` → `/storage1/labdata/plex`)
+
+**Done when:** root disk has comfortable headroom and alerts before it doesn't.
+
+### Phase 1 — Deploy the stack
+
+- `stacks/streaming/docker-compose.yml` + `stack.env`, `jellyfin` only
+- Storage layout per §5; register the stack in Portainer (one-time, per the new-stack procedure)
+- Caddy site blocks; `scripts/gen-bookmarks.ps1`
+- Libraries: Movies, TV, Anime (Anime with anime metadata providers)
+- Verify `check-portainer-stack-env.ps1` before pushing
+
+**Done when:** Jellyfin reachable at `jellyfin.ginnoir.com`, libraries scanned, no users.
+**Rollback:** delete the stack.
+
+### Phase 2 — Parity
+
+- Plexyfin: sync collections + artwork from PMS, then **remove the plugin**
+- **Trickplay generation** — full quality, thread-capped (~16-24 of 52), run before any users
+  arrive. Budget ~300-400G on `/storage1` and a multi-day first pass over ~1891 items.
+  Enable NVDEC for this: trickplay is decode + JPEG encode and needs no NVENC, so the P100
+  helps here even though it's useless for live transcode. Schedule when llama-swap is idle.
+- Remaining plugins per §8
+- Sonarr/Radarr/Bazarr → add Jellyfin library-refresh connections (leave the Plex ones for now)
+
+**Done when:** Jellyfin's library visually matches Plex's, trickplay complete.
+
+### Phase 3 — Watch-state sync
+
+- `jellyplex-watched` container, two-way, every 6h
+- Plex admin token + Jellyfin API key in `stack.env`; user-mapping file where names differ
+- Verify against ginnoir's own account in both directions
+
+**Done when:** a play on either server appears on the other within one cycle.
+**Rollback:** stop the container.
+
+### Phase 4 — Pilot
+
+- Configure Wizarr (server connection, wizard steps per device)
+- 3-5 willing users, **at least one each on Roku, Fire TV/Android TV and tvOS**
+- Collect real feedback on playback, transcoding and the onboarding wizard
+
+**Done when:** every target platform has a confirmed working client.
+**Rollback:** pilot users return to Plex, which never stopped.
+
+### Phase 5 — Selective migration
+
+Per D13, this phase is **demand-driven, not a campaign.**
+
+- Wizarr invites issued **individually**, as people ask or as ginnoir chooses
+- Dormant accounts get nothing
+- A short written explainer exists to accompany an invite — sent with invites, not broadcast
+- Both servers live; run at least **4 weeks** to accumulate meaningful Jellystat data
+- Watch transcode load and concurrency; decide on D9
+
+**Done when:** the people who actually watch things are on Jellyfin and it's holding up.
+
+### Phase 6 — Cutover
+
+- Back up `/config/overseerr`, then reconfigure Seerr from Plex to Jellyfin (D10)
+- Jellystat live at `jellystat.ginnoir.com`
+- **Re-check Tautulli** for who is still on Plex and on what. Handle those users
+  individually — Tizen and PlayStation are the weak platforms (3 plays of 600 as of now).
+- Remove Plex connections from Sonarr/Radarr/Bazarr
+- Announce the Plex end date
+
+**Done when:** Seerr requests flow against Jellyfin and nobody depends on Plex.
+**Rollback:** restore `/config/overseerr`.
+
+### Phase 7 — Decommission
+
+- `systemctl stop plexmediaserver && systemctl disable plexmediaserver`
+- **2-week soak** with the service stopped but installed
+- `pacman -Rns plex-media-server`
+- Retire the `tautulli` container and its Caddy block
+- Tarball `/storage1/labdata/plex`, keep 30 days, then reclaim **383G**
+- Update `CLAUDE.md` (drop the external-services Plex section), bookmarks, Uptime Kuma monitors
+
+**Done when:** Plex is gone and nothing references it.
+**Rollback (until the tarball expires):** reinstall the package, restore the data dir.
+
+## 11. Risks
+
+| Risk | Severity | Mitigation |
+|---|---|---|
+| Root disk at 97% | **high** | Phase 0 exists solely for this; monitor + alert added |
+| Public Jellyfin is new attack surface Plex's relay hid | medium | Jellyfin's built-in brute-force lockout, strong admin credential, no anonymous access, monitoring |
+| CPU transcode capacity at peak | medium | measured in Phase 5 via Jellystat; Arc A380 escape hatch (D9) |
+| Seerr in-place conversion unsupported upstream | medium | config backup first, fresh-config fallback documented |
+| Plexyfin is destructive | medium | run once in Phase 2, then remove (D5) |
+| Trickplay generation starves live transcoding | low | thread-capped, runs before users arrive, NVDEC-assisted |
+| Intro Skipper 10.11 compatibility | low | verify at install; it's optional polish |
+| Holdouts on Tizen / PlayStation | low | 3 of 600 plays; handled individually at Phase 6 with fresh data |
+
+## 12. Out of scope
+
+- Music, Books and Stash content — Jellyfin serves Movies, TV and Anime only
+- SSO for Jellyfin (D3)
+- Migrating Seerr request history
+- Replacing the P100 or restructuring GPU allocation
+- The wider `/storage1` redundancy problem (tracked separately)

portainer-compose.yml

@@ -149,6 +149,29 @@ services:
     labels:
       - "com.centurylabs.watchtower.enable=false"
 
+  gitea-runner-ci:
+    container_name: gitea_runner_ci
+    image: gitea/act_runner:latest
+    restart: unless-stopped
+    networks:
+      - gitea_internal
+    environment:
+      - GITEA_INSTANCE_URL=https://gitea.ginnoir.com
+      - GITEA_RUNNER_REGISTRATION_TOKEN=${GITEA_RUNNER_REGISTRATION_TOKEN}
+      - GITEA_RUNNER_NAME=valhalla-ci-runner-2
+      - GITEA_RUNNER_LABELS=ubuntu-latest:docker://node:24-bookworm,node-24:docker://node:24-bookworm
+      - CONFIG_FILE=/data/config.yaml
+    volumes:
+      - /config/gitea/act_runner_ci:/data
+      - /var/run/docker.sock:/var/run/docker.sock
+      - /home/ginnoir/valhalla-lab:/valhalla-lab
+      - /config/caddy:/config/caddy
+    depends_on:
+      gitea:
+        condition: service_started
+    labels:
+      - "com.centurylabs.watchtower.enable=false"
+
   registry:
     container_name: registry
     image: registry:2

scripts/check_uptime.js

@@ -1,10 +1,27 @@
 const fs = require('fs');
 const path = require('path');
 
+async function fetchWithRetry(url, options = {}, retries = 3, backoff = 2000) {
+  for (let i = 0; i < retries; i++) {
+    try {
+      const response = await fetch(url, options);
+      if (response.ok) return response;
+      if (response.status >= 500) {
+        throw new Error(`HTTP ${response.status}: ${response.statusText}`);
+      }
+      return response;
+    } catch (err) {
+      if (i === retries - 1) throw err;
+      console.warn(`Fetch to ${url} failed (attempt ${i + 1}/${retries}): ${err.message}. Retrying in ${backoff}ms...`);
+      await new Promise(resolve => setTimeout(resolve, backoff));
+    }
+  }
+}
+
 async function checkStatus() {
   try {
     // 1. Fetch status page HTML
-    const htmlResponse = await fetch('https://uptime.ginnoir.com/status/default');
+    const htmlResponse = await fetchWithRetry('https://uptime.ginnoir.com/status/default');
     if (!htmlResponse.ok) {
       throw new Error(`Failed to fetch status page: ${htmlResponse.statusText}`);
     }
@@ -34,7 +51,7 @@ async function checkStatus() {
     }
 
     // 2. Fetch heartbeat JSON
-    const heartbeatResponse = await fetch('https://uptime.ginnoir.com/api/status-page/heartbeat/default');
+    const heartbeatResponse = await fetchWithRetry('https://uptime.ginnoir.com/api/status-page/heartbeat/default');
     if (!heartbeatResponse.ok) {
       throw new Error(`Failed to fetch heartbeat: ${heartbeatResponse.statusText}`);
     }

scripts/check_uptime_to_obsidian.js

@@ -0,0 +1,179 @@
+const fs = require('fs');
+const path = require('path');
+
+async function fetchWithRetry(url, options = {}, retries = 3, backoff = 2000) {
+  for (let i = 0; i < retries; i++) {
+    try {
+      const response = await fetch(url, options);
+      if (response.ok) return response;
+      if (response.status >= 500) {
+        throw new Error(`HTTP ${response.status}: ${response.statusText}`);
+      }
+      return response;
+    } catch (err) {
+      if (i === retries - 1) throw err;
+      console.warn(`Fetch to ${url} failed (attempt ${i + 1}/${retries}): ${err.message}. Retrying in ${backoff}ms...`);
+      await new Promise(resolve => setTimeout(resolve, backoff));
+    }
+  }
+}
+
+async function checkStatus() {
+  try {
+    // 1. Fetch status page HTML
+    const htmlResponse = await fetchWithRetry('https://uptime.ginnoir.com/status/default');
+    if (!htmlResponse.ok) {
+      throw new Error(`Failed to fetch status page: ${htmlResponse.statusText}`);
+    }
+    const html = await htmlResponse.text();
+
+    // Extract window.preloadData = ...;
+    const preloadRegex = /window\.preloadData\s*=\s*({.*?});/s;
+    const match = html.match(preloadRegex);
+    if (!match) {
+      throw new Error("Could not find window.preloadData in HTML");
+    }
+
+    let preloadData;
+    try {
+      preloadData = new Function(`return ${match[1]};`)();
+    } catch (e) {
+      throw new Error(`Failed to parse preloadData: ${e.message}`);
+    }
+
+    const monitors = [];
+    if (preloadData && preloadData.publicGroupList) {
+      for (const group of preloadData.publicGroupList) {
+        if (group.monitorList) {
+          monitors.push(...group.monitorList);
+        }
+      }
+    }
+
+    // 2. Fetch heartbeat JSON
+    const heartbeatResponse = await fetchWithRetry('https://uptime.ginnoir.com/api/status-page/heartbeat/default');
+    if (!heartbeatResponse.ok) {
+      throw new Error(`Failed to fetch heartbeat: ${heartbeatResponse.statusText}`);
+    }
+    const heartbeats = await heartbeatResponse.json();
+
+    // 3. Map status and print summary
+    const monitorMap = {};
+    for (const monitor of monitors) {
+      monitorMap[monitor.id] = {
+        name: monitor.name,
+        type: monitor.type,
+        status: 'UNKNOWN',
+        ping: null,
+        lastCheck: null,
+        msg: ''
+      };
+    }
+
+    const heartbeatList = heartbeats.heartbeatList || {};
+    for (const id in heartbeatList) {
+      const list = heartbeatList[id];
+      if (list && list.length > 0) {
+        const latest = list[list.length - 1];
+        if (monitorMap[id]) {
+          monitorMap[id].status = latest.status === 1 ? 'UP' : 'DOWN';
+          monitorMap[id].ping = latest.ping;
+          monitorMap[id].lastCheck = latest.time;
+          monitorMap[id].msg = latest.msg || '';
+        }
+      }
+    }
+
+    // Generate summary
+    const monitorValues = Object.values(monitorMap);
+    const total = monitorValues.length;
+    const up = monitorValues.filter(m => m.status === 'UP').length;
+    const down = monitorValues.filter(m => m.status === 'DOWN').length;
+    const unknown = monitorValues.filter(m => m.status === 'UNKNOWN').length;
+
+    let summaryMd = `### Uptime Status Summary (Checked at ${new Date().toLocaleString()})\n\n`;
+    
+    if (down > 0) {
+      summaryMd += `⚠️ **Status: Degraded (${down}/${total} services DOWN)**\n\n`;
+    } else if (up === total) {
+      summaryMd += `✅ **Status: Healthy (All ${total} services UP)**\n\n`;
+    } else {
+      summaryMd += `ℹ️ **Status: Mixed (UP: ${up}, DOWN: ${down}, UNKNOWN: ${unknown})**\n\n`;
+    }
+
+    if (down > 0) {
+      summaryMd += `#### 🚨 DOWN Services:\n`;
+      monitorValues.filter(m => m.status === 'DOWN').forEach(m => {
+        summaryMd += `- **${m.name}** (${m.type}) - ${m.msg || 'No message'} (Last checked: ${m.lastCheck})\n`;
+      });
+      summaryMd += `\n`;
+    }
+
+    summaryMd += `#### 📋 Service Statuses:\n`;
+    summaryMd += `| Service | Status | Latency (ms) | Last Check |\n`;
+    summaryMd += `| :--- | :---: | :---: | :--- |\n`;
+    
+    // Sort services by name
+    monitorValues.sort((a, b) => a.name.localeCompare(b.name));
+    
+    for (const m of monitorValues) {
+      const statusIcon = m.status === 'UP' ? '🟢 UP' : m.status === 'DOWN' ? '🔴 DOWN' : '⚪ UNKNOWN';
+      const pingText = m.ping !== null ? `${m.ping} ms` : 'N/A';
+      summaryMd += `| ${m.name} | ${statusIcon} | ${pingText} | ${m.lastCheck || 'N/A'} |\n`;
+    }
+
+    // Output to stdout
+    console.log(summaryMd);
+
+    // 1. Write the latest report to Uptime Status Report.md
+    const reportPath = 'c:/Users/MattC/Documents/Obsidian Vault/Homelab/Uptime Status Report.md';
+    const reportContent = `---
+project: Homelab
+type: status
+status: current
+tags: [monitoring, uptime]
+updated: ${new Date().toISOString().split('T')[0]}
+---
+
+# Live Uptime Status Report
+
+${summaryMd}
+`;
+    fs.writeFileSync(reportPath, reportContent, 'utf8');
+    console.log(`Wrote status report to ${reportPath}`);
+
+    // 2. Append to today's session log
+    const todayStr = new Date().toISOString().split('T')[0];
+    const sessionLogDir = 'c:/Users/MattC/Documents/Obsidian Vault/Homelab/Sessions';
+    if (!fs.existsSync(sessionLogDir)) {
+      fs.mkdirSync(sessionLogDir, { recursive: true });
+    }
+    const sessionLogPath = path.join(sessionLogDir, `${todayStr}-uptime-status-monitoring-schedule.md`);
+
+    let sessionContent = '';
+    if (!fs.existsSync(sessionLogPath)) {
+      sessionContent = `# Session Log — ${todayStr} — Uptime Status Monitoring Schedule
+
+## What was done
+- Automatically checked uptime status page and updated reports.
+
+## Daily Log of Checks
+`;
+    } else {
+      sessionContent = fs.readFileSync(sessionLogPath, 'utf8');
+    }
+
+    const timeStr = new Date().toLocaleTimeString();
+    const statusText = down > 0 ? `🚨 DEGRADED (${down}/${total} services DOWN)` : `✅ Healthy (${total}/${total} services UP)`;
+    const logEntry = `\n### Check at ${timeStr}\n- **Status**: ${statusText}\n`;
+
+    fs.writeFileSync(sessionLogPath, sessionContent + logEntry, 'utf8');
+    console.log(`Appended check entry to ${sessionLogPath}`);
+
+  } catch (error) {
+    console.error(`Error checking status: ${error.message}`);
+    process.exit(1);
+  }
+}
+
+checkStatus();

stacks/authentik/docker-compose.yml

@@ -8,7 +8,7 @@
 services:
   authentik-server:
     container_name: authentik_server
-    image: ghcr.io/goauthentik/server:2026.5.2
+    image: ghcr.io/goauthentik/server:2026.5.3
     restart: unless-stopped
     command: server
     networks: [authentik, edge]
@@ -23,6 +23,11 @@ services:
       - ./blueprints:/blueprints/custom:ro
     ports:
       - "9200:9000"
+    logging:
+      driver: json-file
+      options:
+        max-size: "10m"
+        max-file: "3"
     depends_on:
       authentik-db:
         condition: service_healthy
@@ -31,7 +36,7 @@ services:
 
   authentik-worker:
     container_name: authentik_worker
-    image: ghcr.io/goauthentik/server:2026.5.2
+    image: ghcr.io/goauthentik/server:2026.5.3
     restart: unless-stopped
     command: worker
     networks: [authentik]

stacks/camofox/docker-compose.yml

@@ -0,0 +1,41 @@
+# camofox stack — self-hosted stealth headless browser (Camoufox/Firefox) REST API for the
+# Hermes agent. Chosen over the native Playwright/Browserbase plugins for self-hosted,
+# no-cloud-key, anti-detection browsing on valhalla.
+#
+# No published image: built on-host as camofox-browser:135.0.1-x86_64 via `make build` in the
+# upstream clone (/storage1/hermes/workspace/clones/camofox-browser). pull_policy: never so
+# Portainer's git redeploy uses the local image instead of trying to pull a non-existent
+# registry tag. To make it survive a host rebuild, push to registry.ginnoir.com later (famapp
+# pattern) and swap the image ref.
+#
+# Internal-only: published on the edge host-IP (172.20.0.1:9377) so host-side Hermes can reach
+# it directly, and joined to `edge` so Caddy reaches it by container name. Bearer-auth enforced
+# by camofox itself (CAMOFOX_ACCESS_KEY). Image pinned out of Watchtower.
+services:
+  camofox:
+    image: camofox-browser:135.0.1-x86_64
+    pull_policy: never
+    container_name: camofox
+    restart: unless-stopped
+    labels:
+      - "com.centurylabs.watchtower.enable=false"
+    env_file:
+      - stack.env
+    networks: [edge]
+    # Firefox-based browsers can crash with the default 64MB /dev/shm.
+    shm_size: "1gb"
+    volumes:
+      - /config/camofox/cookies:/data/cookies
+      - /config/camofox/profiles:/data/profiles
+    ports:
+      - "172.20.0.1:9377:9377"
+    healthcheck:
+      test: ["CMD", "curl", "-fsS", "http://localhost:9377/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 5
+      start_period: 40s
+
+networks:
+  edge:
+    external: true

stacks/camofox/stack.env

@@ -0,0 +1,13 @@
+# camofox stack env — committed intentionally (homelab convention; secrets versioned in the
+# private repo). LF line endings (enforced by .gitattributes).
+#
+# CAMOFOX_ACCESS_KEY  — bearer required on all routes except /health, cookie-import, /stop.
+# CAMOFOX_ADMIN_KEY   — required for POST /stop.
+# CAMOFOX_API_KEY     — left UNSET on purpose: keeps the cookie-import endpoint disabled.
+# CAMOFOX_COOKIES_DIR / CAMOFOX_PROFILE_DIR — explicit container paths (avoid home-dir ambiguity);
+#   mapped to /config/camofox/* binds on the host.
+CAMOFOX_ACCESS_KEY=cc9dac55139c97431160468bcfa43bbe0ab862710990f762d04e5881da9574b3
+CAMOFOX_ADMIN_KEY=117606acac488bf9661dab6ed276bc2cb465799eaf5676f2f7576b57e44b0ef4
+CAMOFOX_PORT=9377
+CAMOFOX_COOKIES_DIR=/data/cookies
+CAMOFOX_PROFILE_DIR=/data/profiles

stacks/dev/docker-compose.yml

@@ -57,6 +57,9 @@ services:
       - TZ
       - APP_URL=https://docs.ginnoir.com
       - APP_KEY
+      # stack.env leaks Plane's DATABASE_URL; Laravel prioritizes it over DB_* and
+      # would connect BookStack to postgres_plane. Pin it to BookStack's own MariaDB.
+      - DATABASE_URL=mysql://bookstack:bookstack_d3v_pass@mariadb_bookstack:3306/bookstack
       - DB_HOST=mariadb_bookstack
       - DB_PORT=3306
       - DB_USER=bookstack

stacks/famapp/stack.env

@@ -36,3 +36,11 @@ MINIO_BUCKET=garden
 
 OPENPLANTBOOK_CLIENT_ID=5JdQLab69RhVaF4er7b7D29BCrkf3fo3OEiDsOQa
 OPENPLANTBOOK_CLIENT_SECRET=kxPTrdQPAVta7sjVuIRf5EGIJ1fKvlMlqRSJpQtFxRftJhg7oeGnXzwH7ls6tluQLFUYVDYRMwISiAuKZGZLvkuJzUwFGCwhZw7UtW8OvcTGy4717WeNUinvuCDSsNXP
+
+# LLM assistant (OpenAI-compatible — same as local dev for now)
+LLM_PROVIDER=openai
+LLM_BASE_URL=http://192.168.1.121:8080/v1
+LLM_API_KEY=famapp-unc-f04f7ef0baf68d542d10a501
+LLM_MODEL=auto
+# IANA timezone for assistant relative dates ("Thursday at 2")
+HOUSEHOLD_TIMEZONE=America/Chicago

stacks/llm/docker-compose.yml

@@ -0,0 +1,45 @@
+# llm stack — model-swapping LLM backend for the Hermes agent.
+#
+# llama-swap fronts multiple GGUF models on the single Tesla P100 (16GB). Only one
+# model fits in VRAM at a time, so llama-swap presents all of them via /v1/models
+# and hot-swaps on demand (selecting a different model = a few-second reload). The
+# per-model llama-server commands + args live in llama-swap-config.yaml.
+#
+# The bundled llama.cpp in llama-swap:cuda is build 9803 (5c7c22c3e) — the same
+# build validated on this Pascal card for gemma4 + gpt-oss. Default model and the
+# selectable menu are driven from Hermes (~/.hermes/config.yaml: model.default =
+# gpt-oss-20b; provider valhalla-p100 models: list = the keys in the swap config).
+#
+# Endpoint published on 172.20.0.1:8090 (edge bridge gateway, a host IP) for the
+# host-side Hermes agent. Internal-only; no Caddy, no auth (LAN/host-only).
+# Image is infra-pinned out of Watchtower.
+services:
+  llama-swap:
+    image: ghcr.io/mostlygeek/llama-swap:cuda
+    container_name: llama-swap
+    restart: unless-stopped
+    labels:
+      - "com.centurylabs.watchtower.enable=false"
+    networks: [llm]
+    devices:
+      - "nvidia.com/gpu=0"
+    volumes:
+      - /storage1/labdata/llm/models:/models
+      # Live config is host-managed at /config/llm (Portainer git checkout won't
+      # materialize a relative repo-file bind — it auto-creates a dir). The repo
+      # copy stacks/llm/llama-swap-config.yaml is canonical; on change, copy it to
+      # /config/llm/llama-swap-config.yaml on valhalla (see homelab-apply).
+      - /config/llm/llama-swap-config.yaml:/app/config.yaml:ro
+    ports:
+      - "172.20.0.1:8090:8080"
+    healthcheck:
+      test: ["CMD", "curl", "-fsS", "http://localhost:8080/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 5
+      start_period: 30s
+
+networks:
+  llm:
+    name: llm
+    driver: bridge

stacks/llm/llama-swap-config.yaml

@@ -0,0 +1,58 @@
+# llama-swap model menu for the Hermes backend (single P100, 16GB).
+# llama-swap presents every model below via /v1/models and hot-swaps on demand —
+# only one fits in VRAM at a time, so selecting a different model triggers a
+# brief reload. Default is chosen by Hermes (model.default = gpt-oss-20b).
+#
+# All serve 64k context (Hermes' minimum). Args are the validated Pascal
+# config: q8_0 KV (q4_0 V-cache is pathological on GP100), flash-attn on,
+# --parallel 1 so one sequence gets the full 64k. gpt-oss/gemma4/ornith are
+# natively >=128k so no YaRN/override-kv needed.
+#
+# Excluded: qwen3-30b-a3b-2507 (Q3) — OOMs at 64k in 16GB, so it can't meet
+# Hermes' 64k minimum on this GPU.
+
+healthCheckTimeout: 300
+logLevel: info
+
+macros:
+  # --jinja applies each model's embedded chat template (REQUIRED for gpt-oss'
+  # harmony format, else content comes back empty; harmless/correct for gemma4).
+  common: "-ngl 99 --parallel 1 --ctx-size 65536 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --jinja"
+
+models:
+  "gpt-oss-20b":
+    # MoE ~3.6B active. Fastest prefill (~365 tok/s) -> ~45s cold start on Hermes' 16k prompt.
+    cmd: >
+      /app/llama-server --port ${PORT} --host 0.0.0.0
+      -m /models/gpt-oss-20b-mxfp4.gguf --alias gpt-oss-20b
+      ${common}
+
+  "ornith-1.0-9b":
+    # DeepReinforce Ornith-1.0, dense ~9B on Qwen 3.5 (Q5_K_M). MIT. Agentic-coding
+    # tuned: <think> block (-> reasoning_content under --jinja) + Qwen3 XML tool calls.
+    # Native 256k so no YaRN. Recommended sampling: temp 0.6 / top_p 0.95 / top_k 20.
+    cmd: >
+      /app/llama-server --port ${PORT} --host 0.0.0.0
+      -m /models/ornith-1.0-9b-Q5_K_M.gguf --alias ornith-1.0-9b
+      ${common}
+
+  "gemma-4-26b-a4b":
+    # gemma4 MoE, 4B active / 26B total (UD-Q3_K_M). Quality-leaning; ~147 tok/s prefill.
+    cmd: >
+      /app/llama-server --port ${PORT} --host 0.0.0.0
+      -m /models/gemma-4-26B-A4B-it-UD-Q3_K_M.gguf --alias gemma-4-26b-a4b
+      ${common}
+
+  "gemma-4-12b":
+    # Dense 12B (Q4_K_M). ~85 tok/s prefill.
+    cmd: >
+      /app/llama-server --port ${PORT} --host 0.0.0.0
+      -m /models/gemma-4-12b-it-Q4_K_M.gguf --alias gemma-4-12b
+      ${common}
+
+  "gemma-4-e4b":
+    # Small dense ~4B (Q4_K_M). Lots of VRAM headroom; ~173 tok/s prefill.
+    cmd: >
+      /app/llama-server --port ${PORT} --host 0.0.0.0
+      -m /models/gemma-4-E4B-it-Q4_K_M.gguf --alias gemma-4-e4b
+      ${common}

stacks/llm/stack.env

@@ -0,0 +1,4 @@
+# llm stack secrets — read directly by the container via env_file.
+# llama.cpp's server reads LLAMA_API_KEY from the environment (no --api-key flag,
+# no ${VAR} interpolation), matching the repo's pure-env_file convention.
+LLAMA_API_KEY=fc0c0baea5a9ca4eba94a38c8b7c65153594225afbc0b788eba0652fcfdf0069

stacks/media/docker-compose.yml

@@ -29,6 +29,8 @@ services:
     networks: [media, edge]
     env_file:
       - stack.env
+    ulimits:
+      nofile: { soft: 65536, hard: 65536 }
     volumes:
       - /etc/localtime:/etc/localtime:ro
       - /config/sonarr:/config
@@ -45,6 +47,8 @@ services:
     networks: [media, edge]
     env_file:
       - stack.env
+    ulimits:
+      nofile: { soft: 65536, hard: 65536 }
     volumes:
       - /etc/localtime:/etc/localtime:ro
       - /config/radarr:/config
@@ -60,6 +64,8 @@ services:
     networks: [media, edge]
     env_file:
       - stack.env
+    ulimits:
+      nofile: { soft: 65536, hard: 65536 }
     environment:
       - UMASK_SET=022
     volumes:
@@ -77,6 +83,8 @@ services:
     networks: [media, edge]
     env_file:
       - stack.env
+    ulimits:
+      nofile: { soft: 65536, hard: 65536 }
     environment:
       - UMASK=002
     volumes:
@@ -137,6 +145,8 @@ services:
     networks: [media, edge]
     env_file:
       - stack.env
+    ulimits:
+      nofile: { soft: 65536, hard: 65536 }
     environment:
       - UMASK=002
       - WEBUI_PORTS=6969/tcp

stacks/owncloud/docker-compose.yml

@@ -48,6 +48,11 @@ services:
     environment:
       - MYSQL_DATABASE=owncloud
     command: ["--max-allowed-packet=128M", "--innodb-log-file-size=64M"]
+    logging:
+      driver: json-file
+      options:
+        max-size: "10m"
+        max-file: "3"
     healthcheck:
       # $$ defers expansion to container runtime (env_file supplies the value)
       test: ["CMD-SHELL", "mariadb-admin ping -u root --password=\"$$MYSQL_ROOT_PASSWORD\""]

stacks/photos/docker-compose.yml

@@ -16,6 +16,11 @@ services:
       - /etc/localtime:/etc/localtime:ro
     ports:
       - "2283:2283"
+    logging:
+      driver: json-file
+      options:
+        max-size: "10m"
+        max-file: "3"
     depends_on:
       - immich-redis
       - immich-postgres

stacks/proxy/docker-compose.yml

@@ -19,6 +19,21 @@ services:
     restart: unless-stopped
     labels:
       - "com.centurylabs.watchtower.enable=false"
+    # Public resolvers for ACME DNS-01 zone detection.
+    #
+    # The LAN resolver (OPNsense Unbound, 192.168.1.1) is authoritative for
+    # ginnoir.com via the split-horizon override, but its local-zone only holds
+    # A records — so `SOA ginnoir.com` returns NODATA. certmagic's zone lookup
+    # walks up the label chain looking for an SOA, finds none at ginnoir.com,
+    # climbs to `com.`, and asks Cloudflare for a `com` zone. Result:
+    #   "adding temporary record for zone \"com.\": expected 1 zone, got 0"
+    # and every DNS-01 renewal fails until certs expire.
+    #
+    # Docker's embedded DNS (127.0.0.11) still resolves container names and the
+    # gitea.ginnoir.com alias below; only external lookups go to Cloudflare.
+    dns:
+      - 1.1.1.1
+      - 1.0.0.1
     networks:
       edge:
         aliases:

stacks/proxy/stack.env

@@ -2,3 +2,7 @@
 # Cloudflare API token for Caddy's DNS-01 TLS challenge.
 # Scoped token: Zone > DNS > Edit on the ginnoir.com zone only.
 CF_API_TOKEN=cfut_ijIT9HDjXWKgd0dtnyLqxv8L3deB7hSDivSqquwp905a6c51
+
+# Shared-link gate key — external users visit ?k=<value> once to get a 30-day cookie.
+# Rotate: change value here, push, Portainer redeploys proxy stack, post new URL in Discord.
+SHARE_KEY=8e5d9f1ca20a448b66ed171aea97d172

stacks/remote/docker-compose.yml

@@ -18,7 +18,7 @@ services:
   hbbs:
     container_name: hbbs
     image: rustdesk/rustdesk-server:latest
-    command: hbbs
+    command: hbbs -r 192.168.1.69
     restart: unless-stopped
     env_file:
       - stack.env

stacks/roms/docker-compose.yml

@@ -22,9 +22,9 @@
 
 services:
   romm:
-    # Floating v4 tag (watchtower-tracked). The save-sync (/api/sync/*) + raw
+    # Floating v5 tag (watchtower-tracked). Bumped from :4 (4.9.2) → :5 (5.0.0)
-    # content_hash fixes that once forced a 4.9.0-beta.2 pin are now in stable 4.9.x.
+    # 2026-07-21. Pre-upgrade dump: /config/backup/dumps/romm-pre-5.0.0-20260721.sql
-    image: rommapp/romm:4
+    image: rommapp/romm:5
     container_name: romm
     restart: unless-stopped
     networks: [roms, edge]

stacks/streaming/docker-compose.yml

@@ -0,0 +1,96 @@
+# streaming stack — Jellyfin and its satellites.
+#
+# Storage split is deliberate: SQLite config on the root SSD, all bulk data
+# (metadata, trickplay, cache, transcodes) on /storage1. The root disk is only
+# 194G and has filled twice; nothing here may grow unbounded on it.
+#
+# Jellyfin 10.11 (linuxserver) keeps metadata at /config/data/metadata — NOT
+# /config/metadata. Trickplay is at /config/data/trickplay and is not
+# relocatable from the UI. Transcodes are configured to /transcode.
+#
+# Transcodes are disk-backed, not tmpfs: jellyfin#16608 means Remux and
+# DirectStream jobs finish before the segment cleaner's first check, orphaning
+# segments. Plex already writes transcodes to virtiofs at a 48% transcode rate.
+#
+# Env convention: every service gets stack.env via env_file; var names there are
+# exactly what the containers read. No ${VAR} interpolation, no Portainer UI
+# env vars.
+
+services:
+  jellyfin:
+    container_name: jellyfin
+    image: lscr.io/linuxserver/jellyfin:latest
+    restart: unless-stopped
+    networks: [streaming, edge]
+    env_file:
+      - stack.env
+    devices:
+      - nvidia.com/gpu=0
+    environment:
+      - JELLYFIN_PublishedServerUrl=https://jellyfin.ginnoir.com
+    volumes:
+      - /etc/localtime:/etc/localtime:ro
+      - /config/jellyfin:/config
+      - /storage1/labdata/jellyfin/metadata:/config/data/metadata
+      - /storage1/labdata/jellyfin/cache:/config/cache
+      - /storage1/labdata/jellyfin/trickplay:/config/data/trickplay
+      - /storage1/labdata/jellyfin/transcodes:/transcode
+      - /storage1/Movies:/data/movies:ro
+      - /storage1/TV:/data/tv:ro
+      - /storage1/Anime:/data/anime:ro
+    ports:
+      - "8096:8096"
+
+  jellystat-db:
+    container_name: jellystat-db
+    image: postgres:16-alpine
+    restart: unless-stopped
+    networks: [streaming]
+    env_file:
+      - stack.env
+    labels:
+      - "com.centurylabs.watchtower.enable=false"
+    volumes:
+      - /config/jellystat-db:/var/lib/postgresql/data
+
+  jellystat:
+    container_name: jellystat
+    image: cyfershepard/jellystat:latest
+    restart: unless-stopped
+    networks: [streaming, edge]
+    depends_on:
+      - jellystat-db
+    env_file:
+      - stack.env
+    volumes:
+      - /config/jellystat:/app/backend/backup-data
+    ports:
+      - "3009:3000"
+
+  wizarr:
+    container_name: wizarr
+    image: ghcr.io/wizarrrr/wizarr:latest
+    restart: unless-stopped
+    networks: [streaming, edge]
+    env_file:
+      - stack.env
+    volumes:
+      - /config/wizarr:/data/database
+    ports:
+      - "5690:5690"
+
+  jellyplex-watched:
+    container_name: jellyplex-watched
+    image: luigi311/jellyplex-watched:latest
+    restart: unless-stopped
+    networks: [streaming, edge]
+    env_file:
+      - stack.env
+
+networks:
+  streaming:
+    name: streaming
+    driver: bridge
+  edge:
+    name: edge
+    external: true

stacks/streaming/stack.env

@@ -0,0 +1,34 @@
+# streaming stack config — injected into every service via env_file.
+# Var names are exactly what the containers read; no ${VAR} interpolation,
+# no Portainer UI env vars needed.
+
+# --- shared ---
+TZ=America/Chicago
+PUID=1000
+PGID=1000
+
+# --- jellystat-db (postgres:16-alpine) ---
+POSTGRES_DB=jfstat
+POSTGRES_USER=jellystat
+POSTGRES_PASSWORD=bf2d47d8e1a87baf2b4942704a5acfaf5c2aad336fdd480e
+
+# --- jellystat ---
+POSTGRES_IP=jellystat-db
+POSTGRES_PORT=5432
+JWT_SECRET=0dd9fcdb632277b76977dbada8226e547cc773e1deafc8d778d4038f8d59393b
+
+# --- jellyplex-watched ---
+# Plex is host-native. Plain HTTP on :32400 only answers on loopback; LAN HTTP
+# gets an empty reply. Temporary host socat proxy plex-http-proxy.service
+# exposes loopback Plex HTTP on :32402 for Docker clients (Plexyfin + jellyplex).
+# Tear down that unit after Plex is decommissioned / Plexyfin is done.
+PLEX_URL=http://192.168.1.69:32402
+PLEX_TOKEN=hM9aycswHKSsUrxiHqwq
+JELLYFIN_BASEURL=http://jellyfin:8096
+JELLYFIN_TOKEN=a3d9ee827d5c42298cae087a2c31d3fa
+SYNC_FROM_PLEX_TO_JELLYFIN=True
+SYNC_FROM_JELLYFIN_TO_PLEX=True
+# Start in dry-run. Task 16 flips this to False after the mapping is verified.
+DRYRUN=False
+SLEEP_DURATION=21600
+LOG_LEVEL=INFO