fix(streaming): reach Plex HTTP via host :32402 proxy

Dry-run connected to both servers with matching ginnoir usernames.
Library counts already at/above Plex baseline while the rescan finishes.

.agents/skills/homelab-ssh/SKILL.md

@@ -5,7 +5,7 @@ description: Connect to and run commands on the valhalla homelab server (ginnoir
 
 # homelab-ssh
 
-The production homelab runs on a headless Ubuntu host reachable at `ssh ginnoir@valhalla` (key-based auth; resolves and works from this Windows host directly).
+The production homelab runs on a headless **EndeavourOS (Arch)** host reachable at `ssh ginnoir@valhalla` (key-based auth; resolves and works from this Windows host directly). Package manager is `pacman` — there is no `apt`/`apt-get`. `ginnoir` has passwordless `sudo`.
 
 ## Connect (non-interactive / safe for automation)
 

.claude/skills/homelab-ssh/SKILL.md

@@ -5,7 +5,7 @@ description: Connect to and run commands on the valhalla homelab server (ginnoir
 
 # homelab-ssh
 
-The production homelab runs on a headless Ubuntu host reachable at `ssh ginnoir@valhalla` (key-based auth; resolves and works from this Windows host directly).
+The production homelab runs on a headless **EndeavourOS (Arch)** host reachable at `ssh ginnoir@valhalla` (key-based auth; resolves and works from this Windows host directly). Package manager is `pacman` — there is no `apt`/`apt-get`. `ginnoir` has passwordless `sudo`.
 
 ## Connect (non-interactive / safe for automation)
 

.env

@@ -111,7 +111,8 @@ GITHUB_RUNNER_ACCESS_TOKEN=github_pat_11ACRHQAI0q3n1svrl9jmT_ai2bQlXWSxY8HJjEI6q
 # Used by StackCreateDockerStandaloneRepository and StackGitRedeploy.
 # Pass as RepositoryAuthorizationType:0 (Basic), RepositoryUsername=ginnoir.
 # Runner PAT above does NOT work for this — different scopes.
-GITHUB_STACKS_PAT=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
+# Gitea PAT for Portainer git stacks (env var name kept from GitHub era)
+GITHUB_STACKS_PAT=54ea93904439c2919f1d8dab1ba545e54a2a3e86
 
 # =============================================================
 # DEV STACK

.gitea/workflows/deploy-caddy.yml

@@ -17,6 +17,15 @@ jobs:
 
       - name: Push Caddyfile and reload Caddy
         run: |
+          # Job containers do NOT inherit the runner's /config/caddy bind.
+          # Docker-from-Docker with a host bind is the reliable path: the
+          # volume source is resolved on valhalla, not inside the job container.
+          # (Direct `cp /config/caddy/...` only works on the runner container
+          # itself, which is not where this step runs.)
           apt-get update -qq && apt-get install -y -qq docker.io
-          cp Caddyfile /config/caddy/Caddyfile
+          docker run --rm \
+            -v /config/caddy:/dest \
+            -v "$PWD/Caddyfile:/src/Caddyfile:ro" \
+            alpine:3.20 \
+            cp /src/Caddyfile /dest/Caddyfile
           docker exec caddy caddy reload --config /etc/caddy/Caddyfile

AGENTS.md

@@ -9,9 +9,9 @@ from your tool's user-level instructions (`~/.codex/AGENTS.md` / `~/.gemini/GEMI
 
 ## Essentials (full detail in `CLAUDE.md`)
 
-- **What this is:** deployment config for a live, single-host Docker homelab on a headless Ubuntu
-  server reachable at `ssh ginnoir@valhalla`. ~50 containers in per-domain Portainer-managed stacks.
-  This is **not** application code.
+- **What this is:** deployment config for a live, single-host Docker homelab on a headless
+  **EndeavourOS (Arch)** server reachable at `ssh ginnoir@valhalla` — `pacman`, not `apt`.
+  ~50 containers in per-domain Portainer-managed stacks. This is **not** application code.
 - **The repo is canonical.** Portainer polls `main` every 5 min and redeploys any app stack whose
   `stacks/<domain>/*` files changed. Editing here changes nothing until you `git push`.
 - **Deployment channels:**

CLAUDE.md

@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ## What this repo is
 
-Deployment configuration for a **live, single-host Docker homelab** running on a headless Ubuntu server reachable at `ssh ginnoir@valhalla`. This repo is **not application code** — it is the canonical source for ~50 containers organized into per-domain Portainer-managed stacks.
+Deployment configuration for a **live, single-host Docker homelab** running on a headless **EndeavourOS (Arch)** server reachable at `ssh ginnoir@valhalla` — package manager is `pacman`, there is no `apt`. This repo is **not application code** — it is the canonical source for ~50 containers organized into per-domain Portainer-managed stacks.
 
 **The repo is canonical.** Portainer polls the **Gitea** repo `https://gitea.ginnoir.com/ginnoir/homelabstack` on `main` every 5 min and redeploys any application stack whose source files have changed. Editing here changes nothing until you push (or, for the management plane, run `apply-compose.ps1 -Portainer`). GitHub is a temporary push mirror only.
 
@@ -173,12 +173,15 @@ These run on valhalla but are managed outside of Portainer/Docker.
 
 ### Plex Media Server
 
-- **Install:** deb package (`plexmediaserver`), managed by systemd. Not containerised.
-- **Version:** 1.43.2 (Watchtower doesn't touch it — update via `apt upgrade plexmediaserver` or the Plex UI).
+- **Install:** AUR/pacman package (`plex-media-server`), managed by systemd. Not containerised.
+- **Version:** 1.43.2 (Watchtower doesn't touch it — update via `pacman -Syu plex-media-server` or the Plex UI).
 - **Service:** `systemctl {start,stop,restart,status} plexmediaserver`
 - **Binary:** `/usr/lib/plexmediaserver/`
-- **App data / metadata / DB:** `/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/`
-- **Media libraries:** `/storage1/Media/{Movies,TV,Anime}` — same ZFS pool bind-mounted into the *arr containers.
+- **App data / metadata / DB:** `/var/lib/plex/Plex Media Server/` — note `/var/lib/plex` is a
+  **symlink to `/storage1/labdata/plex`**, so Plex's 383G of app data lives on the ZFS array, not
+  the root disk. (`/var/lib/plexmediaserver/` does not exist.)
+- **Media libraries:** `/storage1/{Movies,TV,Anime}` — `/storage1/Media/*` are symlinks to these.
+  Same ZFS pool bind-mounted into the *arr containers.
 - **Tautulli** (`tautulli` container in the media stack) reads PMS logs from `/storage1/Media` and connects to Plex over the LAN. It is **not** behind Authentik — Plex login is its own auth.
 - **Caddy:** Plex is not reverse-proxied through Caddy (Plex handles its own relay/direct connections). No `plex.ginnoir.com` block exists.
 

Caddyfile

@@ -17,6 +17,18 @@
 	respond @blocked "Access denied" 403
 }
 
+# Same IP gate as (internal_only), but for use INSIDE a route{} block.
+# Caddy sorts the `route` directive ahead of `respond`, so an `import
+# internal_only` sitting above a catch-all `route {}` is dead code — the route
+# matches everything and handles the request before the 403 is ever reached.
+# Inside a route, directives run in written order, so importing this as the
+# first line of the route makes the gate fire. Use this, not internal_only,
+# on any site that wraps its handlers in route{} (i.e. the Authentik sites).
+(internal_gate) {
+	@blocked_ip not remote_ip 192.168.1.0/24 172.16.0.0/12 100.64.0.0/10 127.0.0.1
+	respond @blocked_ip "Access denied" 403
+}
+
 # TB-006 — Authentik forward auth (embedded outpost on authentik-server:9000).
 # Wrap protected sites in `route { ... }` so bypass handles run before forward_auth.
 (authentik_outpost) {
@@ -80,16 +92,24 @@ foundry2.ginnoir.com {
 	reverse_proxy 5etools:80
 }
 
-pf2e.ginnoir.com {
-	root * /srv/aon
-	file_server
-}
-
 # =============================================================
 # FILE STORAGE — internal only (Nextcloud)
 # =============================================================
 files.ginnoir.com {
-	import internal_only
+	# Public share links must resolve for external users; the rest of Nextcloud
+	# stays LAN/tailnet-only. Can't use `import internal_only` here — this needs
+	# the same IP gate with a path-based exemption, so the matcher is inlined.
+	# Blocked = external client AND not a public-share path. The /core, /dist,
+	# /css, /js and theming paths are the assets the share page itself loads;
+	# without them an external visitor gets an unstyled, non-functional page.
+	@blocked {
+		not remote_ip 192.168.1.0/24 172.16.0.0/12 100.64.0.0/10 127.0.0.1
+		not path /s/* /index.php/s/* /public.php /public.php/*
+		not path /apps/files_sharing/* /index.php/apps/files_sharing/* /ocs/v2.php/apps/files_sharing/*
+		not path /core/* /dist/* /css/* /js/* /themes/* /apps/theming/* /index.php/apps/theming/*
+	}
+	respond @blocked "Access denied" 403
+
 	redir /.well-known/carddav /remote.php/dav 301
 	redir /.well-known/caldav /remote.php/dav 301
 	reverse_proxy nextcloud:80 {
@@ -101,11 +121,6 @@ files.ginnoir.com {
 # =============================================================
 # STATIC SITES — public
 # =============================================================
-ffttsystems.ginnoir.com {
-	root * /srv/ffttsystems
-	file_server
-}
-
 wa4.ginnoir.com {
 	root * /srv/wa4
 	file_server
@@ -118,13 +133,24 @@ requests.ginnoir.com {
 	reverse_proxy seerr:5055
 }
 
+# =============================================================
+# STREAMING — public (replaces Plex's own remote access)
+# =============================================================
+jellyfin.ginnoir.com {
+	reverse_proxy jellyfin:8096
+}
+
+wizarr.ginnoir.com {
+	reverse_proxy wizarr:5690
+}
+
 # =============================================================
 # MEDIA MANAGEMENT — internal only
 # =============================================================
 
 sonarr.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle /api/* {
 			reverse_proxy sonarr:8989
@@ -137,8 +163,8 @@ sonarr.ginnoir.com {
 }
 
 radarr.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle /api/* {
 			reverse_proxy radarr:7878
@@ -151,8 +177,8 @@ radarr.ginnoir.com {
 }
 
 bazarr.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle /api/* {
 			reverse_proxy bazarr:6767
@@ -165,8 +191,8 @@ bazarr.ginnoir.com {
 }
 
 prowlarr.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle /api/* {
 			reverse_proxy prowlarr:9696
@@ -179,8 +205,8 @@ prowlarr.ginnoir.com {
 }
 
 tautulli.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle {
 			import authentik_forward_auth
@@ -189,12 +215,23 @@ tautulli.ginnoir.com {
 	}
 }
 
+jellystat.ginnoir.com {
+	route {
+		import internal_gate
+		import authentik_outpost
+		handle {
+			import authentik_forward_auth
+			reverse_proxy jellystat:3000
+		}
+	}
+}
+
 # =============================================================
 # DOWNLOAD CLIENTS — internal only
 # =============================================================
 qbittorrent.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle {
 			import authentik_forward_auth
@@ -204,8 +241,8 @@ qbittorrent.ginnoir.com {
 }
 
 nzbget.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle {
 			import authentik_forward_auth
@@ -215,8 +252,8 @@ nzbget.ginnoir.com {
 }
 
 whisparr.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle /api/* {
 			reverse_proxy whisparr:6969
@@ -229,8 +266,8 @@ whisparr.ginnoir.com {
 }
 
 stash.ginnoir.com {
-	import internal_only
 	route {
+		import internal_gate
 		import authentik_outpost
 		handle {
 			import authentik_forward_auth
@@ -409,6 +446,24 @@ webui.ginnoir.com {
 	reverse_proxy 172.20.0.1:8787
 }
 
+# ComfyUI node editor — inference runs on the Mac at 192.168.1.121 (Metal/MPS).
+# LAN/tailnet only; no Authentik (WebSocket queue/progress breaks under forward_auth).
+#
+# The explicit `resolvers` below is what made DNS-01 work here, not the imgstudio
+# hostname (an earlier comment blamed a "false .com" in comfyui.ginnoir.com —
+# that was wrong). The LAN resolver returns NODATA for `SOA ginnoir.com`, so
+# certmagic's zone lookup climbs to `com.` and Cloudflare rejects it. The caddy
+# container now pins public resolvers stack-wide (stacks/proxy/docker-compose.yml),
+# making this block redundant; kept as belt-and-braces.
+imgstudio.ginnoir.com {
+	import internal_only
+	tls {
+		dns cloudflare {env.CF_API_TOKEN}
+		resolvers 1.1.1.1
+	}
+	reverse_proxy 192.168.1.121:8188
+}
+
 backup.ginnoir.com {
 	route {
 		import authentik_outpost

apply-compose.ps1

@@ -41,8 +41,11 @@ if ($EnvFile) {
 }
 
 if ($Caddy) {
+    # /config/caddy is root-owned on valhalla — scp directly fails with
+    # "Permission denied". Stage via /tmp and sudo-install.
     Write-Host "Pushing Caddyfile ..."
-    scp $caddyLocal "${server}:/config/caddy/Caddyfile"
+    scp $caddyLocal "${server}:/tmp/Caddyfile.new"
+    ssh $server "sudo cp /tmp/Caddyfile.new /config/caddy/Caddyfile && sudo chown root:root /config/caddy/Caddyfile && rm -f /tmp/Caddyfile.new"
     Write-Host "Reloading Caddy ..."
     ssh $server "docker exec caddy caddy reload --config /etc/caddy/Caddyfile"
 }

bookmarks-domains.html

@@ -15,17 +15,16 @@
         <DT><H3>TABLETOP TOOLS (public)</H3>
         <DL><p>
             <DT><A HREF="https://5etools.ginnoir.com">5etools</A>
-            <DT><A HREF="https://pf2e.ginnoir.com">pf2e</A>
         </DL><p>
 
-        <DT><H3>FILE STORAGE — public (Authentik on admin UI; shares + WebDAV bypass) (public)</H3>
+        <DT><H3>FILE STORAGE — internal only (Nextcloud) (internal)</H3>
         <DL><p>
             <DT><A HREF="https://files.ginnoir.com">files</A>
         </DL><p>
 
         <DT><H3>STATIC SITES (public)</H3>
         <DL><p>
-            <DT><A HREF="https://ffttsystems.ginnoir.com">ffttsystems</A>
+            <DT><A HREF="https://wa4.ginnoir.com">wa4</A>
         </DL><p>
 
         <DT><H3>MEDIA REQUESTS (public)</H3>
@@ -33,26 +32,33 @@
             <DT><A HREF="https://requests.ginnoir.com">requests</A>
         </DL><p>
 
-        <DT><H3>MEDIA MANAGEMENT (internal)</H3>
+        <DT><H3>STREAMING — public (replaces Plex's own remote access) (public)</H3>
+        <DL><p>
+            <DT><A HREF="https://jellyfin.ginnoir.com">jellyfin</A>
+            <DT><A HREF="https://wizarr.ginnoir.com">wizarr</A>
+        </DL><p>
+
+        <DT><H3>MEDIA MANAGEMENT (public)</H3>
         <DL><p>
             <DT><A HREF="https://sonarr.ginnoir.com">sonarr</A>
             <DT><A HREF="https://radarr.ginnoir.com">radarr</A>
             <DT><A HREF="https://bazarr.ginnoir.com">bazarr</A>
             <DT><A HREF="https://prowlarr.ginnoir.com">prowlarr</A>
             <DT><A HREF="https://tautulli.ginnoir.com">tautulli</A>
+            <DT><A HREF="https://jellystat.ginnoir.com">jellystat</A>
         </DL><p>
 
-        <DT><H3>DOWNLOAD CLIENTS (internal)</H3>
+        <DT><H3>DOWNLOAD CLIENTS</H3>
         <DL><p>
             <DT><A HREF="https://qbittorrent.ginnoir.com">qbittorrent</A>
-            <DT><A HREF="https://deluge.ginnoir.com">deluge</A>
             <DT><A HREF="https://nzbget.ginnoir.com">nzbget</A>
             <DT><A HREF="https://whisparr.ginnoir.com">whisparr</A>
             <DT><A HREF="https://stash.ginnoir.com">stash</A>
             <DT><A HREF="https://roms.ginnoir.com">roms</A>
             <DT><A HREF="https://romhacks.ginnoir.com">romhacks</A>
-            <DT><A HREF="https://jd.ginnoir.com">jd</A>
-            <DT><A HREF="https://romhacks-files.ginnoir.com">romhacks-files</A>
+            <DT><A HREF="https://jd.ginnoir.com">jd [internal]</A>
+            <DT><A HREF="https://idlegame.ginnoir.com">idlegame [internal]</A>
+            <DT><A HREF="https://romhacks-files.ginnoir.com">romhacks-files [internal]</A>
         </DL><p>
 
         <DT><H3>NOTIFICATIONS &amp; RSS — public (ntfy native auth; FreshRSS HTTP auth via Authentik) (public)</H3>
@@ -85,6 +91,9 @@
             <DT><A HREF="https://portainer.ginnoir.com">portainer</A>
             <DT><A HREF="https://uptime.ginnoir.com">uptime</A>
             <DT><A HREF="https://homarr.ginnoir.com">homarr [internal]</A>
+            <DT><A HREF="https://hermes.ginnoir.com">hermes</A>
+            <DT><A HREF="https://webui.ginnoir.com">webui [internal]</A>
+            <DT><A HREF="https://imgstudio.ginnoir.com">imgstudio [internal]</A>
             <DT><A HREF="https://backup.ginnoir.com">backup</A>
             <DT><A HREF="https://router.ginnoir.com">router [internal]</A>
         </DL><p>
@@ -99,6 +108,13 @@
             <DT><A HREF="https://vault.ginnoir.com">vault</A>
             <DT><A HREF="https://docs.ginnoir.com">docs [internal]</A>
             <DT><A HREF="https://plane.ginnoir.com">plane</A>
+            <DT><A HREF="https://nextcloud.ginnoir.com">nextcloud [internal]</A>
+            <DT><A HREF="https://office.ginnoir.com">office [internal]</A>
+        </DL><p>
+
+        <DT><H3>PHOTOS (internal)</H3>
+        <DL><p>
+            <DT><A HREF="https://photos.ginnoir.com">photos</A>
         </DL><p>
 
         <DT><H3>NOTES / PKM — internal only (LiveSync over LAN or tailnet) (internal)</H3>

bookmarks-ports.html

@@ -18,10 +18,7 @@
 
         <DT><H3>Dev</H3>
         <DL><p>
-            <DT><A HREF="ssh://git@valhalla:2222">gitea :2222</A>
-            <DT><A HREF="http://valhalla:3030">gitea :3030</A>
             <DT><A HREF="http://valhalla:8443">code_server</A>
-            <DT><A HREF="http://valhalla:5000">registry</A>
             <DT><A HREF="http://valhalla:4224">dbx</A>
             <DT><A HREF="http://valhalla:6875">bookstack :6875</A>
         </DL><p>
@@ -42,13 +39,12 @@
 
         <DT><H3>Media</H3>
         <DL><p>
-            <DT><A HREF="http://valhalla:8112">deluge</A>
             <DT><A HREF="http://valhalla:6789">nzbget</A>
             <DT><A HREF="http://valhalla:8989">sonarr</A>
             <DT><A HREF="http://valhalla:7878">radarr</A>
             <DT><A HREF="http://valhalla:6767">bazarr</A>
             <DT><A HREF="http://valhalla:9696">prowlarr</A>
-            <DT><A HREF="http://valhalla:5055">overseerr</A>
+            <DT><A HREF="http://valhalla:5055">seerr</A>
             <DT><A HREF="http://valhalla:8181">tautulli</A>
             <DT><A HREF="http://valhalla:3232">qbittorrent :3232</A>
             <DT><A HREF="http://valhalla:6881">qbittorrent :6881 (TCP/UDP)</A>
@@ -78,6 +74,11 @@
             <DT><A HREF="http://valhalla:8080">owncloud_server</A>
         </DL><p>
 
+        <DT><H3>Photos</H3>
+        <DL><p>
+            <DT><A HREF="http://valhalla:2283">immich-server</A>
+        </DL><p>
+
         <DT><H3>Proxy</H3>
         <DL><p>
             <DT><A HREF="http://valhalla:80">caddy :80</A>
@@ -108,5 +109,12 @@
             <DT><A HREF="http://valhalla:8997">romm :8997</A>
         </DL><p>
 
+        <DT><H3>Streaming</H3>
+        <DL><p>
+            <DT><A HREF="http://valhalla:8096">jellyfin</A>
+            <DT><A HREF="http://valhalla:3009">jellystat :3009</A>
+            <DT><A HREF="http://valhalla:5690">wizarr</A>
+        </DL><p>
+
     </DL><p>
 </DL><p>

docs/2026-06-28-gpt-oss-20b-ollama-benchmark.md

@@ -0,0 +1,84 @@
+# gpt-oss-20b — `.73` Ollama box vs valhalla P100 (2026-06-28)
+
+**TL;DR:** The same model (`gpt-oss-20b`) runs **~1.3–1.5× faster on generation** and
+**~1.6–2× faster on prefill** on ginnoir's `192.168.1.73` Ollama box than on valhalla's
+Tesla P100 llama-swap backend — *despite* the `.73` box partially offloading to CPU at 64K
+context. Tool-calls and code outputs are correct on both. For interactive Hermes use the
+`.73` box is the better backend whenever it's powered on; the P100 remains the always-on
+fallback. **Caveat:** at very deep context (~43K tokens) `.73` generation drops to ~8 tok/s
+(KV no longer fully GPU-resident).
+
+## Setup
+
+- **Endpoint:** `http://192.168.1.73:11434` — Ollama **0.30.11**, model `gpt-oss:20b`,
+  quant **MXFP4** (OpenAI's native gpt-oss 4-bit format), reported 20.9B params.
+- **Serving config:** `num_ctx 65536` (matches the P100's 64K and Hermes' hard ≥64K
+  requirement), sampling `temperature 0.6 / top_p 0.95 / top_k 20` (identical to the
+  2026-06-27 Ornith/P100 run).
+- **Measured via** Ollama's native `/api/chat` (exact `prompt_eval`/`eval` token counts +
+  durations). Cold model load at 64K ctx took **~65 s** (one-time).
+- **P100 baseline** = the gpt-oss-20b column from `docs/2026-06-27-ornith-9b-benchmark.md`
+  (llama-swap, q8/q8 KV, `--parallel 1`, `--jinja`, 64K).
+- GPU on `.73` **could not be identified** — no SSH (port 22 filtered). Inferred from VRAM
+  behavior (below): a fast but VRAM-limited card (or unified-memory host).
+
+## Results
+
+| Test | Metric | **.73 Ollama (MXFP4)** | **valhalla P100 (GGUF)** | `.73` advantage |
+|---|---|---|---|---|
+| Tool call | valid `tool_calls`? | ✅ `get_weather({"city":"Tokyo"})` | ✅ identical | tie (both correct) |
+| Codegen (`merge_intervals`) | correct? | ✅ correct, non-mutating, type-hinted | ✅ correct (mutates input) | `.73` slightly cleaner |
+| Debug (`second_largest`) | correct? | ✅ correct single-pass, handles dup max | ✅ correct | tie |
+| Deep-prefill | answered? | ✅ "quick brown fox" | ✅ (capped probe) | tie |
+| **Gen speed** | tok/s — tool_call | **33.1** | 23.0 | **1.44×** |
+| **Gen speed** | tok/s — codegen | **27.4** | 21.7 | **1.26×** |
+| **Gen speed** | tok/s — debug | **27.6** | 18.9 | **1.46×** |
+| **Prefill** | tok/s — shallow (~100–135 tok) | **240–287** | 133–188 | ~1.5–1.8× |
+| **Prefill** | tok/s — deep | **2542** (n=43.8k) | 155 (n=23.4k) | far faster (diff depths) |
+| Gen @ deep ctx | tok/s | 8.4 (n=43.8k ctx) | 12.7 (n=23.4k ctx) | **P100 wins at depth** |
+| Reasoning verbosity | think chars (codegen/debug) | 1760 / 1317 | 1671 / 2118 | comparable |
+| VRAM @ 64K | on-GPU / total | **8.82 / 14.16 GB** (partial CPU offload) | ~12.6 GB (100% GPU) | — |
+
+## Reading the numbers
+
+- **Shallow/typical depth is where `.73` wins decisively.** All three real tasks (tool-call,
+  codegen, debug) run at shallow context, and `.73` generates at **27–33 tok/s vs the P100's
+  ~19–23** — roughly the difference between "comfortable" and "sluggish" for an interactive
+  agent loop. Prefill is also ~1.5–1.8× faster, so first-token latency improves too.
+- **The `.73` box is VRAM-limited, not compute-limited.** At 64K ctx only **8.82 GB of the
+  14.16 GB** working set sits in VRAM — the rest (weights tail + deep KV) spills to system
+  RAM. It still beats the full-GPU P100, which means the card itself is much faster than the
+  P100; with more VRAM (or a smaller `num_ctx`) it would pull further ahead.
+- **The one place the P100 wins: very deep context.** At ~43K resident tokens, `.73`
+  generation falls to **8.4 tok/s** because the KV cache is partly in CPU RAM (memory-
+  bandwidth-bound attention). The P100 holds its whole 64K KV in VRAM and degrades more
+  gracefully (12.7 tok/s at 23K). In practice Hermes' steady-state prompt is ~16K, so this
+  rarely bites — but long sessions on `.73` will slow down more than on the P100.
+- **Quant differs**, so this isn't a pure hardware A/B: `.73` runs MXFP4 (gpt-oss's native,
+  near-lossless 4-bit) while the P100 GGUF quant is whatever llama-swap pulled. Both are
+  genuine gpt-oss-20b and both produced correct outputs; no quality regression observed.
+
+## Verdict for the Hermes backend
+
+- **Prefer `.73` when it's up.** It's the faster daily driver for gpt-oss-20b at the depths
+  Hermes actually runs at. Switch in-session with `/model --provider ollama --model gpt-oss:20b`.
+- **Keep the P100 (`valhalla-p100`) as the always-on default.** It's a container on the
+  24/7 server; the `.73` box may be a desktop/workstation that isn't always powered. The
+  P100 also degrades more gracefully at very deep context.
+- **If you want `.73` to be strictly better,** drop its `num_ctx` toward what Hermes needs
+  (it hard-requires ≥64K, so you can't go below that for Hermes) **or** put gpt-oss on a
+  bigger-VRAM card there — eliminating the CPU spill would lift both prefill and deep-context
+  generation.
+
+## Caveats
+
+- Small hand-written suite (4 tasks), not SWE-bench — measures latency/throughput and basic
+  correctness, not deep code quality.
+- The codegen prompt lost its back-ticked tokens to shell quoting during the run (prompt_n 95
+  vs the P100's 113); the model still produced a correct `merge_intervals`, and gen tok/s is
+  prompt-content-independent, so the speed comparison stands.
+- Deep-prefill rows use different context depths (43.8K on `.73` vs 23.4K on P100), so the
+  prefill-tok/s cells aren't directly comparable — read them as "each box's deep-prefill rate
+  at that depth," not a head-to-head ratio.
+- Raw responses saved on valhalla at `/tmp/ollama-bench/` (one `.json` per task); P100
+  baselines at `/tmp/ornith-bench/`.

docs/2026-06-28-ollama-73-fleet-benchmark.md

@@ -0,0 +1,87 @@
+# `.73` Ollama fleet benchmark — all 9 models (2026-06-28)
+
+**TL;DR:** Throughput across every model on `192.168.1.73`. Generation speed spans an
+**~5× range** — from `gemma4:e4b` at **~93 tok/s** down to the big `qwen3:30b-a3b` at
+**~18 tok/s**. **All 9 models emit valid tool-calls.** For an interactive agent backend the
+sweet spot is **`gpt-oss:20b` (~29 tok/s)** or **`gemma4:12b` (~48 tok/s)** if 12B quality
+suffices; the 30B-class Qwen MoEs are the slowest here (heavy CPU offload at 64K on this
+VRAM-limited box). **Caveat:** code-correctness for the heavy *thinking* models is
+indeterminate — they used the whole 768-token gen cap reasoning and never emitted code (see
+Caveats); re-run with a bigger budget to judge quality.
+
+## Setup
+- Endpoint `http://192.168.1.73:11434`, Ollama 0.30.11. Each model served at **`num_ctx
+  65536`** (Hermes' ≥64K requirement), sampling `temp 0.6 / top_p 0.95 / top_k 20`.
+- Native `/api/chat` timings. 4 tasks: tool-call, codegen (`merge_intervals`), debug
+  (`second_largest`), deep-prefill (~16K-token filler). Gen capped: 256 / 768 / 768 / 128.
+- Same `.73` box as the gpt-oss head-to-head in
+  `docs/2026-06-28-gpt-oss-20b-ollama-benchmark.md` (GPU still unidentified — no SSH).
+
+## Generation speed (tok/s) — the headline
+
+Average of the three real tasks (tool-call / codegen / debug), sorted fastest first:
+
+| Model | avg gen t/s | tool | codegen | debug | deep-ctx gen | cold load s | tool-call? |
+|---|--:|--:|--:|--:|--:|--:|:--:|
+| **gemma4:e4b** | **92.7** | 91.1 | 93.3 | 93.6 | 82.6 | 24.8 | ✅ |
+| **gemma4:12b** | **48.2** | 46.8 | 49.0 | 48.8 | 45.7 | 10.6 | ✅ |
+| **gpt-oss:20b** | **29.4** | 32.7 | 27.9 | 27.7 | 29.9 | 0.4¹ | ✅ |
+| **qwen3.6:35b-a3b** | **27.3** | 29.1 | 26.4 | 26.4 | 28.2 | 35.9 | ✅ |
+| **gemma4:26b** | **25.7** | 27.6 | 25.0 | 24.5 | 26.5 | 53.8 | ✅ |
+| **glm-4.7-flash** | **21.2** | 24.2 | 19.7 | 19.7 | 21.4 | 34.8 | ✅ |
+| **qwen3-vl:30b-a3b** | **19.1** | 22.7 | 16.5 | 18.2 | 20.6 | 29.8 | ✅ |
+| **qwen3-coder:30b** | **19.0** | 22.8 | 17.1 | 17.2 | 20.4 | 25.3 | ✅ |
+| **qwen3:30b-a3b** | **17.6** | 19.5 | 16.6 | 16.6 | 18.5 | 25.0 | ✅ |
+
+¹ gpt-oss was already resident from the prior run; real cold load is ~65 s.
+
+## Prefill speed (tok/s)
+
+| Model | shallow (~100 tok) | deep (~16K tok) |
+|---|--:|--:|
+| gemma4:e4b | 1193–1799 | 7283 |
+| gemma4:12b | 667–1078 | 3318 |
+| gpt-oss:20b | 281–379 | 2655 |
+| qwen3:30b-a3b | 63–149 | 1030 |
+| qwen3-coder:30b | 76–270 | 979 |
+| gemma4:26b | 110–127² | 977 |
+| qwen3.6:35b-a3b | 78–209 | 620 |
+| glm-4.7-flash | 70–163 | 649 |
+| qwen3-vl:30b-a3b | 68–127 | 588 |
+
+² gemma4:26b's first request after load measured 11.7 t/s (cold-cache artifact); ignore.
+
+## What stands out
+- **The two small gemmas are in a different league.** `gemma4:e4b` (~93 t/s) and
+  `gemma4:12b` (~48 t/s) are dense but small, so they sit fully on GPU and fly. If a 4B/12B
+  is smart enough for the job, they're the most responsive options by far.
+- **gpt-oss:20b is the best "big-brain, still-fast" pick** (~29 t/s) — MoE ~3.6B active keeps
+  it quick despite 20B total. `qwen3.6:35b-a3b` nearly matches it (~27 t/s) and may be
+  stronger; worth A/B-ing on real tasks.
+- **The 30B-a3b Qwen trio is the slowest** (~17–19 t/s). Same "3B-active" MoE label, but
+  larger total weights → more spills to CPU RAM at 64K on this VRAM-limited box, dragging
+  generation below gpt-oss. `qwen3-coder` being this slow undercuts it as a *fast* coding
+  model here.
+- **Every model tool-calls.** All 9 emitted a valid `get_weather({"city":"Tokyo"})`, so any
+  of them can drive Hermes' tool loop.
+
+## Correctness (partial — see caveat)
+- **Confirmed correct** code on the non-/light-thinking models that finished within the cap:
+  `gpt-oss:20b`, `gemma4:12b`, `qwen3-vl:30b-a3b`, `qwen3-coder:30b` (clean `def`,
+  `reason=stop` or code present), plus `gemma4:26b` & `qwen3:30b-a3b` on the task each
+  finished.
+- **Indeterminate** (truncated mid-reasoning, `content=0`, `reason=length`): `glm-4.7-flash`
+  (both), `gemma4:e4b` (both), `qwen3.6:35b-a3b` (both), `gemma4:26b` (codegen),
+  `qwen3:30b-a3b` (debug). These spent all 768 gen tokens in the `thinking` channel — **not
+  wrong, just unfinished.** A re-run at `num_predict ~3072` is needed to grade their output.
+
+## Caveats
+- The 768-token gen cap was too low for heavy chain-of-thought models — it bounds runtime but
+  truncates their answers. Speed (tok/s) is unaffected and valid; code *quality* for the
+  truncated set is not measured here.
+- Per-model VRAM split not captured (models unload after 2 min `keep_alive`); only
+  gpt-oss-20b is known (8.82 GB on-GPU / 14.16 GB total at 64K → partial CPU offload). The
+  slow 30B-class numbers are consistent with heavier offload.
+- Small hand-written suite, not SWE-bench. Quants are each model's Ollama default.
+- Raw per-task responses on valhalla at `/tmp/ollama-bench-all/` (`summary.json` + one JSON
+  per model/task); progress log `/tmp/ollama-bench-all/progress.txt`.

docs/superpowers/specs/2026-08-05-jellyfin-migration-design.md

@@ -0,0 +1,327 @@
+# Jellyfin migration — design
+
+**Date:** 2026-08-05
+**Status:** approved, not yet implemented
+**Supersedes:** `plans/homelab-improvements/tasks/TB-027-jellyfin.md` (stub — resolve it to "Do it" and link here)
+
+---
+
+## 1. Why
+
+Plex now requires a Plex Pass (server owner) or a Remote Watch Pass (viewer) for remote
+playback of personal media. Enforcement began on Roku in late 2025 and reaches Fire TV,
+Android TV, Google TV, Apple TV and third-party clients through 2026. valhalla's Plex
+server has **no Plex Pass**, so every remote viewer is being pushed toward paying for
+access to a server they don't own.
+
+Jellyfin has no such gate. This spec replaces host-native Plex with containerised Jellyfin,
+managed in this repo like everything else.
+
+## 2. Current state (measured 2026-08-05, not assumed)
+
+### Plex
+
+| Fact | Value |
+|---|---|
+| Install | AUR `plex-media-server` 1.43.2, systemd unit `plexmediaserver`, runs as `plex:plex` |
+| Binary | `/usr/lib/plexmediaserver/` |
+| Data dir | `/var/lib/plex` → **symlink to `/storage1/labdata/plex`** (virtiofs, *not* the root disk) |
+| Data size | **383G total** — `Media` 249G, `localhost` 116G, `Metadata` 15G, `Plug-in Support` 2.1G |
+| Reverse proxy | none — Plex handles its own relay/direct connect |
+
+> `CLAUDE.md` documents the data dir as `/var/lib/plexmediaserver/...`. **That path does not
+> exist.** Correcting it is a Phase 0 task.
+
+### Libraries
+
+Served from `/storage1`, exposed to the \*arrs through `/storage1/Media/*` symlinks:
+
+| Library | Real path | Items |
+|---|---|---|
+| Movies | `/storage1/Movies` | 1357 |
+| TV | `/storage1/TV` | 173 |
+| Anime | `/storage1/Anime` | 361 |
+
+`/storage1/Music`, `/storage1/Books` and `/storage1/LinuxISOs` (Stash's) are **out of scope**.
+
+### Usage (Tautulli, trailing 90 days / 600-play sample)
+
+| Metric | Value |
+|---|---|
+| Users | 37 registered, 29 flagged active — but **many are dormant in practice** |
+| Stream decisions | 90 direct play · 3 direct stream · **84 transcode (~48%)** |
+| Peak concurrent streams | **7** |
+| Transcode drivers | audio 23/25 · video 15/25 · **subtitle 0/25** |
+| Source content | h264 18 / hevc 7 · **100% SDR** |
+| Platforms | Android 281 · tvOS 102 · browsers 53 · Roku 39 · iOS 18 · Tizen 2 · PlayStation 1 |
+
+Two workloads that would have sunk a CPU-only plan are **absent**: no subtitle burn-in and
+no HDR tone-mapping. The residual load is 1080p SDR, which 52 Xeon threads can carry.
+
+### Host constraints
+
+| Constraint | Detail |
+|---|---|
+| 🔴 Root disk | `/dev/sda2` 194G, **179G used, 5.7G free (97%)**. `/config` lives here. Docker holds 123.5G of images, 51G reclaimable. |
+| `/storage1` | 94T, 78T used, **17T free**, virtiofs from the Proxmox host |
+| GPU | Tesla P100-PCIE-16GB. **GP100 has no NVENC** — decode-only per NVIDIA's support matrix. Currently held by llama-swap. |
+| CPU / RAM | 52 threads · 94G RAM (69G available) |
+| Docker GPU | CDI configured (`/etc/cdi/nvidia.yaml`) |
+
+## 3. Decisions
+
+| # | Decision | Rationale |
+|---|---|---|
+| D1 | New Portainer stack `stacks/streaming/` | Jellyfin config changes must not redeploy sonarr/radarr/qbittorrent. Plex-era cleanup stays contained. |
+| D2 | Public `jellyfin.ginnoir.com` via Caddy | Replaces Plex's relay. Tailscale-only or `share_gate` would break Roku/Fire TV/tvOS clients. |
+| D3 | **No SSO for Jellyfin** | `jellyfin-plugin-sso` completes OIDC only inside a browser. Roku, Android TV/Fire TV and Swiftfin cannot use it at all — and that is the entire user base. Native Jellyfin accounts. **Deliberate exception to the project auth standard.** |
+| D4 | Parallel run, then retire | Plex untouched and running through Phase 6. No failure before Phase 7 costs anything but time. |
+| D5 | Plexyfin once, then removed | Copies the curated 15G of posters/backdrops and rebuilds collections from PMS. It deletes and replaces artwork wholesale on every run, so leaving it installed would destroy later Jellyfin customisations. |
+| D6 | JellyPlex-Watched, two-way, scheduled | Users can move between servers freely during the overlap. |
+| D7 | Jellystat replaces Tautulli | Closest 1:1 equivalent. Tautulli is Plex-only and dies with Plex. |
+| D8 | CPU-only transcoding, measured | P100 cannot encode. Workload is 1080p SDR, no burn-in, no tone-mapping. Jellystat measures the truth. |
+| D9 | Escape hatch: Intel Arc A310/A380 | ~$100-150, no session limit, AV1. Only if Phase 5 data shows CPU strain. |
+| D10 | Seerr converted in place at cutover | `seerr-team/seerr` already supports Jellyfin natively. In-place conversion is unsupported upstream ([seerr#2522](https://github.com/seerr-team/seerr/issues/2522)) — mitigated by config backup and a fresh-config fallback. |
+| D11 | Transcode dir on disk, **not tmpfs** | [jellyfin#16608](https://github.com/jellyfin/jellyfin/issues/16608): the segment cleaner first checks at 20s and the throttler at 5s, but Remux/DirectStream jobs finish in 2-7s, so cleanup never fires and segments orphan. DirectStream (copy video + transcode audio) is 8 of 25 sampled sessions — routine here, not an edge case. Plex already writes transcodes to virtiofs at a 48% transcode rate without trouble. |
+| D12 | Trickplay at full quality | Plex's 365G of BIF thumbnails is **format-incompatible** with Jellyfin's tiled JPEG trickplay. None of it is reusable; Jellyfin regenerates from scratch. |
+| D13 | Wizarr invites issued **manually and selectively** | Most Plex users are dormant. No bulk invite generation, no scripted account creation, no broadcast announcement. ginnoir sends individual invites as people ask or as he chooses. |
+
+## 4. Architecture
+
+New Portainer stack `stacks/streaming/` — private `streaming` network plus `edge`, all
+config via `env_file: stack.env` per the repo convention, no Portainer UI env vars.
+
+| Container | Image | Purpose |
+|---|---|---|
+| `jellyfin` | `lscr.io/linuxserver/jellyfin` | media server (PUID/PGID from `stack.env`) |
+| `jellystat` | `cyfershepard/jellystat` | Tautulli replacement |
+| `jellystat-db` | `postgres:16-alpine` | pinned + `com.centurylabs.watchtower.enable=false` per infra convention |
+| `wizarr` | `ghcr.io/wizarrrr/wizarr` | invite + guided per-device onboarding, used ad-hoc |
+| `jellyplex-watched` | `luigi311/jellyplex-watched` | scheduled two-way watch-state sync |
+
+### Ordering constraint
+
+Plexyfin and JellyPlex-Watched both read from the **live PMS API**. Everything that pulls
+from Plex must complete before Plex is decommissioned. This is the hard sequencing rule of
+the whole migration.
+
+## 5. Storage layout
+
+The root disk is the primary hazard, so Jellyfin's bulky data is split off deliberately.
+
+| Container path | Host path | Filesystem | Rationale |
+|---|---|---|---|
+| `/config` | `/config/jellyfin` | root SSD | SQLite DBs — better fsync semantics, easy to back up. A few GB. |
+| `/config/metadata` | `/storage1/labdata/jellyfin/metadata` | virtiofs | artwork + NFO, grows unbounded |
+| `/config/data/trickplay` | `/storage1/labdata/jellyfin/trickplay` | virtiofs | **~300-400G.** 10.11 moved trickplay here and it is *not* relocatable from the UI, hence an explicit sub-bind. |
+| `/config/cache` | `/storage1/labdata/jellyfin/cache` | virtiofs | image cache |
+| `/transcode` | `/storage1/labdata/jellyfin/transcodes` | virtiofs | see D11 |
+| `/data/movies` | `/storage1/Movies` | virtiofs, **`:ro`** | Jellyfin never writes to the library |
+| `/data/tv` | `/storage1/TV` | virtiofs, **`:ro`** | |
+| `/data/anime` | `/storage1/Anime` | virtiofs, **`:ro`** | |
+
+Net effect: only a few GB ever lands on the 194G root disk.
+
+### Note on SQLite and virtiofs
+
+Plex has run its SQLite library DB on virtiofs here for months without incident, so this is
+proven workable — the root-SSD placement is a preference, not a hard requirement.
+
+It carries one accepted trade-off. If `/storage1` fails to mount again (as on 2026-07-06),
+Jellyfin with its DB on root will **start up healthy against an empty library** and flag the
+entire catalogue as missing — the RomM failure mode. With the DB on `/storage1` it would
+simply fail to start, which is the louder and safer failure. `storage1-guard.service`
+already blocks guest start when `/storage1` isn't real, so root SSD stands.
+
+## 6. Caddy
+
+```caddy
+# =============================================================
+# STREAMING — public
+# =============================================================
+jellyfin.ginnoir.com {
+	reverse_proxy jellyfin:8096
+}
+
+wizarr.ginnoir.com {
+	reverse_proxy wizarr:5690
+}
+
+# =============================================================
+# STREAMING STATS — internal only
+# =============================================================
+jellystat.ginnoir.com {
+	route {
+		import internal_gate
+		import authentik_outpost
+		handle {
+			import authentik_forward_auth
+			reverse_proxy jellystat:3000
+		}
+	}
+}
+```
+
+`jellyfin` and `wizarr` are public by design — remote users and invite recipients reach them
+from anywhere. `jellystat` follows the tautulli/sonarr pattern: `internal_gate` first inside
+the `route` block (an `import internal_only` above a catch-all route is dead code), then
+Authentik forward auth.
+
+In Jellyfin → Networking, register the Caddy container as a known proxy and enable forwarded
+headers, so real client IPs reach Jellystat and the remote-vs-LAN bitrate rules.
+
+## 7. Metadata strategy
+
+1. Jellyfin scans and fetches fresh from TMDB/TVDB — the same sources Plex used, so baseline
+   quality is identical.
+2. **Plexyfin runs once**, overwriting posters/backdrops with the curated Plex artwork and
+   rebuilding collections. This is where the 15G `Metadata` investment is preserved.
+3. **Plexyfin is then removed.** See D5.
+
+Plex's 365G of `Media` + `localhost` (BIF preview thumbnails) is not reusable — see D12.
+
+## 8. Plugins
+
+**Install**
+
+| Plugin | Purpose | Note |
+|---|---|---|
+| Plexyfin | collections + curated artwork from PMS | 3rd-party repo. Temporary — remove after Phase 2. |
+| Intro Skipper | skip intros/recaps/credits | **Verify 10.11 compatibility at install.** The original is unmaintained; use the active fork. |
+| AniDB / AniList | anime metadata | Anime library only |
+| Trakt | per-user scrobbling | optional |
+
+**Deliberately skipped**
+
+| Plugin | Why |
+|---|---|
+| `jellyfin-plugin-sso` | browser-only OIDC — useless for Roku/Fire TV/tvOS (D3) |
+| Open Subtitles | Bazarr already writes subtitles to disk and does it better |
+| Playback Reporting | Jellystat supersedes it |
+
+## 9. Transcoding
+
+CPU-only on 52 threads: `veryfast` preset, per-session thread cap, throttling enabled,
+**segment deletion on with a ~360s keep window**. Measured worst case is 7 concurrent streams
+at ~48% transcode, 1080p SDR, no burn-in, no tone-mapping.
+
+Jellyfin's Android client (the heaviest platform here at 281 plays) direct-plays audio codecs
+Plex's client refuses, so the transcode rate may **fall** after migration. Jellystat confirms
+or refutes this during Phase 5.
+
+If it does strain: Intel Arc A310/A380 (D9). Needs a PCIe slot, an R730XD shutdown, and an
+IOMMU passthrough to the valhalla VM — check the new card's IOMMU group, as only the P100's
+group 7 is known clean.
+
+## 10. Phases
+
+Plex stays running and untouched through Phase 6.
+
+### Phase 0 — Disk remediation and baseline
+
+The root disk is at 97%. Nothing else may proceed until it isn't.
+
+- `docker image prune` / `builder prune` — ~51G reclaimable
+- Audit the remaining 179G on `/dev/sda2`; identify anything else structurally growing
+- Uptime Kuma monitor on root-disk free space + ntfy alert (this is the second root fill)
+- Correct the Plex paths in `CLAUDE.md` (`/var/lib/plex` → `/storage1/labdata/plex`)
+
+**Done when:** root disk has comfortable headroom and alerts before it doesn't.
+
+### Phase 1 — Deploy the stack
+
+- `stacks/streaming/docker-compose.yml` + `stack.env`, `jellyfin` only
+- Storage layout per §5; register the stack in Portainer (one-time, per the new-stack procedure)
+- Caddy site blocks; `scripts/gen-bookmarks.ps1`
+- Libraries: Movies, TV, Anime (Anime with anime metadata providers)
+- Verify `check-portainer-stack-env.ps1` before pushing
+
+**Done when:** Jellyfin reachable at `jellyfin.ginnoir.com`, libraries scanned, no users.
+**Rollback:** delete the stack.
+
+### Phase 2 — Parity
+
+- Plexyfin: sync collections + artwork from PMS, then **remove the plugin**
+- **Trickplay generation** — full quality, thread-capped (~16-24 of 52), run before any users
+  arrive. Budget ~300-400G on `/storage1` and a multi-day first pass over ~1891 items.
+  Enable NVDEC for this: trickplay is decode + JPEG encode and needs no NVENC, so the P100
+  helps here even though it's useless for live transcode. Schedule when llama-swap is idle.
+- Remaining plugins per §8
+- Sonarr/Radarr/Bazarr → add Jellyfin library-refresh connections (leave the Plex ones for now)
+
+**Done when:** Jellyfin's library visually matches Plex's, trickplay complete.
+
+### Phase 3 — Watch-state sync
+
+- `jellyplex-watched` container, two-way, every 6h
+- Plex admin token + Jellyfin API key in `stack.env`; user-mapping file where names differ
+- Verify against ginnoir's own account in both directions
+
+**Done when:** a play on either server appears on the other within one cycle.
+**Rollback:** stop the container.
+
+### Phase 4 — Pilot
+
+- Configure Wizarr (server connection, wizard steps per device)
+- 3-5 willing users, **at least one each on Roku, Fire TV/Android TV and tvOS**
+- Collect real feedback on playback, transcoding and the onboarding wizard
+
+**Done when:** every target platform has a confirmed working client.
+**Rollback:** pilot users return to Plex, which never stopped.
+
+### Phase 5 — Selective migration
+
+Per D13, this phase is **demand-driven, not a campaign.**
+
+- Wizarr invites issued **individually**, as people ask or as ginnoir chooses
+- Dormant accounts get nothing
+- A short written explainer exists to accompany an invite — sent with invites, not broadcast
+- Both servers live; run at least **4 weeks** to accumulate meaningful Jellystat data
+- Watch transcode load and concurrency; decide on D9
+
+**Done when:** the people who actually watch things are on Jellyfin and it's holding up.
+
+### Phase 6 — Cutover
+
+- Back up `/config/overseerr`, then reconfigure Seerr from Plex to Jellyfin (D10)
+- Jellystat live at `jellystat.ginnoir.com`
+- **Re-check Tautulli** for who is still on Plex and on what. Handle those users
+  individually — Tizen and PlayStation are the weak platforms (3 plays of 600 as of now).
+- Remove Plex connections from Sonarr/Radarr/Bazarr
+- Announce the Plex end date
+
+**Done when:** Seerr requests flow against Jellyfin and nobody depends on Plex.
+**Rollback:** restore `/config/overseerr`.
+
+### Phase 7 — Decommission
+
+- `systemctl stop plexmediaserver && systemctl disable plexmediaserver`
+- **2-week soak** with the service stopped but installed
+- `pacman -Rns plex-media-server`
+- Retire the `tautulli` container and its Caddy block
+- Tarball `/storage1/labdata/plex`, keep 30 days, then reclaim **383G**
+- Update `CLAUDE.md` (drop the external-services Plex section), bookmarks, Uptime Kuma monitors
+
+**Done when:** Plex is gone and nothing references it.
+**Rollback (until the tarball expires):** reinstall the package, restore the data dir.
+
+## 11. Risks
+
+| Risk | Severity | Mitigation |
+|---|---|---|
+| Root disk at 97% | **high** | Phase 0 exists solely for this; monitor + alert added |
+| Public Jellyfin is new attack surface Plex's relay hid | medium | Jellyfin's built-in brute-force lockout, strong admin credential, no anonymous access, monitoring |
+| CPU transcode capacity at peak | medium | measured in Phase 5 via Jellystat; Arc A380 escape hatch (D9) |
+| Seerr in-place conversion unsupported upstream | medium | config backup first, fresh-config fallback documented |
+| Plexyfin is destructive | medium | run once in Phase 2, then remove (D5) |
+| Trickplay generation starves live transcoding | low | thread-capped, runs before users arrive, NVDEC-assisted |
+| Intro Skipper 10.11 compatibility | low | verify at install; it's optional polish |
+| Holdouts on Tizen / PlayStation | low | 3 of 600 plays; handled individually at Phase 6 with fresh data |
+
+## 12. Out of scope
+
+- Music, Books and Stash content — Jellyfin serves Movies, TV and Anime only
+- SSO for Jellyfin (D3)
+- Migrating Seerr request history
+- Replacing the P100 or restructuring GPU allocation
+- The wider `/storage1` redundancy problem (tracked separately)

portainer-compose.yml

@@ -149,6 +149,29 @@ services:
     labels:
       - "com.centurylabs.watchtower.enable=false"
 
+  gitea-runner-ci:
+    container_name: gitea_runner_ci
+    image: gitea/act_runner:latest
+    restart: unless-stopped
+    networks:
+      - gitea_internal
+    environment:
+      - GITEA_INSTANCE_URL=https://gitea.ginnoir.com
+      - GITEA_RUNNER_REGISTRATION_TOKEN=${GITEA_RUNNER_REGISTRATION_TOKEN}
+      - GITEA_RUNNER_NAME=valhalla-ci-runner-2
+      - GITEA_RUNNER_LABELS=ubuntu-latest:docker://node:24-bookworm,node-24:docker://node:24-bookworm
+      - CONFIG_FILE=/data/config.yaml
+    volumes:
+      - /config/gitea/act_runner_ci:/data
+      - /var/run/docker.sock:/var/run/docker.sock
+      - /home/ginnoir/valhalla-lab:/valhalla-lab
+      - /config/caddy:/config/caddy
+    depends_on:
+      gitea:
+        condition: service_started
+    labels:
+      - "com.centurylabs.watchtower.enable=false"
+
   registry:
     container_name: registry
     image: registry:2

scripts/check_uptime.js

@@ -1,10 +1,27 @@
 const fs = require('fs');
 const path = require('path');
 
+async function fetchWithRetry(url, options = {}, retries = 3, backoff = 2000) {
+  for (let i = 0; i < retries; i++) {
+    try {
+      const response = await fetch(url, options);
+      if (response.ok) return response;
+      if (response.status >= 500) {
+        throw new Error(`HTTP ${response.status}: ${response.statusText}`);
+      }
+      return response;
+    } catch (err) {
+      if (i === retries - 1) throw err;
+      console.warn(`Fetch to ${url} failed (attempt ${i + 1}/${retries}): ${err.message}. Retrying in ${backoff}ms...`);
+      await new Promise(resolve => setTimeout(resolve, backoff));
+    }
+  }
+}
+
 async function checkStatus() {
   try {
     // 1. Fetch status page HTML
-    const htmlResponse = await fetch('https://uptime.ginnoir.com/status/default');
+    const htmlResponse = await fetchWithRetry('https://uptime.ginnoir.com/status/default');
     if (!htmlResponse.ok) {
       throw new Error(`Failed to fetch status page: ${htmlResponse.statusText}`);
     }
@@ -34,7 +51,7 @@ async function checkStatus() {
     }
 
     // 2. Fetch heartbeat JSON
-    const heartbeatResponse = await fetch('https://uptime.ginnoir.com/api/status-page/heartbeat/default');
+    const heartbeatResponse = await fetchWithRetry('https://uptime.ginnoir.com/api/status-page/heartbeat/default');
     if (!heartbeatResponse.ok) {
       throw new Error(`Failed to fetch heartbeat: ${heartbeatResponse.statusText}`);
     }

scripts/check_uptime_to_obsidian.js

@@ -0,0 +1,179 @@
+const fs = require('fs');
+const path = require('path');
+
+async function fetchWithRetry(url, options = {}, retries = 3, backoff = 2000) {
+  for (let i = 0; i < retries; i++) {
+    try {
+      const response = await fetch(url, options);
+      if (response.ok) return response;
+      if (response.status >= 500) {
+        throw new Error(`HTTP ${response.status}: ${response.statusText}`);
+      }
+      return response;
+    } catch (err) {
+      if (i === retries - 1) throw err;
+      console.warn(`Fetch to ${url} failed (attempt ${i + 1}/${retries}): ${err.message}. Retrying in ${backoff}ms...`);
+      await new Promise(resolve => setTimeout(resolve, backoff));
+    }
+  }
+}
+
+async function checkStatus() {
+  try {
+    // 1. Fetch status page HTML
+    const htmlResponse = await fetchWithRetry('https://uptime.ginnoir.com/status/default');
+    if (!htmlResponse.ok) {
+      throw new Error(`Failed to fetch status page: ${htmlResponse.statusText}`);
+    }
+    const html = await htmlResponse.text();
+
+    // Extract window.preloadData = ...;
+    const preloadRegex = /window\.preloadData\s*=\s*({.*?});/s;
+    const match = html.match(preloadRegex);
+    if (!match) {
+      throw new Error("Could not find window.preloadData in HTML");
+    }
+
+    let preloadData;
+    try {
+      preloadData = new Function(`return ${match[1]};`)();
+    } catch (e) {
+      throw new Error(`Failed to parse preloadData: ${e.message}`);
+    }
+
+    const monitors = [];
+    if (preloadData && preloadData.publicGroupList) {
+      for (const group of preloadData.publicGroupList) {
+        if (group.monitorList) {
+          monitors.push(...group.monitorList);
+        }
+      }
+    }
+
+    // 2. Fetch heartbeat JSON
+    const heartbeatResponse = await fetchWithRetry('https://uptime.ginnoir.com/api/status-page/heartbeat/default');
+    if (!heartbeatResponse.ok) {
+      throw new Error(`Failed to fetch heartbeat: ${heartbeatResponse.statusText}`);
+    }
+    const heartbeats = await heartbeatResponse.json();
+
+    // 3. Map status and print summary
+    const monitorMap = {};
+    for (const monitor of monitors) {
+      monitorMap[monitor.id] = {
+        name: monitor.name,
+        type: monitor.type,
+        status: 'UNKNOWN',
+        ping: null,
+        lastCheck: null,
+        msg: ''
+      };
+    }
+
+    const heartbeatList = heartbeats.heartbeatList || {};
+    for (const id in heartbeatList) {
+      const list = heartbeatList[id];
+      if (list && list.length > 0) {
+        const latest = list[list.length - 1];
+        if (monitorMap[id]) {
+          monitorMap[id].status = latest.status === 1 ? 'UP' : 'DOWN';
+          monitorMap[id].ping = latest.ping;
+          monitorMap[id].lastCheck = latest.time;
+          monitorMap[id].msg = latest.msg || '';
+        }
+      }
+    }
+
+    // Generate summary
+    const monitorValues = Object.values(monitorMap);
+    const total = monitorValues.length;
+    const up = monitorValues.filter(m => m.status === 'UP').length;
+    const down = monitorValues.filter(m => m.status === 'DOWN').length;
+    const unknown = monitorValues.filter(m => m.status === 'UNKNOWN').length;
+
+    let summaryMd = `### Uptime Status Summary (Checked at ${new Date().toLocaleString()})\n\n`;
+    
+    if (down > 0) {
+      summaryMd += `⚠️ **Status: Degraded (${down}/${total} services DOWN)**\n\n`;
+    } else if (up === total) {
+      summaryMd += `✅ **Status: Healthy (All ${total} services UP)**\n\n`;
+    } else {
+      summaryMd += `ℹ️ **Status: Mixed (UP: ${up}, DOWN: ${down}, UNKNOWN: ${unknown})**\n\n`;
+    }
+
+    if (down > 0) {
+      summaryMd += `#### 🚨 DOWN Services:\n`;
+      monitorValues.filter(m => m.status === 'DOWN').forEach(m => {
+        summaryMd += `- **${m.name}** (${m.type}) - ${m.msg || 'No message'} (Last checked: ${m.lastCheck})\n`;
+      });
+      summaryMd += `\n`;
+    }
+
+    summaryMd += `#### 📋 Service Statuses:\n`;
+    summaryMd += `| Service | Status | Latency (ms) | Last Check |\n`;
+    summaryMd += `| :--- | :---: | :---: | :--- |\n`;
+    
+    // Sort services by name
+    monitorValues.sort((a, b) => a.name.localeCompare(b.name));
+    
+    for (const m of monitorValues) {
+      const statusIcon = m.status === 'UP' ? '🟢 UP' : m.status === 'DOWN' ? '🔴 DOWN' : '⚪ UNKNOWN';
+      const pingText = m.ping !== null ? `${m.ping} ms` : 'N/A';
+      summaryMd += `| ${m.name} | ${statusIcon} | ${pingText} | ${m.lastCheck || 'N/A'} |\n`;
+    }
+
+    // Output to stdout
+    console.log(summaryMd);
+
+    // 1. Write the latest report to Uptime Status Report.md
+    const reportPath = 'c:/Users/MattC/Documents/Obsidian Vault/Homelab/Uptime Status Report.md';
+    const reportContent = `---
+project: Homelab
+type: status
+status: current
+tags: [monitoring, uptime]
+updated: ${new Date().toISOString().split('T')[0]}
+---
+
+# Live Uptime Status Report
+
+${summaryMd}
+`;
+    fs.writeFileSync(reportPath, reportContent, 'utf8');
+    console.log(`Wrote status report to ${reportPath}`);
+
+    // 2. Append to today's session log
+    const todayStr = new Date().toISOString().split('T')[0];
+    const sessionLogDir = 'c:/Users/MattC/Documents/Obsidian Vault/Homelab/Sessions';
+    if (!fs.existsSync(sessionLogDir)) {
+      fs.mkdirSync(sessionLogDir, { recursive: true });
+    }
+    const sessionLogPath = path.join(sessionLogDir, `${todayStr}-uptime-status-monitoring-schedule.md`);
+
+    let sessionContent = '';
+    if (!fs.existsSync(sessionLogPath)) {
+      sessionContent = `# Session Log — ${todayStr} — Uptime Status Monitoring Schedule
+
+## What was done
+- Automatically checked uptime status page and updated reports.
+
+## Daily Log of Checks
+`;
+    } else {
+      sessionContent = fs.readFileSync(sessionLogPath, 'utf8');
+    }
+
+    const timeStr = new Date().toLocaleTimeString();
+    const statusText = down > 0 ? `🚨 DEGRADED (${down}/${total} services DOWN)` : `✅ Healthy (${total}/${total} services UP)`;
+    const logEntry = `\n### Check at ${timeStr}\n- **Status**: ${statusText}\n`;
+
+    fs.writeFileSync(sessionLogPath, sessionContent + logEntry, 'utf8');
+    console.log(`Appended check entry to ${sessionLogPath}`);
+
+  } catch (error) {
+    console.error(`Error checking status: ${error.message}`);
+    process.exit(1);
+  }
+}
+
+checkStatus();

stacks/authentik/docker-compose.yml

@@ -8,7 +8,7 @@
 services:
   authentik-server:
     container_name: authentik_server
-    image: ghcr.io/goauthentik/server:2026.5.2
+    image: ghcr.io/goauthentik/server:2026.5.3
     restart: unless-stopped
     command: server
     networks: [authentik, edge]
@@ -36,7 +36,7 @@ services:
 
   authentik-worker:
     container_name: authentik_worker
-    image: ghcr.io/goauthentik/server:2026.5.2
+    image: ghcr.io/goauthentik/server:2026.5.3
     restart: unless-stopped
     command: worker
     networks: [authentik]

stacks/dev/docker-compose.yml

@@ -57,6 +57,9 @@ services:
       - TZ
       - APP_URL=https://docs.ginnoir.com
       - APP_KEY
+      # stack.env leaks Plane's DATABASE_URL; Laravel prioritizes it over DB_* and
+      # would connect BookStack to postgres_plane. Pin it to BookStack's own MariaDB.
+      - DATABASE_URL=mysql://bookstack:bookstack_d3v_pass@mariadb_bookstack:3306/bookstack
       - DB_HOST=mariadb_bookstack
       - DB_PORT=3306
       - DB_USER=bookstack

stacks/famapp/stack.env

@@ -36,3 +36,11 @@ MINIO_BUCKET=garden
 
 OPENPLANTBOOK_CLIENT_ID=5JdQLab69RhVaF4er7b7D29BCrkf3fo3OEiDsOQa
 OPENPLANTBOOK_CLIENT_SECRET=kxPTrdQPAVta7sjVuIRf5EGIJ1fKvlMlqRSJpQtFxRftJhg7oeGnXzwH7ls6tluQLFUYVDYRMwISiAuKZGZLvkuJzUwFGCwhZw7UtW8OvcTGy4717WeNUinvuCDSsNXP
+
+# LLM assistant (OpenAI-compatible — same as local dev for now)
+LLM_PROVIDER=openai
+LLM_BASE_URL=http://192.168.1.121:8080/v1
+LLM_API_KEY=famapp-unc-f04f7ef0baf68d542d10a501
+LLM_MODEL=auto
+# IANA timezone for assistant relative dates ("Thursday at 2")
+HOUSEHOLD_TIMEZONE=America/Chicago

stacks/proxy/docker-compose.yml

@@ -19,6 +19,21 @@ services:
     restart: unless-stopped
     labels:
       - "com.centurylabs.watchtower.enable=false"
+    # Public resolvers for ACME DNS-01 zone detection.
+    #
+    # The LAN resolver (OPNsense Unbound, 192.168.1.1) is authoritative for
+    # ginnoir.com via the split-horizon override, but its local-zone only holds
+    # A records — so `SOA ginnoir.com` returns NODATA. certmagic's zone lookup
+    # walks up the label chain looking for an SOA, finds none at ginnoir.com,
+    # climbs to `com.`, and asks Cloudflare for a `com` zone. Result:
+    #   "adding temporary record for zone \"com.\": expected 1 zone, got 0"
+    # and every DNS-01 renewal fails until certs expire.
+    #
+    # Docker's embedded DNS (127.0.0.11) still resolves container names and the
+    # gitea.ginnoir.com alias below; only external lookups go to Cloudflare.
+    dns:
+      - 1.1.1.1
+      - 1.0.0.1
     networks:
       edge:
         aliases:

stacks/roms/docker-compose.yml

@@ -22,9 +22,9 @@
 
 services:
   romm:
-    # Floating v4 tag (watchtower-tracked). The save-sync (/api/sync/*) + raw
-    # content_hash fixes that once forced a 4.9.0-beta.2 pin are now in stable 4.9.x.
-    image: rommapp/romm:4
+    # Floating v5 tag (watchtower-tracked). Bumped from :4 (4.9.2) → :5 (5.0.0)
+    # 2026-07-21. Pre-upgrade dump: /config/backup/dumps/romm-pre-5.0.0-20260721.sql
+    image: rommapp/romm:5
     container_name: romm
     restart: unless-stopped
     networks: [roms, edge]

stacks/streaming/docker-compose.yml

@@ -0,0 +1,96 @@
+# streaming stack — Jellyfin and its satellites.
+#
+# Storage split is deliberate: SQLite config on the root SSD, all bulk data
+# (metadata, trickplay, cache, transcodes) on /storage1. The root disk is only
+# 194G and has filled twice; nothing here may grow unbounded on it.
+#
+# Jellyfin 10.11 (linuxserver) keeps metadata at /config/data/metadata — NOT
+# /config/metadata. Trickplay is at /config/data/trickplay and is not
+# relocatable from the UI. Transcodes are configured to /transcode.
+#
+# Transcodes are disk-backed, not tmpfs: jellyfin#16608 means Remux and
+# DirectStream jobs finish before the segment cleaner's first check, orphaning
+# segments. Plex already writes transcodes to virtiofs at a 48% transcode rate.
+#
+# Env convention: every service gets stack.env via env_file; var names there are
+# exactly what the containers read. No ${VAR} interpolation, no Portainer UI
+# env vars.
+
+services:
+  jellyfin:
+    container_name: jellyfin
+    image: lscr.io/linuxserver/jellyfin:latest
+    restart: unless-stopped
+    networks: [streaming, edge]
+    env_file:
+      - stack.env
+    devices:
+      - nvidia.com/gpu=0
+    environment:
+      - JELLYFIN_PublishedServerUrl=https://jellyfin.ginnoir.com
+    volumes:
+      - /etc/localtime:/etc/localtime:ro
+      - /config/jellyfin:/config
+      - /storage1/labdata/jellyfin/metadata:/config/data/metadata
+      - /storage1/labdata/jellyfin/cache:/config/cache
+      - /storage1/labdata/jellyfin/trickplay:/config/data/trickplay
+      - /storage1/labdata/jellyfin/transcodes:/transcode
+      - /storage1/Movies:/data/movies:ro
+      - /storage1/TV:/data/tv:ro
+      - /storage1/Anime:/data/anime:ro
+    ports:
+      - "8096:8096"
+
+  jellystat-db:
+    container_name: jellystat-db
+    image: postgres:16-alpine
+    restart: unless-stopped
+    networks: [streaming]
+    env_file:
+      - stack.env
+    labels:
+      - "com.centurylabs.watchtower.enable=false"
+    volumes:
+      - /config/jellystat-db:/var/lib/postgresql/data
+
+  jellystat:
+    container_name: jellystat
+    image: cyfershepard/jellystat:latest
+    restart: unless-stopped
+    networks: [streaming, edge]
+    depends_on:
+      - jellystat-db
+    env_file:
+      - stack.env
+    volumes:
+      - /config/jellystat:/app/backend/backup-data
+    ports:
+      - "3009:3000"
+
+  wizarr:
+    container_name: wizarr
+    image: ghcr.io/wizarrrr/wizarr:latest
+    restart: unless-stopped
+    networks: [streaming, edge]
+    env_file:
+      - stack.env
+    volumes:
+      - /config/wizarr:/data/database
+    ports:
+      - "5690:5690"
+
+  jellyplex-watched:
+    container_name: jellyplex-watched
+    image: luigi311/jellyplex-watched:latest
+    restart: unless-stopped
+    networks: [streaming, edge]
+    env_file:
+      - stack.env
+
+networks:
+  streaming:
+    name: streaming
+    driver: bridge
+  edge:
+    name: edge
+    external: true

stacks/streaming/stack.env

@@ -0,0 +1,34 @@
+# streaming stack config — injected into every service via env_file.
+# Var names are exactly what the containers read; no ${VAR} interpolation,
+# no Portainer UI env vars needed.
+
+# --- shared ---
+TZ=America/Chicago
+PUID=1000
+PGID=1000
+
+# --- jellystat-db (postgres:16-alpine) ---
+POSTGRES_DB=jfstat
+POSTGRES_USER=jellystat
+POSTGRES_PASSWORD=bf2d47d8e1a87baf2b4942704a5acfaf5c2aad336fdd480e
+
+# --- jellystat ---
+POSTGRES_IP=jellystat-db
+POSTGRES_PORT=5432
+JWT_SECRET=0dd9fcdb632277b76977dbada8226e547cc773e1deafc8d778d4038f8d59393b
+
+# --- jellyplex-watched ---
+# Plex is host-native. Plain HTTP on :32400 only answers on loopback; LAN HTTP
+# gets an empty reply. Temporary host socat proxy plex-http-proxy.service
+# exposes loopback Plex HTTP on :32402 for Docker clients (Plexyfin + jellyplex).
+# Tear down that unit after Plex is decommissioned / Plexyfin is done.
+PLEX_URL=http://192.168.1.69:32402
+PLEX_TOKEN=hM9aycswHKSsUrxiHqwq
+JELLYFIN_BASEURL=http://jellyfin:8096
+JELLYFIN_TOKEN=a3d9ee827d5c42298cae087a2c31d3fa
+SYNC_FROM_PLEX_TO_JELLYFIN=True
+SYNC_FROM_JELLYFIN_TO_PLEX=True
+# Start in dry-run. Task 16 flips this to False after the mapping is verified.
+DRYRUN=False
+SLEEP_DURATION=21600
+LOG_LEVEL=INFO