From d24498e4404019557393585d5b69b3d989da1518 Mon Sep 17 00:00:00 2001 From: ginnoir Date: Wed, 5 Aug 2026 12:53:29 -0500 Subject: [PATCH] docs: add Jellyfin-replaces-Plex migration design MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plex now gates remote playback behind Plex Pass / Remote Watch Pass and valhalla has no Plex Pass, so remote viewers are being pushed to pay for access to a server they don't own. Design covers a new stacks/streaming Portainer stack (jellyfin, jellystat + postgres, wizarr, jellyplex-watched), a storage layout that keeps bulk data off the 97%-full root disk, and eight phases with Plex left running and untouched through Phase 6. Findings that shaped it, measured rather than assumed: - root disk is at 97% (5.7G free) and /config lives on it — Phase 0 - Tesla P100 is GP100: no NVENC, decode only, so transcoding is CPU-bound - 48% of streams transcode, peak 7 concurrent, but 1080p SDR with no subtitle burn-in and no HDR tone-mapping, which 52 threads can carry - /var/lib/plex is a symlink to /storage1/labdata/plex holding 383G; the 15G Metadata dir is reusable via Plexyfin, the 365G of BIF preview thumbnails is not (format-incompatible with Jellyfin trickplay) - jellyfin-plugin-sso is browser-only, so Roku/Fire TV/tvOS cannot use it — native Jellyfin accounts, a deliberate exception to the auth standard Also notes that CLAUDE.md documents a Plex data path that does not exist. Co-Authored-By: Claude Opus 5 --- .../2026-08-05-jellyfin-migration-design.md | 327 ++++++++++++++++++ 1 file changed, 327 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-05-jellyfin-migration-design.md diff --git a/docs/superpowers/specs/2026-08-05-jellyfin-migration-design.md b/docs/superpowers/specs/2026-08-05-jellyfin-migration-design.md new file mode 100644 index 0000000..d94b8cb --- /dev/null +++ b/docs/superpowers/specs/2026-08-05-jellyfin-migration-design.md @@ -0,0 +1,327 @@ +# Jellyfin migration — design + +**Date:** 2026-08-05 +**Status:** approved, not yet implemented +**Supersedes:** `plans/homelab-improvements/tasks/TB-027-jellyfin.md` (stub — resolve it to "Do it" and link here) + +--- + +## 1. Why + +Plex now requires a Plex Pass (server owner) or a Remote Watch Pass (viewer) for remote +playback of personal media. Enforcement began on Roku in late 2025 and reaches Fire TV, +Android TV, Google TV, Apple TV and third-party clients through 2026. valhalla's Plex +server has **no Plex Pass**, so every remote viewer is being pushed toward paying for +access to a server they don't own. + +Jellyfin has no such gate. This spec replaces host-native Plex with containerised Jellyfin, +managed in this repo like everything else. + +## 2. Current state (measured 2026-08-05, not assumed) + +### Plex + +| Fact | Value | +|---|---| +| Install | AUR `plex-media-server` 1.43.2, systemd unit `plexmediaserver`, runs as `plex:plex` | +| Binary | `/usr/lib/plexmediaserver/` | +| Data dir | `/var/lib/plex` → **symlink to `/storage1/labdata/plex`** (virtiofs, *not* the root disk) | +| Data size | **383G total** — `Media` 249G, `localhost` 116G, `Metadata` 15G, `Plug-in Support` 2.1G | +| Reverse proxy | none — Plex handles its own relay/direct connect | + +> `CLAUDE.md` documents the data dir as `/var/lib/plexmediaserver/...`. **That path does not +> exist.** Correcting it is a Phase 0 task. + +### Libraries + +Served from `/storage1`, exposed to the \*arrs through `/storage1/Media/*` symlinks: + +| Library | Real path | Items | +|---|---|---| +| Movies | `/storage1/Movies` | 1357 | +| TV | `/storage1/TV` | 173 | +| Anime | `/storage1/Anime` | 361 | + +`/storage1/Music`, `/storage1/Books` and `/storage1/LinuxISOs` (Stash's) are **out of scope**. + +### Usage (Tautulli, trailing 90 days / 600-play sample) + +| Metric | Value | +|---|---| +| Users | 37 registered, 29 flagged active — but **many are dormant in practice** | +| Stream decisions | 90 direct play · 3 direct stream · **84 transcode (~48%)** | +| Peak concurrent streams | **7** | +| Transcode drivers | audio 23/25 · video 15/25 · **subtitle 0/25** | +| Source content | h264 18 / hevc 7 · **100% SDR** | +| Platforms | Android 281 · tvOS 102 · browsers 53 · Roku 39 · iOS 18 · Tizen 2 · PlayStation 1 | + +Two workloads that would have sunk a CPU-only plan are **absent**: no subtitle burn-in and +no HDR tone-mapping. The residual load is 1080p SDR, which 52 Xeon threads can carry. + +### Host constraints + +| Constraint | Detail | +|---|---| +| 🔴 Root disk | `/dev/sda2` 194G, **179G used, 5.7G free (97%)**. `/config` lives here. Docker holds 123.5G of images, 51G reclaimable. | +| `/storage1` | 94T, 78T used, **17T free**, virtiofs from the Proxmox host | +| GPU | Tesla P100-PCIE-16GB. **GP100 has no NVENC** — decode-only per NVIDIA's support matrix. Currently held by llama-swap. | +| CPU / RAM | 52 threads · 94G RAM (69G available) | +| Docker GPU | CDI configured (`/etc/cdi/nvidia.yaml`) | + +## 3. Decisions + +| # | Decision | Rationale | +|---|---|---| +| D1 | New Portainer stack `stacks/streaming/` | Jellyfin config changes must not redeploy sonarr/radarr/qbittorrent. Plex-era cleanup stays contained. | +| D2 | Public `jellyfin.ginnoir.com` via Caddy | Replaces Plex's relay. Tailscale-only or `share_gate` would break Roku/Fire TV/tvOS clients. | +| D3 | **No SSO for Jellyfin** | `jellyfin-plugin-sso` completes OIDC only inside a browser. Roku, Android TV/Fire TV and Swiftfin cannot use it at all — and that is the entire user base. Native Jellyfin accounts. **Deliberate exception to the project auth standard.** | +| D4 | Parallel run, then retire | Plex untouched and running through Phase 6. No failure before Phase 7 costs anything but time. | +| D5 | Plexyfin once, then removed | Copies the curated 15G of posters/backdrops and rebuilds collections from PMS. It deletes and replaces artwork wholesale on every run, so leaving it installed would destroy later Jellyfin customisations. | +| D6 | JellyPlex-Watched, two-way, scheduled | Users can move between servers freely during the overlap. | +| D7 | Jellystat replaces Tautulli | Closest 1:1 equivalent. Tautulli is Plex-only and dies with Plex. | +| D8 | CPU-only transcoding, measured | P100 cannot encode. Workload is 1080p SDR, no burn-in, no tone-mapping. Jellystat measures the truth. | +| D9 | Escape hatch: Intel Arc A310/A380 | ~$100-150, no session limit, AV1. Only if Phase 5 data shows CPU strain. | +| D10 | Seerr converted in place at cutover | `seerr-team/seerr` already supports Jellyfin natively. In-place conversion is unsupported upstream ([seerr#2522](https://github.com/seerr-team/seerr/issues/2522)) — mitigated by config backup and a fresh-config fallback. | +| D11 | Transcode dir on disk, **not tmpfs** | [jellyfin#16608](https://github.com/jellyfin/jellyfin/issues/16608): the segment cleaner first checks at 20s and the throttler at 5s, but Remux/DirectStream jobs finish in 2-7s, so cleanup never fires and segments orphan. DirectStream (copy video + transcode audio) is 8 of 25 sampled sessions — routine here, not an edge case. Plex already writes transcodes to virtiofs at a 48% transcode rate without trouble. | +| D12 | Trickplay at full quality | Plex's 365G of BIF thumbnails is **format-incompatible** with Jellyfin's tiled JPEG trickplay. None of it is reusable; Jellyfin regenerates from scratch. | +| D13 | Wizarr invites issued **manually and selectively** | Most Plex users are dormant. No bulk invite generation, no scripted account creation, no broadcast announcement. ginnoir sends individual invites as people ask or as he chooses. | + +## 4. Architecture + +New Portainer stack `stacks/streaming/` — private `streaming` network plus `edge`, all +config via `env_file: stack.env` per the repo convention, no Portainer UI env vars. + +| Container | Image | Purpose | +|---|---|---| +| `jellyfin` | `lscr.io/linuxserver/jellyfin` | media server (PUID/PGID from `stack.env`) | +| `jellystat` | `cyfershepard/jellystat` | Tautulli replacement | +| `jellystat-db` | `postgres:16-alpine` | pinned + `com.centurylabs.watchtower.enable=false` per infra convention | +| `wizarr` | `ghcr.io/wizarrrr/wizarr` | invite + guided per-device onboarding, used ad-hoc | +| `jellyplex-watched` | `luigi311/jellyplex-watched` | scheduled two-way watch-state sync | + +### Ordering constraint + +Plexyfin and JellyPlex-Watched both read from the **live PMS API**. Everything that pulls +from Plex must complete before Plex is decommissioned. This is the hard sequencing rule of +the whole migration. + +## 5. Storage layout + +The root disk is the primary hazard, so Jellyfin's bulky data is split off deliberately. + +| Container path | Host path | Filesystem | Rationale | +|---|---|---|---| +| `/config` | `/config/jellyfin` | root SSD | SQLite DBs — better fsync semantics, easy to back up. A few GB. | +| `/config/metadata` | `/storage1/labdata/jellyfin/metadata` | virtiofs | artwork + NFO, grows unbounded | +| `/config/data/trickplay` | `/storage1/labdata/jellyfin/trickplay` | virtiofs | **~300-400G.** 10.11 moved trickplay here and it is *not* relocatable from the UI, hence an explicit sub-bind. | +| `/config/cache` | `/storage1/labdata/jellyfin/cache` | virtiofs | image cache | +| `/transcode` | `/storage1/labdata/jellyfin/transcodes` | virtiofs | see D11 | +| `/data/movies` | `/storage1/Movies` | virtiofs, **`:ro`** | Jellyfin never writes to the library | +| `/data/tv` | `/storage1/TV` | virtiofs, **`:ro`** | | +| `/data/anime` | `/storage1/Anime` | virtiofs, **`:ro`** | | + +Net effect: only a few GB ever lands on the 194G root disk. + +### Note on SQLite and virtiofs + +Plex has run its SQLite library DB on virtiofs here for months without incident, so this is +proven workable — the root-SSD placement is a preference, not a hard requirement. + +It carries one accepted trade-off. If `/storage1` fails to mount again (as on 2026-07-06), +Jellyfin with its DB on root will **start up healthy against an empty library** and flag the +entire catalogue as missing — the RomM failure mode. With the DB on `/storage1` it would +simply fail to start, which is the louder and safer failure. `storage1-guard.service` +already blocks guest start when `/storage1` isn't real, so root SSD stands. + +## 6. Caddy + +```caddy +# ============================================================= +# STREAMING — public +# ============================================================= +jellyfin.ginnoir.com { + reverse_proxy jellyfin:8096 +} + +wizarr.ginnoir.com { + reverse_proxy wizarr:5690 +} + +# ============================================================= +# STREAMING STATS — internal only +# ============================================================= +jellystat.ginnoir.com { + route { + import internal_gate + import authentik_outpost + handle { + import authentik_forward_auth + reverse_proxy jellystat:3000 + } + } +} +``` + +`jellyfin` and `wizarr` are public by design — remote users and invite recipients reach them +from anywhere. `jellystat` follows the tautulli/sonarr pattern: `internal_gate` first inside +the `route` block (an `import internal_only` above a catch-all route is dead code), then +Authentik forward auth. + +In Jellyfin → Networking, register the Caddy container as a known proxy and enable forwarded +headers, so real client IPs reach Jellystat and the remote-vs-LAN bitrate rules. + +## 7. Metadata strategy + +1. Jellyfin scans and fetches fresh from TMDB/TVDB — the same sources Plex used, so baseline + quality is identical. +2. **Plexyfin runs once**, overwriting posters/backdrops with the curated Plex artwork and + rebuilding collections. This is where the 15G `Metadata` investment is preserved. +3. **Plexyfin is then removed.** See D5. + +Plex's 365G of `Media` + `localhost` (BIF preview thumbnails) is not reusable — see D12. + +## 8. Plugins + +**Install** + +| Plugin | Purpose | Note | +|---|---|---| +| Plexyfin | collections + curated artwork from PMS | 3rd-party repo. Temporary — remove after Phase 2. | +| Intro Skipper | skip intros/recaps/credits | **Verify 10.11 compatibility at install.** The original is unmaintained; use the active fork. | +| AniDB / AniList | anime metadata | Anime library only | +| Trakt | per-user scrobbling | optional | + +**Deliberately skipped** + +| Plugin | Why | +|---|---| +| `jellyfin-plugin-sso` | browser-only OIDC — useless for Roku/Fire TV/tvOS (D3) | +| Open Subtitles | Bazarr already writes subtitles to disk and does it better | +| Playback Reporting | Jellystat supersedes it | + +## 9. Transcoding + +CPU-only on 52 threads: `veryfast` preset, per-session thread cap, throttling enabled, +**segment deletion on with a ~360s keep window**. Measured worst case is 7 concurrent streams +at ~48% transcode, 1080p SDR, no burn-in, no tone-mapping. + +Jellyfin's Android client (the heaviest platform here at 281 plays) direct-plays audio codecs +Plex's client refuses, so the transcode rate may **fall** after migration. Jellystat confirms +or refutes this during Phase 5. + +If it does strain: Intel Arc A310/A380 (D9). Needs a PCIe slot, an R730XD shutdown, and an +IOMMU passthrough to the valhalla VM — check the new card's IOMMU group, as only the P100's +group 7 is known clean. + +## 10. Phases + +Plex stays running and untouched through Phase 6. + +### Phase 0 — Disk remediation and baseline + +The root disk is at 97%. Nothing else may proceed until it isn't. + +- `docker image prune` / `builder prune` — ~51G reclaimable +- Audit the remaining 179G on `/dev/sda2`; identify anything else structurally growing +- Uptime Kuma monitor on root-disk free space + ntfy alert (this is the second root fill) +- Correct the Plex paths in `CLAUDE.md` (`/var/lib/plex` → `/storage1/labdata/plex`) + +**Done when:** root disk has comfortable headroom and alerts before it doesn't. + +### Phase 1 — Deploy the stack + +- `stacks/streaming/docker-compose.yml` + `stack.env`, `jellyfin` only +- Storage layout per §5; register the stack in Portainer (one-time, per the new-stack procedure) +- Caddy site blocks; `scripts/gen-bookmarks.ps1` +- Libraries: Movies, TV, Anime (Anime with anime metadata providers) +- Verify `check-portainer-stack-env.ps1` before pushing + +**Done when:** Jellyfin reachable at `jellyfin.ginnoir.com`, libraries scanned, no users. +**Rollback:** delete the stack. + +### Phase 2 — Parity + +- Plexyfin: sync collections + artwork from PMS, then **remove the plugin** +- **Trickplay generation** — full quality, thread-capped (~16-24 of 52), run before any users + arrive. Budget ~300-400G on `/storage1` and a multi-day first pass over ~1891 items. + Enable NVDEC for this: trickplay is decode + JPEG encode and needs no NVENC, so the P100 + helps here even though it's useless for live transcode. Schedule when llama-swap is idle. +- Remaining plugins per §8 +- Sonarr/Radarr/Bazarr → add Jellyfin library-refresh connections (leave the Plex ones for now) + +**Done when:** Jellyfin's library visually matches Plex's, trickplay complete. + +### Phase 3 — Watch-state sync + +- `jellyplex-watched` container, two-way, every 6h +- Plex admin token + Jellyfin API key in `stack.env`; user-mapping file where names differ +- Verify against ginnoir's own account in both directions + +**Done when:** a play on either server appears on the other within one cycle. +**Rollback:** stop the container. + +### Phase 4 — Pilot + +- Configure Wizarr (server connection, wizard steps per device) +- 3-5 willing users, **at least one each on Roku, Fire TV/Android TV and tvOS** +- Collect real feedback on playback, transcoding and the onboarding wizard + +**Done when:** every target platform has a confirmed working client. +**Rollback:** pilot users return to Plex, which never stopped. + +### Phase 5 — Selective migration + +Per D13, this phase is **demand-driven, not a campaign.** + +- Wizarr invites issued **individually**, as people ask or as ginnoir chooses +- Dormant accounts get nothing +- A short written explainer exists to accompany an invite — sent with invites, not broadcast +- Both servers live; run at least **4 weeks** to accumulate meaningful Jellystat data +- Watch transcode load and concurrency; decide on D9 + +**Done when:** the people who actually watch things are on Jellyfin and it's holding up. + +### Phase 6 — Cutover + +- Back up `/config/overseerr`, then reconfigure Seerr from Plex to Jellyfin (D10) +- Jellystat live at `jellystat.ginnoir.com` +- **Re-check Tautulli** for who is still on Plex and on what. Handle those users + individually — Tizen and PlayStation are the weak platforms (3 plays of 600 as of now). +- Remove Plex connections from Sonarr/Radarr/Bazarr +- Announce the Plex end date + +**Done when:** Seerr requests flow against Jellyfin and nobody depends on Plex. +**Rollback:** restore `/config/overseerr`. + +### Phase 7 — Decommission + +- `systemctl stop plexmediaserver && systemctl disable plexmediaserver` +- **2-week soak** with the service stopped but installed +- `pacman -Rns plex-media-server` +- Retire the `tautulli` container and its Caddy block +- Tarball `/storage1/labdata/plex`, keep 30 days, then reclaim **383G** +- Update `CLAUDE.md` (drop the external-services Plex section), bookmarks, Uptime Kuma monitors + +**Done when:** Plex is gone and nothing references it. +**Rollback (until the tarball expires):** reinstall the package, restore the data dir. + +## 11. Risks + +| Risk | Severity | Mitigation | +|---|---|---| +| Root disk at 97% | **high** | Phase 0 exists solely for this; monitor + alert added | +| Public Jellyfin is new attack surface Plex's relay hid | medium | Jellyfin's built-in brute-force lockout, strong admin credential, no anonymous access, monitoring | +| CPU transcode capacity at peak | medium | measured in Phase 5 via Jellystat; Arc A380 escape hatch (D9) | +| Seerr in-place conversion unsupported upstream | medium | config backup first, fresh-config fallback documented | +| Plexyfin is destructive | medium | run once in Phase 2, then remove (D5) | +| Trickplay generation starves live transcoding | low | thread-capped, runs before users arrive, NVDEC-assisted | +| Intro Skipper 10.11 compatibility | low | verify at install; it's optional polish | +| Holdouts on Tizen / PlayStation | low | 3 of 600 plays; handled individually at Phase 6 with fresh data | + +## 12. Out of scope + +- Music, Books and Stash content — Jellyfin serves Movies, TV and Anime only +- SSO for Jellyfin (D3) +- Migrating Seerr request history +- Replacing the P100 or restructuring GPU allocation +- The wider `/storage1` redundancy problem (tracked separately)