# romhacks stack

Automated ingestion of Pokémon rom-hacks from a Discord server's per-generation
threads into the RomM library.

```
romhacks (orchestrator)              jdownloader (jlesage)        RomM (roms stack)
  DiscordChatExporter  ── JSON ──▶   FolderWatch reads .crawljob,   scans
  parse links + art    ── crawljob ▶ resolves ~all hosts,   ──────▶ /storage1/Emulation
  stage metadata       ── art ─────▶ downloads to /output
```

## State: INERT

Ships idle and safe. It will not export or download until armed:
- `DISCORD_TOKEN` / `GUILD_ID` empty → orchestrator loop idles (see `run.sh`).
- `DRY_RUN=true` → even configured, it parses + stages metadata but writes **no**
  `.crawljob`, so JDownloader fetches nothing.

## Why this design

- **Member-only Discord access** → no official bot possible. DiscordChatExporter
  (DCE) does a **read-only** export with a **user token**. That violates Discord
  ToS, so this is wired for an **alt account** (joined to the server purely for
  this) to keep any ban risk off the main account.
- **Links span many file hosts** → JDownloader2 auto-resolves hundreds of hosts
  and handles waits/captcha via its noVNC UI; far more robust than per-host glue.
- **Art + blurb** → DCE `--media` downloads inline thread images; the orchestrator
  stages them with the thread name/description for a later RomM custom-cover
  hand-off (pairs with the `deck-sgdb` art pipeline).

## Arming checklist

1. **Alt account**: create it, join the server, grab its user token → `DISCORD_TOKEN`
   in `stack.env`. Set `GUILD_ID` (right-click server → Copy Server ID; Developer
   Mode on).
2. **Channels**: fill `orchestrator/channels.json` with the per-generation channel
   (or forum) IDs. Add any new file hosts you see to `hosts`.
3. **Register the stack** in Portainer once (new git stacks need a one-time
   registration — see memory `portainer-new-stack-registration`). Build is on:
   Portainer builds `homelab/romhacks-orchestrator` from `./orchestrator`.
4. **JDownloader FolderWatch**: open the noVNC UI at `http://valhalla:8998`
   (set `VNC_PASSWORD` first), Settings → Extensions → **Folder Watch** → enable,
   add watch folder `/watch`. This is what consumes the orchestrator's crawljobs.
5. **Dry run**: leave `DRY_RUN=true`, let one cycle run, inspect
   `/storage1/labdata/romhacks/metadata/*` — confirm names, blurbs, art, and that
   the extracted `links` look like real ROM links (not random URLs).
6. **Arm**: flip `DRY_RUN=false`. Downloads land in
   `/storage1/labdata/romhacks/incoming/<game>/`.
7. **Caddy (optional, internal-only)** — expose the JD UI on the LAN:
   ```
   jd.ginnoir.com {
       import internal_only
       reverse_proxy jdownloader:5800
   }
   ```
   then `./scripts/gen-bookmarks.ps1` and push (runner reloads Caddy).
8. **RomM hand-off** (next phase, not built yet): sort `incoming/<game>` into the
   right `/storage1/Emulation/<platform>` folder and push the staged art to RomM's
   custom-cover API.

## Editing config

`channels.json` and the scripts are **baked into the image** (Portainer's git-stack
checkout doesn't reliably bind sibling repo files — see the compose header). So a
change there needs a **push + rebuild**; if a content-only edit doesn't trigger a
rebuild, use Portainer → the stack → **Re-pull and redeploy**.

## Paths

| Host path | Purpose |
|---|---|
| `/config/romhacks/exports` | DCE JSON + downloaded media (SSD) |
| `/config/romhacks/state` | processed-message-id dedupe (SSD) |
| `/config/romhacks/crawljobs` | orchestrator → JDownloader FolderWatch (`/watch`) |
| `/config/romhacks/jd` | JDownloader config (SSD) |
| `/storage1/labdata/romhacks/incoming` | JDownloader downloads (ZFS) |
| `/storage1/labdata/romhacks/metadata` | staged name/blurb/art per game (ZFS) |

## Gotchas

- **ToS / ban risk** is real but low for read-only export. Use the alt account.
- **Pre-patched ROMs vs patches**: bundled base-game ROMs are piracy; IPS/BPS
  patches are broadly tolerated. Prefer patch links where the thread offers both.
- DCE flags / dll path are pinned in `orchestrate.py`; verify on first run (the
  code globs for the dll if the pinned path moves between image versions).