ginnoir
ba207a0fcd
Inert by default: empty token keeps the orchestrator idle and DRY_RUN=true suppresses downloads. Orchestrator exports per-generation forum threads via DiscordChatExporter, extracts download links + art, stages metadata, and writes JDownloader crawljobs. Armed here with an alt account token and all 11 romhack forums, still in DRY_RUN pending first-cycle validation.
84 lines
3.8 KiB
YAML
84 lines
3.8 KiB
YAML
# romhacks stack — automated Pokémon rom-hack ingestion from a Discord server's
|
|
# per-generation threads into the RomM library.
|
|
#
|
|
# FLOW
|
|
# romhacks (orchestrator) jdownloader (jlesage) RomM (roms stack)
|
|
# ┌───────────────────────┐ ┌──────────────────┐ ┌──────────────┐
|
|
# │ DiscordChatExporter ──▶│ JSON │ FolderWatch picks│ files │ scans │
|
|
# │ parse links + art ───▶│ crawljob│ up .crawljob, │ ───────▶ │ /storage1/ │
|
|
# │ stage metadata ───▶│ ──────▶ │ resolves ~all │ │ Emulation │
|
|
# └───────────────────────┘ │ hosts, downloads │ └──────────────┘
|
|
# └──────────────────┘
|
|
#
|
|
# WHY THIS SHAPE
|
|
# - Member-only access => no official Discord bot is possible. DiscordChatExporter
|
|
# (DCE) does a READ-ONLY export with a USER token. Self-token use violates
|
|
# Discord ToS, so this is wired for an ALT account (see stack.env) to isolate
|
|
# ban risk from the main account.
|
|
# - The download links span many file hosts (Mega/Drive/MediaFire/Pixeldrain/...),
|
|
# so JDownloader2 (jlesage image) does the actual fetching — it auto-resolves
|
|
# hundreds of hosts and handles waits/captcha via its noVNC UI.
|
|
# - DCE `--media` downloads the inline thread images (box art / screenshots); the
|
|
# orchestrator stages those + the thread blurb as metadata for a later RomM
|
|
# custom-cover hand-off (dovetails with the deck-sgdb art pipeline).
|
|
#
|
|
# INERT BY DEFAULT. Does nothing until armed: DISCORD_TOKEN/GUILD_ID are empty and
|
|
# DRY_RUN=true, so even if deployed it idles and never enqueues a download. See
|
|
# README.md for the arming checklist.
|
|
#
|
|
# BUILD (not bind) for the orchestrator: Portainer git-stack checkouts do NOT
|
|
# reliably materialize sibling repo files at relative FILE bind paths (Docker then
|
|
# auto-creates a directory — see the `share` stack note). So the orchestrator's
|
|
# scripts + channels.json are baked into a built image from ./orchestrator instead
|
|
# of bind-mounted. Editing channels.json => push => Portainer rebuild (use
|
|
# "Re-pull and redeploy" if a content-only change doesn't trigger a rebuild).
|
|
#
|
|
# Tiered binds: working state (exports/state/crawljobs + JD config) -> /config
|
|
# (SSD); bulk output + art -> /storage1/labdata/romhacks (ZFS, root-auto-created).
|
|
#
|
|
# Only jdownloader joins `edge` (Caddy proxies its noVNC UI, internal-only).
|
|
|
|
services:
|
|
romhacks:
|
|
build:
|
|
context: ./orchestrator
|
|
image: homelab/romhacks-orchestrator:latest
|
|
container_name: romhacks
|
|
restart: unless-stopped
|
|
networks: [romhacks]
|
|
env_file:
|
|
- stack.env
|
|
volumes:
|
|
- /config/romhacks/exports:/exports
|
|
- /config/romhacks/state:/state
|
|
- /config/romhacks/crawljobs:/crawljobs
|
|
- /storage1/labdata/romhacks/metadata:/metadata
|
|
|
|
jdownloader:
|
|
image: jlesage/jdownloader-2:latest
|
|
container_name: jdownloader
|
|
restart: unless-stopped
|
|
networks: [romhacks, edge]
|
|
env_file:
|
|
- stack.env
|
|
environment:
|
|
# root so it can write to the ZFS-tier /output bind (dirs auto-created as root)
|
|
- USER_ID=0
|
|
- GROUP_ID=0
|
|
volumes:
|
|
- /config/romhacks/jd:/config
|
|
# shared with the orchestrator: it drops .crawljob files here; the JD
|
|
# FolderWatch extension must be enabled + pointed at /watch (one-time, README).
|
|
- /config/romhacks/crawljobs:/watch
|
|
- /storage1/labdata/romhacks/incoming:/output
|
|
ports:
|
|
- "8998:5800"
|
|
|
|
networks:
|
|
romhacks:
|
|
name: romhacks
|
|
driver: bridge
|
|
edge:
|
|
name: edge
|
|
external: true
|