Compare commits

...
2 Commits
Author SHA1 Message Date
ginnoirandClaude Opus 4.8 809600a04d feat: ROM-hack dedup, conflict resolution, and vault library_path resync
Catalog cleanup pass across gb/gbc/gba/nds/3ds:
- Standardize nonstandard hack filenames to `Pokemon - <Name> [Hack].<ext>`;
  remove verified byte-identical duplicates (conform-rom-names.sh).
- Resolve 9 version-conflicts via side-by-side emulator comparison
  (resolve-version-conflicts.sh): promote newer builds, keep genuine
  distinct hacks, fix a mislabeled Dark Rising slot (was the Digimon
  crossover "Worlds Collide"), and rebuild Mega Power 5.77 + HeartGold
  Generations v2.0 fresh from their patches.
- Resync 286 vault notes' `library_path` to the real on-disk [Hack] files
  and fix 15 stale platform fields (resync-vault-library-paths.py).
- Regenerate MOC tables + wiki/catalog.json (396 notes).

Also includes pending Playnite PC-hack setup/repair scripts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 14:29:05 -05:00
ginnoirandClaude Opus 4.8 ae56d101b7 refactor: standardize hack filenames to [Hack] and flatten to platform root
Switch the library naming convention from "Pokemon - <Name> (Hack)" to
"Pokemon - <Name> [Hack][version][tags]", and place hacks in the platform
root (roms/<plat>/) instead of a Hacks/ subfolder. Applied across the live
valhalla library (272 hack/fan-game files) via scripts/standardize-names.py.

- romhack-import.py / romhack-fetch.py: emit [Hack], write to roms/<plat>/
- romhack-pc-migrate.py: emit [Hack]
- build-romhack-vault.py (archived): update documented library_path
- standardize-names.py: new one-shot, self-fetching standardizer (idempotent;
  pipes \n-only bytes over SSH to avoid Windows CRLF corrupting filenames)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 18:08:31 -05:00
14 changed files with 867 additions and 454 deletions
+3
View File
@@ -22,3 +22,6 @@ Thumbs.db
.scrape/ .scrape/
.scrape_lib/ .scrape_lib/
scripts/__pycache__/ scripts/__pycache__/
# Per-machine ROM compare/patch scratch (side-by-side emulator runs, patch builds).
.compare/
+2 -2
View File
@@ -10,7 +10,7 @@ valhalla homelab library. Spun off from the `homelabstack` repo on 2026-06-06.
> or the `vault` skill. New hacks are added there **manually** (see the vault's > or the `vault` skill. New hacks are added there **manually** (see the vault's
> `_Claude.md` checklist). This repo is just the tooling. > `_Claude.md` checklist). This repo is just the tooling.
The ROMs live on valhalla under `/storage1/Emulation/roms/<platform>/Hacks/`, The ROMs live on valhalla under `/storage1/Emulation/roms/<platform>/`,
served by the RomM stack (`roms.ginnoir.com`) — deployed from `homelabstack` served by the RomM stack (`roms.ginnoir.com`) — deployed from `homelabstack`
(`stacks/roms/`). (`stacks/roms/`).
@@ -76,7 +76,7 @@ vault `Wanted` list.
They are served through the **same RomM stack** as the console hacks, under the They are served through the **same RomM stack** as the console hacks, under the
`windows` platform (RomM's "Microsoft Windows"; note `pc` is excluded in RomM's `windows` platform (RomM's "Microsoft Windows"; note `pc` is excluded in RomM's
`config.yml`, `windows` is not). Each hack is a single archive in the platform `config.yml`, `windows` is not). Each hack is a single archive in the platform
**root**`Pokemon - <Hack> (Hack).zip` — NOT in a subfolder (a subfolder would **root**`Pokemon - <Hack> [Hack].zip` — NOT in a subfolder (a subfolder would
make RomM treat the whole thing as one multi-file game). make RomM treat the whole thing as one multi-file game).
```bash ```bash
+2 -2
View File
@@ -319,7 +319,7 @@ def gen_hack_note(h):
f'status: "{status}"', f'status: "{status}"',
f"type: {yaml_list(types)}", f"type: {yaml_list(types)}",
f'generation: "{gen}"', f'generation: "{gen}"',
f'library_path: "roms/{plat}/Hacks/{libname}"', f'library_path: "roms/{plat}/{libname}"',
f'source: "{url or lookup_url(title)}"', f'source: "{url or lookup_url(title)}"',
f"added: {ADDED}", f"added: {ADDED}",
f"tags: {yaml_list(tags)}", f"tags: {yaml_list(tags)}",
@@ -343,7 +343,7 @@ def gen_hack_note(h):
f"- Lookup: {lookup_url(title)}", f"- Lookup: {lookup_url(title)}",
"", "",
"## In the library", "## In the library",
f"- `roms/{plat}/Hacks/{libname}` on valhalla", f"- `roms/{plat}/{libname}` on valhalla",
f"- Patch/source artifact archived under `/storage1/igir/romhacks/patches/`", f"- Patch/source artifact archived under `/storage1/igir/romhacks/patches/`",
"", "",
"[[Index|← back to directory]]", "[[Index|← back to directory]]",
+89
View File
@@ -0,0 +1,89 @@
#!/usr/bin/env bash
# Conform nonstandard ROM-hack filenames to the `Pokemon - <Name> [Hack].<ext>`
# standard, and remove verified byte-identical duplicates. Runs on valhalla.
# DRY_RUN=1 bash conform-rom-names.sh # preview (default)
# DRY_RUN=0 bash conform-rom-names.sh # execute
set -u
ROOT=/storage1/Emulation/roms
DRY_RUN=${DRY_RUN:-1}
del(){ # del <platform> <file> — delete only if it still exists
local f="$ROOT/$1/$2"
if [ ! -f "$f" ]; then echo "SKIP del (missing): $1/$2"; return; fi
if [ "$DRY_RUN" = 1 ]; then echo "DEL $1/$2"; else rm -- "$f" && echo "DEL $1/$2"; fi
}
mv1(){ # mv1 <platform> <src> <dst> — rename only if dst does NOT exist
local s="$ROOT/$1/$2" d="$ROOT/$1/$3"
if [ ! -f "$s" ]; then echo "SKIP mv (missing src): $1/$2"; return; fi
if [ -e "$d" ]; then echo "SKIP mv (dst exists): $1/$3"; return; fi
if [ "$DRY_RUN" = 1 ]; then echo "MV $1/$2 -> $3"; else mv -n -- "$s" "$d" && echo "MV $1/$2 -> $3"; fi
}
echo "=== DELETE: verified byte-identical duplicates of an existing [Hack] file ==="
del gba "sors v1.3 .gba"
del gba "Pokemon FireRed Rocket Edition.gba"
del gba "Pokemon SOTS (v2.1.2).gba"
del gba "U_E_Inclement_Emerald_Beta_1.1.3.gba"
del gba "flora-sky-main-completed.gba"
del gba "Giratina Strikes Back Full.gba"
del gba "PK EMR Kaizo v2.0.gba"
echo
echo "=== RENAME: GBA ==="
mv1 gba "CHROME1.5.gba" "Pokemon - Chrome [Hack].gba"
mv1 gba "DarkVioletFinal2018V1.gba" "Pokemon - Dark Violet [Hack].gba"
mv1 gba "Energized v1.0.21.gba" "Pokemon - Energized [Hack].gba"
mv1 gba "Exceeded v11.5.gba" "Pokemon - Exceeded [Hack].gba"
mv1 gba "Hisui Red.gba" "Pokemon - Hisui Red [Hack].gba"
mv1 gba "Leaf Green Extreme Randomizer.gba" "Pokemon - Leaf Green Extreme Randomizer [Hack].gba"
mv1 gba "Moon_Galaxy_Completed.gba" "Pokemon - Moon Galaxy [Hack].gba"
mv1 gba "PKMN EMR PARTY RANDOMIZER.GBA" "Pokemon - Emerald Party Randomizer [Hack].gba"
mv1 gba "PerfectE v5.4 PSS.gba" "Pokemon - Perfect Emerald [Hack].gba"
mv1 gba "Pokemon Fire Red 898 Randomizer v1.2.3.gba" "Pokemon - Fire Red 898 Randomizer [Hack].gba"
mv1 gba "Pokemon Fusion Origins Ver 3.8.gba" "Pokemon - Fusion Origins [Hack].gba"
mv1 gba "Pokemon Ultimate Fusion.gba" "Pokemon - Ultimate Fusion [Hack].gba"
mv1 gba "Pokemon_Infernal_Legend_beta_2.1.GBA" "Pokemon - Infernal Legend [Hack].gba"
mv1 gba "Pokemon_Sunset_Orange_a1.0.gba" "Pokemon - Sunset Orange [Hack].gba"
mv1 gba "QuetzalSpanishAlpha8v4.gba" "Pokemon - Quetzal (Spanish) [Hack].gba"
mv1 gba "autumn orange v1.2 casual mode.gba" "Pokemon - Autumn Orange [Hack].gba"
mv1 gba "fr multiverse v1.6.gba" "Pokemon - FireRed Multiverse [Hack].gba"
mv1 gba "saffron-d2.gba" "Pokemon - Saffron [Hack].gba"
mv1 gba "the-unown-king.gba" "Pokemon - The Unown King [Hack].gba"
mv1 gba "wings of chaos alpha v2.5.gba" "Pokemon - Wings of Chaos [Hack].gba"
echo
echo "=== RENAME: 3DS ==="
mv1 3ds "Pokemon Binary Sun v1d3d1 - Patched by Ducumon.3ds" "Pokemon - Binary Sun [Hack].3ds"
mv1 3ds "Pokemon Omega Ruby QoL.cxi" "Pokemon - Omega Ruby QoL [Hack].cxi"
echo
echo "=== RENAME: NDS ==="
mv1 nds "Rainbow-Gold-v1.5.nds" "Pokemon - Rainbow Gold [Hack].nds"
mv1 nds "goldenshield.nds" "Pokemon - Golden Shield [Hack].nds"
mv1 nds "Pokemon Definitive HeartGold.nds" "Pokemon - Definitive HeartGold [Hack].nds"
mv1 nds "Pokemon Horror White v1.nds" "Pokemon - Horror White [Hack].nds"
mv1 nds "Pokemon Moon Black 2 (v4.2.3).nds" "Pokemon - Moon Black 2 [Hack].nds"
mv1 nds "Pokemon Black 2 Redux Complete (v1.4.1).nds" "Pokemon - Black 2 Redux [Hack].nds"
echo
echo "=== RENAME: phase 2 — web/header-identified hacks ==="
# Hyper EMR LA = Hyper Emerald v5.7 "Lost Artifacts" (Destvol et al.)
mv1 gba "Hyper EMR LA v5.7 bugfix 2.gba" "Pokemon - Hyper Emerald Lost Artifacts [Hack].gba"
# header BPEE/FireRed-base confirmations below
mv1 gba "darkfireb2.1.3.gba" "Pokemon - DarkFire [Hack].gba"
mv1 gba "MioPidgey.gba" "Pokemon - MioPidgey [Hack].gba"
mv1 gba "re universe.gba" "Pokemon - Re Universe [Hack].gba"
mv1 gba "TRE Johto Release April 30, 2025.gba" "Pokemon - Team Rocket Edition (DragonsDen) [Hack].gba"
# Elysium ships split A/B; this file is the Part B half
mv1 gba "Elysium Part B (v2.0).gba" "Pokemon - Elysium (Part B) [Hack].gba"
# ScalretFinal: product code CTR-P-ECLA = Alpha Sapphire base; "Scalret" = Scarlet
mv1 3ds "ScalretFinal.3ds" "Pokemon - Scarlet [Hack].3ds"
echo
echo "=== RENAME: phase 3 — user-confirmed IDs ==="
mv1 gba "Spirits.gba" "Pokemon - Spirits of the Storm [Hack].gba"
mv1 nds "EotS-v1.01.nds" "Pokemon - Mystery Dungeon - Explorers of the Spirit [Hack].nds"
mv1 gba "all-in-v1.0.gba" "Pokemon - All In [Hack].gba"
echo
echo "Done (DRY_RUN=$DRY_RUN)."
+67
View File
@@ -0,0 +1,67 @@
<#
.SYNOPSIS
Undo Playnite damage from setup-playnite-pc-hack.ps1 LiteDB writes.
Fixes:
1. Removes broken GameScannerConfig (null emulator, scoop\apps) from scanners.db
2. Clears active library filter that restricts view to RomM only
3. Disables pc_windows RomM mapping until emulator is recreated in Playnite UI
Playnite must be fully closed before running.
#>
$ErrorActionPreference = 'Stop'
if (Get-Process -Name 'Playnite*' -ErrorAction SilentlyContinue) {
throw 'Close Playnite completely, then re-run.'
}
$playniteConfig = Join-Path $env:LOCALAPPDATA 'Playnite\config.json'
$scannersDb = Join-Path $env:LOCALAPPDATA 'Playnite\library\scanners.db'
$rommConfig = Join-Path $env:LOCALAPPDATA 'Playnite\ExtensionsData\9700aa21-447d-41b4-a989-acd38f407d9f\config.json'
$liteDbDll = Join-Path $env:LOCALAPPDATA 'Playnite\LiteDB.dll'
$orphanEmuId = 'fa12bc5e-dbb9-4d21-b1e6-a67c2fa6a513'
$nullEmuId = '00000000-0000-0000-0000-000000000000'
# --- 1. Remove broken emulator scanner ---
Add-Type -Path $liteDbDll
$db = [LiteDB.LiteDatabase]::new("Filename=$scannersDb")
$col = $db.GetCollection('GameScannerConfig')
$removed = 0
foreach ($doc in @($col.FindAll())) {
$emuId = if ($doc['EmulatorId'].IsGuid) { $doc['EmulatorId'].AsGuid.ToString() } else { $doc['EmulatorId'].AsString }
$dir = if ($doc.ContainsKey('Directory')) { $doc['Directory'].AsString } else { '' }
if ($emuId -eq $nullEmuId -or $dir -like '*\scoop\apps') {
$col.Delete($doc['_id']) | Out-Null
$removed++
Write-Host "Removed broken scanner: Directory=$dir EmulatorId=$emuId"
}
}
$db.Dispose()
if ($removed -eq 0) { Write-Host 'No broken GameScannerConfig entries found.' }
# --- 2. Clear RomM-only library filter ---
$cfg = Get-Content -Raw $playniteConfig | ConvertFrom-Json
if ($cfg.FilterSettings.Library -and $cfg.FilterSettings.Library.Ids -and $cfg.FilterSettings.Library.Ids.Count -gt 0) {
$ids = @($cfg.FilterSettings.Library.Ids)
Write-Host "Clearing library filter (was: $($ids -join ', '))"
$cfg.FilterSettings.Library.Ids = @()
}
$cfg | ConvertTo-Json -Depth 20 | Set-Content -Encoding UTF8 $playniteConfig
Write-Host 'Playnite config: library filter cleared.'
# --- 3. Disable orphan pc_windows RomM mapping ---
$romm = Get-Content -Raw $rommConfig | ConvertFrom-Json
$pc = $romm.Mappings | Where-Object { $_.PlatformId -eq 'pc_windows' } | Select-Object -First 1
if ($pc) {
if ($pc.EmulatorId -eq $orphanEmuId) {
$pc.Enabled = $false
Write-Host "Disabled pc_windows RomM mapping (orphan emulator $orphanEmuId)."
Write-Host 'Re-enable after adding Pokemon PC Hack Launcher in Playnite UI.'
} else {
Write-Host "pc_windows mapping uses emulator $($pc.EmulatorId) - left enabled."
}
$romm | ConvertTo-Json -Depth 8 -Compress | Set-Content -Encoding UTF8 $rommConfig
}
Write-Host "`nDone. Open Playnite - full library should be visible again."
Write-Host 'Emulated-game scan errors on startup should be gone.'
+29
View File
@@ -0,0 +1,29 @@
# Rebuild emulators.db from readable BSON docs (fixes LiteDB page corruption).
$ErrorActionPreference = 'Stop'
Get-Process -Name 'Playnite*' -ErrorAction SilentlyContinue | Stop-Process -Force
Start-Sleep -Seconds 2
$lib = Join-Path $env:LOCALAPPDATA 'Playnite\library'
$src = Join-Path $lib 'emulators.db'
$new = Join-Path $lib 'emulators.db.new'
$bak = Join-Path $lib 'emulators.db.broken'
Add-Type -Path (Join-Path $env:LOCALAPPDATA 'Playnite\LiteDB.dll')
if (Test-Path $new) { Remove-Item -Force $new }
$srcDb = [LiteDB.LiteDatabase]::new("Filename=$src")
$dstDb = [LiteDB.LiteDatabase]::new("Filename=$new")
$srcCol = $srcDb.GetCollection('Emulator')
$dstCol = $dstDb.GetCollection('Emulator')
$n = 0
foreach ($doc in $srcCol.FindAll()) {
$dstCol.Insert($doc) | Out-Null
$n++
}
$srcDb.Dispose()
$dstDb.Dispose()
Move-Item -Force $src $bak
Move-Item -Force $new $src
Write-Host "Rebuilt emulators.db from $n documents."
Write-Host "Old file: $bak"
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env bash
# Resolve the side-by-side version-conflict decisions (pairs 1-4,6-8; 5 & 9 were
# swapped live via fresh patches). Guarded: del only if present, mv only if dst free.
# DRY_RUN=1 bash resolve-version-conflicts.sh (default)
# DRY_RUN=0 bash resolve-version-conflicts.sh
set -u
G=/storage1/Emulation/roms/gba
DRY_RUN=${DRY_RUN:-1}
del(){ local f="$1/$2"; if [ ! -f "$f" ]; then echo "SKIP del (missing): $2"; return; fi
if [ "$DRY_RUN" = 1 ]; then echo "DEL $2"; else rm -- "$f" && echo "DEL $2"; fi; }
mv1(){ local s="$1/$2" d="$1/$3"; if [ ! -f "$s" ]; then echo "SKIP mv (missing src): $2"; return; fi
if [ -e "$d" ]; then echo "SKIP mv (dst exists): $3"; return; fi
if [ "$DRY_RUN" = 1 ]; then echo "MV $2 -> $3"; else mv -n -- "$s" "$d" && echo "MV $2 -> $3"; fi; }
echo "=== deletes (older/superseded loose copies) ==="
del "$G" "Pokemon - Unbound [Hack].gba" # pair2: older 2.1.1, replaced by loose 2.1.1.1
del "$G" "supermariomon v1.3.gba" # pair6: older than catalog 1.5
del "$G" "HnS_v1.0.gba" # pair7: older than catalog 1.2.1
del "$G" "advanceredux 31-3-25 (2).gba" # pair8: 2020 build, catalog is 2026
echo
echo "=== pair3 Dark Rising: fix the mislabeled Digimon file, then promote the real DR1 ==="
mv1 "$G" "Pokemon - Dark Rising [Hack].gba" "Pokemon vs Digimon - Worlds Collide [Hack].gba"
mv1 "$G" "PokemonDarkRising1Complete.GBA" "Pokemon - Dark Rising [Hack].gba"
echo
echo "=== pair2 Unbound: promote the newer loose 2.1.1.1 ==="
mv1 "$G" "Pokemon Unbound.gba" "Pokemon - Unbound [Hack].gba"
echo
echo "=== pair1 GS Chronicles: keep both, disambiguate the Gold inc build ==="
mv1 "$G" "Pokemon GS Chronicles (Beta 2.7.1).gba" "Pokemon - GS Chronicles (Gold inc) [Hack].gba"
echo
echo "=== pair4 Dark Worship: keep both, conform the Super Dark Worship name ==="
mv1 "$G" "Super Dark Worship Oficial.gba" "Pokemon - Super Dark Worship [Hack].gba"
echo
echo "Done (DRY_RUN=$DRY_RUN)."
+131
View File
@@ -0,0 +1,131 @@
#!/usr/bin/env python3
"""Resync each vault hack note's `library_path` to the actual on-disk ROM file.
The [Hack] filename standardization renamed ROMs on disk but left the vault
notes' `library_path` frontmatter pointing at old names/paths. This matches each
note to its real file (by normalized title, falling back to the old path's
basename) and rewrites `library_path` + any exact body copy of the old value.
Read-only by default; pass --apply to write. ROM listings are read from
/tmp/romsync/<plat>.txt (one filename per line), pulled from valhalla.
"""
import os, re, sys, glob
VAULT = r"C:\Users\MattC\Documents\Obsidian Vault"
HACKS = os.path.join(VAULT, "Pokémon ROM Hacks", "Hacks")
LISTS = os.path.join(os.path.dirname(__file__), "..", ".scrape", "romsync")
PLAT = {"GBA": "gba", "GBC": "gbc", "GB": "gb", "NDS": "nds", "3DS": "3ds", "PC": "windows"}
APPLY = "--apply" in sys.argv
def norm(s):
s = os.path.splitext(s)[0].lower()
s = re.sub(r"\[[^\]]*\]", "", s) # drop [Hack], [2.0.5], [Full], ...
s = re.sub(r"\([^)]*\)", "", s) # drop (Gold inc), (Spanish), ...
s = re.sub(r"pok[eé]?mon", "", s) # drop the franchise prefix/word
s = re.sub(r"[^a-z0-9]", "", s) # keep bare digits (Dark Rising 2)
return s
# ---- build per-platform file index: normkey -> [filenames] ----
idx = {}
for path in glob.glob(os.path.join(LISTS, "*.txt")):
plat = os.path.splitext(os.path.basename(path))[0]
idx[plat] = {}
for line in open(path, encoding="utf-8"):
fn = line.strip()
if not fn or not re.search(r"\.(gba|gbc|gb|nds|3ds|cci|cxi|zip)$", fn, re.I):
continue
idx[plat].setdefault(norm(fn), []).append(fn)
def fm_get(text, key):
m = re.search(rf"(?m)^{key}:\s*(.*)$", text)
return m.group(1).strip().strip('"').strip("'") if m else None
FUZZY = "--fuzzy" in sys.argv # also write the proposed fuzzy/cross-platform matches
def find_exact_all(key):
"""Exact normalized-key match across ALL platforms (catches wrong platform field)."""
hits = []
for p, files in idx.items():
for fn in files.get(key, []):
hits.append((p, fn))
return hits
# Notes whose only fuzzy candidate is a different/ambiguous hack — held for manual
# review, never auto-applied even with --fuzzy.
FUZZY_HOLD = {
"Bloody Fusion.md", "Too Many Types.md", "HeartGold Generation.md",
"FireRed Reignited LeafGreen Regrown.md", "Kanto Complete.md",
"Mystery Dungeon - Wigglytuff's Bizarre Adventure.md",
}
def find_fuzzy(nk):
"""One key is a substring of the other; require a [Hack] file; must be unique.
Skip very short keys (e.g. a hack literally named 'Z') to avoid spurious hits."""
if len(nk) < 4: return []
hits = []
for p, files in idx.items():
for k, fns in files.items():
if len(k) < 4 or not nk: continue
if nk in k or k in nk:
for fn in fns:
if "[hack]" in fn.lower():
hits.append((p, fn))
return sorted(set(hits))
changed = unchanged = unmatched = ambiguous = 0
report = {"CHANGED": [], "UNMATCHED": [], "AMBIGUOUS": [], "FUZZY": []}
for md in sorted(glob.glob(os.path.join(HACKS, "*.md"))):
text = open(md, encoding="utf-8").read()
title = fm_get(text, "title") or os.path.splitext(os.path.basename(md))[0]
platform = (fm_get(text, "platform") or "").upper()
cur = fm_get(text, "library_path") or ""
plat = PLAT.get(platform)
note = os.path.basename(md)
if not plat or plat not in idx:
continue
def write_path(newpath):
if not APPLY: return
new = re.sub(r"(?m)^(library_path:\s*).*$", lambda m: m.group(1) + newpath, text, count=1)
# Update body copies ONLY when the old value was an emulation-rom path;
# never touch /storage1/labdata/... served-files links.
if cur and cur.startswith(("roms/", "/storage1/Emulation/")):
new = new.replace(cur, newpath)
open(md, "w", encoding="utf-8", newline="").write(new)
nk = norm(title)
# 1) exact match within the declared platform (auto-apply)
cand = idx[plat].get(nk) or (idx[plat].get(norm(os.path.basename(cur))) if cur else None)
if cand and len(cand) == 1:
newpath = f"roms/{plat}/{cand[0]}"
if cur == newpath: unchanged += 1; continue
changed += 1; report["CHANGED"].append((note, cur, newpath)); write_path(newpath); continue
if cand and len(cand) > 1:
ambiguous += 1; report["AMBIGUOUS"].append((note, cand)); continue
# 2) cross-platform exact, then fuzzy substring -> PROPOSAL (apply only with --fuzzy)
prop = find_exact_all(nk) or find_fuzzy(nk)
if len(prop) == 1:
p2, fn = prop[0]; newpath = f"roms/{p2}/{fn}"
if cur == newpath: unchanged += 1; continue
report["FUZZY"].append((note, cur, newpath))
if FUZZY and note not in FUZZY_HOLD: changed += 1; write_path(newpath)
continue
unmatched += 1; report["UNMATCHED"].append((note, platform, cur))
summary = (f"{'APPLIED' if APPLY else 'DRY-RUN'}{' +FUZZY' if FUZZY else ''}: {changed} changed, "
f"{unchanged} already correct, {len(report['FUZZY'])} fuzzy proposals, "
f"{unmatched} unmatched (no on-disk file — likely Wanted/gated), {ambiguous} ambiguous")
lines = [summary, ""]
for tag in ("AMBIGUOUS", "FUZZY", "CHANGED", "UNMATCHED"):
rows = report[tag]
if not rows: continue
lines.append(f"--- {tag} ({len(rows)}) ---")
for r in rows:
if tag in ("CHANGED", "FUZZY"): lines.append(f" {r[0]}\n {r[1] or ''} -> {r[2]}")
elif tag == "AMBIGUOUS": lines.append(f" {r[0]} -> {r[1]}")
else: lines.append(f" {r[0]} [{r[1]}] (cur: {r[2] or ''})")
lines.append("")
rep = os.path.join(os.path.dirname(__file__), "..", ".scrape", "resync-report.txt")
open(rep, "w", encoding="utf-8").write("\n".join(lines))
print(summary)
print("full report ->", os.path.normpath(rep))
+3 -3
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Acquire, patch, validate, and place worthwhile Pokémon ROM-hacks into the """Acquire, patch, validate, and place worthwhile Pokémon ROM-hacks into the
EmuDeck library at /storage1/Emulation/roms/<platform>/Hacks/ on valhalla. EmuDeck library at /storage1/Emulation/roms/<platform>/ on valhalla.
Each hack is one MANIFEST entry naming its source. Two source kinds: Each hack is one MANIFEST entry naming its source. Two source kinds:
- "rom": a pre-patched full ROM is downloaded and placed as-is (validated). - "rom": a pre-patched full ROM is downloaded and placed as-is (validated).
@@ -242,7 +242,7 @@ def main():
raise RuntimeError(f"validation: {verr}") raise RuntimeError(f"validation: {verr}")
crc = zlib.crc32(open(src, "rb").read()) & 0xffffffff crc = zlib.crc32(open(src, "rb").read()) & 0xffffffff
if place: if place:
dst_dir = os.path.join(ROMS, plat, "Hacks") dst_dir = os.path.join(ROMS, plat)
os.makedirs(dst_dir, exist_ok=True) os.makedirs(dst_dir, exist_ok=True)
shutil.copy2(src, os.path.join(dst_dir, out_name)) shutil.copy2(src, os.path.join(dst_dir, out_name))
detail += " | PLACED" detail += " | PLACED"
@@ -253,7 +253,7 @@ def main():
for hk, pl, st, dt, nm in rows: for hk, pl, st, dt, nm in rows:
print(f"[{st:4}] {pl:4} {hk:<20} {dt}") print(f"[{st:4}] {pl:4} {hk:<20} {dt}")
if st == "OK": if st == "OK":
print(f" -> {pl}/Hacks/{nm}") print(f" -> {pl}/{nm}")
print(f"\nplaced: {'YES (--place)' if place else 'NO (dry run; pass --place)'}") print(f"\nplaced: {'YES (--place)' if place else 'NO (dry run; pass --place)'}")
if __name__ == "__main__": if __name__ == "__main__":
+10 -10
View File
@@ -2,12 +2,12 @@
"""Import a drop-folder of mixed Pokémon hack files into the valhalla library. """Import a drop-folder of mixed Pokémon hack files into the valhalla library.
Handles a folder containing any mix of: Handles a folder containing any mix of:
- completed ROMs (.gba/.gbc/.gb/.nds) -> validated, clean-named, copied to roms/<plat>/Hacks/ - completed ROMs (.gba/.gbc/.gb/.nds) -> validated, clean-named, copied to roms/<plat>/
- patches (.ips/.bps/.ups/.xdelta) -> applied to an owned base, placed in Hacks/, archived - patches (.ips/.bps/.ups/.xdelta) -> applied to an owned base, placed in roms/<plat>/, archived
- documentation (.pdf/.txt/.png/.md) -> archived under PATCHES/_docs/ - documentation (.pdf/.txt/.png/.md) -> archived under PATCHES/_docs/
- archives (.zip/.rar/.7z) -> extracted and recursed (same rules), source kept - archives (.zip/.rar/.7z) -> extracted and recursed (same rules), source kept
Naming matches the rest of the library: "Pokemon - <Hack> (Hack).<ext>" Naming matches the rest of the library: "Pokemon - <Hack> [Hack].<ext>"
(version/junk parentheticals are stripped from the source filename). (version/junk parentheticals are stripped from the source filename).
Base ROM for a patch is auto-detected: BPS/UPS embed source CRC32 so the correct Base ROM for a patch is auto-detected: BPS/UPS embed source CRC32 so the correct
@@ -89,7 +89,7 @@ NAME_OVERRIDES = [
] ]
# Files we will NOT auto-place (unidentified / need user to name). Reported instead. # Files we will NOT auto-place (unidentified / need user to name). Reported instead.
SKIP_NAME_SUBSTR = ["beta 15 + expansion"] SKIP_NAME_SUBSTR = ["beta 15 + expansion"]
# Known base ROMs in a drop folder — place under roms/<plat>/, not Hacks/ # Known base ROMs in a drop folder — placed under roms/<plat>/ (un-tagged, no [Hack])
BASE_ROM_FILES = [ BASE_ROM_FILES = [
("fire red (j)", "gba", "Pokemon - Fire Red (J) (V1.0).gba"), ("fire red (j)", "gba", "Pokemon - Fire Red (J) (V1.0).gba"),
] ]
@@ -205,7 +205,7 @@ def place_rom(src, plat, hackname, origin):
if is_skip_name(origin): if is_skip_name(origin):
report.append(("ROM", "NEEDS-ID", origin, "unidentified hack — tell me the name", "")) report.append(("ROM", "NEEDS-ID", origin, "unidentified hack — tell me the name", ""))
return return
out_name = f"Pokemon - {hackname} (Hack).{plat}" out_name = f"Pokemon - {hackname} [Hack].{plat}"
err, padto = validate(src, plat) err, padto = validate(src, plat)
if err: if err:
report.append(("ROM", "FAIL", origin, err, "")); return report.append(("ROM", "FAIL", origin, err, "")); return
@@ -213,11 +213,11 @@ def place_rom(src, plat, hackname, origin):
if padto and len(data) < padto: if padto and len(data) < padto:
data += b"\xff" * (padto - len(data)) data += b"\xff" * (padto - len(data))
if PLACE: if PLACE:
d = os.path.join(ROMS, plat, "Hacks"); os.makedirs(d, exist_ok=True) d = os.path.join(ROMS, plat); os.makedirs(d, exist_ok=True)
with open(os.path.join(d, out_name), "wb") as f: with open(os.path.join(d, out_name), "wb") as f:
f.write(data) f.write(data)
report.append(("ROM", "OK", origin, f"{len(data):,}b crc={zlib.crc32(data)&0xffffffff:08x}", report.append(("ROM", "OK", origin, f"{len(data):,}b crc={zlib.crc32(data)&0xffffffff:08x}",
f"{plat}/Hacks/{out_name}")) f"{plat}/{out_name}"))
def try_apply(patch, base_key, out): def try_apply(patch, base_key, out):
@@ -314,8 +314,8 @@ def process_file(path, origin=None):
def place_pc_archive(path, hackname, origin, variant=None): def place_pc_archive(path, hackname, origin, variant=None):
tag = variant or detect_variant(path) or "" tag = variant or detect_variant(path) or ""
suffix = f" {tag}" if tag else "" suffix = tag if tag else ""
out_name = f"Pokemon - {hackname} (Hack){suffix}.zip" out_name = f"Pokemon - {hackname} [Hack]{suffix}.zip"
dest_dir = pc_dest_dir(tag) dest_dir = pc_dest_dir(tag)
dest = os.path.join(dest_dir, out_name) dest = os.path.join(dest_dir, out_name)
ext = os.path.splitext(path)[1].lower() ext = os.path.splitext(path)[1].lower()
@@ -352,7 +352,7 @@ def place_pc_archive(path, hackname, origin, variant=None):
def place_pc_installer(path, hackname, origin): def place_pc_installer(path, hackname, origin):
"""Wrap a standalone .exe installer in a zip so RomM can serve it.""" """Wrap a standalone .exe installer in a zip so RomM can serve it."""
tag = detect_variant(path) or "[Installer]" tag = detect_variant(path) or "[Installer]"
out_name = f"Pokemon - {hackname} (Hack) {tag}.zip" out_name = f"Pokemon - {hackname} [Hack]{tag}.zip"
dest_dir = pc_dest_dir(tag) dest_dir = pc_dest_dir(tag)
dest = os.path.join(dest_dir, out_name) dest = os.path.join(dest_dir, out_name)
tmp_zip = dest + ".part" tmp_zip = dest + ".part"
+2 -2
View File
@@ -8,7 +8,7 @@ and were never folded into RomM. This script places them as single archives into
the `windows` RomM platform so RomM catalogs them and the Playnite plugin can the `windows` RomM platform so RomM catalogs them and the Playnite plugin can
download + extract + launch them like any other game. download + extract + launch them like any other game.
Target: /storage1/Emulation/roms/windows/Pokemon - <Hack> (Hack).<ext> Target: /storage1/Emulation/roms/windows/Pokemon - <Hack> [Hack].<ext>
- `windows` is the RomM "Microsoft Windows" platform (NOT excluded in RomM's - `windows` is the RomM "Microsoft Windows" platform (NOT excluded in RomM's
config.yml; `pc` IS excluded, so do not use it). config.yml; `pc` IS excluded, so do not use it).
- Files go in the platform ROOT (mirrors the console-hack convention); a - Files go in the platform ROOT (mirrors the console-hack convention); a
@@ -166,7 +166,7 @@ def main():
src_ext = os.path.splitext(chosen)[1].lower() src_ext = os.path.splitext(chosen)[1].lower()
will_repack = repack and src_ext in REPACK_EXT will_repack = repack and src_ext in REPACK_EXT
out_ext = ".zip" if will_repack else src_ext out_ext = ".zip" if will_repack else src_ext
out_name = f"Pokemon - {clean_name(title)} (Hack){out_ext}" out_name = f"Pokemon - {clean_name(title)} [Hack]{out_ext}"
dest = os.path.join(DEST, out_name) dest = os.path.join(DEST, out_name)
note = " [repack rar/7z->zip]" if will_repack else ( note = " [repack rar/7z->zip]" if will_repack else (
" [RAR as-is; extract manually if Playnite balks]" if src_ext == ".rar" else "") " [RAR as-is; extract manually if Playnite balks]" if src_ext == ".rar" else "")
+31 -105
View File
@@ -1,127 +1,53 @@
<# <#
.SYNOPSIS .SYNOPSIS
Configure Playnite + RomM plugin for Pokémon PC hacks (windows / pc_windows). Configure RomM plugin mapping for PC hacks. Emulator must be added in Playnite UI (see below).
Playnite must be fully closed before running. Playnite must be fully closed before running (only touches RomM plugin config.json).
#> #>
$ErrorActionPreference = 'Stop' $ErrorActionPreference = 'Stop'
$PlayniteDir = Join-Path $env:LOCALAPPDATA 'Playnite' $RommConfig = Join-Path $env:LOCALAPPDATA 'Playnite\ExtensionsData\9700aa21-447d-41b4-a989-acd38f407d9f\config.json'
$LiteDbDll = Join-Path $PlayniteDir 'LiteDB.dll'
$EmuDb = Join-Path $PlayniteDir 'library\emulators.db'
$RommConfig = Join-Path $PlayniteDir 'ExtensionsData\9700aa21-447d-41b4-a989-acd38f407d9f\config.json'
$Launcher = 'C:\Users\MattC\Documents\pokemon\scripts\launch-pc-hack.ps1'
$DestPath = 'C:\Users\MattC\roms\windows' $DestPath = 'C:\Users\MattC\roms\windows'
$EmuName = 'Pokemon PC Hack Launcher'
$PlatformSpec = 'pc_windows' $PlatformSpec = 'pc_windows'
$ProfileName = 'Default'
$ProfileId = [guid]::NewGuid().ToString()
if (Get-Process -Name 'Playnite*' -ErrorAction SilentlyContinue) { if (-not (Test-Path $RommConfig)) { throw "RomM config not found: $RommConfig" }
throw 'Close Playnite completely, then re-run this script.'
}
foreach ($p in @($LiteDbDll, $Launcher, $RommConfig)) {
if (-not (Test-Path $p)) { throw "Missing: $p" }
}
$pwshCmd = Get-Command pwsh -ErrorAction SilentlyContinue $emu = Get-Content -Raw $RommConfig | ConvertFrom-Json | ForEach-Object { $_.Mappings } |
if ($pwshCmd) { $pwsh = $pwshCmd.Source } else { $pwsh = (Get-Command powershell).Source } Where-Object { $_.PlatformId -eq $PlatformSpec } | Select-Object -First 1
Add-Type -Path $LiteDbDll if (-not $emu -or -not $emu.EmulatorId) {
Write-Host @'
$installDir = Split-Path -Parent $Launcher Add the Playnite emulator first (Library -> Configure Emulators -> Add):
$arguments = "-ExecutionPolicy Bypass -File `"$Launcher`" `"{ImagePath}`""
function New-BsonArray([object[]]$items) { Name: Pokemon PC Hack Launcher
$list = New-Object 'System.Collections.Generic.List[LiteDB.BsonValue]' Install: C:\Users\MattC\Documents\pokemon\scripts
foreach ($item in $items) { Profile: Default
if ($item -is [LiteDB.BsonDocument]) { $list.Add($item) } Platforms: PC (Windows)
elseif ($item -is [LiteDB.BsonValue]) { $list.Add($item) } Extensions: zip
else { $list.Add([LiteDB.BsonValue]$item) } Executable: C:\Program Files\PowerShell\7\pwsh.exe
} Arguments: -ExecutionPolicy Bypass -File "C:\Users\MattC\Documents\pokemon\scripts\launch-pc-hack.ps1" "{ImagePath}"
return [LiteDB.BsonArray]::new($list) Working dir: {InstallDir}
}
$profileDoc = [LiteDB.BsonDocument]::new() Then in RomM plugin settings, map pc_windows -> that emulator, destination
$profileDoc['_id'] = [LiteDB.BsonValue][guid]::Parse($ProfileId) '@ + $DestPath + @', Auto-extract ON.
$profileDoc['Name'] = $ProfileName
$profileDoc['Platforms'] = New-BsonArray @($PlatformSpec)
$profileDoc['ImageExtensions'] = New-BsonArray @('zip')
$profileDoc['Executable'] = $pwsh
$profileDoc['Arguments'] = $arguments
$profileDoc['WorkingDirectory'] = '{InstallDir}'
$profileDoc['UseShellExecute'] = $false
$profileDoc['OverrideDefaultArgs'] = $false
$profileDoc['ScriptStartup'] = $false
$profileDoc['ScriptGameImport'] = $false
$profileDoc['TrackingMode'] = 0
$profileDoc['TrackingPath'] = $null
$db = [LiteDB.LiteDatabase]::new("Filename=$EmuDb") Or re-run this script after the emulator exists (it reads EmulatorId from an existing pc_windows mapping).
try {
$col = $db.GetCollection('Emulator')
$existing = $col.FindOne([LiteDB.Query]::EQ('Name', $EmuName))
if ($existing) {
$emuDoc = $existing
$emuId = $existing['_id'].AsGuid.ToString()
if ($emuId -eq '00000000-0000-0000-0000-000000000000') {
$emuId = $existing['_id'].AsString
}
$profiles = $emuDoc['CustomProfiles'].AsArray
if ($profiles.Count -gt 0) {
$p0 = $profiles[0]
if ($p0['_id'].IsGuid) { $ProfileId = $p0['_id'].AsGuid.ToString() }
else { $ProfileId = $p0['_id'].AsString }
$profileDoc['_id'] = [LiteDB.BsonValue][guid]::Parse($ProfileId)
}
Write-Host "Updating existing emulator '$EmuName' ($emuId)"
} else {
$emuId = [guid]::NewGuid().ToString()
$emuDoc = [LiteDB.BsonDocument]::new()
$emuDoc['_id'] = [LiteDB.BsonValue][guid]::Parse($emuId)
$profileDoc['_id'] = [LiteDB.BsonValue][guid]::Parse($ProfileId)
Write-Host "Creating emulator '$EmuName' ($emuId)"
}
$emuDoc['Name'] = $EmuName '@
$emuDoc['InstallDir'] = $installDir exit 1
$emuDoc['BuiltInConfigId'] = $null
$profilesList = New-Object 'System.Collections.Generic.List[LiteDB.BsonValue]'
$profilesList.Add($profileDoc)
$emuDoc['CustomProfiles'] = [LiteDB.BsonArray]::new($profilesList)
$col.Upsert($emuDoc) | Out-Null
} finally {
$db.Dispose()
} }
$cfg = Get-Content -Raw $RommConfig | ConvertFrom-Json $cfg = Get-Content -Raw $RommConfig | ConvertFrom-Json
$mapping = $cfg.Mappings | Where-Object { $_.PlatformId -eq $PlatformSpec } | Select-Object -First 1 $m = $cfg.Mappings | Where-Object { $_.PlatformId -eq $PlatformSpec } | Select-Object -First 1
if ($mapping) { if (-not $m) {
$mapping.EmulatorId = $emuId Write-Host 'No pc_windows mapping yet — add it in RomM plugin settings after creating the emulator.'
$mapping.EmulatorProfileId = $ProfileId exit 1
$mapping.DestinationPath = $DestPath
$mapping.AutoExtract = $true
$mapping.Enabled = $true
$mapping.SyncSaves = $false
Write-Host 'Updated RomM mapping for pc_windows.'
} else {
$cfg.Mappings += [pscustomobject]@{
MappingId = [guid]::NewGuid()
Enabled = $true
AutoExtract = $true
UseM3u = $false
SyncSaves = $false
SaveStrategy = 0
SaveDirOverride = $null
EmulatorId = $emuId
EmulatorProfileId = $ProfileId
PlatformId = $PlatformSpec
DestinationPath = $DestPath
}
Write-Host 'Added RomM mapping for pc_windows.'
} }
$m.DestinationPath = $DestPath
$m.AutoExtract = $true
$m.Enabled = $true
$m.SyncSaves = $false
New-Item -ItemType Directory -Force -Path $DestPath | Out-Null New-Item -ItemType Directory -Force -Path $DestPath | Out-Null
$cfg | ConvertTo-Json -Depth 8 -Compress | Set-Content -Encoding UTF8 $RommConfig $cfg | ConvertTo-Json -Depth 8 -Compress | Set-Content -Encoding UTF8 $RommConfig
Write-Host "Playnite configured. ROM download dir: $DestPath" Write-Host "RomM pc_windows mapping updated (destination $DestPath, AutoExtract on)."
Write-Host "Open Playnite -> Menu -> Library -> Import RomM library (windows games)."
+93
View File
@@ -0,0 +1,93 @@
#!/usr/bin/env python3
"""
Standardize valhalla Hack-library filenames to:
Pokemon - <Name> [Hack][<version>][<functional tags>].<ext>
Rule-based + self-fetching: queries valhalla for every file still tagged
"(Hack)" anywhere under the roms tree and rewrites it.
- mechanical: " (Hack)" -> " [Hack]" (preserves other (...) like (FRLG+))
- OVERRIDES: platform-port fan-games whose stem carries "-<ver>-<os>"
get the version pulled into [ver] and the OS kept as a tag.
- SKIP: non-game patcher utilities are left untouched.
Dry-run prints OLD -> NEW; --apply runs `mv -n` over SSH.
Idempotent: already-[Hack] files don't match the (Hack) query, so re-runs
are no-ops.
"""
import subprocess, sys, shlex, os
APPLY = "--apply" in sys.argv
HOST = "valhalla"
ROOT = "/storage1/Emulation/roms"
# Non-game utilities — leave alone.
SKIP = {
"Pokemon - DeltaPatcherLite (Hack) [Installer].zip",
"Pokemon - Patcher (Hack) [Installer].zip",
}
# Stem-encoded version/OS ports -> canonical [Hack][ver][OS].
OVERRIDES = {
"Pokemon - Reborn-19.5.0-linux (Hack) [Linux].zip": "Pokemon - Reborn [Hack][19.5.0][Linux].zip",
"Pokemon - Rejuvenation-13.5.0-linux (Hack) [Linux].zip": "Pokemon - Rejuvenation [Hack][13.5.0][Linux].zip",
"Pokemon - Reborn-19.5.0-macos (Hack) [macOS].zip": "Pokemon - Reborn [Hack][19.5.0][macOS].zip",
"Pokemon - Rejuvenation-13.5.0-macos (Hack) [macOS].zip": "Pokemon - Rejuvenation [Hack][13.5.0][macOS].zip",
}
def new_name(base):
if base in SKIP:
return None
if base in OVERRIDES:
return OVERRIDES[base]
if " (Hack)" in base:
return base.replace(" (Hack)", " [Hack]")
return None
def fetch():
cmd = f"find {shlex.quote(ROOT)} -type f -name '*(Hack)*'"
out = subprocess.run(["ssh", HOST, cmd], capture_output=True, text=True, check=True).stdout
return sorted(p for p in out.splitlines() if p.strip())
def main():
paths = fetch()
cmds, special, mech, skipped = [], [], 0, []
for full in paths:
d, base = os.path.dirname(full), os.path.basename(full)
nn = new_name(base)
if nn is None:
skipped.append(base)
continue
cmds.append(f"mv -n -- {shlex.quote(full)} {shlex.quote(d + '/' + nn)}")
if base in OVERRIDES:
special.append((base, nn))
else:
mech += 1
print(f"Found {len(paths)} files still tagged (Hack).\n")
print(f"Mechanical (Hack)->[Hack] swaps : {mech}")
print(f"Special version/OS overrides : {len(special)}")
for o, n in special:
print(f" {o}\n -> {n}")
print(f"Skipped (non-game utilities) : {len(skipped)}")
for s in skipped:
print(f" {s}")
if not APPLY:
print("\n(dry-run — pass --apply to execute over SSH)")
return
# NB: pipe bytes with explicit \n line endings. On Windows, text=True wraps
# stdin in a TextIOWrapper that rewrites \n -> \r\n, which bash would append
# as a stray CR onto every destination filename. Encoding ourselves avoids it.
script = "set -e\n" + "\n".join(cmds) + "\n"
r = subprocess.run(["ssh", HOST, "bash -s"], input=script.encode("utf-8"))
print(f"\napplied {len(cmds)} renames (exit {r.returncode})")
sys.exit(r.returncode)
if __name__ == "__main__":
main()
+365 -329
View File
File diff suppressed because it is too large Load Diff