Files

T

ginnoir b7ff9b2411 docs: add R730XD post-migration task plan

2026-06-27 05:05:30 -05:00

22 KiB

Raw Permalink Blame History

R730XD Post-Migration Host Tasks — Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Close out the five remaining post-migration items on the Proxmox host valhalla-dev: a permanent fix for virtiofsd FD exhaustion, the hookscript bug, GPU thermal safety, container GPU access, and retiring the last failing pool disk.

Architecture: These are five independent host-ops tasks against a live single-host Proxmox box (no application code, no test suite). Each is self-contained and can be executed in any order, with two dependencies noted below. "Tests" here are verification commands with expected output; "rollback" replaces "revert commit." Most changes are on the Proxmox host (.68) or the valhalla VM (.69), neither of which is tracked in this git repo — persistence is via on-host files + the vault migration note [[Server Migration (Proxmox R730XD)]].

Tech Stack: Proxmox VE (Debian) host, virtiofsd 1.13.2, ZFS (storage1 stripe), EndeavourOS guest VM (id 100), NVIDIA Tesla P100 + driver 580.159.04 + nvidia-ctk 1.19.1, systemd, ipmitool fan control.

Access & conventions (read first)

Host (valhalla-dev, Proxmox): wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "<cmd>"' — the WSL id_ed25519 (ginnoir@TELLUS) key is authorized for root.
Guest (valhalla VM): wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "<cmd>"' (sudo is NOPASSWD).
For multi-line remote scripts use the heredoc form: wsl.exe -- bash -lc 'ssh ... root@192.168.1.68 bash -s' <<'EOF' … EOF (avoids shell-variable mangling through the WSL→ssh layers).
GateGuard: the first Bash and every Write/Edit requires stating the user request + what the op does/affects before it runs.

Recommended sequence & dependencies

Task 1 (virtiofsd --inode-file-handles) and Task 2 (hookscript fix) are both the FD-exhaustion fix — do them together. Task 1 is the real fix; Task 2 repairs the fallback. Both require a VM restart, so batch them into one VM bounce.
Task 3 (GPU fan safety) is a hard prerequisite for Task 4 (container GPU) being used under load — never run a sustained CUDA job until Task 3 is live (passive P100 + Dell-auto fans that can't see GPU temp = overheat risk).
Task 5 (replace sdb) is fully independent and starts a ~1–1.5 day resilver — run it when power is stable.

Task 1: Permanent virtiofsd FD fix — `--inode-file-handles=prefer`

Problem: virtiofsd holds one O_PATH file descriptor per inode the guest touches on /storage1. With ~78 TB of media scanned by *arr/Plex it climbed to ~1,000,000 open FDs and the guest got Too many open files on new /storage1 access. --inode-file-handles=prefer makes virtiofsd use name_to_handle_at/open_by_handle_at instead, keeping near-zero FDs. Proxmox 9 has no config option to pass this flag, so we inject it via a dpkg-divert wrapper around the virtiofsd binary (survives package updates).

Targets (host .68):

Divert: /usr/libexec/virtiofsd → /usr/libexec/virtiofsd.real
Create: /usr/libexec/virtiofsd (wrapper script)
Restart: VM 100 (cold)
Step 1: Baseline — capture the current launch args and FD count

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
echo "=== current virtiofsd cmdline ==="
for p in $(pgrep virtiofsd); do tr '\0' ' ' </proc/$p/cmdline; echo; done
echo "=== worker FD count (the one that climbs) ==="
for p in $(pgrep virtiofsd); do echo "pid $p fds=$(ls /proc/$p/fd 2>/dev/null | wc -l)"; done
EOF

Expected: two virtiofsd processes, args include --shared-dir=/storage1 --xattr; worker FD count is whatever it has grown to (could be tens of thousands+). Record it.

Step 2: Confirm the binary supports the flag

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "/usr/libexec/virtiofsd --help 2>&1 | grep -- --inode-file-handles"'

Expected: a line containing --inode-file-handles=<INODE_FILE_HANDLES>. (Confirmed present on 1.13.2.)

Step 3: Create the diversion and the wrapper

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
set -e
# Move the real binary aside, persistently across apt upgrades
dpkg-divert --add --rename --divert /usr/libexec/virtiofsd.real /usr/libexec/virtiofsd
# Write a wrapper that injects the flag, then execs the real binary with all original args
cat > /usr/libexec/virtiofsd <<'WRAP'
#!/bin/bash
# Proxmox-launched virtiofsd wrapper: force inode-file-handles to stop FD growth.
# See docs/superpowers/plans/2026-06-26-r730xd-post-migration-tasks.md Task 1.
exec /usr/libexec/virtiofsd.real --inode-file-handles=prefer "$@"
WRAP
chmod 0755 /usr/libexec/virtiofsd
echo "=== verify ==="
dpkg-divert --list /usr/libexec/virtiofsd
ls -l /usr/libexec/virtiofsd /usr/libexec/virtiofsd.real
EOF

Expected: diversion listed (diversion of /usr/libexec/virtiofsd to /usr/libexec/virtiofsd.real), wrapper is a 0755 regular file, .real is the original ELF binary.

Step 4: Cold-restart VM 100 so a new virtiofsd launches via the wrapper

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
qm shutdown 100 --timeout 180 && qm status 100
qm start 100 && qm status 100
EOF

Expected: status: stopped then status: running.

Step 5: VERIFY — new virtiofsd carries the flag and FDs stay low

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
echo "=== flag present in launch args? ==="
for p in $(pgrep virtiofsd); do tr '\0' ' ' </proc/$p/cmdline | grep -o -- '--inode-file-handles=prefer' && echo " (pid $p OK)"; done
echo "=== /storage1 still works in guest ==="
ssh -o BatchMode=yes -o ConnectTimeout=8 -i /root/.ssh/id_ed25519 ginnoir@192.168.1.69 'ls /storage1 >/dev/null 2>&1 && echo storage1-OK || echo storage1-FAIL' 2>/dev/null || echo "(set up host->guest key per Task 3, or check from the admin box)"
EOF

Expected: --inode-file-handles=prefer (pid … OK) for the worker. If the host→guest key isn't set yet, verify /storage1 from the admin box instead: wsl.exe -- bash -lc 'ssh ... ginnoir@192.168.1.69 "ls /storage1 >/dev/null && echo OK"'.

Step 6: Soak check — FD count after the library gets re-scanned

Re-run Step 1's FD-count command after a Plex/*arr scan cycle (or ~an hour of normal use). Expected: worker FD count stays in the hundreds/low-thousands, not climbing toward 1M. This confirms the fix.

Rollback:

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
rm -f /usr/libexec/virtiofsd
dpkg-divert --remove --rename /usr/libexec/virtiofsd
qm shutdown 100 --timeout 180 && qm start 100
EOF

Persist: copy the wrapper to this repo under docs/host/usr-libexec-virtiofsd.wrapper (reference only) and note the diversion in the vault migration page.

Task 2: Fix the `virtiofsd-limits.sh` hookscript bug

Problem: qm start 100 logs hookscript error for 100 on post-start: /var/lib/vz/snippets/virtiofsd-limits.sh: line 19: exit: 0: numeric argument required. Line 19 is exit 0 — the "0 not numeric" error means a stray carriage return (\r) is attached (exit $'0\r'). The script is the FD-limit fallback (raises virtiofsd nofile to 10M); after Task 1 it's a belt-and-suspenders safety net, but it should run cleanly.

Targets (host .68):

Rewrite: /var/lib/vz/snippets/virtiofsd-limits.sh (clean LF, correct content)
Step 1: Confirm the CRLF/stray-char hypothesis

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "file /var/lib/vz/snippets/virtiofsd-limits.sh; grep -c $'"'"'\r'"'"' /var/lib/vz/snippets/virtiofsd-limits.sh"'

Expected: file reports ... with CRLF line terminators and/or a non-zero \r count. (If zero, the bug is a different stray char — proceed to Step 2 anyway; the rewrite fixes it regardless.)

Step 2: Rewrite the script cleanly (LF only)

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
cp /var/lib/vz/snippets/virtiofsd-limits.sh /var/lib/vz/snippets/virtiofsd-limits.sh.bak
cat > /var/lib/vz/snippets/virtiofsd-limits.sh <<'SCRIPT'
#!/bin/bash
# Fallback: raise virtiofsd open-file limit after VM 100 starts.
# Primary fix is the --inode-file-handles=prefer wrapper (Task 1); this is a safety net.
VMID="$1"
PHASE="$2"
[ "$VMID" = "100" ] && [ "$PHASE" = "post-start" ] || exit 0
sleep 3
pgrep virtiofsd | while read -r pid; do
    prlimit --pid "$pid" --nofile=10000000:10000000 \
      && logger -t virtiofsd-limits "raised nofile for PID $pid to 10M"
done
exit 0
SCRIPT
chmod 0755 /var/lib/vz/snippets/virtiofsd-limits.sh
# guarantee no CRLF crept back in
sed -i 's/\r$//' /var/lib/vz/snippets/virtiofsd-limits.sh
echo "=== syntax check ==="
bash -n /var/lib/vz/snippets/virtiofsd-limits.sh && echo "syntax OK"
file /var/lib/vz/snippets/virtiofsd-limits.sh
EOF

Expected: syntax OK and file reports a plain Bourne-Again shell script, ASCII text executable (no CRLF).

Step 3: VERIFY — dry-run the hook and confirm no error

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "/var/lib/vz/snippets/virtiofsd-limits.sh 100 post-start; echo exit=$?"'

Expected: exit=0, no numeric argument required error. (It will also bump the live virtiofsd nofile to 10M — harmless.)

Step 4: VERIFY at next real start (do this opportunistically with Task 1's restart)

After any qm start 100, check the journal:

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "journalctl -u qemu-server@100 -n 30 --no-pager | grep -iE \"hookscript|virtiofsd-limits\" | tail"'

Expected: no hookscript error; a virtiofsd-limits: raised nofile … logger line.

Rollback: cp /var/lib/vz/snippets/virtiofsd-limits.sh.bak /var/lib/vz/snippets/virtiofsd-limits.sh

Task 3: GPU thermal safety — wire P100 temp into the fan daemon

Problem: The P100 is passively cooled. Fans are on Dell-auto, which cannot read the GPU's temperature (non-Dell card), and the custom valhalla-thermal-monitor.service (the CPU/GPU-reactive controller with a watchdog→Dell-auto failsafe) is currently disabled (shut off during the June panics). The daemon already supports GPU temp via a VM_SSH env var (gpu_max() runs $VM_SSH 'nvidia-smi --query-gpu=temperature.gpu …'), but VM_SSH is unset → gpu=off. We set up host→guest SSH, point VM_SSH at it, and re-enable the daemon so GPU temp drives the fans. This must be live before any sustained GPU workload.

Targets:

Create (host): /root/.ssh/id_ed25519 keypair (if absent) for host→guest SSH
Modify (guest .69): ~ginnoir/.ssh/authorized_keys (append host pubkey)
Modify (host): valhalla-thermal-monitor.service drop-in with Environment=VM_SSH=…
Enable + start: valhalla-thermal-monitor.service
Step 1: Generate a host→guest SSH key (host side)

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
[ -f /root/.ssh/id_ed25519 ] || ssh-keygen -t ed25519 -N "" -C "valhalla-dev-host->vm" -f /root/.ssh/id_ed25519
echo "=== host pubkey ==="
cat /root/.ssh/id_ed25519.pub
EOF

Expected: prints an ssh-ed25519 … valhalla-dev-host->vm pubkey. Copy this line.

Step 2: Authorize that key on the guest (bridge via the admin/WSL key)

Substitute <HOST_PUBKEY> with the line from Step 1:

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "mkdir -p ~/.ssh && chmod 700 ~/.ssh && grep -qxF \"<HOST_PUBKEY>\" ~/.ssh/authorized_keys 2>/dev/null || echo \"<HOST_PUBKEY>\" >> ~/.ssh/authorized_keys; chmod 600 ~/.ssh/authorized_keys; echo done"'

Expected: done.

Step 3: VERIFY host→guest SSH + nvidia-smi works non-interactively

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "ssh -o BatchMode=yes -o ConnectTimeout=5 -i /root/.ssh/id_ed25519 ginnoir@192.168.1.69 \"nvidia-smi --query-gpu=temperature.gpu --format=csv,noheader,nounits\""'

Expected: a bare integer like 56. If it errors, fix the key before continuing — do not enable the daemon without a working GPU temp source.

Step 4: Set VM_SSH on the service via a drop-in

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
mkdir -p /etc/systemd/system/valhalla-thermal-monitor.service.d
cat > /etc/systemd/system/valhalla-thermal-monitor.service.d/gpu.conf <<'DROP'
[Service]
Environment=VM_SSH=ssh -o BatchMode=yes -o ConnectTimeout=5 -i /root/.ssh/id_ed25519 ginnoir@192.168.1.69
DROP
systemctl daemon-reload
echo "=== drop-in ==="; systemctl cat valhalla-thermal-monitor.service | grep -i VM_SSH
EOF

Expected: the Environment=VM_SSH=… line echoed back.

Step 5: Enable + start the daemon

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "systemctl enable --now valhalla-thermal-monitor.service; systemctl is-active valhalla-thermal-monitor.service"'

Expected: active.

Step 6: VERIFY — GPU temp is in the curve, and the failsafe is intact

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "journalctl -t thermal-monitor -n 8 --no-pager"'

Expected: lines like cpu=NNC gpu=NNC -> fan NN% with a real number for gpu= (not na), and the startup line shows gpu=on.

Step 7: VERIFY the stop/crash failsafe returns fans to Dell-auto

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "systemctl cat valhalla-thermal-monitor.service | grep -iE \"ExecStopPost|trap\"; echo --- ; grep -nE \"0x30 0x30 0x01 0x01|trap\" /usr/local/sbin/valhalla-thermal-monitor.sh"'

Expected: confirms an ExecStopPost/trap that issues ipmitool raw 0x30 0x30 0x01 0x01 (return to Dell auto). This was previously proven; just re-confirm it's present.

Rollback:

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "systemctl disable --now valhalla-thermal-monitor.service; rm -f /etc/systemd/system/valhalla-thermal-monitor.service.d/gpu.conf; systemctl daemon-reload; ipmitool raw 0x30 0x30 0x01 0x01"'

(The final ipmitool call forces fans back to Dell auto.)

Task 4: Container GPU access (nvidia-ctk + CDI) in the guest

Problem: The guest has nvidia-ctk 1.19.1 and a working driver, but Docker has only the runc runtime and no CDI spec, so containers can't use the P100. Generate a CDI spec and enable CDI in Docker so workloads request the GPU with --device nvidia.com/gpu=all. Prerequisite: Task 3 must be live before running any non-trivial GPU container. Restarting Docker bounces all ~73 containers — do it in a maintenance window.

Targets (guest .69):

Create: /etc/cdi/nvidia.yaml (CDI spec)
Modify: /etc/docker/daemon.json (enable CDI feature)
Restart: docker service
Step 1: Baseline — Docker version (CDI needs Engine ≥ 25) and current runtimes

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "docker version --format \"{{.Server.Version}}\"; docker info 2>/dev/null | grep -iE \"Runtimes|Default Runtime\""'

Expected: a server version. If ≥ 25.0, use CDI (Steps 2–5). If older, use the classic nvidia runtime instead (see Alternative below).

Step 2: Generate the CDI spec

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "sudo mkdir -p /etc/cdi && sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml && sudo nvidia-ctk cdi list"'

Expected: nvidia-ctk cdi list shows nvidia.com/gpu=all (and nvidia.com/gpu=0).

Step 3: Enable the CDI feature in Docker

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 bash -s' <<'EOF'
sudo install -d /etc/docker
# merge "features.cdi=true" into daemon.json (create if absent)
sudo python3 - <<'PY'
import json,os
p="/etc/docker/daemon.json"
d=json.load(open(p)) if os.path.exists(p) and os.path.getsize(p) else {}
d.setdefault("features",{})["cdi"]=True
json.dump(d,open(p,"w"),indent=2)
print(open(p).read())
PY
EOF

Expected: prints a daemon.json containing "features": { "cdi": true }.

Step 4: Restart Docker (MAINTENANCE WINDOW — bounces all containers)

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "sudo systemctl restart docker && sleep 20 && echo running:$(docker ps -q|wc -l)"'

Expected: Docker restarts; running container count climbs back toward ~72 over the next minute.

Step 5: VERIFY — a CUDA container sees the P100

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "docker run --rm --device nvidia.com/gpu=all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi --query-gpu=name --format=csv,noheader"'

Expected: Tesla P100-PCIE-16GB. Watch the host thermal log (Task 3) during any longer test to confirm fans respond to GPU temp.

Alternative (Docker < 25, classic runtime):

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 ginnoir@192.168.1.69 "sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker && docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi -L"'

Expected: GPU 0: Tesla P100-PCIE-16GB (UUID: …).

Rollback: remove /etc/cdi/nvidia.yaml, revert daemon.json (drop features.cdi), sudo systemctl restart docker.

Task 5: Replace the last failing disk `sdb` with the staged 4 TB

Problem: storage1 is still DEGRADED because sdb (4 TB WD Red, serial WD-WCC4E4TSK1S2, "too many errors") is failing. A new 4 TB disk (serial V6HXW23W) is already installed in the old Slot 2 (currently sdc, not in the pool). zpool replace swaps it in. This starts a ~1–1.5 day resilver over a no-redundancy stripe — run only when power is stable, and keep sdb seated during the resilver (pulling it first loses its data).

Targets (host .68): storage1 pool — no files.

Step 1: Resolve the new disk's stable by-id (do NOT use bare sdc — letters shuffle on reboot)

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "ls -l /dev/disk/by-id/ | grep -iE \"V6HXW23W\" | grep -vE \"part\""'

Expected: one or more by-id symlinks for serial V6HXW23W (e.g. wwn-0x… / ata-…V6HXW23W). Pick the wwn-… or ata-… whole-disk link → call it $NEWDISK (full path /dev/disk/by-id/<that>).

Step 2: Pre-flight — confirm target is sdb and new disk is unused

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 bash -s' <<'EOF'
echo "=== sdb serial (must be WD-WCC4E4TSK1S2) ==="; lsblk -dno NAME,SERIAL | awk '$1=="sdb"'
echo "=== new disk NOT in pool ==="; zpool status -P storage1 | grep -q "V6HXW23W" && echo "ALREADY IN POOL?!" || echo "new disk not in pool (good)"
echo "=== pool baseline ==="; zpool status storage1 | grep -E "state:|scan:"
EOF

Expected: sdb serial is WD-WCC4E4TSK1S2; new disk not in pool; no active scan.

Step 3: Kick off the replace (substitute $NEWDISK from Step 1)

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "zpool replace storage1 sdb /dev/disk/by-id/<NEWDISK>; echo exit=$?; zpool status storage1 | grep -A4 replacing"'

Expected: exit=0; status now shows a replacing-N vdev with sdb (old) and the new disk (resilvering).

Step 4: VERIFY at completion (poll periodically — reuse the existing resilver-watch cadence)

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "zpool status storage1"'

Expected at completion: scan: resilvered … with N errors, the replacing vdev is gone, sdb is evicted, and the new disk is a normal member. Pool state should move toward ONLINE (it may still report the 7 pre-existing data errors — see Step 5).

Step 5: After resilver — clear stale errors if no live counters remain

wsl.exe -- bash -lc 'ssh -o BatchMode=yes -i ~/.ssh/id_ed25519 root@192.168.1.68 "zpool status -v storage1"'

If only the 3 known media files remain and all device READ/WRITE/CKSUM are 0, optionally delete + re-grab those files via the *arr stack, then zpool clear storage1 to reset the error state. Expected after clear + a scrub: state: ONLINE, errors: No known data errors.

Step 6: Physically remove the evicted sdb (optional, later)

Use the SES locate-LED method to confirm the bay before pulling (host must be up): match sdb's SAS address to its sg_ses --dev-slot-num and sg_ses --dev-slot-num=<N> --set=ident /dev/sg12. Verify the drive label reads WD-WCC4E4TSK1S2 before full removal.

Rollback (only before/early in resilver): zpool detach storage1 <NEWDISK> cancels the replace and keeps the original sdb in service.

Self-review notes

Coverage: all five requested items have a task (fan safety = Task 3, container GPU = Task 4, hookscript = Task 2, sdb = Task 5, virtiofs permanent fix = Task 1). ✅
Dependencies flagged: Task 3 before Task 4 under load; Tasks 1+2 share a VM restart.
Stable identifiers: disk ops use serial/by-id, never bare sdX (letters shuffled across the week's reboots).
Each task has explicit verification + rollback. No zpool/Docker/systemd change lacks a confirmation command.

22 KiB Raw Permalink Blame History Unescape Escape