ginnoirandClaude Opus 4.8 4aa2cd9468 fix(llm): q8_0 V-cache (q4_0 tanked generation to 1.3 tok/s on Pascal)
The q4_0 V-cache + flash-attention path is pathological on the GP100: 1.28
tok/s generation at 5-8% GPU util. q8_0 V-cache gives 9.2 tok/s and still fits
64k context in 16GB (15.3GB used, ~950MB free).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 17:04:17 -05:00
2026-06-14 22:53:22 -05:00
2026-06-02 21:32:30 -05:00
2026-06-02 21:32:30 -05:00
S
Description
Mirror of ginnoir/homelabstack (primary on GitHub).
Readme MIT
753 KiB
Languages
Python 54.5%
Shell 18.5%
PowerShell 16%
JavaScript 5.5%
HTML 4.7%
Other 0.7%