Keep the SPIR-V of shaders nobody read for 30 seconds compressed in memory.

The shader memory cache held 777 MiB of SPIR-V for 7115 shaders for the whole session. A background thread now
compresses the SPIR-V of each shader no reader pinned for shader_compress_idle_s seconds (zstd level 1, 26% of the
bytes on the test install's 44,151 modules) and frees the plain copy; readers pin it through gfx_shader_spirv_pin and
get it decoded back. What the drawing thread compares at pipeline misses and draws, the module's FNV-1a and its Binding
decorations, is kept with the shader, so module_binding_missing, the water corrections and the DLSS, crumbs and skip
lists decode nothing. A compute pipeline created at a dispatch and a mesh shader's first pipeline key still decode on
that thread, and the memory report counts those decodes apart, with the SPIR-V as held against uncompressed and the
shaders held compressed. shader_compress = 0 keeps it plain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
chrg6
2026-11-13 07:48:06 -05:00
parent 71e52159f9
commit 3573a96f46
14 changed files with 411 additions and 134 deletions

View File

@@ -534,14 +534,13 @@ static uint64_t shader_hash(const GfxShader *s) {
if (!s) return 0;
for (int i = 0; i < n; i++)
if (tab[i].s == s) return tab[i].h;
uint64_t words = 0, h = 0xcbf29ce484222325ull;
const uint32_t *w = gfx_shader_spirv(s, &words);
const uint8_t *b = (const uint8_t *)w;
for (uint64_t i = 0; i < words * 4; i++) h = (h ^ b[i]) * 0x100000001b3ull;
uint64_t words = 0, h = gfx_shader_spirv_fnv(s);
const uint32_t *w = gfx_shader_spirv_pin(s, &words);
char path[600];
snprintf(path, sizeof path, "%s/sh_%016llx.spv", dir, (unsigned long long)h);
FILE *f = fopen(path, "wb");
if (f) fwrite(w, 4, (size_t)words, f), fclose(f);
gfx_shader_spirv_unpin(s);
snprintf(path, sizeof path, "%s/sh_%016llx.code", dir, (unsigned long long)h); // the guest program it came from
gfx_shader_write_code(s, path);
if (n < 1024) tab[n].s = s, tab[n].h = h, n++;