From 1a206434e002f58ae7e0d310c56af6714b76c65e Mon Sep 17 00:00:00 2001 From: chrg6 Date: Thu, 12 Nov 2026 22:30:02 -0500 Subject: [PATCH] Build the mesh pipelines' stage libraries ahead of their draws, at the preparation and at boot. The preparation and the boot build the stage libraries of single shaders. A mesh pipeline's pre-rasterization library needs its mesh stage and its task stage together, and its fragment shader library depends on what that mesh stage leaves unwritten, so only a pipeline record names what they are made of. Every listed mesh pipeline the preparation builds and every recorded one the boot queues now notes its two shader libraries, each key once. The preparation builds them on every core after the listed pipelines and before the pipeline cache is saved, inside the marker that covers a driver dying on a listed pipeline. The boot builds them on one thread at idle priority beside the game, behind the memory gate, with the rest of the library prebuild (lib_prebuild = 0 leaves it out). Each is destroyed once built, which leaves it in the pipeline caches and the driver's own, as for the single shaders' libraries. A mesh pipeline of known stages that a draw meets with other targets, another blend state or sample count then finds its libraries in the caches and is linked, where it was compiled whole. The recorded and listed mesh pipelines are still built whole ahead of play, also on a machine that leaves the rest of the shared list to its first draw. Co-Authored-By: Claude Fable 5.2 --- docs/architecture.md | 4 +- runtime/vk_pipeline.c | 156 ++++++++++++++++++++++++++++++++++++++++- tools/pipeline_list.py | 6 +- 3 files changed, 162 insertions(+), 4 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 4bed3247..3b7b6af5 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -186,7 +186,9 @@ keeps per-core data without locks and relies on that, so the host models it (`ru and mesh stages make one pre-rasterization library, no vertex input library is linked (a mesh pipeline has no such state), and the fragment shader and fragment output libraries are the vertex pipelines' where their keys allow (a mesh pipeline's fragment library differs where the mesh stage leaves an input unwritten, and on a device that - sets a shading rate per draw). With `gpl_mesh = 0`, and wherever the start's + sets a shading rate per draw). Their libraries need the mesh and task stage together, so they are made from the + pipeline records and not from the single prepared shaders: the preparation and every boot build those of the + listed and recorded mesh pipelines into the caches (`mesh_pre_note`). With `gpl_mesh = 0`, and wherever the start's log says so, a mesh pipeline has no libraries and is built whole, 0.2 to 1.1 s the first time. Compute pipelines are compiled outside the pipeline lock, and one compile serves every thread that needs it. The boot builds the pipeline of every compute shader play recorded or the shader preparation listed, but keeps only those of the shaders diff --git a/runtime/vk_pipeline.c b/runtime/vk_pipeline.c index d7c2524e..97583b1c 100644 --- a/runtime/vk_pipeline.c +++ b/runtime/vk_pipeline.c @@ -5556,6 +5556,140 @@ void vk_pipeline_prewarm(void) { __atomic_store_n(&pw_phase, 5, __ATOMIC_RELEASE); } +// ---- mesh pipelines' stage libraries ahead of their draws +// The shader preparation and the boot build the stage libraries of single shaders (lb_library): one vertex or one +// fragment shader is all such a library holds. A mesh pipeline's pre-rasterization library holds its mesh stage and +// its task stage together, and its fragment shader library depends on what that mesh stage leaves unwritten (FsKey), +// so these are taken from the pipeline records, which name all three shaders: every recorded mesh pipeline the boot +// queues (prewarm_run) and every listed one the preparation builds (seed_pipelines) notes its two shader libraries +// here, each key once. They are built once and destroyed, as the single shaders' libraries are, which leaves them in +// the pipeline caches and the driver's own: a mesh pipeline of the same stages that a draw meets later (other +// targets, another blend state or sample count, or a recorded one the boot has not come to) then finds its libraries +// there and is linked in milliseconds, where it was compiled whole before. +// - preparation (mesh_pre_run, from seed_pipelines): on every core, before the pipeline cache is saved, so that +// file holds them too; they are as many as the list has mesh stage pairs, not the tens of thousands of single +// shaders' libraries that are kept out of it. +// - boot (mesh_pre_start, from lb_start): one thread beside the game at idle priority, behind the memory gate +// (gate_wait); NX2NAVI_LIB_PREBUILD=0 leaves it out with the rest of that prebuild. +typedef struct { + VkPipeState s; + GKey k; + int kind; // LIB_PR or LIB_FS + uint64_t h; // key_hash of the key + uint32_t size; + union { PrKey pr; FsKey fs; } key; +} MeshPre; +static struct { + MeshPre *e; + uint32_t n, cap, n_pr; // n_pr: the pre-rasterization libraries among them + volatile int64_t next; + volatile int32_t stop; + int boot; // the builder runs beside the game: idle priority, the memory gate + PlatThread *th; // the boot's builder + PlatMutex stat; + uint64_t built, have, fail; // have: a draw had made the library by the time its turn came + double ms, max, t0; +} MP = {.stat = PLAT_MUTEX_INIT}; + +// Notes the two shader libraries of mesh pipeline s (key k), a pipeline with stage libraries (mesh_libs), for +// mesh_pre_worker. Caller holds P.lock; no builder runs yet. +static void mesh_pre_note(const VkPipeState *s, const GKey *k) { + ViKey v; + PrKey pk; + FsKey fk; + FoKey ok; + lib_keys(s, k, &v, &pk, &fk, &ok); + const void *key[2] = {&pk, &fk}; + const uint32_t size[2] = {sizeof pk, sizeof fk}; + for (int i = 0; i < 2; i++) { + int kind = i ? LIB_FS : LIB_PR; + uint64_t h = key_hash(key[i], size[i]); + uint32_t j = 0; + while (j < MP.n && !(MP.e[j].kind == kind && MP.e[j].h == h && !memcmp(&MP.e[j].key, key[i], size[i]))) j++; + if (j < MP.n) continue; + if (MP.n == MP.cap) { + uint32_t cap = MP.cap ? MP.cap * 2 : 64; + MeshPre *t = realloc(MP.e, cap * sizeof *t); + if (!t) return; + MP.e = t, MP.cap = cap; + } + MeshPre *e = &MP.e[MP.n++]; + memset(e, 0, sizeof *e); + e->s = *s, e->k = *k, e->kind = kind, e->h = h, e->size = size[i]; + memcpy(&e->key, key[i], size[i]); + MP.n_pr += kind == LIB_PR; + } +} + +static void mesh_pre_worker(void *arg) { + (void)arg; + for (;;) { + int idle = MP.boot && !bf_waiting(); // as the boot's other builders: normal priority while the start waits + plat_thread_priority(idle ? PLAT_PRIO_IDLE : PLAT_PRIO_NORMAL); + int64_t i = plat_inc64(&MP.next) - 1; + if (i >= (int64_t)MP.n || MP.stop) break; + MeshPre *e = &MP.e[i]; + double held = 0; + if (MP.boot && !gate_wait("mesh library prebuild", &MP.stop, &held)) { + char g[256]; + if (!MP.stop) + rt_log("vk_pipeline: mesh library prebuild: ends here, at %lld of %u libraries: no commit headroom for " + "another build (%s). The others are built when a draw first needs them", (long long)i, MP.n, + gate_text(g, sizeof g)); + break; + } + const char *why = NULL; + Bindings B = {0}; + plat_thread_priority(PLAT_PRIO_NORMAL); // P.lock is never held at idle priority: the replay takes it + plat_mutex_lock(&P.lock); + int have = lib_find(e->h, &e->key, e->size) != NULL; + Layout *l = have ? NULL : mesh_pair_layout(&e->s, &B, &why); + VkPipelineLayout lay = l ? half_layout(l, e->kind == LIB_FS) : VK_NULL_HANDLE; + plat_mutex_unlock(&P.lock); + if (idle) plat_thread_priority(PLAT_PRIO_IDLE); + VkPipeline pipe = VK_NULL_HANDLE; + double t0 = now_ms(), ms = 0; + if (lay) pipe = lib_create(e->kind, &e->s, &e->k, lay, &e->key); + if (pipe) { + ms = now_ms() - t0; + vkDestroyPipeline(P.device, pipe, NULL); + __atomic_fetch_add(&cache_merges, 1, __ATOMIC_RELAXED); // autosave_once: no pipeline count tells of it + } + if (MP.boot) gate_leave(held); + plat_mutex_lock(&MP.stat); + if (have) MP.have++; + else if (!pipe) MP.fail++; + else MP.built++, MP.ms += ms, MP.max = ms > MP.max ? ms : MP.max; + plat_mutex_unlock(&MP.stat); + } + if (MP.boot) + rt_log("vk_pipeline: mesh library prebuild %s: of the %u shader libraries of the recorded mesh pipelines (%u of " + "task and mesh stages, %u fragment shader) %llu built into the caches (avg %.1f ms, longest %.0f ms; a " + "few milliseconds each when the caches held them), %llu made by a draw first, %llu failed; %.0f ms", + MP.stop ? "stopped" : "done", MP.n, MP.n_pr, MP.n - MP.n_pr, (unsigned long long)MP.built, + MP.built ? MP.ms / (double)MP.built : 0.0, MP.max, (unsigned long long)MP.have, + (unsigned long long)MP.fail, now_ms() - MP.t0); +} + +// Boot: the noted libraries on one thread beside the game. +static void mesh_pre_start(void) { + if (!MP.n || MP.th || MP.next) return; + MP.boot = 1, MP.t0 = now_ms(); + MP.th = plat_thread_new(mesh_pre_worker, NULL, "mesh lib prebuild"); +} + +// Preparation: the noted libraries on up to `threads` threads; returns when all are built. +static void mesh_pre_run(int threads) { + if (!MP.n) return; + PlatThread *t[16]; + int n = 0; + MP.boot = 0, MP.t0 = now_ms(); + for (int i = 0; i < threads && i < 16 && (uint32_t)i < MP.n; i++) + if ((t[n] = plat_thread_new(mesh_pre_worker, NULL, "mesh lib prebuild"))) n++; + if (!n) mesh_pre_worker(NULL); + for (int i = 0; i < n; i++) plat_thread_join(t[i]); +} + // NX2NAVI_PIPE_SEED_BOOT (auto, the default; 1; 0): whether the boot builds the pipelines of the shared list that this // install has never drawn (keys_open_seed), 1, or leaves them to their first draw, 0. Built at boot they are there // when a draw first needs them, which is what the list is for, and every one is a pipeline the driver holds for the @@ -5682,6 +5816,8 @@ static void prewarm_run(void) { if (!is_mesh(&s)) { // the state of the single shaders' libraries (lb_state): vertex pipelines decide it pr_hist[(k.clamp & 1) | (k.poly_mode & 3) << 1 | (k.minus_w & 1) << 3]++; fs_hist[k.samples >= 8 ? 3 : k.samples >= 4 ? 2 : k.samples >= 2 ? 1 : 0]++; + } else if (mesh_libs()) { // a mesh pipeline's libraries are built with its own state (mesh_pre_start) + mesh_pre_note(&s, &k); } if (!at_boot && i >= K.own_n && !is_mesh(&s)) { // of the shared list alone: made at its first draw (seed_boot) listed++; @@ -7382,6 +7518,7 @@ static void lb_start(const uint32_t pr_hist[16], const uint32_t fs_hist[4]) { const char *te = cfg_get("NX2NAVI_LIB_PREBUILD_THREADS"); // no fixed default: 2 when unset (1 under 4 CPUs) const char *be = cfg_get("NX2NAVI_LIB_PREBUILD_BIG"); // SPIR-V words; LB_BIG_WORDS when unset, 0 no limit if (!A.on || !cfg_bool("NX2NAVI_LIB_PREBUILD")) return; + mesh_pre_start(); // the recorded mesh pipelines' libraries lb_state(pr_hist, fs_hist); // the libraries' state, also for lw_start if (BF.on) BF.lw = 1; // after the compute pipelines, with the start waiting (bf_wait) else lw_start(); @@ -7456,6 +7593,8 @@ static void lb_start(const uint32_t pr_hist[16], const uint32_t fs_hist[4]) { static void lb_stop(void) { lw_stop(); + plat_xchg32(&MP.stop, 1); + if (MP.th) plat_thread_join(MP.th), MP.th = NULL; plat_xchg32(&LB.stop, 1); for (int i = 0; i < LB.threads; i++) if (LB.th[i]) plat_thread_join(LB.th[i]), LB.th[i] = NULL; @@ -7623,7 +7762,8 @@ static void lb_warm_prepared(const char *dir) { // (vk_caps_gpl: AMD's Windows driver for RDNA 2 and older) a vertex pipeline's outputs come from its own pair (no // independent layouts), so a listed one always is the one a draw asks for. A mesh pipeline, which has no stage // libraries, otherwise leaves its draws out of the picture for its whole compile (0.2 to 1.1 s each on a first visit -// of a forest, 18 of them). +// of a forest, 18 of them). Where mesh pipelines have stage libraries (mesh_libs) the listed ones are still built +// whole, and the libraries of their stages are built after them (mesh_pre_run), for the mesh pipelines no list names. // A driver whose compiler crashes on one of these would crash at every preparation: seed-pipelines-running.txt marks // the step, and a run that finds the marker of a run that died skips the step for good on this folder (the pipelines // are then built at their first draw, as without a list) until the marker is deleted. @@ -7715,6 +7855,7 @@ static int seed_pipelines(const char *dir) { } GKey k; gkey_fill(&s, &k); + if (is_mesh(&s) && mesh_libs()) mesh_pre_note(&s, &k); uint64_t hk = key_hash(&k, sizeof k); int q = 0; if (pipe_find(&k, sizeof k, hk)) known++; @@ -7742,6 +7883,19 @@ static int seed_pipelines(const char *dir) { while (K.left) plat_cond_wait(&done_cv, &P.lock); // those a pool worker took uint32_t built = K.built - built0, failed = K.failed - failed0; plat_mutex_unlock(&P.lock); + // The listed mesh pipelines' stage libraries, before the save and inside the marker: the file then holds them, + // and a driver that dies on one is met by the next preparation as one that died on a listed pipeline. + mesh_pre_run(cpus); + if (MP.n) { + char ml[320]; + snprintf(ml, sizeof ml, "prepare-pipelines: %u stage libraries of the listed mesh pipelines (%u of task and mesh " + "stages, %u fragment shader): %llu built (avg %.1f ms, longest %.0f ms), %llu failed; %.0f ms", MP.n, + MP.n_pr, MP.n - MP.n_pr, (unsigned long long)MP.built, MP.built ? MP.ms / (double)MP.built : 0.0, MP.max, + (unsigned long long)MP.fail, now_ms() - MP.t0); + printf("%s\n", ml); + fflush(stdout); + rt_log("vk_pipeline: %s", ml); + } int saved = cache_save(5000, 0) > 0; // explicit preparation waits for a moment with no build running plat_file_delete(mark); char msg[640]; diff --git a/tools/pipeline_list.py b/tools/pipeline_list.py index e10751cb..39400a0f 100644 --- a/tools/pipeline_list.py +++ b/tools/pipeline_list.py @@ -4,8 +4,10 @@ A game folder records every graphics pipeline a run builds in pipeline-keys.bin keys": the pipeline state with its shaders named by input id) and every shader variant in shader-answers.bin (runtime/gfx_shader.c "answers met in play": stage, code hash, input id, translator answers). The next start of that folder builds its recorded pipelines ahead of play; another install, or the same one after its caches were removed, -meets each of them in play first. A mesh pipeline met that way has no stage libraries to be linked from, so its draws -are left out of the picture for its whole compile, 0.2 to 1.1 s. +meets each of them in play first. A mesh pipeline met that way is compiled then, 0.2 to 1.1 s with its draws left out +of the picture: all of it where mesh pipelines have no stage libraries, and otherwise the library of its task and +mesh stages, which is most of it and which only a pipeline record names (runtime/vk_pipeline.c "mesh pipelines' stage +libraries ahead of their draws"). This tool folds the two files of any number of game folders into one list file; a list file made before can stand in for a folder, so a list grows by the folders played since. Beside the shader variants the pipelines name, the list