diff --git a/docs/building.md b/docs/building.md index 7126091c..e177e585 100644 --- a/docs/building.md +++ b/docs/building.md @@ -127,7 +127,17 @@ of them. Without this step the first launch compiles shaders and pipelines during play and stutters until the caches fill. The AMD driver keeps its pipeline cache per exe path and holds a limited amount: preparing another build folder can - evict this one's. `nx2navi.exe --warm-shaders` rebuilds it from the shader cache. + evict this one's. `nx2navi.exe --warm-shaders` rebuilds it from the shader cache. A driver update empties the + driver's cache too, and a change to what a stage library is built with gives every library another entry in it. + So an update that carries the prepared shaders over runs `--warm-shaders` (`tools/make_game.py` + `warm_driver_cache`: seconds while the libraries are there, minutes when they are not), and the game itself checks + at every start: it builds a sample of the prepared fragment libraries and, when their median time says the cache + is cold, builds the whole prepared list into the driver's cache again on idle-priority threads while it runs + (`runtime/vk_pipeline.c` `lw_start`, `NX2NAVI_LIB_WARM`; `shader-warm.txt` next to the caches says where that + stands). With the cache cold every pipeline a draw first needs compiles in play: on an RX 7900 XT a library took + 93 ms (vertex) and 242 ms (fragment) on average against 0.3 and 0.6 ms from the cache, and a first walk through a + village skipped 11,180 draws and made 94 draws wait 5.5 s in all, against 1,605 skipped (1,473 of them in the + scene's first frames) and 60 ms of waits with the cache warm. When the driver's shader compiler crashes on a variant (NVIDIA 615.78.08 on an RTX 5080 does, issue #30), `prepare_shaders.py` runs the driver step again with `--warm-shaders`, up to three times, without the variants the driver was building at the crash. `shader-prepare-skip.txt` next to the exe lists them with the GPU and driver diff --git a/docs/releases/v0.6.2.md b/docs/releases/v0.6.2.md index 2e96eae0..153ca7ef 100644 --- a/docs/releases/v0.6.2.md +++ b/docs/releases/v0.6.2.md @@ -15,6 +15,15 @@ Draft notes for the release after 0.6.1. The version number is not decided yet. ## Fixed +- Ground, walls and characters that appeared and disappeared, black or unlit frames, and stalls of up to two seconds + wherever something was shown for the first time (a new area, a cutscene). The graphics driver's cache had lost the + shader libraries prepared at install, so every new pipeline compiled during play: after a driver update, or after + an update of NX2navi that changes how the libraries are built (this one does on AMD RX 7000 and RX 9000 cards, + where it also ends the 20 second start). The game now checks the driver's cache at every start and fills it again in the background by + itself, an update fills it again before you play, draws wait a moment for a pipeline that only needs linking + instead of being left out, and lighting passes are no longer left out at all. +- Textures out of view for 11 seconds at 165 fps (30 seconds were meant) could show a blank stand-in for a few frames + when they came back while memory was short. The wait is now two minutes of real time at any frame rate. - Settings menu, mouse: a setting changes only on the two framed arrows beside its value, and the arrows stay in one place. A click anywhere on a row used to step the setting, and the left arrow moved with the value's width, so clicking through the values landed on the row and stepped the wrong way. Holding the button on an arrow repeats. diff --git a/nx2navi.ini.example b/nx2navi.ini.example index f915c9df..f48db1d1 100644 --- a/nx2navi.ini.example +++ b/nx2navi.ini.example @@ -71,8 +71,15 @@ # Draws whose pipeline is still building: default waits on a reserved worker; draw is the previous scheduling. # pipe_wait = default # Which draws are dropped instead of waiting for their pipeline: strict (only draws into per-frame passes), any (also -# draws into passes whose target a recent frame rendered) or fullscreen (those and full-screen draws). -# pipe_skip = fullscreen +# draws into passes whose target a recent frame rendered; full-screen draws wait, up to 50 ms a frame) or fullscreen +# (those and full-screen draws: no wait, but a lighting or fog pass that is dropped shows as a black or unlit frame). +# pipe_skip = any +# How many milliseconds the draws of one frame may wait, in all, for a missing pipeline to be linked from the shader +# libraries in the graphics driver's cache before they are dropped (0 to 100; 0: dropped at once, as before). +# pipe_link_ms = 6 +# The graphics driver's cache of shader libraries is checked at every start and, when it lost them (a driver update, +# a cache the driver emptied), filled again in the background while the game runs. 0: no check; force: at every start. +# lib_warm = 1 # How long (ms) a draw into a target no recent frame rendered waits for a pipeline built for the first time, # then is skipped. The default waits, as such a target (a lookup table, a new area's target) can stay empty if # skipped; a lower value caps the stall. diff --git a/runtime/cfg.c b/runtime/cfg.c index 7c03acd2..3edea66f 100644 --- a/runtime/cfg.c +++ b/runtime/cfg.c @@ -47,7 +47,7 @@ static const char *const known[] = { "HDR_EXPOSURE", "HDR_FLARE", "HDR_GAMMA", "HDR_GAME_NITS", "HDR_GRADE", "HDR_HIGHLIGHT_SAT", "HDR_HIGHLIGHTS", "HDR_HUE", "HDR_LIVE", "HDR_PEAK_NITS", "HDR_SATURATION", "HDR_SHADOWS", "HDR_TONEMAP", "HDR_UI_NITS", "HIGHFPS_FIX", "INPUT", "INPUT_RECORD", "INPUT_SCRIPT", "LANG", "LIB_PREBUILD", - "BIG_BUILD_FREE_MB", "BIG_BUILD_MAX_MB", "LIB_PREBUILD_BIG", "LIB_PREBUILD_BIG_BG", "LIB_PREBUILD_KEEP", "LIB_PREBUILD_THREADS", "LOCALE", "LOD_GUARD", "LSTICK_DEADZONE", "MEMORY", "MEM_CENSUS", "MENU_LANG", "MENU_SOUND_VOLUME", "MENU_TEST", + "BIG_BUILD_FREE_MB", "BIG_BUILD_MAX_MB", "LIB_PREBUILD_BIG", "LIB_PREBUILD_BIG_BG", "LIB_PREBUILD_KEEP", "LIB_PREBUILD_THREADS", "LIB_WARM", "PIPE_LINK_MS", "LOCALE", "LOD_GUARD", "LSTICK_DEADZONE", "MEMORY", "MEM_CENSUS", "MENU_LANG", "MENU_SOUND_VOLUME", "MENU_TEST", "MENU_CURSOR", "MENU_CURSOR_LOG", "MENU_TEST_CLICK", "MENU_TEST_DIR", "MENU_TEST_SET", "MINIMAL_LAYOUT", "MICROPHONE", "MIRROR_GAP", "MODS_DIR", "MODS_DISABLED", "MODS_ORDER", "MODS_REBUILD", "MONITOR", "MOTION", "MOUSE_MODE", "MOUSE_DIRECT_PATCH", "MOUSE_SENS", "MOVIE_HOST", "MOVIE_PACE", "MVOBJ_ALL", "MVOBJ_BANKS", "MVOBJ_EQUAL", "MVOBJ_EXACT", "MVOBJ_EXEC", "MVOBJ_NOC4", "MVOBJ_ONLY", "MVOBJ_TIMES", "MVPREV_CHECK", "MVPREV_GROW", "MVPREV_SKIP", "MVPREV_TRUST", "MV_CHECK", "MV_SURVEY", @@ -111,7 +111,7 @@ static const struct { const char *key, *section; } sections[] = { {"ASTC_GPU_DESWIZZLE", "performance"}, {"ASTC_SCRATCH_KEEP", "performance"}, {"ASTC_ARENA", "performance"}, {"POOL_CREATE_BUDGET", "performance"}, {"POOL_CREATE_MS", "performance"}, {"TIMER_1MS", "performance"}, {"PIPE_WAIT", "performance"}, {"PIPE_SKIP", "performance"}, {"ONESHOT_BUDGET", "performance"}, - {"LIB_PREBUILD", "performance"}, {"LIB_PREBUILD_THREADS", "performance"}, {"PIPE_PREBUILD", "performance"}, + {"LIB_PREBUILD", "performance"}, {"LIB_PREBUILD_THREADS", "performance"}, {"LIB_WARM", "performance"}, {"PIPE_LINK_MS", "performance"}, {"PIPE_PREBUILD", "performance"}, {"PREWARM_XLATE", "performance"}, {"GPL", "performance"}, {"ASTC", "performance"}, {"MEMORY", "performance"}, {"TEX_ALIAS", "performance"}, {"TEX_DISUSE", "performance"}, {"TEX_REUSE", "performance"}, {"TEX_VRAM_PRESSURE", "performance"}, {"ASTC_BC", "performance"}, {"LOD_GUARD", "performance"}, {"FIX_C0", "performance"}, {"FIX_ZERO", "performance"}, {"BIND_SKIP", "performance"}, {"SHADER_CACHE_FLUSH", "performance"}, {"SHADER_CARRY", "performance"}, {"SHADER_MEMO", "performance"}, {"PIPE_MEMO", "performance"}, {"TEX_MEMO", "performance"}, {"DESC_TEMPLATE", "performance"}, @@ -208,8 +208,10 @@ static const CfgMeta metas[] = { // held submissions handed to the queue together (vk_sync.c, submit batching) {"SUBMIT_BATCH", "4", "1..16", RESTART}, // shaders and pipelines - {"PIPE_WAIT", "default", "default|draw", RESTART}, {"PIPE_SKIP", "fullscreen", "strict|any|fullscreen", RESTART}, + {"PIPE_WAIT", "default", "default|draw", RESTART}, {"PIPE_SKIP", "any", "strict|any|fullscreen", RESTART}, + {"PIPE_LINK_MS", "6", "0..100", RESTART}, // vk_exec_draw.c link_budget {"LIB_PREBUILD", "1", "0|1", RESTART}, {"LIB_PREBUILD_THREADS", "", "1..64", RESTART}, + {"LIB_WARM", "1", "0|1|force", RESTART}, // vk_pipeline.c lw_start: the driver's cache of stage libraries, warmed when cold {"PIPE_PREBUILD", "1", "0|1", RESTART}, {"PREWARM_XLATE", "", "1..64", RESTART}, {"GPL", "1", "0|1", RESTART}, {"LIB_PREBUILD_BIG_BG", "1", "0|1", RESTART}, {"PIPE_FEEDBACK", "1", "0|1|2", RESTART}, {"PIPELINE_CACHE_MAX_MB", "512", "0..65536", RESTART}, {"PIPE_BINARIES", "0", "0|1", RESTART}, {"ONESHOT_WAIT_MS", "30000", "0..60000", RESTART}, {"ONESHOT_BUDGET", "0", "0|1", RESTART}, {"PIPE_BINARIES_MAX_MB", "1024", "0..65536", RESTART}, {"BIG_BUILD_FREE_MB", "8192", "0..262144", RESTART}, diff --git a/runtime/vk_exec_draw.c b/runtime/vk_exec_draw.c index 23fbc3ed..49001d1c 100644 --- a/runtime/vk_exec_draw.c +++ b/runtime/vk_exec_draw.c @@ -3249,23 +3249,53 @@ static void foliage_shadow_cull(const VkExec *ex, VkPipeState *s) { // buffer or g-buffer with a colour target the frame before did not use: half-resolution effect buffers in fights). // Passes with none (volumes, LUTs, shadow layers seen for the first time) still wait, and so do passes whose only // recent target is the depth buffer while a colour target is rendered for the first time (item icon captures). -// fullscreen (2, default): as any, and full-screen draws into such passes are skipped instead of waiting from their +// Full-screen draws into such passes wait from their budget of 50 ms a presented frame. The default: a skipped +// full-screen draw is a lighting, fog or tone mapping pass that leaves its target as it was, which shows as a black +// or unlit frame with the interface intact, and with the stage libraries in the driver's cache the wait is a +// millisecond (vk_pipeline.c lw_start keeps them there). +// fullscreen (2): as any, and full-screen draws into such passes are skipped instead of waiting from their // budget. A full-screen pass whose stage libraries are cold in the driver cache took 86 ms on average (up to 194 ms) // in a fight on an RX 9070: three frames in a row each spent the 50 ms budget, and some of the draws were skipped -// after all. Skipping costs the pass for the frames until the pipeline is ready. +// after all. Skipping costs the pass for the frames until the pipeline is ready. It was the default until the +// libraries were kept warm. +// Any other draw whose pipeline is missing waits for its fast link from a small budget per presented frame +// (link_budget) before it is skipped. static int pipe_skip_mode(void) { static int mode = -1; if (mode < 0) { const char *e = cfg_value("NX2NAVI_PIPE_SKIP"); mode = !strcmp(e, "strict") || !strcmp(e, "0") ? 0 : !strcmp(e, "fullscreen") || !strcmp(e, "2") ? 2 : 1; static const char *const what[3] = {"only draws into per-frame passes (NX2NAVI_PIPE_SKIP=strict)", - "draws into passes with a target the previous frame rendered", + "draws into passes with a target the previous frame rendered; full-screen " + "draws wait (NX2NAVI_PIPE_SKIP=any)", "those and full-screen draws (NX2NAVI_PIPE_SKIP=fullscreen)"}; rt_log("vk_exec: draws skipped while their pipeline builds: %s", what[mode]); } return mode; } +// NX2NAVI_PIPE_LINK_MS (default 6; 0 = off): how long the draws of one presented frame may wait, in all, for the fast +// link of a pipeline that is missing, LINK_DRAW_MS at most each. A draw into a per-frame pass used to be skipped at +// once and for as long as its pipeline built, also when its stage libraries were in the driver's cache and the link +// took under a millisecond: geometry seen for the first time (ground, a wall, a character) was missing for a frame or +// more and then appeared. With the libraries warm the wait ends within the budget and nothing is skipped; with them +// cold it runs out after a few draws and the rest are skipped as before, so a burst of cold pipelines costs a frame +// this much and no more. Measured on an RX 7900 XT, first walk through a village with no pipeline recorded: 2,349 +// draws skipped without the budget, with the libraries warm in both runs. +#define LINK_DRAW_MS 3u +static uint64_t *link_budget(void) { + static uint64_t frame = ~0ull, left, per_frame = ~0ull; + if (per_frame == ~0ull) { + long long ms = cfg_int("NX2NAVI_PIPE_LINK_MS"); + per_frame = (uint64_t)(ms < 0 ? 0 : ms > 100 ? 100 : ms) * 1000000ull; + rt_log("vk_exec: draws wait up to %llu ms a frame in all for the fast link of a missing pipeline " + "(NX2NAVI_PIPE_LINK_MS)", (unsigned long long)(per_frame / 1000000)); + } + uint64_t f = __atomic_load_n(&gfx_counters.frames, __ATOMIC_RELAXED); + if (f != frame) frame = f, left = per_frame; + return &left; +} + // The previous draw's shaders, kept while its program, bound pipeline state, the vertex and fragment stages' first 32 // texture and image handles and every slot's resolution (vk_texture_slots_gen, read unlocked: a registration racing // with it is ordered after this draw, as answers_for's own check would order it) stay the same: answers_for reads @@ -3461,6 +3491,7 @@ static int prepare_draw_ex(VkExec *ex, int prim, int32_t count, int restart, int uint32_t wait_ms = 0; uint64_t t0 = 0; uint64_t *budget = NULL; // the per-frame budget a timed wait is charged to + int linking = 0; // the wait is the short one for a fast link (link_budget) int large = (uint64_t)ex->pass_w * ex->pass_h >= ONESHOT_LARGE_PX; if (oneshot > 0 && oneshot_budget_on()) { budget = oneshot_budget(large); @@ -3474,6 +3505,12 @@ static int prepare_draw_ex(VkExec *ex, int prim, int32_t count, int restart, int wait_ms = (uint32_t)(wait_left_ns / 1000000); if (wait_ms) t0 = kern_now_ns(); budget = &wait_left_ns; + } else { + budget = link_budget(); + wait_ms = (uint32_t)(*budget / 1000000); + if (wait_ms > LINK_DRAW_MS) wait_ms = LINK_DRAW_MS; + linking = wait_ms != 0; + if (!wait_ms) budget = NULL; } // The previous draw's pipeline while its state repeats (PM, NX2NAVI_PIPE_MEMO=0 turns it off): the cache key is made // from the state alone, and a pipeline once found only changes from its fast link to its optimised build, read @@ -3485,13 +3522,21 @@ static int prepare_draw_ex(VkExec *ex, int prim, int32_t count, int restart, int info = PM.info, pending = 0; p = __atomic_load_n(&info->pipeline, __ATOMIC_ACQUIRE); } else { - w0 = t0 ? t0 : wait_ms ? kern_now_ns() : 0; // the clock is read only for draws that may wait - p = vk_pipeline_graphics_async(&s, &info, &pending, wait_ms); + if (linking) { // nearly every draw finds its pipeline: the clock is read only for one that did not + p = vk_pipeline_graphics_async(&s, &info, &pending, 0); + if (pending) { + t0 = kern_now_ns(); + p = vk_pipeline_graphics_async(&s, &info, &pending, wait_ms); + } + } else { + w0 = t0 ? t0 : wait_ms ? kern_now_ns() : 0; // the clock is read only for draws that may wait + p = vk_pipeline_graphics_async(&s, &info, &pending, wait_ms); + } if (t0 && budget) { uint64_t spent = kern_now_ns() - t0; *budget = spent < *budget ? *budget - spent : 0; } - why = oneshot > 0 ? "one-shot, shader" : wait_ms ? "full-screen" : NULL; + why = oneshot > 0 ? "one-shot, shader" : linking ? NULL : wait_ms ? "full-screen" : NULL; if (pending && oneshot < 0 && !vk_exec_targets_recent(ex, skip >= 1)) { perf_count(PERF_CNT_ONESHOT_WAIT); why = "one-shot"; diff --git a/runtime/vk_pipeline.c b/runtime/vk_pipeline.c index 229887be..a30f99b8 100644 --- a/runtime/vk_pipeline.c +++ b/runtime/vk_pipeline.c @@ -4748,12 +4748,189 @@ static void lb_state(const uint32_t pr_hist[16], const uint32_t fs_hist[4]) { LB.samples = (uint8_t)(1u << bf); } +// ---- the driver's cache of stage libraries, warmed again during play +// +// The preparation (lb_warm_prepared) builds every prepared variant's stage library once, so the driver's disk cache +// holds it and the library a draw needs later takes a few milliseconds instead of a compile: 3 ms against 100 ms for +// a vertex and 340 ms for a fragment library on an RX 7900 XT (driver 26.10.1). The driver's cache does not outlive +// everything, though: a driver update empties it, the AMD driver keeps a limited amount and drops old entries, and a +// change to what a library is built with (the dynamic states it lists, as when dynamic vertex input was turned off for +// this driver) gives every library another entry. After any of them each pipeline a draw first needs compiles cold: +// the draw is skipped for the frames that takes, or waits for it. Measured on that card with the cache cold, a walk +// through a village not seen before skipped 11,180 draws while their pipelines built and 94 draws waited 5.5 s in +// all, one of them 1.9 s. +// +// So every start checks: a sample of the prepared fragment libraries is built (lw_probe), and when their median time +// says the cache is cold, the whole prepared list is built again into the driver's cache on a few idle-priority +// threads while the game runs (lw_worker), as the preparation does but without stopping play. shader-warm.txt beside +// the caches says where that stands: "warming" while it runs, so a start after a run that ended meanwhile goes on, +// and "done" with the driver and a count of the times the cache was cold again right after a finished warm. At two in +// a row the driver does not keep stage libraries at all and warming stops being tried (NX2NAVI_LIB_WARM=0 turns it off; +// =force warms at every start). +#define LW_SAMPLE 12 +#define LW_COLD_MS 15.0 +#define LW_MAX_THREADS 6 +static struct { + GfxWarmEnt *e; + size_t n; + volatile int64_t next, done; + volatile int32_t running, stop, started; + PlatThread *th[LW_MAX_THREADS + 1]; + int threads, strikes; + double t0; + char stamp[PLAT_PATH_MAX * 2 + 32], id[64]; +} LW; + +static void lw_stamp(const char *state, int strikes) { + char line[128]; + int n = snprintf(line, sizeof line, "%s %s %d\n", state, LW.id, strikes); + if (!plat_file_write_atomic(LW.stamp, line, (size_t)n)) rt_log("vk_pipeline: cannot write %s", LW.stamp); +} + +// One prepared vertex or fragment variant's library into the driver's cache; its time in ms, negative when the +// variant is on the skip list, not in the shader cache or of another stage. +static double lw_build(const GfxWarmEnt *e) { + if (e->stage != GFX_STAGE_VS && e->stage != GFX_STAGE_FS) return -1; + if (skip_listed(e->iid)) return -1; + int owned = 0; + const GfxShader *sh = gfx_shader_load_private(e->iid, &owned); + if (!sh) return -1; + double t0 = now_ms(); + int st = gfx_shader_stage(sh); + if (st == GFX_STAGE_VS || st == GFX_STAGE_FS) lb_library(sh, st == GFX_STAGE_VS, 0); + double ms = now_ms() - t0; + if (owned) gfx_shader_release_private(sh); + return ms; +} + +static void lw_worker(void *arg) { + (void)arg; + plat_thread_priority(PLAT_PRIO_IDLE); + for (;;) { + size_t i = (size_t)plat_inc64(&LW.next) - 1; + if (i >= LW.n || LW.stop) break; + lw_build(&LW.e[i]); + int64_t d = plat_inc64(&LW.done); + if (d % 8000 == 0) + rt_log("vk_pipeline: library warm: %lld of %zu prepared variants in %.0f s", (long long)d, LW.n, + (now_ms() - LW.t0) / 1000); + } + if (!plat_dec32(&LW.running) && !LW.stop) { + lw_stamp("done", LW.strikes); + rt_log("vk_pipeline: library warm: done, %zu prepared variants in %.0f s on %d idle threads; new pipelines " + "link from the driver's cache from here on", LW.n, (now_ms() - LW.t0) / 1000, LW.threads); + } +} + +// The median time of LW_SAMPLE prepared fragment libraries spread over the list; negative without that many. +static double lw_probe(void) { + double ms[LW_SAMPLE]; + int got = 0; + size_t step = LW.n / LW_SAMPLE ? LW.n / LW_SAMPLE : 1, at = (size_t)(plat_now_ns() / 1000) % step; + for (size_t i = at; i < LW.n && got < LW_SAMPLE && !LW.stop; i += step) + for (size_t j = i; j < i + step && j < LW.n; j++) { // the next fragment variant from here + if (LW.e[j].stage != GFX_STAGE_FS) continue; + double t = lw_build(&LW.e[j]); + if (t >= 0) ms[got++] = t; + break; + } + if (got < LW_SAMPLE / 2) return -1; + for (int i = 1; i < got; i++) + for (int j = i; j > 0 && ms[j] < ms[j - 1]; j--) { + double t = ms[j]; + ms[j] = ms[j - 1], ms[j - 1] = t; + } + return ms[got / 2]; +} + +static void lw_main(void *arg) { + (void)arg; + plat_thread_priority(PLAT_PRIO_BELOW_NORMAL); + char was[16] = "", id[64] = ""; + int strikes = 0; + uint64_t size = 0; + char *f = (char *)plat_file_read(LW.stamp, &size); + if (f && size < 200) { + char line[200]; + memcpy(line, f, size), line[size] = 0; + if (sscanf(line, "%15s %63s %d", was, id, &strikes) < 2) was[0] = 0; + } + free(f); + int same = !strcmp(id, LW.id), resume = same && !strcmp(was, "warming"); + const char *mode = cfg_value("NX2NAVI_LIB_WARM"); + double median = resume ? 0 : lw_probe(); + if (LW.stop) return; + int cold = median > LW_COLD_MS || !strcmp(mode, "force"); + if (!resume && !cold) { + if (same && strikes) lw_stamp("done", 0); // the driver kept them this time + rt_log("vk_pipeline: library warm: the driver's cache has the prepared libraries (a sample's median %.1f ms)", + median); + return; + } + if (!resume && same && strikes >= 2 && strcmp(mode, "force")) { + rt_log("vk_pipeline: library warm: the prepared libraries are cold again (a sample's median %.0f ms) after two " + "finished warms on this driver: it does not keep them, so pipelines compile when a draw first needs " + "them (NX2NAVI_LIB_WARM=force warms anyway)", median); + return; + } + LW.strikes = resume ? strikes : same && !strcmp(was, "done") ? strikes + 1 : 0; + lw_stamp("warming", LW.strikes); + int cpus = (int)plat_cpu_usable(), t = cpus / 4; + t = t < 2 ? 2 : t > LW_MAX_THREADS ? LW_MAX_THREADS : t; + if (resume) + rt_log("vk_pipeline: library warm: going on with the warm an earlier run began: %zu prepared variants on %d " + "idle threads", LW.n, t); + else + rt_log("vk_pipeline: library warm: the driver's cache does not have the prepared libraries (a sample's median " + "%.0f ms; a driver update or a change of what they are built with): building %zu prepared variants " + "into it on %d idle threads while the game runs. Until it is done, a pipeline a draw first needs may " + "compile, which skips that draw for some frames or makes it wait", median, LW.n, t); + LW.t0 = now_ms(); + LW.running = t; + int started = 0; + for (int i = 0; i < t; i++) + if ((LW.th[1 + started] = plat_thread_new(lw_worker, NULL, "lib warm"))) started++; + LW.threads = started; + if (plat_add32(&LW.running, started - t) == 0 && !started) rt_log("vk_pipeline: library warm: no thread started"); +} + +// Boot, once the library state is known (lb_state): the prepared list is read and the check starts on its own thread. +static void lw_start(void) { + const char *mode = cfg_value("NX2NAVI_LIB_WARM"); + if (!P.gpl || !strcmp(mode, "0") || plat_xchg32(&LW.started, 1)) return; + char dir[PLAT_PATH_MAX * 2], path[PLAT_PATH_MAX * 2 + 32]; + if (!cache_dir(dir, sizeof dir)) return; + snprintf(path, sizeof path, "%s/shader-prepared.bin", dir); + snprintf(LW.stamp, sizeof LW.stamp, "%s/shader-warm.txt", dir); + uint64_t size = 0; + uint8_t *f = plat_file_read(path, &size); + size_t n = f && size >= 16 ? (size_t)(size - 16) / 36 : 0; + LW.e = n ? calloc(n, sizeof *LW.e) : NULL; + for (size_t i = 0; LW.e && i < n; i++) + memcpy(LW.e[i].iid, f + 16 + 36 * i + 4, 32), LW.e[i].stage = f[16 + 36 * i], LW.e[i].prio = f[16 + 36 * i + 1]; + free(f); + if (!LW.e) return; // nothing prepared (a build without the preparation step) + LW.n = n; + VkPhysicalDeviceProperties pp; + vkGetPhysicalDeviceProperties(vk_ctx()->phys, &pp); + snprintf(LW.id, sizeof LW.id, "%04x:%04x:%08x", pp.vendorID, pp.deviceID, pp.driverVersion); + LW.th[0] = plat_thread_new(lw_main, NULL, "lib warm check"); +} + +static void lw_stop(void) { + plat_xchg32(&LW.stop, 1); + for (int i = 0; i <= LW_MAX_THREADS; i++) + if (LW.th[i]) plat_thread_join(LW.th[i]), LW.th[i] = NULL; +} + // Boot: compute shaders of the warm list (recorded and prepared), then the first KEEP recorded graphics variants. static void lb_start(const uint32_t pr_hist[16], const uint32_t fs_hist[4]) { const char *e = cfg_value("NX2NAVI_LIB_PREBUILD"), *ke = cfg_get("NX2NAVI_LIB_PREBUILD_KEEP"); const char *te = cfg_get("NX2NAVI_LIB_PREBUILD_THREADS"); // no fixed default: 2 when unset (1 under 4 CPUs) const char *be = cfg_get("NX2NAVI_LIB_PREBUILD_BIG"); // SPIR-V words; LB_BIG_WORDS when unset, 0 no limit if (!A.on || !cfg_bool("NX2NAVI_LIB_PREBUILD")) return; + lb_state(pr_hist, fs_hist); // the libraries' state, also for lw_start + lw_start(); LB.big_words = be && *be ? strtoull(be, NULL, 10) : LB_BIG_WORDS; char dir[PLAT_PATH_MAX * 2]; if (cache_dir(dir, sizeof dir)) skip_init(dir); @@ -4773,7 +4950,6 @@ static void lb_start(const uint32_t pr_hist[16], const uint32_t fs_hist[4]) { return; } LB.n = n; - lb_state(pr_hist, fs_hist); LB.idle = 1; int cpus = (int)plat_cpu_usable(); // Two threads by default, not half the CPUs. On a 32-thread CPU with NVIDIA's driver, 16 builders kept the driver's @@ -4802,6 +4978,7 @@ static void lb_start(const uint32_t pr_hist[16], const uint32_t fs_hist[4]) { } static void lb_stop(void) { + lw_stop(); plat_xchg32(&LB.stop, 1); for (int i = 0; i < LB.threads; i++) if (LB.th[i]) plat_thread_join(LB.th[i]), LB.th[i] = NULL; diff --git a/runtime/vk_texture.c b/runtime/vk_texture.c index 5812141f..15c65edf 100644 --- a/runtime/vk_texture.c +++ b/runtime/vk_texture.c @@ -1058,8 +1058,27 @@ int vk_texture_commit_low(void) { return __atomic_load_n(&commit_low, __ATOMIC_R // content exists only in its image waits GPU_STALE_FRAMES, or GAP_TIMES the longest it stayed released before the game // took it back when that is longer. An overwritten entry has no content left to wait for (overwrite_others): it goes // with the intact ones, whether or not a descriptor still names it. -#define STALE_FRAMES 120 -#define GPU_STALE_FRAMES 1800 +// +// The waits here and DISUSE_FRAMES below are counts of presents, chosen at 60 presents a second: two seconds, thirty, +// two minutes. The same count is less time at a higher frame rate: at 165 presents a second 1800 of them are 11 s, so +// a render target the game came back to after 12 s was gone and a texture out of view for 11 s was retired. Each +// count is therefore taken times the present rate over 60, measured over the last two seconds and never under 1 +// (presents_for): the waits are the same time at any frame rate, and no shorter where frames come slower than 60. +static uint64_t presents_for(uint64_t at_60) { // tex_lock held + static uint64_t t0, f0; + static double times = 1; + uint64_t now = plat_now_ns(), f = __atomic_load_n(&gfx_counters.frames, __ATOMIC_RELAXED); + if (!t0) { + t0 = now, f0 = f; + } else if (now - t0 >= 2000000000ull) { + double rate = (double)(f - f0) * 1e9 / (double)(now - t0) / 60.0; + times = rate > 1 ? rate : 1; + t0 = now, f0 = f; + } + return (uint64_t)((double)at_60 * times); +} +#define STALE_FRAMES presents_for(120) +#define GPU_STALE_FRAMES presents_for(1800) #define GAP_TIMES 8 static int idle(const Tex *e, uint64_t now) { return look_round - e->use_round >= IDLE_ROUNDS && now - e->use_frame >= STALE_FRAMES; @@ -1074,8 +1093,9 @@ static uint64_t stale_after(const Tex *e) { static void drop_stale(uint64_t keep, uint64_t *count) { uint64_t now = __atomic_load_n(&gfx_counters.frames, __ATOMIC_RELAXED); int dropped = 0; + const uint64_t stale = STALE_FRAMES; for (int any = 0; any < 2; any++) - for (Tex *e = rel_head, *next; e && now - e->rel_frame >= STALE_FRAMES; e = next) { // in release order + for (Tex *e = rel_head, *next; e && now - e->rel_frame >= stale; e = next) { // in release order if (dropped == PRESSURE_STEP || rel_bytes <= keep) return; next = e->rel_next; if (!e->bytes || (!any && !comes_back_intact(e) && !e->overwritten)) continue; @@ -1171,7 +1191,12 @@ static uint64_t evict_idle(uint64_t want, uint64_t used, uint64_t budget) { // // marks missed. NX2NAVI_TEX_DISUSE=2 retires and never destroys (stand-ins only, nothing freed): it tells the marks and // stand-ins from the freeing as the cause of a fault. Only while every module that indexes the pool was marked and two // reads arrived: an unmarked shader could sample what the marks call unused. -#define DISUSE_FRAMES 1800 +// Two minutes, not the thirty seconds it was: a texture out of view that long and then seen again shows its stand-in +// for the frames until its mark is read, which is a flash of missing ground or a dark object each time the camera +// turns back, and a batch of such textures gave back 0.4 to 31 MiB in a recorded session. The textures of an area +// left behind, which are what the memory grows by, still go two minutes after it. +#define DISUSE_SECONDS 120 +#define DISUSE_FRAMES presents_for(DISUSE_SECONDS * 60) #define HOLDS_MAX 65536 static nvn_hmap holds = {.klen = sizeof(TexKey)}; // value: 1 static uint32_t marks_reads; @@ -1240,10 +1265,10 @@ static uint64_t evict_disused(uint64_t want) { want -= retiring_bytes; Tex *v[EVICT_STEP]; int nv = 0; - uint64_t now = __atomic_load_n(&gfx_counters.frames, __ATOMIC_RELAXED); + uint64_t now = __atomic_load_n(&gfx_counters.frames, __ATOMIC_RELAXED), disuse = DISUSE_FRAMES; for (uint32_t i = 0; i < by_key.cap; i++) { // the EVICT_STEP least recently sampled candidates, oldest first Tex *e = by_key.vals[i]; - if (!e || e->released || e->retiring || !e->bytes || !comes_back_intact(e) || now - e->samp_frame < DISUSE_FRAMES) + if (!e || e->released || e->retiring || !e->bytes || !comes_back_intact(e) || now - e->samp_frame < disuse) continue; int at = nv; while (at > 0 && v[at - 1]->samp_frame > e->samp_frame) at--; @@ -1289,7 +1314,7 @@ static uint64_t evict_disused(uint64_t want) { disuse_logs++; rt_log("vk: %d textures no shader sampled for %d s or more evicted (%.1f MiB), least recently sampled first; their " "images go once the texture pool sets no longer name them, and one sampled again is made again within a " - "few frames", n, DISUSE_FRAMES / 60, freed / 1048576.0); + "few frames", n, DISUSE_SECONDS, freed / 1048576.0); } return retiring_bytes; } diff --git a/tests/launcher_test.py b/tests/launcher_test.py index 9db77f43..db895a39 100644 --- a/tests/launcher_test.py +++ b/tests/launcher_test.py @@ -1425,6 +1425,36 @@ class MakeGameTest(Tmp): self.assertIn('stay in use', buf.getvalue()) self.assertIn('shader cache packed', buf.getvalue()) + def test_carried_update_warms_the_drivers_cache(self): + """An update that carries the shaders over runs the game's --warm-shaders, in the game folder; a run that + fails is said and does not stop the build, and without a built game or a preparation nothing runs.""" + calls = [] + for name, value in (('shader_carry', lambda out: True), + ('shader_run', lambda cmd, cwd=None: calls.append((cmd, cwd)) or self.rc)): + self.addCleanup(setattr, self.mg, name, getattr(self.mg, name)) + setattr(self.mg, name, value) + self.rc = 0 + self.out.mkdir(parents=True, exist_ok=True) + with contextlib.redirect_stdout(io.StringIO()): + self.mg.step_shaders(self.out, update=True) + self.assertEqual(calls, []) # no exe, no prepared list + (self.out / self.mg.EXE_NAME).write_bytes(b'MZ') + with contextlib.redirect_stdout(io.StringIO()): + self.mg.step_shaders(self.out, update=True) + self.assertEqual(calls, []) # never prepared: nothing to build again + (self.out / 'shader-prepared.bin').write_bytes(bytes(16)) + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + self.mg.step_shaders(self.out, update=True) + self.assertEqual(calls, [([str(self.out / self.mg.EXE_NAME), '--warm-shaders'], str(self.out))]) + self.assertIn("driver's cache warm", buf.getvalue()) + self.rc = 3 + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + self.mg.step_shaders(self.out, update=True) # no exception: the game warms it itself + self.assertIn('did not finish (3)', buf.getvalue()) + self.assertEqual(len(calls), 2) + def test_romfs_failure_stops_the_build(self): ran, lifting, failed = [], threading.Event(), threading.Event() diff --git a/tools/make_game.py b/tools/make_game.py index a607c595..eb7e4cd9 100644 --- a/tools/make_game.py +++ b/tools/make_game.py @@ -1372,30 +1372,12 @@ def shader_carry(out): return (out / 'shader-prepared.bin').is_file() and bool(cache_identities(out)) -def step_shaders(out, update=False): - """Every shader of the game files into the game's shader cache and the driver's own cache, before the first start. - Its output is read as it comes: the counts nx2navi.exe prints become progress events, the other lines go on to - the log, and while nothing moves an event every few seconds keeps saying the step is running and for how long. - update: after a kit update the shaders prepared before stay in use and the game translates them again in the - background (shader_carry), so the update does not wait the minutes a full preparation takes.""" - # The game translates with the copy beside it. The compile step refreshes that copy, but a kit that changes only - # the translator runs this step again alone (step_key), which would then prepare with the old one. Only a kit that - # has one: the compile step, which always runs first, stops a build without it. - for d in (KIT / 'bin', KIT / 'build' / 'xlate'): - if (d / XLATE_NAME).is_file(): - note_cache_translator(out) - for name in (XLATE_NAME, 'nx2navi_xlate.id'): - copy_if_changed(d / name, out / name) - break - if update and shader_carry(out): - say(' the shaders prepared before this update stay in use; the game translates them again in the background') - if loose_shaders(out) >= PACK_LOOSE_MIN: - pack_shader_cache(out) - return - cmd = [sys.executable, '-u', str(KIT / 'tools' / 'prepare_shaders.py'), str(out), '--romfs', str(out / 'romfs'), - '--exefs', str(out / 'exefs'), '--no-list', '--exe', EXE_NAME] - proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, encoding='utf-8', - errors='replace', bufsize=1) +def shader_run(cmd, cwd=None): + """Runs a shader preparation command and returns its exit code. Its output is read as it comes: the counts + nx2navi.exe prints become progress events, the other lines go on to the log, and while nothing moves an event + every few seconds keeps saying the step is running and for how long.""" + proc = subprocess.Popen(cmd, cwd=cwd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, + encoding='utf-8', errors='replace', bufsize=1) t0 = time.monotonic() state = {'phase': None, 'done': 0, 'total': 0, 'moved': t0} lock = threading.Lock() @@ -1435,6 +1417,56 @@ def step_shaders(out, update=False): proc.stdout.close() stop.set() beat.join() + return rc + + +def warm_driver_cache(out): + """After an update that carries the shaders over: every prepared stage library is built into the graphics driver's + cache again (`nx2navi.exe --warm-shaders`). It takes seconds while the driver still has them. It takes minutes when + the update changed what a library is built with, which gives each one another entry in the driver's cache: without + this run every pipeline a draw first needed then compiled in play, which skipped that draw for some frames or made + it wait (on an RX 7900 XT 100 to 340 ms a library against 3 ms from the cache, after dynamic vertex input was + turned off for that driver). A run that fails is not an error of the build: the game checks the cache at every + start and warms it in the background itself (runtime/vk_pipeline.c lw_start).""" + exe = out / EXE_NAME + if not exe.is_file() or not (out / 'shader-prepared.bin').is_file(): + return + say(" the prepared shaders are built into the graphics driver's cache again") + t0 = time.monotonic() + try: + rc = shader_run([str(exe), '--warm-shaders'], cwd=str(out)) + except OSError as e: + rc = str(e) + if rc: + say(f' that did not finish ({rc}); the game does it in the background while it runs') + else: + say(f" driver's cache warm in {time.monotonic() - t0:.0f} s") + + +def step_shaders(out, update=False): + """Every shader of the game files into the game's shader cache and the driver's own cache, before the first start + (shader_run reports its progress). + update: after a kit update the shaders prepared before stay in use and the game translates them again in the + background (shader_carry), so the update does not wait the minutes a full preparation takes; the driver's cache + gets their stage libraries again (warm_driver_cache).""" + # The game translates with the copy beside it. The compile step refreshes that copy, but a kit that changes only + # the translator runs this step again alone (step_key), which would then prepare with the old one. Only a kit that + # has one: the compile step, which always runs first, stops a build without it. + for d in (KIT / 'bin', KIT / 'build' / 'xlate'): + if (d / XLATE_NAME).is_file(): + note_cache_translator(out) + for name in (XLATE_NAME, 'nx2navi_xlate.id'): + copy_if_changed(d / name, out / name) + break + if update and shader_carry(out): + say(' the shaders prepared before this update stay in use; the game translates them again in the background') + if loose_shaders(out) >= PACK_LOOSE_MIN: + pack_shader_cache(out) + warm_driver_cache(out) + return + cmd = [sys.executable, '-u', str(KIT / 'tools' / 'prepare_shaders.py'), str(out), '--romfs', str(out / 'romfs'), + '--exefs', str(out / 'exefs'), '--no-list', '--exe', EXE_NAME] + rc = shader_run(cmd) if rc: fail('preparing the shaders failed (the lines above say why); the game still runs, compiling shaders as it ' 'meets them. Run the build again to retry this step')