From 2dd363b86ee5ae61c3604e5a76b5ee1de523326c Mon Sep 17 00:00:00 2001 From: chrg6 Date: Sat, 14 Nov 2026 11:33:02 -0500 Subject: [PATCH] Count the texture cache's short waits in seconds below 60 presents a second too. The cache counts its short waits in presents (PRESENTS(n): n presents at 60 a second) and scales them by the presents a second. Above 60 it always did. Below 60 the rate stopped at 60, so the 120 presents a released texture stays out of the VRAM pressure's reach (STALE_FRAMES) were 4 s at 30 presents a second and 5.7 s at 21. The game lets go of textures by the second, as the player moves, so what that window holds grows with its length. A tester's log (RTX 4070 SUPER, 2560x1440, DLSS Quality with frame generation, texture detail Full, 0.10.2-rc4): the card went to 11,198 MiB against a budget of 10,600 while the frame rate fell from 63 to 21 fps. The released textures held 2,644 MiB against their bound of 384, and 2,105 MiB of that was released too lately to go (442 MiB when the pressure began). On an RTX 5090 with the budget taken as 10,600 MiB and the tester's output settings, a walk through Hyrule Field at a 30 fps limit: the released-too-lately textures were all in their first 120 presents (the new census line below) and held 383 and 429 MiB on average at the memory reports of two runs, against 292 MiB at 60 fps. With this change they are in their first 60 presents and hold 209 MiB. Textures made again after a drop: 295 and 523 before, 511 after. Not shown: the tester's 21 fps and a card that is really over its budget, which this card does not reach; the memory in use as the driver counts it stayed at 9.8 GiB in the capped runs. The rate now follows the presents down to 15 a second (vk_tex_release.h trl_rate_sample, trl_rate, trl_presents), once eight seconds in a row were under 60, and never in a run's first eight seconds; one slow second among fast ones still shortens nothing. Every wait PRESENTS counts follows it. NX2NAVI_TEX_RATE_LOW=0 stops the rate at 60 as before, and one line says when the rate crosses 60. Values at 60 presents a second and above are the ones before. The census of the released textures over their bound gets a second line: what the ones released too lately wait for (their first presents after the release, content only the GPU made, the gap before a take-back with or without a lookup still reaching them). It is what showed which wait held the memory. tests/tex_release_check has the rate's checks and a table of the wait by presents a second; they fail against the rule before. Co-Authored-By: Claude Fable 5.2 --- docs/releases/v0.11.0.md | 7 ++++++ nx2navi.ini.example | 7 ++++++ runtime/cfg.c | 2 +- runtime/vk_tex_release.h | 33 +++++++++++++++++++++++++ runtime/vk_texture.c | 52 ++++++++++++++++++++++++++++++++------- tests/tex_release_check.c | 50 +++++++++++++++++++++++++++++++++++++ 6 files changed, 141 insertions(+), 10 deletions(-) diff --git a/docs/releases/v0.11.0.md b/docs/releases/v0.11.0.md index 9a754417..be4b1b95 100644 --- a/docs/releases/v0.11.0.md +++ b/docs/releases/v0.11.0.md @@ -85,6 +85,13 @@ cards). it was found in): the first memory report asked Windows for a summary of the game's heap, and that summary holds every allocation of the game while it is taken. It is now taken before play only (`mem_census = 1` asks for it in play too, for a diagnosis). +- **Video memory staying over the card's budget once the frame rate falls under 60 fps** (reported on an RTX 4070 + SUPER at 2560x1440 with DLSS and frame generation, where the frame rate went from 63 to 21 fps): a texture the game + has just let go of is kept out of the video memory pressure's reach for 2 seconds, counted in frames. Under 60 fps + the count did not shrink, so the 2 seconds were 4 at 30 fps and nearly 6 at 21, and the textures waiting them out + held 2.1 GB on that card. The wait is 2 seconds at any frame rate from 15 fps up now. On our own PC at a 30 fps + limit those textures held 209 MB on average where they held about 400. Not yet confirmed on a card that is really + short of video memory: please report how it goes (`tex_rate_low = 0` brings the old count back). - **Crashes and hangs found in a review of the whole runtime**; none was seen in our own play, and each fix has a check of its own: - a crash when the game read memory the lazy write-back was just handing to the GPU (NVIDIA and Intel cards); diff --git a/nx2navi.ini.example b/nx2navi.ini.example index 5fec9d25..23f4e521 100644 --- a/nx2navi.ini.example +++ b/nx2navi.ini.example @@ -432,6 +432,13 @@ # game does this all the time: 52,000 second uploads in three minutes without it, 600 with it. 0: only the small ones, # as before (for a comparison; nothing should look different). Read at start. # tex_same_big = 1 +# The texture cache counts its short waits in frames: a texture the game let go of may be dropped under video memory +# pressure 2 seconds later, one nothing sampled for 2 seconds counts as idle, and so on. Above 60 fps those frame +# counts always grew with the frame rate, so that the seconds stayed. Below 60 fps they did not shrink: the 2 seconds +# were 4 at 30 fps and nearly 6 at 21, and the textures waiting them out held about three times the video memory on a +# card that was already over its budget, which kept the frame rate down. 1 (the default) counts the seconds below 60 +# fps too, down to 15 fps, once the game has been under 60 fps for 8 seconds; 0: as before. Read at start. +# tex_rate_low = 1 # Texture compaction (experimental). Video memory is taken in blocks (vma_block_mb), and a block goes back to the # driver only once everything in it is freed: after a long session the blocks hold textures scattered between holes, # and the game holds a gigabyte or two of video memory it does not use. 1: textures no shader sampled for ten seconds diff --git a/runtime/cfg.c b/runtime/cfg.c index da2947df..72cd5c1b 100644 --- a/runtime/cfg.c +++ b/runtime/cfg.c @@ -66,7 +66,7 @@ static const char *const known[] = { "MVOBJ_ALL", "MVOBJ_BANKS", "MVOBJ_EQUAL", "MVOBJ_EXACT", "MVOBJ_EXEC", "MVOBJ_NOC4", "MVOBJ_ONLY", "MVOBJ_TIMES", "MVPREV_CHECK", "MVPREV_GROW", "MVPREV_SKIP", "MVPREV_TRUST", "MV_CHECK", "MV_SURVEY", "HANDLE_CENSUS", - "PIPE_SEED_BOOT", "ONE_PER_CORE", "OPERATION_MODE", "OUTPUT_RES", "PAD_LAYOUT", "PACE_HZ", "PASS_CRUMBS", "PATCH_OUTPUT", "PATCH_RES_FIXED", "PATCH_SCENE", "PATCH_WIDE_ASPECT", "PATCH_WIDE_HUD", "PATCH_WIDE_MOVIE", "PATCH_WIDE_OUT", "PATCH_FREECAM", "PATCH_OCCLUSION_OFF", "PATCH_WIDE_SAVE", "PATCH_WIDE_SCENE", "PCSS", "PCSS_K", "PERF_OVERLAY", "PERF_SLOW_MS", "PIPELINE_CACHE", "PIPELINE_CACHE_MAX_MB", "PIPELINE_CACHE_PLAY", "PIPELINE_CACHE_PLAY_MB", "PIPE_BINARIES", "PIPE_BINARIES_MAX_MB", "PIPE_FEEDBACK", "PIPE_MEMO", "PIPE_PREBUILD", "PIPE_SEED", "PIPE_SKIP", "PIPE_SYNC", "PIPE_WAIT", "POWER_FULL", "TEX_SAME", "TEX_SAME_BIG", "TEX_FLAT3D", "TEX_KEEP_TAKEN", "TEX_HARD_ORDER", "TEX_STALE_KEPT", "ASTC_SCRATCH_TRIM", "PRESENT_HELD", "HELD_SYNC", + "PIPE_SEED_BOOT", "ONE_PER_CORE", "OPERATION_MODE", "OUTPUT_RES", "PAD_LAYOUT", "PACE_HZ", "PASS_CRUMBS", "PATCH_OUTPUT", "PATCH_RES_FIXED", "PATCH_SCENE", "PATCH_WIDE_ASPECT", "PATCH_WIDE_HUD", "PATCH_WIDE_MOVIE", "PATCH_WIDE_OUT", "PATCH_FREECAM", "PATCH_OCCLUSION_OFF", "PATCH_WIDE_SAVE", "PATCH_WIDE_SCENE", "PCSS", "PCSS_K", "PERF_OVERLAY", "PERF_SLOW_MS", "PIPELINE_CACHE", "PIPELINE_CACHE_MAX_MB", "PIPELINE_CACHE_PLAY", "PIPELINE_CACHE_PLAY_MB", "PIPE_BINARIES", "PIPE_BINARIES_MAX_MB", "PIPE_FEEDBACK", "PIPE_MEMO", "PIPE_PREBUILD", "PIPE_SEED", "PIPE_SKIP", "PIPE_SYNC", "PIPE_WAIT", "POWER_FULL", "TEX_SAME", "TEX_SAME_BIG", "TEX_FLAT3D", "TEX_KEEP_TAKEN", "TEX_RATE_LOW", "TEX_HARD_ORDER", "TEX_STALE_KEPT", "ASTC_SCRATCH_TRIM", "PRESENT_HELD", "HELD_SYNC", "PRESENT_ORDER", "PREPARE_PIPELINES", "PREWARM_FULL", "PREWARM_FULL_S", "PREWARM_WAIT_MS", "PREWARM_XLATE", "PROBE_MOVIE", "PROBE_MOVIE_SHOT", "PROFILE", "PROFILE_CHAINS", "PROFILE_DIR", "PROFILE_FRAMES", "PROFILE_REPLAY", "PROFILE_SLOW", "POOL_FAST", "POOL_GROUP", "POOL_GROUP_CHECK", "POOL_PRESNAP", "POOL_PRESNAP_RUNS", diff --git a/runtime/vk_tex_release.h b/runtime/vk_tex_release.h index b3a26832..dfa05d4e 100644 --- a/runtime/vk_tex_release.h +++ b/runtime/vk_tex_release.h @@ -139,3 +139,36 @@ static inline uint32_t trl_clock_age(const TrlClock *c, uint64_t frame, uint32_t uint32_t s = c->sec[lo % TRL_CLOCK_N]; return now_sec > s ? now_sec - s : 0; } + +// The rate the short waits go by. vk_texture.c counts them in presents (PRESENTS(n): n presents at 60 a second: how +// long after its release a texture may go, how long without a sample makes one idle) and scales them by the presents +// a second, kept as a multiple of 60 in 1/256. Above 60 they always were scaled. Below 60 they were not: the rate +// stopped at 60, so "120 presents" was 4 s at 30 presents a second and 5.7 s at 21, and every texture the game let go +// of stayed out of the VRAM pressure's reach that much longer. The game lets go by the second, as the player moves, so +// what the window holds grows with its length: in a tester's log (RTX 4070 SUPER at 2560x1440, 2026-11-14) it held +// 442 MiB at 60 presents a second and 2,105 MiB at 21, on a card then 600 MiB over its budget. The rate follows the +// presents down to TRL_RATE_MIN now (low 1; 0: stops at 60, as before). +#define TRL_RATE_60 256u +#define TRL_RATE_MIN 64u // 15 presents a second: fewer count as 15 +#define TRL_RATE_MAX 2048u // 480 presents a second +static inline uint32_t trl_rate_sample(uint64_t presents, uint64_t ns, int low) { + if (!ns) return TRL_RATE_60; + uint64_t r = presents * 256ull * 1000000000ull / (ns * 60); + uint32_t least = low ? TRL_RATE_MIN : TRL_RATE_60; + return r < least ? least : r > TRL_RATE_MAX ? TRL_RATE_MAX : (uint32_t)r; +} + +// The rate of the last n samples (one a second): the highest of them, so that a slow second or a few (an area load, a +// movie's start) shorten nothing, and only n slow seconds in a row do. A sample not taken yet (0) counts as 60 a +// second: the first n seconds of a run never go below it. +static inline uint32_t trl_rate(const uint32_t *ring, int n) { + uint32_t m = TRL_RATE_MIN; + for (int i = 0; i < n; i++) { + uint32_t v = ring[i] ? ring[i] : TRL_RATE_60; + if (v > m) m = v; + } + return m; +} + +// n presents at 60 a second, in presents at the rate: the same wall time. +static inline uint64_t trl_presents(uint64_t n, uint32_t rate) { return n * rate >> 8; } diff --git a/runtime/vk_texture.c b/runtime/vk_texture.c index a3b770db..b91eccf2 100644 --- a/runtime/vk_texture.c +++ b/runtime/vk_texture.c @@ -179,11 +179,16 @@ static uint32_t submit_epoch = 1; // bumped per queue submit: draw bindings ch // were set at 60 presents a second. Without a limit the game presents 85 to 240 a second (a menu), and "120 presents" // was then half a second and "30 s without a sample" 7 to 20 s: under memory pressure textures went three times as // readily as meant, and the ones still on screen showed their dark stand-ins until they were made again. -// tex_rate is the presents per second as a multiple of 60 in 1/256 (256 at 60 or fewer), the highest of the last RATE_N -// seconds so that one slow second does not shorten the thresholds; PRESENTS(n) is n presents at 60 in today's presents. +// tex_rate is the presents per second as a multiple of 60 in 1/256, the highest of the last RATE_N seconds so that one +// slow second does not shorten the thresholds; PRESENTS(n) is n presents at 60 in today's presents. +// NX2NAVI_TEX_RATE_LOW (default 1; 0: as before): the rate follows the presents below 60 a second too, down to 15 +// (vk_tex_release.h trl_rate_sample). It stopped at 60, so at 21 presents a second a texture stayed out of the VRAM +// pressure's reach for 5.7 s after its release and not 2, and the released ones held 2.1 GiB of a 12 GiB card that +// was over its budget. #define RATE_N 8 -static uint32_t tex_rate = 256; -#define PRESENTS(n) ((uint64_t)(n) * tex_rate >> 8) +#define RATE_SAY 16 +static uint32_t tex_rate = TRL_RATE_60; +#define PRESENTS(n) trl_presents((uint64_t)(n), tex_rate) // The wall clock of the presents (vk_tex_release.h TrlClock): rate_tick notes the present count about once a second, // so that the released textures' hold (drop_aged) and the take-back ages (vk_texture_reuse_log) count seconds, not @@ -215,12 +220,20 @@ static void rate_tick(void) { // each submission, tex_lock held if (!t0) t0 = t, f0 = f, pclock_t0 = t, trl_clock_tick(&pclock, f, 0); if (t - t0 < 1000000000ull) return; trl_clock_tick(&pclock, f, (uint32_t)(play_ns / 1000000000ull)); - uint64_t r = (f - f0) * 256ull * 1000000000ull / ((t - t0) * 60); - ring[at++ % RATE_N] = r < 256 ? 256 : r > 2048 ? 2048 : (uint32_t)r; + static int low = -1, said; + if (low < 0) low = rt_knob("TEX_RATE_LOW", "1") != 0; + ring[at++ % RATE_N] = trl_rate_sample(f - f0, t - t0, low); t0 = t, f0 = f; - uint32_t m = 256; - for (int i = 0; i < RATE_N; i++) - if (ring[i] > m) m = ring[i]; + uint32_t m = trl_rate(ring, RATE_N); + // one line at each crossing of 60, RATE_SAY times a run: the log then shows which waits the drops went by + if ((m < TRL_RATE_60) != (tex_rate < TRL_RATE_60) && said < RATE_SAY) { + said++; + if (m < TRL_RATE_60) + rt_log("vk: the game presents %u frames a second, fewer than 60, for %d seconds: the textures' waits count " + "their seconds in these (2 s are %llu presents, not 120; NX2NAVI_TEX_RATE_LOW=0: 120)", + m * 60 >> 8, RATE_N, (unsigned long long)trl_presents(120, m)); + else rt_log("vk: the game presents 60 frames a second or more again: the textures' waits count 60 or more"); + } tex_rate = m; } // Mirrored textures the GPU wrote into, sorted by address (vk_texture_cpu_stale, vk_texture_write_back). Changed under @@ -3425,6 +3438,12 @@ static void rel_census(uint64_t cap, int room) { // tex_lock held enum { YOUNG, SLOT, SLOT_OVER, SAMPLED, LOOKED, NEXT, KINDS }; uint64_t by[KINDS] = {0}, gpu = 0; int n[KINDS] = {0}, ngpu = 0; + // the young ones by what they wait for: fewer than STALE_FRAMES since the release; content only the GPU made + // (GPU_STALE_FRAMES or the gap); intact and waiting out the gap (stale_after) with nothing reaching them, which + // the VRAM pressure takes; intact, waiting it out and still reached + enum { Y_NEW, Y_GPU, Y_GAP, Y_REACHED, YKINDS }; + uint64_t yby[YKINDS] = {0}, gap_max = 0; + int yn[YKINDS] = {0}; for (const Tex *e = rel_head; e; e = e->rel_next) { if (!e->bytes) continue; int shown = e->ph && pref[e->ph - 1].refs; @@ -3434,6 +3453,12 @@ static void rel_census(uint64_t cap, int room) { // tex_lock held : !e->overwritten && !idle(e, now) ? LOOKED : NEXT; by[k] += e->bytes, n[k]++; if (!comes_back_intact(e)) gpu += e->bytes, ngpu++; + if (k == YOUNG) { + int y = now - e->rel_frame < STALE_FRAMES ? Y_NEW + : !comes_back_intact(e) && !e->overwritten ? Y_GPU : idle(e, now) ? Y_GAP : Y_REACHED; + yby[y] += e->bytes, yn[y]++; + if (y >= Y_GAP && stale_after(e) > gap_max) gap_max = stale_after(e); + } } const double M = 1048576.0; rt_log("vk: released textures over their bound%s: %d entries, %.0f MiB of %.0f MiB (%d of them, %.0f MiB, hold content " @@ -3443,6 +3468,15 @@ static void rel_census(uint64_t cap, int room) { // tex_lock held room ? " (kept past it while the card has room)" : "", nreleased, rel_bytes / M, cap / M, ngpu, gpu / M, n[YOUNG], by[YOUNG] / M, n[SAMPLED], by[SAMPLED] / M, n[SLOT], by[SLOT] / M, n[SLOT_OVER], by[SLOT_OVER] / M, n[LOOKED], by[LOOKED] / M, n[NEXT], by[NEXT] / M); + if (n[YOUNG]) + rt_log("vk: the ones released too lately wait: %d, %.0f MiB, the first %llu presents after a release (2 s at the " + "%u presents a second the waits count); %d, %.0f MiB, with content only the GPU made, %llu presents or their own " + "gap; %d, %.0f MiB, intact, for 8 times their longest gap before a take-back (%llu presents at most) with " + "nothing reaching them: the VRAM pressure takes these; %d, %.0f MiB, intact, for their gap and still " + "reached", + yn[Y_NEW], yby[Y_NEW] / M, (unsigned long long)STALE_FRAMES, tex_rate * 60 >> 8, yn[Y_GPU], yby[Y_GPU] / M, + (unsigned long long)GPU_STALE_FRAMES, yn[Y_GAP], yby[Y_GAP] / M, (unsigned long long)gap_max, yn[Y_REACHED], + yby[Y_REACHED] / M); } static void sweep_pressure(void) { // tex_lock held diff --git a/tests/tex_release_check.c b/tests/tex_release_check.c index 42ae2257..1f883648 100644 --- a/tests/tex_release_check.c +++ b/tests/tex_release_check.c @@ -60,6 +60,55 @@ static void clock_checks(void) { "short run: %u %u %u", trl_clock_age(&c2, 50, 10), trl_clock_age(&c2, 0, 10), trl_clock_age(&c2, 251, 10)); } +// ---- the rate the short waits go by (trl_rate_sample, trl_rate, trl_presents) +static void rate_checks(void) { + const uint64_t S = 1000000000ull; + CHECK(trl_rate_sample(60, S, 1) == 256 && trl_rate_sample(60, S, 0) == 256, "60 presents a second"); + CHECK(trl_rate_sample(240, S, 1) == 1024 && trl_rate_sample(240, S, 0) == 1024, "240 a second"); + CHECK(trl_rate_sample(1000, S, 1) == TRL_RATE_MAX, "1000 a second counts as 480"); + CHECK(trl_rate_sample(30, S, 1) == 128, "30 a second: %u", trl_rate_sample(30, S, 1)); + CHECK(trl_rate_sample(21, S, 1) == 89, "21 a second: %u", trl_rate_sample(21, S, 1)); + CHECK(trl_rate_sample(5, S, 1) == TRL_RATE_MIN && trl_rate_sample(0, S, 1) == TRL_RATE_MIN, "under 15 a second: 15"); + CHECK(trl_rate_sample(30, S, 0) == 256 && trl_rate_sample(0, S, 0) == 256, "the rule before: never under 60"); + CHECK(trl_rate_sample(100, 0, 1) == 256, "no time passed: 60"); + CHECK(trl_rate_sample(63, 3 * S, 1) == 89, "21 a second over three seconds: %u", trl_rate_sample(63, 3 * S, 1)); + // the highest of the samples; one not taken yet counts as 60, so a run's first seconds never go below it + enum { N = 8 }; + uint32_t ring[N] = {0}; + CHECK(trl_rate(ring, N) == 256, "no sample yet: %u", trl_rate(ring, N)); + for (int i = 0; i < N - 1; i++) ring[i] = trl_rate_sample(21, S, 1); + CHECK(trl_rate(ring, N) == 256, "seven slow seconds and one not sampled: %u", trl_rate(ring, N)); + ring[N - 1] = trl_rate_sample(21, S, 1); + CHECK(trl_rate(ring, N) == 89, "eight slow seconds: %u", trl_rate(ring, N)); + ring[3] = trl_rate_sample(60, S, 1); + CHECK(trl_rate(ring, N) == 256, "one second at 60 among them: %u", trl_rate(ring, N)); + ring[3] = trl_rate_sample(144, S, 1); + CHECK(trl_rate(ring, N) == trl_rate_sample(144, S, 1), "one second at 144 among them"); + for (int i = 0; i < N; i++) ring[i] = trl_rate_sample(120, S, 1); + ring[5] = trl_rate_sample(2, S, 1); // an area load: one second with two presents + CHECK(trl_rate(ring, N) == 512, "a slow second among fast ones shortens nothing: %u", trl_rate(ring, N)); + // 120 presents at 60 a second are 2 s at every rate from 15 a second up: STALE_FRAMES, the wait after a release + static const uint32_t fps[] = {15, 21, 30, 45, 60, 90, 144, 240}; + printf("the wait after a release (120 presents at 60 a second), by the presents a second:\n"); + printf(" presents/s before: presents, seconds now: presents, seconds\n"); + for (unsigned i = 0; i < sizeof fps / sizeof *fps; i++) { + uint64_t was = trl_presents(120, trl_rate_sample(fps[i], S, 0)), now = trl_presents(120, trl_rate_sample(fps[i], S, 1)); + double was_s = (double)was / fps[i], now_s = (double)now / fps[i]; + printf(" %10u %16llu %7.2f %13llu %7.2f\n", fps[i], (unsigned long long)was, was_s, + (unsigned long long)now, now_s); + CHECK(now_s > 1.9 && now_s <= 2.0, "%u presents a second: %.2f s", fps[i], now_s); + if (fps[i] >= 60) CHECK(was == now, "%u presents a second: %llu presents, %llu before", fps[i], + (unsigned long long)now, (unsigned long long)was); + } + // under 15 a second the wait is 15 a second's: 30 presents, longer than 2 s + CHECK(trl_presents(120, trl_rate_sample(10, S, 1)) == 30, "10 presents a second: 30 presents"); + // the tester's log (vk_tex_release.h): 21 presents a second. Before, 120 presents were 5.7 s there + double before = (double)trl_presents(120, trl_rate_sample(21, S, 0)) / 21; + CHECK(before > 5.7 && before < 5.72, "the rule before at 21 a second: %.2f s", before); + // the longer waits scale the same way: GPU_STALE_FRAMES (1800 presents at 60, 30 s) + CHECK(trl_presents(1800, trl_rate_sample(30, S, 1)) == 900, "30 s at 30 presents a second: 900 presents"); +} + // ---- the rules static void rule_checks(void) { CHECK(trl_hold(TRL_HOLD_S, TRL_STORAGE) == 60 && trl_hold(TRL_HOLD_S, TRL_GPU) == 120, "holds 60 and 120 s"); @@ -335,6 +384,7 @@ static void hard_checks(void) { int main(void) { clock_checks(); + rate_checks(); rule_checks(); taken_back_checks(); session_checks();