mirror of
https://github.com/tangerineturtle1/NX2navi.git
synced 2026-11-17 15:49:57 -04:00
Keep the driver's cache of stage libraries warm, link a missing pipeline instead of skipping its draw, and count the texture cache's waits in time.
The preparation builds every stage library once so the driver's disk cache holds it and a library a draw needs later takes under a millisecond. Turning dynamic vertex input off for AMD's RDNA 3 and newer driver changed the dynamic states every library lists, which gave each one another entry in the driver's cache, and nothing built them again: an update carried the prepared shaders over without the driver step. On an RX 7900 XT (26.10.1) all 42,142 prepared libraries were cold, 93 ms a vertex and 242 ms a fragment library on average. With no pipeline recorded, a walk through a village skipped 11,180 draws while their pipelines built and made 94 draws wait 5.5 s in all, four of them 1.0 to 2.0 s; a player's session log shows up to 114,834 draws skipped in 600 frames and 181 waits of 251 ms on average. Skipped ground and walls appear and disappear, and a skipped lighting pass is a black frame with the interface intact. - The game checks the driver's cache at every start (vk_pipeline.c lw_start): a sample of the prepared fragment libraries is built, and when their median time says the cache is cold the whole prepared list is built into it again on idle-priority threads while the game runs. shader-warm.txt records a warm in progress, so the next start goes on with one a run ended, and stops the tries on a driver that does not keep the libraries. This also covers a driver update and a cache the driver emptied. - An update that carries the prepared shaders over runs the game's --warm-shaders (tools/make_game.py warm_driver_cache): 11 s with the libraries there, about 8 minutes with all of them cold on that card. - A draw whose pipeline is missing waits for its fast link from a budget of 6 ms a presented frame, 3 ms at most each (NX2NAVI_PIPE_LINK_MS), before it is skipped: with warm libraries the link is done within that. - Full-screen draws wait from their 50 ms budget again instead of being skipped (NX2NAVI_PIPE_SKIP = any is the default; fullscreen stays selectable). With warm libraries the wait is a millisecond. The same walk afterwards: libraries 0.3 and 0.7 ms on average, 1,605 draws skipped, 1,473 of them in the scene's first 600 frames and 8 to 80 in each 600 after, and 19 waits of 60 ms in all. First visits to four other saves (a mountain pass, the Lost Woods, Kokiri Forest, a river) waited 12 to 84 ms in all except the Lost Woods, where mesh shader pipelines, which have no stage libraries, and vertex libraries that need added outputs still compiled once: 23 waits, 2.1 s in all, the longest 776 ms. The texture cache counted its waits in presents chosen at 60 a second: at 165 presents a second the 1800 that stood for thirty seconds were 11 s, so a released render target went sooner than meant and a texture out of view for 11 s was retired and showed its stand-in for some frames when it came back. The counts are now taken times the present rate over 60 (presents_for), and a texture is retired after two minutes without a sample instead of thirty seconds. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
@@ -1372,30 +1372,12 @@ def shader_carry(out):
|
||||
return (out / 'shader-prepared.bin').is_file() and bool(cache_identities(out))
|
||||
|
||||
|
||||
def step_shaders(out, update=False):
|
||||
"""Every shader of the game files into the game's shader cache and the driver's own cache, before the first start.
|
||||
Its output is read as it comes: the counts nx2navi.exe prints become progress events, the other lines go on to
|
||||
the log, and while nothing moves an event every few seconds keeps saying the step is running and for how long.
|
||||
update: after a kit update the shaders prepared before stay in use and the game translates them again in the
|
||||
background (shader_carry), so the update does not wait the minutes a full preparation takes."""
|
||||
# The game translates with the copy beside it. The compile step refreshes that copy, but a kit that changes only
|
||||
# the translator runs this step again alone (step_key), which would then prepare with the old one. Only a kit that
|
||||
# has one: the compile step, which always runs first, stops a build without it.
|
||||
for d in (KIT / 'bin', KIT / 'build' / 'xlate'):
|
||||
if (d / XLATE_NAME).is_file():
|
||||
note_cache_translator(out)
|
||||
for name in (XLATE_NAME, 'nx2navi_xlate.id'):
|
||||
copy_if_changed(d / name, out / name)
|
||||
break
|
||||
if update and shader_carry(out):
|
||||
say(' the shaders prepared before this update stay in use; the game translates them again in the background')
|
||||
if loose_shaders(out) >= PACK_LOOSE_MIN:
|
||||
pack_shader_cache(out)
|
||||
return
|
||||
cmd = [sys.executable, '-u', str(KIT / 'tools' / 'prepare_shaders.py'), str(out), '--romfs', str(out / 'romfs'),
|
||||
'--exefs', str(out / 'exefs'), '--no-list', '--exe', EXE_NAME]
|
||||
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, encoding='utf-8',
|
||||
errors='replace', bufsize=1)
|
||||
def shader_run(cmd, cwd=None):
|
||||
"""Runs a shader preparation command and returns its exit code. Its output is read as it comes: the counts
|
||||
nx2navi.exe prints become progress events, the other lines go on to the log, and while nothing moves an event
|
||||
every few seconds keeps saying the step is running and for how long."""
|
||||
proc = subprocess.Popen(cmd, cwd=cwd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True,
|
||||
encoding='utf-8', errors='replace', bufsize=1)
|
||||
t0 = time.monotonic()
|
||||
state = {'phase': None, 'done': 0, 'total': 0, 'moved': t0}
|
||||
lock = threading.Lock()
|
||||
@@ -1435,6 +1417,56 @@ def step_shaders(out, update=False):
|
||||
proc.stdout.close()
|
||||
stop.set()
|
||||
beat.join()
|
||||
return rc
|
||||
|
||||
|
||||
def warm_driver_cache(out):
|
||||
"""After an update that carries the shaders over: every prepared stage library is built into the graphics driver's
|
||||
cache again (`nx2navi.exe --warm-shaders`). It takes seconds while the driver still has them. It takes minutes when
|
||||
the update changed what a library is built with, which gives each one another entry in the driver's cache: without
|
||||
this run every pipeline a draw first needed then compiled in play, which skipped that draw for some frames or made
|
||||
it wait (on an RX 7900 XT 100 to 340 ms a library against 3 ms from the cache, after dynamic vertex input was
|
||||
turned off for that driver). A run that fails is not an error of the build: the game checks the cache at every
|
||||
start and warms it in the background itself (runtime/vk_pipeline.c lw_start)."""
|
||||
exe = out / EXE_NAME
|
||||
if not exe.is_file() or not (out / 'shader-prepared.bin').is_file():
|
||||
return
|
||||
say(" the prepared shaders are built into the graphics driver's cache again")
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
rc = shader_run([str(exe), '--warm-shaders'], cwd=str(out))
|
||||
except OSError as e:
|
||||
rc = str(e)
|
||||
if rc:
|
||||
say(f' that did not finish ({rc}); the game does it in the background while it runs')
|
||||
else:
|
||||
say(f" driver's cache warm in {time.monotonic() - t0:.0f} s")
|
||||
|
||||
|
||||
def step_shaders(out, update=False):
|
||||
"""Every shader of the game files into the game's shader cache and the driver's own cache, before the first start
|
||||
(shader_run reports its progress).
|
||||
update: after a kit update the shaders prepared before stay in use and the game translates them again in the
|
||||
background (shader_carry), so the update does not wait the minutes a full preparation takes; the driver's cache
|
||||
gets their stage libraries again (warm_driver_cache)."""
|
||||
# The game translates with the copy beside it. The compile step refreshes that copy, but a kit that changes only
|
||||
# the translator runs this step again alone (step_key), which would then prepare with the old one. Only a kit that
|
||||
# has one: the compile step, which always runs first, stops a build without it.
|
||||
for d in (KIT / 'bin', KIT / 'build' / 'xlate'):
|
||||
if (d / XLATE_NAME).is_file():
|
||||
note_cache_translator(out)
|
||||
for name in (XLATE_NAME, 'nx2navi_xlate.id'):
|
||||
copy_if_changed(d / name, out / name)
|
||||
break
|
||||
if update and shader_carry(out):
|
||||
say(' the shaders prepared before this update stay in use; the game translates them again in the background')
|
||||
if loose_shaders(out) >= PACK_LOOSE_MIN:
|
||||
pack_shader_cache(out)
|
||||
warm_driver_cache(out)
|
||||
return
|
||||
cmd = [sys.executable, '-u', str(KIT / 'tools' / 'prepare_shaders.py'), str(out), '--romfs', str(out / 'romfs'),
|
||||
'--exefs', str(out / 'exefs'), '--no-list', '--exe', EXE_NAME]
|
||||
rc = shader_run(cmd)
|
||||
if rc:
|
||||
fail('preparing the shaders failed (the lines above say why); the game still runs, compiling shaders as it '
|
||||
'meets them. Run the build again to retry this step')
|
||||
|
||||
Reference in New Issue
Block a user