The game draws its scene into one of a few pooled targets of the same size and moves to another whenever its pass list changes, while the camera and the scene carry on. vk_dlaa.c target_changed started a new history at every such change, so DLSS and FSR 3.1 began again from one jittered frame each time: a few soft, grainy frames with dots over the whole picture. The history is the upscaler's own and not in the target, so it now stays where the frame has valid motion vectors (the camera of this frame and of the last one); without them a new history starts as before. The log line of a change says which of the two happened, and the 300-frame line counts the changes with the history kept. NX2NAVI_DLAA_TARGET_KEEP=0 gives a new history at every change.
From pull request #538, whose author counted 35 resets in a walk of a minute through Kokiri Forest before and 1 after. Here it is compiled and the settings probe passes; it was not run in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A pipeline cache over the size limit was dropped at every start and, since the file is no longer written in play, never replaced: every start began empty. The play file is now folded in at the next start whatever its size when the file beside it is over the limit.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
auto turns the fallback on for a device without VK_EXT_mesh_shader, 1 for every device (testing), 0 never.
mesh_fallback_mb sizes the buffers the passes write. Nothing draws through it yet.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Two descriptors can name one CPU-written storage: one whose content the cache takes from the CPU, and a second with a texture flag for which it does not. The second has no upload of its own and its slot shows the first one's image while that entry is live. Once the first is released, a lookup of the second's slot makes it an entry of its own, start_at destroys the released first one, and the new image starts cleared and is never uploaded; the slot shows it from then on, also after the first descriptor is registered again. The same happens when both are released and the texture pool refresh looks the second's slot up before the next texture arrives in that memory.
With tex_split_fill = 1 (the default) start_at gives such an entry the storage's bytes: by a copy from the released CPU-written entry it is about to destroy when that image is current and alias_compatible allows, else by an upload from the storage. The image then follows the storage as a CPU-written texture's does until the GPU or WriteTexels writes it; 2 fills it once, 0 is the behaviour before. Depth, stencil and multisample images, pools without CPU access and storage over 32 MiB are left out, and an entry that is to be filled gets a new image instead of a released entry's, since a reused image goes through its first transition after the upload batch. The perf trace counts the copies, the fills with their bytes and time, the fills made again after a write and the entries left empty.
--probe-vkmem makes both sequences with the three values and reads the second descriptor's image back: zeros with 0, the storage's bytes with 1, and with 2 the bytes of the texture that was in the memory before when both descriptors were let go of. One case makes the entry inside a recorded submission, through a hook that lets the probe stand in for a replay, and there the content comes by the copy.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The library creates its pipelines itself and names no pipeline cache. It is now handed pipeline creates of the runtime's in the place of the device's two, and a create without a cache is asked of the pipeline cache first and compiled into the play cache otherwise. ffx_pipe_cache = 0 turns it off. --probe-ffx-cache makes one upscaling context without a window or the game and says where its pipelines came from, and tests/ffx_cache_check.py runs it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The fence's wait cost about 4.5 % of the frames a second, and the player who reported the flicker still sees it and black frames with it. wb_split = 0 keeps the fence's wait for whoever wants to compare.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
pipeline-cache.bin was copied out of the driver and written whole at every save after any pipeline was created, also one the cache had held: tonight's logs show three saves of 358.1 MB in a run that added 0.3 MB, and 302 to 306 MB written in each of seven sessions that added about 0.5 MB each. The library warm in play built its 37,572 stage libraries into the same cache, which grew it by 1.6 GB: a save of 2026.7 MB in play, a file over pipeline_cache_max_mb, and the next start dropped it whole and compiled the pipelines it had held.
cache_save now asks a cache for the size of its data first and neither copies nor writes one whose size is what its file holds. What a run compiles goes into a second cache with its own file, pipeline-cache-play.bin (pipeline_cache_play, default 1; needs pipelineCreationCacheControl): pipe_create_in asks the first cache with VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED_BIT and builds into the play cache only when that says a compile is needed, and in play only the play file is saved. At a start the play file is read back and merged into the first cache in memory; once it reached pipeline_cache_play_mb (64) it is folded in for good: pipeline-cache.bin is written once, before play, and the play file removed. The preparation modes fold it at any size and build into the one file as before. The library warm's threads create with no cache of ours, so their libraries reach the driver's own cache alone.
Checked with --probe-prewarm on a copy of a dev folder's caches (3,212 recorded pipelines, 333 compute shaders), no game run: with a filled pipeline-cache.bin every pipeline came from it and neither file was written; with none and a cold driver cache the compiles went into pipeline-cache-play.bin (75.9 MB written, pipeline-cache.bin not created); the next start merged it and wrote pipeline-cache.bin once (191.1 MB) and removed it; kept below the fold size, a play file of 68.8 MB gave 1,445 pipelines at 0.3 ms from the first cache where 470 hit without the merge, and was not written again. Not checked in play: the saves during a session, the library warm without a cache, and drivers other than NVIDIA's.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
For a day that wait ended on a second timeline, when the GPU had run the submission's own commands (wb_split = 1,
then the default), on the thought that it only left out write-backs nothing needed yet. It left out the completion
thread's side effects for the submissions before it as well, and the game reads some of those next: the occlusion
counts, counter reports and event values, and the stamps the texture checks read. A frame's first submission is small
and runs right behind the last frame's large one, so the game went on while that one's callbacks were still running.
Measured in Hyrule Field with the camera turning (RTX 5090, lazy write-back on): 2,267 of 2,320 such waits returned
before those side effects had run; they were through 0.45 to 0.8 ms later on average, 12.6 ms at most, and in that
time 770 to 1,015 occlusion counts, 21 to 28 counter reports and 17 event values a frame reached guest memory. The
submission the game waits for wrote nothing back in any of 14 runs, so its own write-backs were never the matter.
A player saw a patch of grass near the camera change from frame to frame there and less of it with the fence's wait.
No scripted run showed such a frame with either wait, so the link to what the player saw is not shown.
The fence's wait takes 0.65 to 0.7 ms more of a frame there, about 6 % at 85 frames a second (single runs beside
other work). wb_split = 1 still gives the other wait, and 2 forces it with shadow storage.
NX2NAVI_TRACE=perf now counts what a wait on that timeline leaves behind and the completion thread's time by callback.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit 4d108b706e72c001ddbe20608f06469885df584b)
The game held 23.9 to 24.4 GB of commit in three sessions on an RTX 5090 at 3840x2160, flat over the session, on a
machine whose commit stood at 66 of 73 GB with it. A machine that cannot give it that much pages, drops textures or
loses the device, and its log said nothing about memory until then.
mem_start_note reads the system's commit when a game starts and writes one warning line when what is free, plus what
the process already holds, is below mem_need_mb (24576 by default, 0 for no warning). The menu has no place for a
start-up notice (its only toasts are the co-op page's), so the line is in the log alone.
tests/mem_need_check.py checks the rule and, through --probe-memneed, the answer and the warning line with the
machine's own commit.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Only the large compute shaders' background builder looked at the system's commit before it started a build, and it
saw nothing of what the builds beside it were about to take. The boot's recorded pipelines (up to 64 builders), the
pool's background builds during play, the boot's compute pipelines and stage libraries and the library warm asked
nobody.
Each of them now asks the gate before a build. A build that starts reserves an estimate of the driver compiler's
working memory until it ends (build_reserve_mb, 64 MB; 4096 MB for a large compute shader), and the gate admits a
build only while the commit headroom holds the part kept free (build_free_mb, 4096 MB), the reservations and its own.
A refused boot builder ends and leaves its pipelines to the pool; the pool leaves a background build queued and asks
again every two seconds; the stage library prebuild and the library warm wait up to 30 seconds and then end, and the
warm goes on at the next start. A pipeline a draw or a dispatch waits for is never refused. The preparation of the
pipeline list runs without the gate: no game runs beside it.
tests/build_gate_check.py checks the gate's arithmetic and, through --probe-buildgate, its answers with the machine's
own commit. What the builders do when refused is not exercised by a test.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A lost device used to end the process from whichever submit or wait noticed it first, while the present only logged it
and went on and the other threads kept calling the driver. The completion thread could take the submissions' timeline
as reached and run the callbacks of submissions that never finished.
vk_device_lost_check now takes the result of every submit, present, acquire and wait. The first thread to see the loss
sets a flag, writes the report and ends the process as a fatal error does (the log written, a save commit under way
finished), with an exit code of its own; later threads stop in that call, and a watchdog ends the process after ten
seconds should the first thread hang. The replay, its submit, the uploads, the present and the lazy fetch read the flag
and return without touching the device. The completion thread asks the device with a call of its own before a new
value counts and runs no callback once the flag is set.
The executable's --probe-devlost and tests/device_lost_check.py check this without a lost device, and
device_lost_at in nx2navi.ini ends a real run the same way.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With the retiring on, a player's session lost the ground of Hyrule Field and Lon Lon Ranch (6,663 retired, 928 back), and with the two holes of the way back closed it lost it again on arriving in a village (1,441 retired, 82 back; neither fix acted once). With the retiring off the same play kept its ground. What keeps the stand-in in place is not found; a draw that samples it once into a picture the game keeps would, and no mark undoes that. tex_disuse_evict = 1 turns the retiring back on; the shaders keep their marks, so no pipeline changes either way. Released textures and idle ones the game no longer names still give way when memory runs short, so a PC that is short of memory has less to give back than before.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
mouse_mode = direct is the default, but on 1.1.1 it did nothing: only the 1.4.3 patch table had the sites, so the mouse always went through the stick emulation, which gives no turn at slow mouse speeds and tops out at about 107 degrees a second. The 1.1.1 table now has the set mouse_direct (on by default): at main+0x37c45c, where the camera's input update holds the processed stick in s1 and s0, a host call performs the displaced store and adds the raw mouse counts to both, so the mouse meets no dead zone and no curve; at main+0x37c4f0 a second call clears the idle bit for an update the mouse moved in. The counts are divided by the frames the game steps per update and by the stick camera speed, so the same hand movement turns the same angle at every frame rate. The Controls tab gets the mouse mode and mouse sensitivity rows, and mouse_mode = stick or NX2NAVI_MOUSE_DIRECT_PATCH=0 gives the old behaviour.
Checked here: the generator lifts the 1.1.1 game with the two sites (it compares each site's original word), the lifted game compiles and links with the runtime, and tests/mouse_direct_check.py passes its 1.1.1 harness (register contract, displaced store and load, equal turn at every frame rate and stick speed, the reset pause, capture and mode). Not checked here: the game was not run, so aiming with the mouse was not tried, and the differential tests against Unicorn were not run because no Python on this PC has Unicorn. The contributor measured 0.0800 degrees per count on both axes in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit bb8fecfbe43517006eafd7be9c22e8dbe9601de7)
Without the other players drawn in the world, which are unfinished. No relay address is built in, and with co-op off nothing opens a socket. Built on Windows and tested against a relay on this PC only; two games in one room and any real network have not been tried.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A second reading of the commits taken in from pull requests found these, each traced in the code:
- heap_lazy: the camera speed's search read the whole heap, which with the setting on committed every page and undid what it saves; a scan now steps over the pages the lazy heap never committed (mem_untouched_run). tests/frame_step_check.py and tests/save_share_check.py no longer linked (frame_rate.c and fs_host.c call the new memory functions; the second also missed mem_probing before); both pass again. The settings text says that with the setting on a PC out of commit ends the game in play, not at start, and the census comment is back over its own function.
- Pipeline libraries on AMD's older Windows driver: the condition is the driver for RDNA 2 and older (RX 400 to RX 6000), not RX 5000 and RX 6000 only, and the texts named a price the code does not have. With libraries off a new pipeline is built by a worker while its draw is left out, and the start waits for every recorded pipeline instead of 20 s at most. The texts say so, and that the finding is one player's on one card. The menu strings nothing shows are as they were.
- Page file hint: it comes only when Windows sizes every page file, and it names the initial size and the automatic management that must be unticked first.
- Frame generation note: it keeps "needs an RTX 40 or newer" for the players it is right for and points the others to the log; the bridge's line says it is Streamline's general support check.
- Comments and settings texts that still described the old behaviour: the process commit bound, the wait with shadow storage, the scratch's least size, the action prompt under auto.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit 19f03aa01b26d6e17fb5fc70ad987aa8c68e8225)
Verified by runs in the audit: the Market's periodic stall (a pool slot was listed once per refresh pass instead of once; the list fell from 160,385 entries to at most 3,192 and no frame over 20 ms remained while standing), orderly exits delete the pipeline prebuild's running marker (left behind, the next start wrote a skip file and every later start skipped the prebuild), the log and crash report name an area a save asks for that the game does not have, and the menu atlas hash no longer overflows a signed multiply. The other fixes compile and pass the regression route but their faults were never reproduced: see AUDIT.md section 13 for each.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This line logs every setting nx2navi.ini sets, and a bug report takes a copy of nx2navi.ini along: with co-op both would have carried a room's secret, the relay's address and the player's name to whoever reads the report, and the secret lets anyone into the room. One list (runtime/cfg_private.h: the room's secret, the offer number, the relay, the name, the key a maker removes, the menu's test hook) now serves the log's line, "Copy system info" and the report's copy of the file, in the game and in the launcher: such a key reads "(set, not shown)", every other byte of the file stays.
Also: the last leftovers of the restorations that were left out with the puppets (their settings prefix and section), main.c's byte order mark as it was, and two comments that cited files this tree does not have.
Checked: with a room, a relay and a name in nx2navi.ini the log's line shows all three as set and none of their values; the same text gives the same bytes through the game's scrub and the launcher's (a launcher test, and the C function built alone); the launcher tests, the co-op probe, the relay tests and the tab check pass.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Each player keeps their own game and save. One player makes a room, the others join it with its secret, and story flags and, by the room's rules, items, songs, quest items, small keys (a shared pool), rupees and ammunition reach every save. The layer is a host thread with one UDP socket to a relay the players run themselves (tools/coop_relay.py, Python's standard library only); the messages between the players are sealed with a key from the room's secret (XChaCha20-Poly1305, Monocypher 4.0.2 vendored with its licence and a row in THIRD_PARTY_NOTICES.md), every change is signed by the player who made it, a player accepts a room by its members' keys and the key of its rules, the save is copied to coop-backup before anything is applied, and values are set through the game's own routines between frames. The Co-op tab makes, joins, accepts, removes and leaves, and the menu takes typed text for the name, the relay and the secret. This is the contributor's pull request #518 on this line.
Nothing reaches the network while co-op is off: the socket is opened, and the relay's name looked up, only after co-op is on, a relay is set and a room is made or joined. There is no default relay.
Left out of the pull request: the other players drawn in the world (the "puppets") with the copied engine they need. They were unfinished, added two patch sites to the game's per-frame update that could not be put through the differential tests here, and their file helpers clashed with the ones this line already has. So no position is sent: the POS message's codec stays, nothing sends it.
Changed against the pull request:
- The menu's test hook drives the Co-op tab only when it comes from the environment. From nx2navi.ini it could turn co-op on, join a room and accept its offer.
- The log's label for a room is hashed from the secret instead of being its first three characters.
- A game that has not saved yet has no save to copy, and takes nothing from the room until it has saved once; that used to count as a copy made.
- A PING's token no longer carries the PC's uptime.
- On Windows the maker's key file and its folder get an access list with the player's account alone.
- The tests that link the platform layer get the socket library, and the save test a stand-in for the title screen's hook.
Checked on Windows, where the pull request had never been compiled: the runtime builds and links; --probe-coop passes (sockets on 127.0.0.1, the codec against its byte vectors, a fuzz of every parser, two sessions through a hostile mock relay, a relay restart, the sync engine); tests/coop_relay_test.py passes its 61 tests, with the probe through the real relay on 127.0.0.1 at 0 and 30 % loss; tests/coop_page_check.py passes and its deliberate fault fails 48 checks. Not checked: two games in one room (one game runs at a time here), any real network, the tab in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Loading a save in Hyrule Field at night lost the device about 2 s after the load in 7 of about 22 loads on an RTX 5090 (a GPU read fault at the start of a large buffer, in or right after one of the game's compute dispatches). With the lazy write-back off it loaded 10 times of 10, and with only the piece release off 15 times of 15. How the piece release leads to the fault is not known; until it is, a CPU touch releases its whole range again, as before the town performance pass. wb_lazy_chunk = 1 turns it back on.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The two sides both made the wait after a frame's first submission cover the write-backs under shadow storage; the rule is kept in one place, vk_sync_wb_split.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
NX2NAVI_SHADOW_CHECK=1 logs with the shadow line, every 60 frames: the whole copy-ins that overwrote a store a
shader made through the device address table, and the pools they fell on; the write-backs that moved bytes into
a page the CPU had written since its copy-in; the bytes both sides changed since their base copy; what the
copy-ins cost by kind, and which pools and flag sets took them. The stores are found by content: a whole
copy-in leaves a hash of the page, and the next one compares the shadow page with it first.
NX2NAVI_SHADOW_TABLE_STORES=1 keeps such stores: a page found stored into gets a base copy, so copy-ins take
only the CPU's bytes, and the fence of every submission that uses the table writes the stores back to guest
memory. --probe-vkmem checks it with a GPU write that nothing notes. It is off by default. The overwrite it
removes was the suspected cause of the flicker on Linux and is not: standing outdoors (RTX 5090 with
NX2NAVI_SHADOW=1) it happens exactly four times a frame, on ten pages in pools of 20 and 44 KiB, with the copy
that precedes the frame's stores, and no frame came out wrong with it or without it in 1,800 frames each; the
hashes cost 1.8 ms a frame of the replay (67 against 75 fps); and a count the CPU resets before the write-back
has landed would keep counting, where the whole copy-in resets it every time.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With shadow storage (the Linux default) what the GPU wrote reaches guest memory in the fence's completion
callback, later than an import's store on the GPU. The wait after a frame's first submission returned at the
executed timeline, before that, so the game rewrote what those passes had written and the write-back then put
the GPU's older bytes back over the rewrite. The whole 3D scene came out brighter for single frames, each of
them a frame whose main work was held at its CPU event wait and replayed late.
Standing outdoors with NX2NAVI_SHADOW=1 on an RTX 5090: 695, 624, 443, 323 and 2 flashed frames of 1,800 in
five runs with the split, none in eight runs without it. The frame rate did not drop (74.6 fps median with the
fix; runs of the build before it gave 71 to 72). NX2NAVI_WB_SPLIT=2 keeps the split with shadow storage, to
measure the fault. That this is the flicker a player reported on Linux (RX 7800 XT, RADV) is not confirmed on
their machine; NX2NAVI_WB_SPLIT=0 gives 0.7.1 the same wait there.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The HDR tonemap gains a mapper built on a finite cone response in LMS: scene grey 0.18 lands where the game's curve puts it and rises through it with the slope of the game's curve, and the hue bend of the game's clamp is given back to bright colours. It takes the scene before the game's exposure gain and the gain itself, which the splice now finds in the game's shader. The luminance mapper stays selectable and the game's own curve stays the third choice. The mapper fits the Rec. 709 gamut here, as the game's scene target holds no negative component. New settings: the tone mapper's mid grey, tone slope match, tone slope and hue clip emulation. The gamma correction on the present applies with the luminance mapper only.
Not part of this: eye adaptation, the anti-aliasing and reconstruction shaders, bloom and local exposure sliders, the output colour space choice and a 16-bit float scene target. Checked by build and numerically against the source of the port; the picture on an HDR display has not been judged. THIRD_PARTY_NOTICES.md names the source.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Windows charges every committed page to the commit limit whether it is used or not, and the game sizes its whole heap up front: in Castle Town after Hyrule Field the heap held 6.8 GiB of commit of which 3.4 GiB had ever been used. With heap_lazy = 1 a change of the heap's size only moves its end, an access to a reserved heap page commits the reserved runs of the 64 KiB around it in the access-fault hook and runs again, and the paths that do not fault commit their range first: mem_thaw (file reads, the host-pointer import), mem_kernel_access (file writes, the frame-rate fix's read) and mem_host_readable (the host's pointer checks). A page another path committed is left as it is, so a frozen, probed or armed page keeps its protection. With trace = perf a line per window gives the faults, the bytes committed and the time they took.
The contributor's runs on a 16 GB PC: 4.67 instead of 6.77 GiB of heap committed and 29.3 instead of 32.9 GiB of process commit in Castle Town, no "system commit low", 12,858 faults in 440 ms over four minutes, the longest 41 ms. With the setting off nothing is hooked and the heap is committed as before.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A new depth or stencil image was cleared in the upload batch, which runs before the whole part being recorded. Bound into pool memory, the image shares its bytes with the live texture at its address, so the clear could wipe depth that earlier draws of the same part still read: the sun's shadowing went fully dark for one frame a few times a minute where the game resizes its shadow maps. The clear now goes in at the replay's position, as it already did for reused images (depth_init_inline, default 1; 0 is the old order). The switch that copied the live image's depth into the new one is removed: every such image was cleared by the game before anything read it, and the copies cost smoothness.
From a contributor's diff and measurements at a town gate: 0 dark frames in 5,423 with the change, 12 in 2,728 without. Checked here on the combined build in Hyrule Field before the Market's gate: every new depth image is started at the replay's position, none in the upload batch, and none had to end an open pass. The dark frames themselves were not counted here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
That driver faults inside vkCreateGraphicsPipelines when it links the stage libraries of a burst of new pipelines. The game asks for one when an enemy dies and its effects are drawn for the first time, so the game ended with code 71 at every kill on an RX 6700 XT and an RX 5700 XT (issue #418), and the player who reported it found that gpl = 0 stops it. Unset, the setting is now off with that driver (vk_caps.amd_sc) and on everywhere else; gpl = 1 turns it on there, gpl = 0 off anywhere, and the caps line of the log says why it is off.
The price on those cards is that a new pipeline is built whole on the draw's path. Not run here: there is no AMD card. On an RTX 5090 the device probe reads "gpl yes" as before and "off by NX2NAVI_GPL=0" with the setting.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Four "mem: sizes" lines give the entries and bytes of what no other line counted: the guest heap, physical maps, kernel region, lazily written ranges and the fixed kernel, setjmp, save-file, RomFS and zstd tables; pool records, mirrors, deferred destructions, upload and staging buffers, the device address table and the texture, sampler and slot caches; image views, descriptor spares, the completion queue, uniform snapshot arenas, recording copies and pins; and pipelines, layouts, stage libraries, job queues, the shader memory cache and the translator processes. Each subsystem reads its own counters under its lock; settings now count the replaced values they keep. Segment and scene lines say "-" for the world clock before it starts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the lazy write-back on, a village and Hyrule Field still merged 20 to 29 MiB a frame back into guest memory at
the end of every submission, inside the GPU work the game's fence waits for. The new log line (mem: lazy write-back
at frame N: written back and not armed since the last line) names the reason by bytes: none of it lay in frozen
regions or on pages that could not be armed; it lay in regions held after a touch. The game's CPU reads a few words
at the start of buffers its shaders fill (12 bytes into ranges of 6.3 and 15.2 MiB among others), about once a
second. Each read gave back the whole armed range and held every 2 MiB region it reached for 1 s, doubling to 32 s,
and nothing was armed in a held region.
A touch now gives back the 64 KiB around its address, fetched alone, and holds only that piece; the rest of the
range stays armed (wb_lazy_chunk, default 1; 0 is the old rule). More than four touches in one region within a
second are taken as a reader going through the range: that range goes whole and its regions are held as before,
and the region's touches go that way at once while it keeps being touched within a minute. Each of the first eight
touches of a log window is logged under the perf trace with its thread, address, range and what was given back.
Measured on one machine (NVIDIA, uncapped, frame generation off, scripted walks, medians over the windows of 46 s
of world clock):
- village: 121.8, 124.1 and 124.4 fps before; 138.0 and 135.4 with this change; GPU busy 4.8 to 4.15 ms a frame
- Hyrule Field: 101.1 and 113.1 fps before; 131.5 and 131.3 after; GPU busy 6.5 to 4.7 ms
- merged a frame: 20 to 29 MiB before, 1.7 to 2.0 MiB after
- faults on armed pages in 1,000 frames: 14 and 16 before, 17 to 18 after in the village; 19 and 23 before, 17 to
21 after in Hyrule Field; time spent fetching the same (4 to 5 ms in 1,000 frames)
A run with synthetic reads of four armed ranges per log window (wb_lazy_touch = 4, 769 faults) ended without an
error. The picture of every presented frame of a village walk and a field walk (thumbnails, 10,131 and 9,531
frames) had no black frame. The village walk has single frames whose brightness jumps: 14 with the change and 28
without it in about 10,000 frames each; the two looked at, one of each, are jumps of the camera.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The default was off only on AMD's driver branch for RDNA 1 and 2, on the grounds that builds with the default on had been measured on an RX 7900 XT. Those measurements were standing in Hyrule Field and walking through a village; the report this is about is of a game that ended in fights, on an AMD card the player did not name, and fights are where the lazy path is busiest. Nobody has played one on an AMD card under our eyes on either branch, so the exception is now AMD's own driver as a whole (wb_lazy_choice.h, by VkDriverId), until a card of each branch has played fights with wb_lazy = 1 and a log to show. The cost is the open-area frame rate of 0.7.0 on every AMD card under Windows; the RX 7900 XT figures in the draft notes come from builds that had it on by default, which the notes now say. wb_lazy = 1 still turns it on.
--probe-vkdevice prints the line a fatal error or a crash adds to the log, so its wording can be read without a failing run. On an RTX 5090: "fatal: gpu NVIDIA GeForce RTX 5090, driver NVIDIA "621.01" version 621.1.0.0; memory mode mirror, lazy write-back on (the default on Windows (wb_lazy = 0 turns it off)); texture detail not chosen yet (no texture made)", and with the reporting player's nx2navi.ini beside the exe "lazy write-back off (wb_lazy = 0 is set)", after a start whose log names async_compute, vram_safe_mode and exception_handler as no settings of this version.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A player on an AMD card reported crashes in fights and that a line added to nx2navi.ini cured them. That line, async_compute = 1, is no setting of the game, and neither are two more in the same file; the one line of the file that changes how memory reaches the GPU is wb_lazy = 0. A second report, an RX 6600 on 0.7.1, lost its device 3.5 minutes into play with the lazy write-back on. Neither proves it the cause: the player changed several lines at once and sent no log, the RX 6600's last pass mark is inside a draw pass and not a fetch, cards with 8 GB lost their device the same way on NVIDIA before the lazy write-back existed, and reading the fault handler, the fetch, the arming and the buffers' lifetimes found no failure path. What depends on the driver and cannot be checked without the card: pages of guest memory are inaccessible while a host-pointer import of them is bound and written by the GPU, and Vulkan calls run inside the access-fault handler.
The default is now decided once the driver is known (wb_lazy_choice.h, vk_memory.c vk_mem_lazy_decide): on under Windows as before, except on AMD's driver branch for RDNA 1 and 2, which none of us has run it on; AMD's driver for RDNA 3 and newer keeps it on, since builds with that default were measured on an RX 7900 XT. What the player sets wins everywhere, and one line at start says the choice and its reason. On those cards open areas run at the 0.7.0 frame rate again (on an RTX 5090 that was 65 fps against 90 in Hyrule Field; not measured on AMD). tests/wb_lazy_check.c checks the choice by platform, driver and setting without a GPU.
For reports of this kind the log now holds what was missing. At start: every setting nx2navi.ini sets, and in one line the keys that are no settings of this version and do nothing; the settings menu shows those in its bottom row. At a fatal error or a crash: one line with the card, its driver, the memory mode, the lazy write-back with its reason, armed ranges, fetches so far and a fetch in progress, and the texture detail (dispatch.c rt_fatal_note_add, vk_device.c fatal_note), since a report is often the end of a log alone.
--probe-cfg loads the player's file: the three lines are named, and the menu's save keeps them and every other line it does not change byte for byte. Nothing here ran on an AMD card, and no fatal error was provoked to read the new line.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The preparation translates every shader and builds each one's stage library, so a pipeline a draw meets for the first time is linked in 2 to 6 ms. A mesh pipeline has no stage libraries: on an install that never drew it, it compiles whole in play, 0.2 to 1.1 s with its draws left out of the picture (18 of them on a first visit of Kokiri Forest). Whole pipelines were built ahead only from the folder's own pipeline-keys.bin.
tools/pipeline_list.py merge folds the pipeline-keys.bin and shader-answers.bin of game folders that were played into one list file: every key record once, with the answers record of each shader it names. A record names its shaders by input id, which depends on the game's code and the answers alone, and holds the game's pipeline state; what depends on the GPU is derived when the pipeline is built, so a record made on one machine is good on another with the same game version and runtime. Four folders on this PC give 5,432 records, 317 of them mesh pipelines (147 KB of keys).
tools/prepare_shaders.py reads the list (tools/data/pipeline_list.bin, data/pipeline_list.bin in a release folder, or --pipeline-list), adds its shader variants to the seed and writes its records as seed-pipeline-keys.bin. nx2navi.exe --prepare-shaders builds the mesh pipelines among them after the seed is translated and saves the pipeline cache before the stage libraries are built (NX2NAVI_PREPARE_PIPELINES=all builds the vertex pipelines too); --warm-shaders does it again after an update or a new driver, and --prepare-pipelines is that step alone. The pipelines go into the caches and are not kept or added to the folder's own key file, so they cost no start-up time or video memory until a draw meets them. A marker file keeps a driver that crashes on one of them from failing every preparation.
The list is derived from game data and stays out of the repository: check_content.py refuses files recorded from a game folder by their first bytes, and make_kit.py, make_launcher.py and make_release.py carry a list given with --pipeline-list. All three now also carry pipeline_list.py, which prepare_shaders.py imports.
Checked without the game: tests/pipeline_list_test.py, and prepare_shaders.py --no-run on the game files with a list of 317 mesh pipelines (all 266 shader variants found in this game version, 177 new to the seed). The build step in nx2navi.exe compiles and links and has not been run.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Each submission's first sync of the bindless texture pool walked every registered slot, 33,000 in Kokiri Forest, to find the few textures the CPU wrote and the few slots that changed. Three parts of that walk change, each with its own switch (pool_fast, pool_presnap, pool_group; on by default, 0 and ab accepted):
- The freshness check goes through a list of the entries the slots resolved to, with a check by 2 MiB region first; slots that resolved to nothing are looked up again when the game's memory map changes and every eighth submission. The use stamps the idle eviction reads are left from a count of the slots that hold each entry.
- The check takes the write-watch snapshots of its regions first, so unchanged entries pass the region test.
- The GPU-written slots of one address bucket with one descriptor form a group, which the dirty walk leaves alone while its first member is unchanged.
Measured on an RTX 5090 at 2560x1440, standing in Kokiri Forest, in-run A/B over five runs with all three switched (three of them with the barrier change switched too): 93.6 to 100.7, 96.4 to 103.1, 95.8 to 102.6, 99.3 to 107.1 and 91.9 to 99.9 frames a second, the replay thread 0.7 to 0.8 ms a frame shorter; a control's blocks differ by 0.4 %. On a walk with NX2NAVI_POOL_GROUP_CHECK=1 no group that was left alone had missed a write, and screenshots across the A and B blocks show the same picture.
From pull request #390. Not tried: the texture eviction under memory pressure (a 9 GiB budget was never reached on the walk), AMD and Intel drivers, Linux. One of the nine runs with it on ended by itself with exit code 0 and nothing in its log two seconds after the window lost the focus; three repeats of that run did not.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Every pass begin and every compute dispatch recorded a full barrier, which makes the GPU finish everything before each of them. With barrier_game = 1 (NX2NAVI_BARRIER_GAME) a pass after an earlier pass gets a barrier for its attachments only and a dispatch after other dispatches gets none; a change between drawing and compute keeps the full barrier, and so do the game's own barriers and the replay's copies, clears, fills and uniform updates. On an RTX 5090 at 2560x1440 the GPU's time for a frame fell from 5.4 to 4.9 ms standing in Kokiri Forest (two in-run A/B runs, the control's blocks differ by 0.7 %) and from 5.1 to 4.4 ms on a walk through it. The frame rate did not change there, because the replay thread is the limit on that machine; it should rise where the GPU is.
The default is 1 on NVIDIA graphics cards only, and 0 on every other vendor's: the game placed its barriers for an NVIDIA GPU, Vulkan asks for more of them than the game recorded, and no AMD or Intel driver has run this. 34 screenshots across the A and B blocks of one run show the same picture on NVIDIA. NX2NAVI_TRACE=perf logs the barriers a frame by kind.
From pull request #389, which had it on for every vendor; the default by vendor and the settings text are added here. The validation layer's synchronization check could not be run on this machine.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Both are about the time between two frames. Ocarina waits for the previous frame's fence before it lets the next frame's GPU work start: its main thread signals an event from the CPU, and the main queue's commands begin with a wait for it.
The completion thread ran a submission's callbacks before it let the submission's waiters go, and the occlusion counts were among them: one vkGetQueryPoolResults and one vkResetQueryPool per reported query, about a hundred queries a frame in Hyrule Field. On an RX 7900 XT (26.10.1) each call goes into the kernel, 3.9 ms of every frame, so the fence was reached 4 to 6 ms after the GPU had finished with every game thread waiting. Slots come from the ring in order, so one call now reads up to 256 slots from the one asked for, the available results are kept for the segments that follow, and the slots taken are reset in runs (vk_exec_xfer.c oq_take, NX2NAVI_OQ_BATCH). Standing in Hyrule Field at 3440x1440 with FSR: 52.7 fps against 44.6, the longest frame of the measured minute 46 ms against 55; the completion thread spends 0.3 ms a frame in the callback.
When the CPU's signal came a millisecond or two after the main queue's submit call, the submission took its copies, stopped at the wait and was held; the render thread polled the event, presented, and the present replayed the frame under a value the game's fence does not stand for. The next frame then skipped its wait for the GPU and was replayed inside its submit call, and frames of 23 and 15 ms took turns. A submission that begins with a wait for an event the CPU has signalled before now waits for the signal in the submit call, before its copies are taken, 10 ms at most (vk_exec.c vk_exec_gate_wait, NX2NAVI_GATE_WAIT_MS): 54.5 fps and a longest frame of 33 ms in the same place, 58.6 fps with the recorder thread on. With a camera sweep the pictures went out 21.0 ms apart on average with a 99th percentile of 55 ms, against 25.4 and 64. A village went from 48.6 to 53.7 fps and a graveyard from 56.5 to 64.2. 900 consecutive frames of a still scene show no dark or changed frame, with the recorder thread and without.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A contributor traced the one-frame dark shadowing at the castle town gate to depth targets the game remakes at other sizes about a thousand times a minute: the new image started cleared, so everything near the camera read as in shadow for a frame. With depth_alias = 1 a depth-only image of the same format copies the live one's depth for the region both cover. Their three runs each way showed 6, 4 and 0 dark frames without it and none with it; the cost of the copies and other scenes are not measured yet, so it is off by default.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The window that shows a pick-up above the character counts its number up to the new total by a whole step a frame and closes once they are equal. The game rounds that step down, and below one 60 Hz frame per update it was 0: above 60 fps the window stayed until the area was left, and for larger amounts the counting sound was started again every frame. The step is now floored at 1, switchable with pickup_count.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The reports of black and white blocks on Navi, on Link next to her and on bright surfaces seen from inside a house come from RX 5000 and 6000 cards, whose Windows driver compiles with "AMD proprietary shader compiler" (vk_caps.amd_sc); an RX 7900 XT (LLPC) draws the same scenes clean. The game's shaders test for NaN with OpIsNan in 48,717 of an install's 54,959 translated modules, 26 tests each on average, and the translator's guard that keeps NaN out of every float colour output, so that one bad texel does not spread through blur and bloom into blocks, is such a test too. SignedZeroInfNanPreserve asks the driver to keep those tests; whether that compiler does is not known.
With that driver, vk_pipeline.c spirv_nan_bits now turns every 32-bit isnan, scalar or vector, into (bits & 0x7fffffff) > 0x7f800000 on the number's bits, and a float select with a constant whose bits are a NaN into a select of the bits (issue #156), when the module is created. NX2NAVI_VK_NAN_BITS=1 does it on every driver, 0 on none. Run over every module of an install's shader cache, the rewrite changed 1,281,675 tests and 57 selects with every output well formed (each id defined once, below the bound, the uint types declared once); --selftest checks it on a small module, and Link's house with it forced on (RX 7900 XT) draws as without it. Not yet tried on an RX 5000 or 6000 card.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a lost device the driver's fault report gives an address and nothing says what was there. With va_report = 1 (NX2NAVI_VA_REPORT, off by default) the game keeps every address range the driver binds to a buffer or image and every range it unbinds (VK_EXT_device_address_binding_report, runtime/vk_vaddr.c). The report of a lost device then names the ranges that covered each faulting page, or the nearest ones when none did, and vaddr-history.txt beside the log holds the whole history, also when the driver gives no fault report. With 2 the call stack behind each range of 1 MiB or more is kept too. A player sees nothing of this unless the setting is on: without it no extension is enabled and no callback runs.
From pull request #349. Its second part, which stops a frame dump from turning on mesh query statistics, is left out: it changes what a dump writes for everyone who uses one. Built for Windows; no device was lost to try it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A draw whose pipeline is still building is skipped when its pass is rendered every frame and waits otherwise, so that a lookup table or a volume generated once is not left empty. "Every frame" meant "in the last four frames", so after a loading screen or a full-screen menu every pass of the first frame read as new: each draw with a new pipeline waited on the replay thread, up to 30 s each, and a mesh draw, whose pipeline has no fast link, for its whole compile. A player's 0.7.1 log shows it at three area changes: frames of 954, 1015 and 684 ms with 853, 960 and 663 ms of pipeline wait, six mesh draws waiting 52 to 660 ms in passes whose targets had been rendered in thousands of frames.
The target history (now runtime/vk_target_frames.h) also counts the frames each target was rendered in, for as long as it is the same guest texture. From eight frames on its pass is per-frame however long ago the last one was, and its draws are skipped and built by the workers like any other (pipe_skip_steady, on by default). A target rendered for the first time still waits, now from one pool of 50 ms a presented frame that the full-screen draws share (pipe_frame_wait_ms; oneshot_budget = 0 gives the old wait back), a mesh draw 8 ms at most, and the wait for a one-shot draw's shader comes from the same pool (gfx_shader_await takes a limit). A full-screen draw with no picture left to hold waits 40 ms more instead of 500. One frame so waits under 100 ms for pipelines in all.
Every 600 frames with a miss the log says how many draws found a shader or pipeline missing, how long they held the replay thread in all and in the worst frame, by the kind of wait, and how many were skipped at its end.
tests/target_frames_check.py replays the area change of that log against the header. Not yet run in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Eight commits: a draw that misses its pipeline has the rest of its submission read ahead, the library workers link
the pipelines of skipped draws, a pipeline whose fragment shader reads a target of its own pass is fast-linked like
any other, and diagnostics. The merge was clean. Their before and after matrix had not finished when this was merged.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The user asked for this once the flicker fix was measured. The single-frame flicker that kept it off in 0.7.0 came from frames drawn before the game's signal that their data was written, which the previous commit fixes; the lazy write-back only made the frames come faster, so the signal fell into the replay more often. wb_lazy = 0 in nx2navi.ini still turns it off, and an ini that already holds the key keeps its value. Elsewhere than Windows the default stays 0, as its fetch inside the access-fault handler has never run there.
Measured on the build of the previous commit, 1,800 presented frames a run, standing in Hyrule Field by day with DLSS Performance, frame generation 4x and HDR. With the lazy write-back on and eight busy threads beside the game: 38 and 0 single-frame spikes with the fix switched off (408 and 44 frames whose signal came during the replay), 0 and 0 with it on (667 and 724 such frames). With it off and the same load: 0 spikes in four runs, two with the fix and two without, 86 to 140 such frames each. Without load, one run each: 88.5 frames a second with the fix against 93.3 without it with the lazy write-back on (GPU 7.2 against 6.8 ms a frame, 6.53 against 6.48 submissions a frame), 71.0 against 65.6 with it off, so the fix's cost is inside the run-to-run spread; every run ended cleanly without a fatal error. This build was started once with no setting anywhere: the lazy write-back was on, and the Screen tab shows no row for it.
Not measured: play by hand with the lazy write-back on and the fix (the user's sessions so far ran with it off), other areas than Hyrule Field, and more than one run for the frame rates.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>