pipeline-cache.bin was copied out of the driver and written whole at every save after any pipeline was created, also one the cache had held: tonight's logs show three saves of 358.1 MB in a run that added 0.3 MB, and 302 to 306 MB written in each of seven sessions that added about 0.5 MB each. The library warm in play built its 37,572 stage libraries into the same cache, which grew it by 1.6 GB: a save of 2026.7 MB in play, a file over pipeline_cache_max_mb, and the next start dropped it whole and compiled the pipelines it had held.
cache_save now asks a cache for the size of its data first and neither copies nor writes one whose size is what its file holds. What a run compiles goes into a second cache with its own file, pipeline-cache-play.bin (pipeline_cache_play, default 1; needs pipelineCreationCacheControl): pipe_create_in asks the first cache with VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED_BIT and builds into the play cache only when that says a compile is needed, and in play only the play file is saved. At a start the play file is read back and merged into the first cache in memory; once it reached pipeline_cache_play_mb (64) it is folded in for good: pipeline-cache.bin is written once, before play, and the play file removed. The preparation modes fold it at any size and build into the one file as before. The library warm's threads create with no cache of ours, so their libraries reach the driver's own cache alone.
Checked with --probe-prewarm on a copy of a dev folder's caches (3,212 recorded pipelines, 333 compute shaders), no game run: with a filled pipeline-cache.bin every pipeline came from it and neither file was written; with none and a cold driver cache the compiles went into pipeline-cache-play.bin (75.9 MB written, pipeline-cache.bin not created); the next start merged it and wrote pipeline-cache.bin once (191.1 MB) and removed it; kept below the fold size, a play file of 68.8 MB gave 1,445 pipelines at 0.3 ms from the first cache where 470 hit without the merge, and was not written again. Not checked in play: the saves during a session, the library warm without a cache, and drivers other than NVIDIA's.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
For a day that wait ended on a second timeline, when the GPU had run the submission's own commands (wb_split = 1,
then the default), on the thought that it only left out write-backs nothing needed yet. It left out the completion
thread's side effects for the submissions before it as well, and the game reads some of those next: the occlusion
counts, counter reports and event values, and the stamps the texture checks read. A frame's first submission is small
and runs right behind the last frame's large one, so the game went on while that one's callbacks were still running.
Measured in Hyrule Field with the camera turning (RTX 5090, lazy write-back on): 2,267 of 2,320 such waits returned
before those side effects had run; they were through 0.45 to 0.8 ms later on average, 12.6 ms at most, and in that
time 770 to 1,015 occlusion counts, 21 to 28 counter reports and 17 event values a frame reached guest memory. The
submission the game waits for wrote nothing back in any of 14 runs, so its own write-backs were never the matter.
A player saw a patch of grass near the camera change from frame to frame there and less of it with the fence's wait.
No scripted run showed such a frame with either wait, so the link to what the player saw is not shown.
The fence's wait takes 0.65 to 0.7 ms more of a frame there, about 6 % at 85 frames a second (single runs beside
other work). wb_split = 1 still gives the other wait, and 2 forces it with shadow storage.
NX2NAVI_TRACE=perf now counts what a wait on that timeline leaves behind and the completion thread's time by callback.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit 4d108b706e72c001ddbe20608f06469885df584b)
The game held 23.9 to 24.4 GB of commit in three sessions on an RTX 5090 at 3840x2160, flat over the session, on a
machine whose commit stood at 66 of 73 GB with it. A machine that cannot give it that much pages, drops textures or
loses the device, and its log said nothing about memory until then.
mem_start_note reads the system's commit when a game starts and writes one warning line when what is free, plus what
the process already holds, is below mem_need_mb (24576 by default, 0 for no warning). The menu has no place for a
start-up notice (its only toasts are the co-op page's), so the line is in the log alone.
tests/mem_need_check.py checks the rule and, through --probe-memneed, the answer and the warning line with the
machine's own commit.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Only the large compute shaders' background builder looked at the system's commit before it started a build, and it
saw nothing of what the builds beside it were about to take. The boot's recorded pipelines (up to 64 builders), the
pool's background builds during play, the boot's compute pipelines and stage libraries and the library warm asked
nobody.
Each of them now asks the gate before a build. A build that starts reserves an estimate of the driver compiler's
working memory until it ends (build_reserve_mb, 64 MB; 4096 MB for a large compute shader), and the gate admits a
build only while the commit headroom holds the part kept free (build_free_mb, 4096 MB), the reservations and its own.
A refused boot builder ends and leaves its pipelines to the pool; the pool leaves a background build queued and asks
again every two seconds; the stage library prebuild and the library warm wait up to 30 seconds and then end, and the
warm goes on at the next start. A pipeline a draw or a dispatch waits for is never refused. The preparation of the
pipeline list runs without the gate: no game runs beside it.
tests/build_gate_check.py checks the gate's arithmetic and, through --probe-buildgate, its answers with the machine's
own commit. What the builders do when refused is not exercised by a test.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A lost device used to end the process from whichever submit or wait noticed it first, while the present only logged it
and went on and the other threads kept calling the driver. The completion thread could take the submissions' timeline
as reached and run the callbacks of submissions that never finished.
vk_device_lost_check now takes the result of every submit, present, acquire and wait. The first thread to see the loss
sets a flag, writes the report and ends the process as a fatal error does (the log written, a save commit under way
finished), with an exit code of its own; later threads stop in that call, and a watchdog ends the process after ten
seconds should the first thread hang. The replay, its submit, the uploads, the present and the lazy fetch read the flag
and return without touching the device. The completion thread asks the device with a call of its own before a new
value counts and runs no callback once the flag is set.
The executable's --probe-devlost and tests/device_lost_check.py check this without a lost device, and
device_lost_at in nx2navi.ini ends a real run the same way.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With the retiring on, a player's session lost the ground of Hyrule Field and Lon Lon Ranch (6,663 retired, 928 back), and with the two holes of the way back closed it lost it again on arriving in a village (1,441 retired, 82 back; neither fix acted once). With the retiring off the same play kept its ground. What keeps the stand-in in place is not found; a draw that samples it once into a picture the game keeps would, and no mark undoes that. tex_disuse_evict = 1 turns the retiring back on; the shaders keep their marks, so no pipeline changes either way. Released textures and idle ones the game no longer names still give way when memory runs short, so a PC that is short of memory has less to give back than before.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
mouse_mode = direct is the default, but on 1.1.1 it did nothing: only the 1.4.3 patch table had the sites, so the mouse always went through the stick emulation, which gives no turn at slow mouse speeds and tops out at about 107 degrees a second. The 1.1.1 table now has the set mouse_direct (on by default): at main+0x37c45c, where the camera's input update holds the processed stick in s1 and s0, a host call performs the displaced store and adds the raw mouse counts to both, so the mouse meets no dead zone and no curve; at main+0x37c4f0 a second call clears the idle bit for an update the mouse moved in. The counts are divided by the frames the game steps per update and by the stick camera speed, so the same hand movement turns the same angle at every frame rate. The Controls tab gets the mouse mode and mouse sensitivity rows, and mouse_mode = stick or NX2NAVI_MOUSE_DIRECT_PATCH=0 gives the old behaviour.
Checked here: the generator lifts the 1.1.1 game with the two sites (it compares each site's original word), the lifted game compiles and links with the runtime, and tests/mouse_direct_check.py passes its 1.1.1 harness (register contract, displaced store and load, equal turn at every frame rate and stick speed, the reset pause, capture and mode). Not checked here: the game was not run, so aiming with the mouse was not tried, and the differential tests against Unicorn were not run because no Python on this PC has Unicorn. The contributor measured 0.0800 degrees per count on both axes in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit bb8fecfbe43517006eafd7be9c22e8dbe9601de7)
Without the other players drawn in the world, which are unfinished. No relay address is built in, and with co-op off nothing opens a socket. Built on Windows and tested against a relay on this PC only; two games in one room and any real network have not been tried.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A second reading of the commits taken in from pull requests found these, each traced in the code:
- heap_lazy: the camera speed's search read the whole heap, which with the setting on committed every page and undid what it saves; a scan now steps over the pages the lazy heap never committed (mem_untouched_run). tests/frame_step_check.py and tests/save_share_check.py no longer linked (frame_rate.c and fs_host.c call the new memory functions; the second also missed mem_probing before); both pass again. The settings text says that with the setting on a PC out of commit ends the game in play, not at start, and the census comment is back over its own function.
- Pipeline libraries on AMD's older Windows driver: the condition is the driver for RDNA 2 and older (RX 400 to RX 6000), not RX 5000 and RX 6000 only, and the texts named a price the code does not have. With libraries off a new pipeline is built by a worker while its draw is left out, and the start waits for every recorded pipeline instead of 20 s at most. The texts say so, and that the finding is one player's on one card. The menu strings nothing shows are as they were.
- Page file hint: it comes only when Windows sizes every page file, and it names the initial size and the automatic management that must be unticked first.
- Frame generation note: it keeps "needs an RTX 40 or newer" for the players it is right for and points the others to the log; the bridge's line says it is Streamline's general support check.
- Comments and settings texts that still described the old behaviour: the process commit bound, the wait with shadow storage, the scratch's least size, the action prompt under auto.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit 19f03aa01b26d6e17fb5fc70ad987aa8c68e8225)
Verified by runs in the audit: the Market's periodic stall (a pool slot was listed once per refresh pass instead of once; the list fell from 160,385 entries to at most 3,192 and no frame over 20 ms remained while standing), orderly exits delete the pipeline prebuild's running marker (left behind, the next start wrote a skip file and every later start skipped the prebuild), the log and crash report name an area a save asks for that the game does not have, and the menu atlas hash no longer overflows a signed multiply. The other fixes compile and pass the regression route but their faults were never reproduced: see AUDIT.md section 13 for each.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This line logs every setting nx2navi.ini sets, and a bug report takes a copy of nx2navi.ini along: with co-op both would have carried a room's secret, the relay's address and the player's name to whoever reads the report, and the secret lets anyone into the room. One list (runtime/cfg_private.h: the room's secret, the offer number, the relay, the name, the key a maker removes, the menu's test hook) now serves the log's line, "Copy system info" and the report's copy of the file, in the game and in the launcher: such a key reads "(set, not shown)", every other byte of the file stays.
Also: the last leftovers of the restorations that were left out with the puppets (their settings prefix and section), main.c's byte order mark as it was, and two comments that cited files this tree does not have.
Checked: with a room, a relay and a name in nx2navi.ini the log's line shows all three as set and none of their values; the same text gives the same bytes through the game's scrub and the launcher's (a launcher test, and the C function built alone); the launcher tests, the co-op probe, the relay tests and the tab check pass.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Each player keeps their own game and save. One player makes a room, the others join it with its secret, and story flags and, by the room's rules, items, songs, quest items, small keys (a shared pool), rupees and ammunition reach every save. The layer is a host thread with one UDP socket to a relay the players run themselves (tools/coop_relay.py, Python's standard library only); the messages between the players are sealed with a key from the room's secret (XChaCha20-Poly1305, Monocypher 4.0.2 vendored with its licence and a row in THIRD_PARTY_NOTICES.md), every change is signed by the player who made it, a player accepts a room by its members' keys and the key of its rules, the save is copied to coop-backup before anything is applied, and values are set through the game's own routines between frames. The Co-op tab makes, joins, accepts, removes and leaves, and the menu takes typed text for the name, the relay and the secret. This is the contributor's pull request #518 on this line.
Nothing reaches the network while co-op is off: the socket is opened, and the relay's name looked up, only after co-op is on, a relay is set and a room is made or joined. There is no default relay.
Left out of the pull request: the other players drawn in the world (the "puppets") with the copied engine they need. They were unfinished, added two patch sites to the game's per-frame update that could not be put through the differential tests here, and their file helpers clashed with the ones this line already has. So no position is sent: the POS message's codec stays, nothing sends it.
Changed against the pull request:
- The menu's test hook drives the Co-op tab only when it comes from the environment. From nx2navi.ini it could turn co-op on, join a room and accept its offer.
- The log's label for a room is hashed from the secret instead of being its first three characters.
- A game that has not saved yet has no save to copy, and takes nothing from the room until it has saved once; that used to count as a copy made.
- A PING's token no longer carries the PC's uptime.
- On Windows the maker's key file and its folder get an access list with the player's account alone.
- The tests that link the platform layer get the socket library, and the save test a stand-in for the title screen's hook.
Checked on Windows, where the pull request had never been compiled: the runtime builds and links; --probe-coop passes (sockets on 127.0.0.1, the codec against its byte vectors, a fuzz of every parser, two sessions through a hostile mock relay, a relay restart, the sync engine); tests/coop_relay_test.py passes its 61 tests, with the probe through the real relay on 127.0.0.1 at 0 and 30 % loss; tests/coop_page_check.py passes and its deliberate fault fails 48 checks. Not checked: two games in one room (one game runs at a time here), any real network, the tab in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Loading a save in Hyrule Field at night lost the device about 2 s after the load in 7 of about 22 loads on an RTX 5090 (a GPU read fault at the start of a large buffer, in or right after one of the game's compute dispatches). With the lazy write-back off it loaded 10 times of 10, and with only the piece release off 15 times of 15. How the piece release leads to the fault is not known; until it is, a CPU touch releases its whole range again, as before the town performance pass. wb_lazy_chunk = 1 turns it back on.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The two sides both made the wait after a frame's first submission cover the write-backs under shadow storage; the rule is kept in one place, vk_sync_wb_split.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
NX2NAVI_SHADOW_CHECK=1 logs with the shadow line, every 60 frames: the whole copy-ins that overwrote a store a
shader made through the device address table, and the pools they fell on; the write-backs that moved bytes into
a page the CPU had written since its copy-in; the bytes both sides changed since their base copy; what the
copy-ins cost by kind, and which pools and flag sets took them. The stores are found by content: a whole
copy-in leaves a hash of the page, and the next one compares the shadow page with it first.
NX2NAVI_SHADOW_TABLE_STORES=1 keeps such stores: a page found stored into gets a base copy, so copy-ins take
only the CPU's bytes, and the fence of every submission that uses the table writes the stores back to guest
memory. --probe-vkmem checks it with a GPU write that nothing notes. It is off by default. The overwrite it
removes was the suspected cause of the flicker on Linux and is not: standing outdoors (RTX 5090 with
NX2NAVI_SHADOW=1) it happens exactly four times a frame, on ten pages in pools of 20 and 44 KiB, with the copy
that precedes the frame's stores, and no frame came out wrong with it or without it in 1,800 frames each; the
hashes cost 1.8 ms a frame of the replay (67 against 75 fps); and a count the CPU resets before the write-back
has landed would keep counting, where the whole copy-in resets it every time.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With shadow storage (the Linux default) what the GPU wrote reaches guest memory in the fence's completion
callback, later than an import's store on the GPU. The wait after a frame's first submission returned at the
executed timeline, before that, so the game rewrote what those passes had written and the write-back then put
the GPU's older bytes back over the rewrite. The whole 3D scene came out brighter for single frames, each of
them a frame whose main work was held at its CPU event wait and replayed late.
Standing outdoors with NX2NAVI_SHADOW=1 on an RTX 5090: 695, 624, 443, 323 and 2 flashed frames of 1,800 in
five runs with the split, none in eight runs without it. The frame rate did not drop (74.6 fps median with the
fix; runs of the build before it gave 71 to 72). NX2NAVI_WB_SPLIT=2 keeps the split with shadow storage, to
measure the fault. That this is the flicker a player reported on Linux (RX 7800 XT, RADV) is not confirmed on
their machine; NX2NAVI_WB_SPLIT=0 gives 0.7.1 the same wait there.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The HDR tonemap gains a mapper built on a finite cone response in LMS: scene grey 0.18 lands where the game's curve puts it and rises through it with the slope of the game's curve, and the hue bend of the game's clamp is given back to bright colours. It takes the scene before the game's exposure gain and the gain itself, which the splice now finds in the game's shader. The luminance mapper stays selectable and the game's own curve stays the third choice. The mapper fits the Rec. 709 gamut here, as the game's scene target holds no negative component. New settings: the tone mapper's mid grey, tone slope match, tone slope and hue clip emulation. The gamma correction on the present applies with the luminance mapper only.
Not part of this: eye adaptation, the anti-aliasing and reconstruction shaders, bloom and local exposure sliders, the output colour space choice and a 16-bit float scene target. Checked by build and numerically against the source of the port; the picture on an HDR display has not been judged. THIRD_PARTY_NOTICES.md names the source.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Windows charges every committed page to the commit limit whether it is used or not, and the game sizes its whole heap up front: in Castle Town after Hyrule Field the heap held 6.8 GiB of commit of which 3.4 GiB had ever been used. With heap_lazy = 1 a change of the heap's size only moves its end, an access to a reserved heap page commits the reserved runs of the 64 KiB around it in the access-fault hook and runs again, and the paths that do not fault commit their range first: mem_thaw (file reads, the host-pointer import), mem_kernel_access (file writes, the frame-rate fix's read) and mem_host_readable (the host's pointer checks). A page another path committed is left as it is, so a frozen, probed or armed page keeps its protection. With trace = perf a line per window gives the faults, the bytes committed and the time they took.
The contributor's runs on a 16 GB PC: 4.67 instead of 6.77 GiB of heap committed and 29.3 instead of 32.9 GiB of process commit in Castle Town, no "system commit low", 12,858 faults in 440 ms over four minutes, the longest 41 ms. With the setting off nothing is hooked and the heap is committed as before.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A new depth or stencil image was cleared in the upload batch, which runs before the whole part being recorded. Bound into pool memory, the image shares its bytes with the live texture at its address, so the clear could wipe depth that earlier draws of the same part still read: the sun's shadowing went fully dark for one frame a few times a minute where the game resizes its shadow maps. The clear now goes in at the replay's position, as it already did for reused images (depth_init_inline, default 1; 0 is the old order). The switch that copied the live image's depth into the new one is removed: every such image was cleared by the game before anything read it, and the copies cost smoothness.
From a contributor's diff and measurements at a town gate: 0 dark frames in 5,423 with the change, 12 in 2,728 without. Checked here on the combined build in Hyrule Field before the Market's gate: every new depth image is started at the replay's position, none in the upload batch, and none had to end an open pass. The dark frames themselves were not counted here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
That driver faults inside vkCreateGraphicsPipelines when it links the stage libraries of a burst of new pipelines. The game asks for one when an enemy dies and its effects are drawn for the first time, so the game ended with code 71 at every kill on an RX 6700 XT and an RX 5700 XT (issue #418), and the player who reported it found that gpl = 0 stops it. Unset, the setting is now off with that driver (vk_caps.amd_sc) and on everywhere else; gpl = 1 turns it on there, gpl = 0 off anywhere, and the caps line of the log says why it is off.
The price on those cards is that a new pipeline is built whole on the draw's path. Not run here: there is no AMD card. On an RTX 5090 the device probe reads "gpl yes" as before and "off by NX2NAVI_GPL=0" with the setting.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Four "mem: sizes" lines give the entries and bytes of what no other line counted: the guest heap, physical maps, kernel region, lazily written ranges and the fixed kernel, setjmp, save-file, RomFS and zstd tables; pool records, mirrors, deferred destructions, upload and staging buffers, the device address table and the texture, sampler and slot caches; image views, descriptor spares, the completion queue, uniform snapshot arenas, recording copies and pins; and pipelines, layouts, stage libraries, job queues, the shader memory cache and the translator processes. Each subsystem reads its own counters under its lock; settings now count the replaced values they keep. Segment and scene lines say "-" for the world clock before it starts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the lazy write-back on, a village and Hyrule Field still merged 20 to 29 MiB a frame back into guest memory at
the end of every submission, inside the GPU work the game's fence waits for. The new log line (mem: lazy write-back
at frame N: written back and not armed since the last line) names the reason by bytes: none of it lay in frozen
regions or on pages that could not be armed; it lay in regions held after a touch. The game's CPU reads a few words
at the start of buffers its shaders fill (12 bytes into ranges of 6.3 and 15.2 MiB among others), about once a
second. Each read gave back the whole armed range and held every 2 MiB region it reached for 1 s, doubling to 32 s,
and nothing was armed in a held region.
A touch now gives back the 64 KiB around its address, fetched alone, and holds only that piece; the rest of the
range stays armed (wb_lazy_chunk, default 1; 0 is the old rule). More than four touches in one region within a
second are taken as a reader going through the range: that range goes whole and its regions are held as before,
and the region's touches go that way at once while it keeps being touched within a minute. Each of the first eight
touches of a log window is logged under the perf trace with its thread, address, range and what was given back.
Measured on one machine (NVIDIA, uncapped, frame generation off, scripted walks, medians over the windows of 46 s
of world clock):
- village: 121.8, 124.1 and 124.4 fps before; 138.0 and 135.4 with this change; GPU busy 4.8 to 4.15 ms a frame
- Hyrule Field: 101.1 and 113.1 fps before; 131.5 and 131.3 after; GPU busy 6.5 to 4.7 ms
- merged a frame: 20 to 29 MiB before, 1.7 to 2.0 MiB after
- faults on armed pages in 1,000 frames: 14 and 16 before, 17 to 18 after in the village; 19 and 23 before, 17 to
21 after in Hyrule Field; time spent fetching the same (4 to 5 ms in 1,000 frames)
A run with synthetic reads of four armed ranges per log window (wb_lazy_touch = 4, 769 faults) ended without an
error. The picture of every presented frame of a village walk and a field walk (thumbnails, 10,131 and 9,531
frames) had no black frame. The village walk has single frames whose brightness jumps: 14 with the change and 28
without it in about 10,000 frames each; the two looked at, one of each, are jumps of the camera.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The default was off only on AMD's driver branch for RDNA 1 and 2, on the grounds that builds with the default on had been measured on an RX 7900 XT. Those measurements were standing in Hyrule Field and walking through a village; the report this is about is of a game that ended in fights, on an AMD card the player did not name, and fights are where the lazy path is busiest. Nobody has played one on an AMD card under our eyes on either branch, so the exception is now AMD's own driver as a whole (wb_lazy_choice.h, by VkDriverId), until a card of each branch has played fights with wb_lazy = 1 and a log to show. The cost is the open-area frame rate of 0.7.0 on every AMD card under Windows; the RX 7900 XT figures in the draft notes come from builds that had it on by default, which the notes now say. wb_lazy = 1 still turns it on.
--probe-vkdevice prints the line a fatal error or a crash adds to the log, so its wording can be read without a failing run. On an RTX 5090: "fatal: gpu NVIDIA GeForce RTX 5090, driver NVIDIA "621.01" version 621.1.0.0; memory mode mirror, lazy write-back on (the default on Windows (wb_lazy = 0 turns it off)); texture detail not chosen yet (no texture made)", and with the reporting player's nx2navi.ini beside the exe "lazy write-back off (wb_lazy = 0 is set)", after a start whose log names async_compute, vram_safe_mode and exception_handler as no settings of this version.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A player on an AMD card reported crashes in fights and that a line added to nx2navi.ini cured them. That line, async_compute = 1, is no setting of the game, and neither are two more in the same file; the one line of the file that changes how memory reaches the GPU is wb_lazy = 0. A second report, an RX 6600 on 0.7.1, lost its device 3.5 minutes into play with the lazy write-back on. Neither proves it the cause: the player changed several lines at once and sent no log, the RX 6600's last pass mark is inside a draw pass and not a fetch, cards with 8 GB lost their device the same way on NVIDIA before the lazy write-back existed, and reading the fault handler, the fetch, the arming and the buffers' lifetimes found no failure path. What depends on the driver and cannot be checked without the card: pages of guest memory are inaccessible while a host-pointer import of them is bound and written by the GPU, and Vulkan calls run inside the access-fault handler.
The default is now decided once the driver is known (wb_lazy_choice.h, vk_memory.c vk_mem_lazy_decide): on under Windows as before, except on AMD's driver branch for RDNA 1 and 2, which none of us has run it on; AMD's driver for RDNA 3 and newer keeps it on, since builds with that default were measured on an RX 7900 XT. What the player sets wins everywhere, and one line at start says the choice and its reason. On those cards open areas run at the 0.7.0 frame rate again (on an RTX 5090 that was 65 fps against 90 in Hyrule Field; not measured on AMD). tests/wb_lazy_check.c checks the choice by platform, driver and setting without a GPU.
For reports of this kind the log now holds what was missing. At start: every setting nx2navi.ini sets, and in one line the keys that are no settings of this version and do nothing; the settings menu shows those in its bottom row. At a fatal error or a crash: one line with the card, its driver, the memory mode, the lazy write-back with its reason, armed ranges, fetches so far and a fetch in progress, and the texture detail (dispatch.c rt_fatal_note_add, vk_device.c fatal_note), since a report is often the end of a log alone.
--probe-cfg loads the player's file: the three lines are named, and the menu's save keeps them and every other line it does not change byte for byte. Nothing here ran on an AMD card, and no fatal error was provoked to read the new line.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The preparation translates every shader and builds each one's stage library, so a pipeline a draw meets for the first time is linked in 2 to 6 ms. A mesh pipeline has no stage libraries: on an install that never drew it, it compiles whole in play, 0.2 to 1.1 s with its draws left out of the picture (18 of them on a first visit of Kokiri Forest). Whole pipelines were built ahead only from the folder's own pipeline-keys.bin.
tools/pipeline_list.py merge folds the pipeline-keys.bin and shader-answers.bin of game folders that were played into one list file: every key record once, with the answers record of each shader it names. A record names its shaders by input id, which depends on the game's code and the answers alone, and holds the game's pipeline state; what depends on the GPU is derived when the pipeline is built, so a record made on one machine is good on another with the same game version and runtime. Four folders on this PC give 5,432 records, 317 of them mesh pipelines (147 KB of keys).
tools/prepare_shaders.py reads the list (tools/data/pipeline_list.bin, data/pipeline_list.bin in a release folder, or --pipeline-list), adds its shader variants to the seed and writes its records as seed-pipeline-keys.bin. nx2navi.exe --prepare-shaders builds the mesh pipelines among them after the seed is translated and saves the pipeline cache before the stage libraries are built (NX2NAVI_PREPARE_PIPELINES=all builds the vertex pipelines too); --warm-shaders does it again after an update or a new driver, and --prepare-pipelines is that step alone. The pipelines go into the caches and are not kept or added to the folder's own key file, so they cost no start-up time or video memory until a draw meets them. A marker file keeps a driver that crashes on one of them from failing every preparation.
The list is derived from game data and stays out of the repository: check_content.py refuses files recorded from a game folder by their first bytes, and make_kit.py, make_launcher.py and make_release.py carry a list given with --pipeline-list. All three now also carry pipeline_list.py, which prepare_shaders.py imports.
Checked without the game: tests/pipeline_list_test.py, and prepare_shaders.py --no-run on the game files with a list of 317 mesh pipelines (all 266 shader variants found in this game version, 177 new to the seed). The build step in nx2navi.exe compiles and links and has not been run.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Each submission's first sync of the bindless texture pool walked every registered slot, 33,000 in Kokiri Forest, to find the few textures the CPU wrote and the few slots that changed. Three parts of that walk change, each with its own switch (pool_fast, pool_presnap, pool_group; on by default, 0 and ab accepted):
- The freshness check goes through a list of the entries the slots resolved to, with a check by 2 MiB region first; slots that resolved to nothing are looked up again when the game's memory map changes and every eighth submission. The use stamps the idle eviction reads are left from a count of the slots that hold each entry.
- The check takes the write-watch snapshots of its regions first, so unchanged entries pass the region test.
- The GPU-written slots of one address bucket with one descriptor form a group, which the dirty walk leaves alone while its first member is unchanged.
Measured on an RTX 5090 at 2560x1440, standing in Kokiri Forest, in-run A/B over five runs with all three switched (three of them with the barrier change switched too): 93.6 to 100.7, 96.4 to 103.1, 95.8 to 102.6, 99.3 to 107.1 and 91.9 to 99.9 frames a second, the replay thread 0.7 to 0.8 ms a frame shorter; a control's blocks differ by 0.4 %. On a walk with NX2NAVI_POOL_GROUP_CHECK=1 no group that was left alone had missed a write, and screenshots across the A and B blocks show the same picture.
From pull request #390. Not tried: the texture eviction under memory pressure (a 9 GiB budget was never reached on the walk), AMD and Intel drivers, Linux. One of the nine runs with it on ended by itself with exit code 0 and nothing in its log two seconds after the window lost the focus; three repeats of that run did not.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Every pass begin and every compute dispatch recorded a full barrier, which makes the GPU finish everything before each of them. With barrier_game = 1 (NX2NAVI_BARRIER_GAME) a pass after an earlier pass gets a barrier for its attachments only and a dispatch after other dispatches gets none; a change between drawing and compute keeps the full barrier, and so do the game's own barriers and the replay's copies, clears, fills and uniform updates. On an RTX 5090 at 2560x1440 the GPU's time for a frame fell from 5.4 to 4.9 ms standing in Kokiri Forest (two in-run A/B runs, the control's blocks differ by 0.7 %) and from 5.1 to 4.4 ms on a walk through it. The frame rate did not change there, because the replay thread is the limit on that machine; it should rise where the GPU is.
The default is 1 on NVIDIA graphics cards only, and 0 on every other vendor's: the game placed its barriers for an NVIDIA GPU, Vulkan asks for more of them than the game recorded, and no AMD or Intel driver has run this. 34 screenshots across the A and B blocks of one run show the same picture on NVIDIA. NX2NAVI_TRACE=perf logs the barriers a frame by kind.
From pull request #389, which had it on for every vendor; the default by vendor and the settings text are added here. The validation layer's synchronization check could not be run on this machine.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Both are about the time between two frames. Ocarina waits for the previous frame's fence before it lets the next frame's GPU work start: its main thread signals an event from the CPU, and the main queue's commands begin with a wait for it.
The completion thread ran a submission's callbacks before it let the submission's waiters go, and the occlusion counts were among them: one vkGetQueryPoolResults and one vkResetQueryPool per reported query, about a hundred queries a frame in Hyrule Field. On an RX 7900 XT (26.10.1) each call goes into the kernel, 3.9 ms of every frame, so the fence was reached 4 to 6 ms after the GPU had finished with every game thread waiting. Slots come from the ring in order, so one call now reads up to 256 slots from the one asked for, the available results are kept for the segments that follow, and the slots taken are reset in runs (vk_exec_xfer.c oq_take, NX2NAVI_OQ_BATCH). Standing in Hyrule Field at 3440x1440 with FSR: 52.7 fps against 44.6, the longest frame of the measured minute 46 ms against 55; the completion thread spends 0.3 ms a frame in the callback.
When the CPU's signal came a millisecond or two after the main queue's submit call, the submission took its copies, stopped at the wait and was held; the render thread polled the event, presented, and the present replayed the frame under a value the game's fence does not stand for. The next frame then skipped its wait for the GPU and was replayed inside its submit call, and frames of 23 and 15 ms took turns. A submission that begins with a wait for an event the CPU has signalled before now waits for the signal in the submit call, before its copies are taken, 10 ms at most (vk_exec.c vk_exec_gate_wait, NX2NAVI_GATE_WAIT_MS): 54.5 fps and a longest frame of 33 ms in the same place, 58.6 fps with the recorder thread on. With a camera sweep the pictures went out 21.0 ms apart on average with a 99th percentile of 55 ms, against 25.4 and 64. A village went from 48.6 to 53.7 fps and a graveyard from 56.5 to 64.2. 900 consecutive frames of a still scene show no dark or changed frame, with the recorder thread and without.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A contributor traced the one-frame dark shadowing at the castle town gate to depth targets the game remakes at other sizes about a thousand times a minute: the new image started cleared, so everything near the camera read as in shadow for a frame. With depth_alias = 1 a depth-only image of the same format copies the live one's depth for the region both cover. Their three runs each way showed 6, 4 and 0 dark frames without it and none with it; the cost of the copies and other scenes are not measured yet, so it is off by default.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The window that shows a pick-up above the character counts its number up to the new total by a whole step a frame and closes once they are equal. The game rounds that step down, and below one 60 Hz frame per update it was 0: above 60 fps the window stayed until the area was left, and for larger amounts the counting sound was started again every frame. The step is now floored at 1, switchable with pickup_count.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The reports of black and white blocks on Navi, on Link next to her and on bright surfaces seen from inside a house come from RX 5000 and 6000 cards, whose Windows driver compiles with "AMD proprietary shader compiler" (vk_caps.amd_sc); an RX 7900 XT (LLPC) draws the same scenes clean. The game's shaders test for NaN with OpIsNan in 48,717 of an install's 54,959 translated modules, 26 tests each on average, and the translator's guard that keeps NaN out of every float colour output, so that one bad texel does not spread through blur and bloom into blocks, is such a test too. SignedZeroInfNanPreserve asks the driver to keep those tests; whether that compiler does is not known.
With that driver, vk_pipeline.c spirv_nan_bits now turns every 32-bit isnan, scalar or vector, into (bits & 0x7fffffff) > 0x7f800000 on the number's bits, and a float select with a constant whose bits are a NaN into a select of the bits (issue #156), when the module is created. NX2NAVI_VK_NAN_BITS=1 does it on every driver, 0 on none. Run over every module of an install's shader cache, the rewrite changed 1,281,675 tests and 57 selects with every output well formed (each id defined once, below the bound, the uint types declared once); --selftest checks it on a small module, and Link's house with it forced on (RX 7900 XT) draws as without it. Not yet tried on an RX 5000 or 6000 card.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a lost device the driver's fault report gives an address and nothing says what was there. With va_report = 1 (NX2NAVI_VA_REPORT, off by default) the game keeps every address range the driver binds to a buffer or image and every range it unbinds (VK_EXT_device_address_binding_report, runtime/vk_vaddr.c). The report of a lost device then names the ranges that covered each faulting page, or the nearest ones when none did, and vaddr-history.txt beside the log holds the whole history, also when the driver gives no fault report. With 2 the call stack behind each range of 1 MiB or more is kept too. A player sees nothing of this unless the setting is on: without it no extension is enabled and no callback runs.
From pull request #349. Its second part, which stops a frame dump from turning on mesh query statistics, is left out: it changes what a dump writes for everyone who uses one. Built for Windows; no device was lost to try it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A draw whose pipeline is still building is skipped when its pass is rendered every frame and waits otherwise, so that a lookup table or a volume generated once is not left empty. "Every frame" meant "in the last four frames", so after a loading screen or a full-screen menu every pass of the first frame read as new: each draw with a new pipeline waited on the replay thread, up to 30 s each, and a mesh draw, whose pipeline has no fast link, for its whole compile. A player's 0.7.1 log shows it at three area changes: frames of 954, 1015 and 684 ms with 853, 960 and 663 ms of pipeline wait, six mesh draws waiting 52 to 660 ms in passes whose targets had been rendered in thousands of frames.
The target history (now runtime/vk_target_frames.h) also counts the frames each target was rendered in, for as long as it is the same guest texture. From eight frames on its pass is per-frame however long ago the last one was, and its draws are skipped and built by the workers like any other (pipe_skip_steady, on by default). A target rendered for the first time still waits, now from one pool of 50 ms a presented frame that the full-screen draws share (pipe_frame_wait_ms; oneshot_budget = 0 gives the old wait back), a mesh draw 8 ms at most, and the wait for a one-shot draw's shader comes from the same pool (gfx_shader_await takes a limit). A full-screen draw with no picture left to hold waits 40 ms more instead of 500. One frame so waits under 100 ms for pipelines in all.
Every 600 frames with a miss the log says how many draws found a shader or pipeline missing, how long they held the replay thread in all and in the worst frame, by the kind of wait, and how many were skipped at its end.
tests/target_frames_check.py replays the area change of that log against the header. Not yet run in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Eight commits: a draw that misses its pipeline has the rest of its submission read ahead, the library workers link
the pipelines of skipped draws, a pipeline whose fragment shader reads a target of its own pass is fast-linked like
any other, and diagnostics. The merge was clean. Their before and after matrix had not finished when this was merged.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The user asked for this once the flicker fix was measured. The single-frame flicker that kept it off in 0.7.0 came from frames drawn before the game's signal that their data was written, which the previous commit fixes; the lazy write-back only made the frames come faster, so the signal fell into the replay more often. wb_lazy = 0 in nx2navi.ini still turns it off, and an ini that already holds the key keeps its value. Elsewhere than Windows the default stays 0, as its fetch inside the access-fault handler has never run there.
Measured on the build of the previous commit, 1,800 presented frames a run, standing in Hyrule Field by day with DLSS Performance, frame generation 4x and HDR. With the lazy write-back on and eight busy threads beside the game: 38 and 0 single-frame spikes with the fix switched off (408 and 44 frames whose signal came during the replay), 0 and 0 with it on (667 and 724 such frames). With it off and the same load: 0 spikes in four runs, two with the fix and two without, 86 to 140 such frames each. Without load, one run each: 88.5 frames a second with the fix against 93.3 without it with the lazy write-back on (GPU 7.2 against 6.8 ms a frame, 6.53 against 6.48 submissions a frame), 71.0 against 65.6 with it off, so the fix's cost is inside the run-to-run spread; every run ended cleanly without a fatal error. This build was started once with no setting anywhere: the lazy write-back was on, and the Screen tab shows no row for it.
Not measured: play by hand with the lazy write-back on and the fix (the user's sessions so far ran with it off), other areas than Hyrule Field, and more than one run for the frame rates.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Each frame the game sets one of two events to 0 from the CPU, submits its worker queues and the main queue, whose first command waits for that event to be 1, and signals 1 with nvnEventSignal once it has written what the frame's commands read. The recording of a submission takes the VRAM mirror sync and the uniform snapshot at its start and tested the wait a moment later, so a signal that arrived in between passed the wait and the frame was drawn from copies taken before the signal: single frames with the whole scene's shadowing missing (6 to 7 of 255 brighter, the sky and the interface unchanged) or a black scene, more of them the busier the CPU.
Every nvnEventSignal is now numbered and keeps the value it replaced, the recording notes the number before it takes its copies, and a signal with a later number does not count for that submission. The submission stops at the wait like any wait reached before its signal and goes on at once in a submission of its own, whose copies are taken after the signal. NX2NAVI_EVENT_AT_SYNC=0 gives the earlier test. NX2NAVI_EVO_TRACE=1 logs the order in which every frame's submissions, waits and signals were replayed, which is how the frames were found.
Measured with the flight recorder's thumbnails, 1,800 presented frames a run, standing in Hyrule Field by day with the lazy write-back on, DLSS Performance, frame generation 4x and HDR: the build before this change showed 34 single-frame spikes with eight busy threads beside the game, 20 with four and 8 with six, most of them in frames whose signal came during the replay; with this change two runs with eight busy threads showed 0 and 0, with 667 and 724 such frames. A session of the user's with the lazy write-back off ran 25,200 frames without a spike in its three automatic dumps while the change stopped 4,744 wait tests.
Not measured: runs of this same build with the switch off (the two tried failed for other reasons, a device loss during the boot beside another game and a load with cold caches), any run with the lazy write-back off under load, the frame rate and GPU time against the switch off, and the differential tests, which this change does not touch. The flicker with the lazy write-back off that was reported has not been reproduced in a capture; its mechanism is the same signal and occurs there too.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Shaders are looked up and pipelines built when a draw first needs them. The time between nvnProgramSetShaders and a
program's first draw is the head start a build begun at the program's creation would have; the line gives it as
counts under 17 ms, 100 ms, 1 s, 5 s, 30 s and over, mesh programs apart (vk_exec_draw.c prog_lead,
nvn_api_obj.c nvn_prog_made_ns). Off unless asked for.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
It was a constant beside the per-frame budget (pipe_link_ms). With the stage libraries in the driver's cache a fast
link takes 4 to 17 ms for a third of the pipelines a draw misses, so both values decide how many such draws are left
out of a frame; the setting lets that be measured.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The next draw with a new pipeline was only found when the replay reached it, so the draws of an area's first frame,
or of a turn that brings several new materials into view, paid for their pipelines one after another while the
workers that could have built them together sat idle. The submission is recorded before it is replayed: the first
draw of a submission that finds a shader or pipeline missing now has the ops after it read (vk_exec.c vk_exec_ahead)
with the binds that decide a draw's shaders and shader libraries applied to a copy of the replay state. Each draw
there looks its shaders up as the real draw will, which also queues a shader that is not in memory, and names its
state to vk_pipeline_hint, which queues the pre-rasterization and fragment shader libraries that are missing. No
pipeline is entered and no key recorded, so a wrong guess costs two libraries at most. The draw's own lookup then
links on its thread when both libraries are there.
On a first visit to Kokiri Forest a scan read 6,900 ops in 0.78 ms on average (the longest 5.5 ms; a scan ends
after 3 ms), and the scans of the load and the walk named 377 states without a pipeline and queued 104 libraries
for them.
pipe_ahead = 0 (NX2NAVI_PIPE_AHEAD) turns it off.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A draw that is skipped while its pipeline builds leaves the pipeline to the pool: a job is fast-linked first and goes
back in the queue for its optimised build. The pool keeps one worker out of optimised builds; with the other three
inside one (100 to 350 ms each) that worker linked every new pipeline of a burst one after another. The library
workers, which so far served only draws that wait, now take the fast link of a draw-queue job too (vk_pipeline.c
fast_take, lib_worker), and there are as many of them as pool workers (four on sixteen logical processors, two
before).
Measured on a first visit to Kokiri Forest (RTX 5090, libraries in the driver's cache): of the 444 pipelines draws
missed, 180 took over 50 ms to become usable, the longest 1.3 s, and their draws were missing from every frame until
then. With this change and the two beside it (feedback loops, the look-ahead) 23 did, 19 of them mesh pipelines,
which have no fast link.
pipe_fast_workers = 0 (NX2NAVI_PIPE_FAST_WORKERS) gives the old behaviour.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Such a pipeline (soft particles and water reading the depth buffer, full-screen passes that read the target they
write) had no fast link: gpl_link refused it, so its draws were skipped during, or waited for, the whole optimised
build. The flags belong to the fragment output state alone, so the fragment output library carries them and its key
names them (FoKey.fbl); the shader libraries are the ones the same shaders use without a loop.
Measured on a first visit to Kokiri Forest (RTX 5090, stage libraries in the driver's cache, DLSS Frame Generation
on, where no frame is held back and a full-screen draw waits on): full-screen draws of such pipelines waited 267, 42
and 61 ms in the area's first frame and 410 ms in the middle of the walk; fast-linked they wait 2 to 3 ms.
gpl_feedback = 0 in nx2navi.ini (NX2NAVI_GPL_FEEDBACK) gives the old behaviour.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With n under the ring's length each dump overlaps the one before, and a run's frame records can be read back whole
except for its last n frames. NX2NAVI_FLIGHT_AT takes sixteen frame numbers at most, which covered a walk at 60 fps
and lost most of one at 240.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
One conflict, in the build kit's notes, which keep both the shader cache item and the Texture detail text.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>