Commit Graph

449 Commits

Author SHA1 Message Date
chrg6
4ffaf6ea96 Hash a texture's storage only while it is held mapped, at the upload and at the unchanged-bytes check.
Both hashes read guest memory the game may unmap on its own thread; the unchanged-bytes check had no readability test at all.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:26:33 -05:00
chrg6
7ccb958c47 Let a save Commit that is under way finish before the crash report or a fatal error ends the process, for at most two seconds.
The game's Commit returns once the commit is queued, so the game may already have reported the save as done while its thread still copied; ending then left an unfinished copy that the next start rolled back.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:26:01 -05:00
chrg6
5ddff078c3 Count the save Commits under way instead of flagging one, so a second Commit entered beside the first keeps save writes out of its copy.
With a flag the first commit thread's end cleared it while the second still waited for the transaction lock; a write to the save then passed the wait and landed in the working copy the second thread was copying.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:25:39 -05:00
chrg6
5b1413d817 Count what the shadow copy-ins overwrite and cost, and keep stores through the device address table behind a switch.
NX2NAVI_SHADOW_CHECK=1 logs with the shadow line, every 60 frames: the whole copy-ins that overwrote a store a
shader made through the device address table, and the pools they fell on; the write-backs that moved bytes into
a page the CPU had written since its copy-in; the bytes both sides changed since their base copy; what the
copy-ins cost by kind, and which pools and flag sets took them. The stores are found by content: a whole
copy-in leaves a hash of the page, and the next one compares the shadow page with it first.

NX2NAVI_SHADOW_TABLE_STORES=1 keeps such stores: a page found stored into gets a base copy, so copy-ins take
only the CPU's bytes, and the fence of every submission that uses the table writes the stores back to guest
memory. --probe-vkmem checks it with a GPU write that nothing notes. It is off by default. The overwrite it
removes was the suspected cause of the flicker on Linux and is not: standing outdoors (RTX 5090 with
NX2NAVI_SHADOW=1) it happens exactly four times a frame, on ten pages in pools of 20 and 44 KiB, with the copy
that precedes the frame's stores, and no frame came out wrong with it or without it in 1,800 frames each; the
hashes cost 1.8 ms a frame of the replay (67 against 75 fps); and a count the CPU resets before the write-back
has landed would keep counting, where the whole copy-in resets it every time.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:24:50 -05:00
chrg6
175c1947de Pass over the pools with nothing written when a part that uses the device address table syncs the shadows.
Under shadow storage the prologue of every part whose shaders use the table brings the shadow of every pool up
to date. It ran the full per-pool sync for each of them, 10,700 times a frame outdoors (3,772 pools, about
three such parts), though the write-watch snapshots of all pools are taken at once just before. A pool whose
regions hold no stamp newer than its shadow is now passed over with a compare per region: 1,500 pool syncs a
frame are left, and the copy-ins went from 1.32 to 1.01 ms a frame of the replay (RTX 5090 with
NX2NAVI_SHADOW=1, one standing run each way).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:24:50 -05:00
chrg6
2f6ff17651 Give each part of the import check in --probe-vkmem its own write-watch epoch.
Under shadow storage the copy-in reads each region's write-watch snapshot once per submission. The check ran
its two parts in one epoch, so the second part's copy-in did not see the CPU writes made after the first, and
the probe failed with NX2NAVI_SHADOW=1 ("GPU read of CPU-written heap differs"). Each part now starts a new
epoch, as a submission does.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:24:50 -05:00
chrg6
7b009ffc64 Let the wait after a frame's first submission cover the write-backs under shadow storage.
With shadow storage (the Linux default) what the GPU wrote reaches guest memory in the fence's completion
callback, later than an import's store on the GPU. The wait after a frame's first submission returned at the
executed timeline, before that, so the game rewrote what those passes had written and the write-back then put
the GPU's older bytes back over the rewrite. The whole 3D scene came out brighter for single frames, each of
them a frame whose main work was held at its CPU event wait and replayed late.

Standing outdoors with NX2NAVI_SHADOW=1 on an RTX 5090: 695, 624, 443, 323 and 2 flashed frames of 1,800 in
five runs with the split, none in eight runs without it. The frame rate did not drop (74.6 fps median with the
fix; runs of the build before it gave 71 to 72). NX2NAVI_WB_SPLIT=2 keeps the split with shadow storage, to
measure the fault. That this is the flicker a player reported on Linux (RX 7800 XT, RADV) is not confirmed on
their machine; NX2NAVI_WB_SPLIT=0 gives 0.7.1 the same wait there.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:24:50 -05:00
chrg6
d15b4c95aa Count a frame's submissions atomically: several guest threads submit at once from the pause menu's first frame on.
With a lost count the frame's first submission could go without its wait for the GPU, which is there against frames drawn with the next frame's data.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:24:49 -05:00
chrg6
61e1ca8bad Drop a zstd stream's slot by its stream state under one hold of the lock, not by a pointer found earlier.
Between the lookup and the drop another thread could take the spent slot for its own stream; the drop then cleared that thread's slot, and a third thread could take it as free and reset the decoder context still in use.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:24:34 -05:00
chrg6
9be72ae9c0 When the system-event table is full, reuse the slot of an event the guest has destroyed instead of ending the process.
Entries were never removed, and every reload of a save binds two events at new addresses.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:24:12 -05:00
chrg6
4a69a38b48 When the jmp_buf table is full, give the new buffer the slot of the one filled longest ago instead of ending the process.
The table never removes entries and a session keeps meeting new buffer addresses: 12 to 15 more with every reload of a save. A longjmp to a replaced buffer ends the process as one to an unknown buffer always did.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:24:12 -05:00
chrg6
abd2e2c392 Compute the menu grain's lattice hash in unsigned arithmetic: the signed products overflowed, which is undefined.
The bits are the same; the undefined-behaviour sanitizer stopped the start at this line.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:23:24 -05:00
chrg6
fb44809074 Name an area the game asks for and this game version lacks, in the log and in the crash report.
A save that names such an area (a child save with HyruleCastle_Child on 1.1.1) makes the game read through a null area object right after the missing files; the report now says which area it was and that the save cannot be loaded.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:23:12 -05:00
chrg6
e887a1d18b Delete the boot prebuild's running marker at every orderly exit, so closing the game during boot is not taken for a driver crash.
The marker was deleted only when the boot's builds ended. A window closed before that left it, the next start read it as a start that died in the driver's compiler, and every later start on that GPU and driver skipped the prebuild and built each pipeline at its first draw.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:22:27 -05:00
chrg6
b933e3d996 List a texture pool slot without an entry once, so the walk over those slots no longer stalls every 8th frame in the Market.
A slot that resolved kept its place in the list until the next walk, and when it stopped resolving before that walk it was listed again. The list reached 160,000 entries for the 32,768 slots after a minute in the adult Market, and each walk over it held the texture lock for 15 to 30 ms.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:21:39 -05:00
chrg6
f29e34e7a0 Make the observer-response scene mapper the default HDR tone mapper, anchored to the game's mid grey and tone slope.
The HDR tonemap gains a mapper built on a finite cone response in LMS: scene grey 0.18 lands where the game's curve puts it and rises through it with the slope of the game's curve, and the hue bend of the game's clamp is given back to bright colours. It takes the scene before the game's exposure gain and the gain itself, which the splice now finds in the game's shader. The luminance mapper stays selectable and the game's own curve stays the third choice. The mapper fits the Rec. 709 gamut here, as the game's scene target holds no negative component. New settings: the tone mapper's mid grey, tone slope match, tone slope and hue clip emulation. The gamma correction on the present applies with the luminance mapper only.

Not part of this: eye adaptation, the anti-aliasing and reconstruction shaders, bloom and local exposure sliders, the output colour space choice and a 16-bit float scene target. Checked by build and numerically against the source of the port; the picture on an HDR display has not been judged. THIRD_PARTY_NOTICES.md names the source.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:20:31 -05:00
chrg6
7411d31803 Give a thread on a folded virtual core the number of its own core, so that fewer than six cores no longer crash the game.
With fewer than six virtual cores a thread pinned to a core at or above the count runs on core (c mod count), and GetCurrentProcessorNumber told it that virtual core's number. The game keys per-core data by the number without locks, so with four cores a thread pinned to core 5 used core 1's data in turn with core 1's own threads, and the game died within seconds of the first frame in play. The Steam Deck profile sets four cores, so this was every start under that profile (issue #445). The call now answers with one of the thread's own cores that fold onto the virtual core it holds, its ideal core first; with all six cores the answer is as before.

The kernel probe gets a test with two threads pinned to each of the six cores: with 6, 4, 3 and 1 virtual cores no thread sees another core's number and no two see the same at once.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:16:17 -05:00
chrg6
b1d08b06a8 Lay out the action prompt's face buttons as the pad layout has them.
The action prompt at the top right of the HUD draws the four face buttons as a diamond in the console's places: A right, B bottom, X top, Y left. Under pad_layout = xbox, playstation or auto the A action is the bottom button, yet the diamond kept A on the right, so the prompt pointed at the wrong button. At start the runtime now edits the prompt's layout in the shared interface archive for such a layout (the buttons, the dots and the lit crosses move, and the bubbles of X and Y flip) and serves the result from the profile folder; with nintendo, the default, nothing is changed. Every field is found and checked before anything is written, so a layout a mod changed is left as it is, and the archive is packed again with the game's own dictionary at the same unpacked size. One SARC reader and dictionary lookup now serve this, the camera pack and the menu's assets. When the layout changes, the menu's Restart counts the diamond.

tests/action_prompt_check.py passes without game files and on the game's own archive (4 copies, 208 fields, 13 ms at start).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:16:17 -05:00
chrg6
4eea3d1c2f Wait for the write-backs before returning to the game where pools are shadow copies.
With shadow storage (the default on Linux and under Wine) what the GPU wrote reaches the game's memory in the completion thread's fence callback, after the executed timeline that the split wait returns at. The game could then rewrite a pool, and an older write-back overwrote what it wrote: in Kokiri Forest a sky coefficient the game had reset came back, and the world flickered and looked washed out. The wait now covers the write-backs whenever shadow storage is in use; with imported or mirrored pools, the Windows default, nothing changes. The contributor's RX 9070 XT under RADV no longer showed the flicker.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:14:00 -05:00
chrg6
9dc5fa8b36 Check the size of pipeline-cache.bin before reading it.
A file over pipeline_cache_max_mb starts the run empty, but it was read into memory whole before its size was compared, which is the memory the limit is there to spare on a 16 GB machine. The size is now compared first and a file that is dropped is never read. The limit itself stays at 512 MB: sizing it from the machine's memory, as the contributor proposed, was not run by them or here.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:14:00 -05:00
chrg6
10cfbd9ea1 Stop telling an RTX 50 player that frame generation needs an RTX 40 card, and log why it is unavailable.
When Streamline loaded but DLSS Frame Generation was not available, the menu's row read "in use: off, needs an RTX 40 GPU or newer", also on an RTX 5080: the same check fails when the plugin did not load or a function it needs is missing. The row now points to the dlssg lines of the log, where the bridge says whether the plugin loaded and which of its functions it found, and then asks Streamline for its support result on the selected Vulkan device and logs it by name. The query is an optional export, so a bridge DLL of an earlier build keeps working.

The runtime and the bridge (tools/build_dlssg.py, MSVC, Streamline's headers) both build here. The failing case itself was not reproduced: on this RTX 5090 frame generation is available.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:14:00 -05:00
chrg6
dda12051f5 Add heap_lazy (off by default): on Windows the game's heap is committed where it is first used.
Windows charges every committed page to the commit limit whether it is used or not, and the game sizes its whole heap up front: in Castle Town after Hyrule Field the heap held 6.8 GiB of commit of which 3.4 GiB had ever been used. With heap_lazy = 1 a change of the heap's size only moves its end, an access to a reserved heap page commits the reserved runs of the 64 KiB around it in the access-fault hook and runs again, and the paths that do not fault commit their range first: mem_thaw (file reads, the host-pointer import), mem_kernel_access (file writes, the frame-rate fix's read) and mem_host_readable (the host's pointer checks). A page another path committed is left as it is, so a frozen, probed or armed page keeps its protection. With trace = perf a line per window gives the faults, the bytes committed and the time they took.

The contributor's runs on a 16 GB PC: 4.67 instead of 6.77 GiB of heap committed and 29.3 instead of 32.9 GiB of process commit in Castle Town, no "system commit low", 12,858 faults in 440 ms over four minutes, the longest 41 ms. With the setting off nothing is hooked and the heap is committed as before.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:14:00 -05:00
chrg6
73abd6577f Try a smaller allocation before giving up on a texture upload when the video memory is used up.
On an integrated Radeon 890M the ASTC scratch buffer failed to allocate and then a staging buffer ended the run. The scratch asks for 32 MiB whatever the upload needs, and the staging pool grows only in 64 MiB blocks of one memory type. When the device reports it is out of memory, the scratch is now tried again at the size the upload needs, and a staging buffer that does not fit the pool is tried again outside it with the same mapping, where a smaller block or another host-visible type may still have room. The menu's own upload flushes what it wrote, since such a buffer may lie in memory that is not coherent.

Built and run through the memory probe (96 GPU checks pass); the out-of-memory paths themselves were not reached here.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:14:00 -05:00
chrg6
7145c3f208 Say in the log when the system's commit runs low under a page file that Windows sizes itself.
With the page file left to Windows the commit limit stays near the installed memory plus what the file held at the start, and on a 16 GB PC with a 16 GB card it ran low in Hyrule Field at 32 GB, where a fixed page file of 32 GB or more gave 49 GB and no low commit. The first "system commit low" line of a run is now followed, once, by a line that names the cause and the Windows setting, and the build kit's guide gives the same advice for 16 GB PCs. plat_page_file_managed reads the PagingFiles value once; on this PC (page file sized by Windows) it returns 1.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:14:00 -05:00
chrg6
3ea3f5baa8 Leave the video memory in use out of this process's commit bound on Windows with a discrete card.
Windows charges a card's memory to the process's commit, and the bound on that commit defaults to the installed memory, at least 24576 MB. On a 16 GB PC with a 16 GB card (RTX 5060 Ti) the process stood at 29 to 31 GB in Hyrule Field, 9 to 10 GB of it video memory, so the bound held for the whole area while the system had commit to spare: textures gave way and the ground flickered white. The bound now compares the commit less the device-local memory in use, which is on the card and not in RAM or the page file and is bounded by the video memory rules already; the contributor's runs then never reached the bound and dropped no texture. The log line gives both figures.

A PC whose process commit is large mostly through video memory is no longer caught by this bound; the system's own figure still is. Not reproduced here: this PC has more memory than the bound.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:14:00 -05:00
chrg6
8bc23ea06e Clear a new depth image where the game makes it, not ahead of the part being recorded, and drop the depth copy switch.
A new depth or stencil image was cleared in the upload batch, which runs before the whole part being recorded. Bound into pool memory, the image shares its bytes with the live texture at its address, so the clear could wipe depth that earlier draws of the same part still read: the sun's shadowing went fully dark for one frame a few times a minute where the game resizes its shadow maps. The clear now goes in at the replay's position, as it already did for reused images (depth_init_inline, default 1; 0 is the old order). The switch that copied the live image's depth into the new one is removed: every such image was cleared by the game before anything read it, and the copies cost smoothness.

From a contributor's diff and measurements at a town gate: 0 dark frames in 5,423 with the change, 12 in 2,728 without. Checked here on the combined build in Hyrule Field before the Market's gate: every new depth image is started at the replay's position, none in the upload batch, and none had to end an open pass. The dark frames themselves were not counted here.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:10:34 -05:00
chrg6
093cf949df Open the recording device only when the game starts microphone capture.
The game opens two input streams while it starts and leaves both stopped through ordinary play, and each open took the system's default recording device and held it, paused, until the game ended: Windows shows the microphone as in use for the whole session, and a Bluetooth headset can drop to its call profile. An open now checks that a default input exists and prepares the stream and its bounded buffer; the device is opened at StartInputStream and released at stop or close, after which the game can still read or purge what was buffered. A start that is refused returns an input error and can be tried again.

tests/audio_input_check.py did not compile on this line (it had no stand-in for the microphone setting); with the change it builds and passes: idle streams hold no device, a refused start can be retried, a stop releases the device and keeps the unread frames, two streams are independent. Not checked with a real microphone in the game.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 22:59:02 -05:00
chrg6
6750e0bee1 Turn graphics pipeline libraries off by default on AMD's Windows driver for RX 5000 and RX 6000 cards.
That driver faults inside vkCreateGraphicsPipelines when it links the stage libraries of a burst of new pipelines. The game asks for one when an enemy dies and its effects are drawn for the first time, so the game ended with code 71 at every kill on an RX 6700 XT and an RX 5700 XT (issue #418), and the player who reported it found that gpl = 0 stops it. Unset, the setting is now off with that driver (vk_caps.amd_sc) and on everywhere else; gpl = 1 turns it on there, gpl = 0 off anywhere, and the caps line of the log says why it is off.

The price on those cards is that a new pipeline is built whole on the draw's path. Not run here: there is no AMD card. On an RTX 5090 the device probe reads "gpl yes" as before and "off by NX2NAVI_GPL=0" with the setting.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 22:57:58 -05:00
chrg6
6b68516fb1 Count the page generations handed out in the perf trace's size lines, so a run shows how fast a session uses up the 32-bit generation counter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 22:48:27 -05:00
chrg6
d2afdfa0f4 Under the perf trace, name the guest threads that submit and present on each queue, and log a submit or present that starts while another runs.
The synchronous replay's front end takes no lock of its own and relies on one such call at a time; a run now shows whether that holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 22:35:29 -05:00
chrg6
78223225f4 Mark DLSS and DLSS frame generation feature creates as work under the perf trace, with the device idle wait of releasing the kept features before them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 22:34:54 -05:00
chrg6
ab4bd8f8f9 Give a slow frame's line under the perf trace its drain wait and replay time, so a stall inside the replay or behind the present thread is no longer read as unnamed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 22:34:30 -05:00
chrg6
ded13958de Log the sizes of the caches, pools and tables under the perf trace, every 600 presents and with each memory snapshot.
Four "mem: sizes" lines give the entries and bytes of what no other line counted: the guest heap, physical maps, kernel region, lazily written ranges and the fixed kernel, setjmp, save-file, RomFS and zstd tables; pool records, mirrors, deferred destructions, upload and staging buffers, the device address table and the texture, sampler and slot caches; image views, descriptor spares, the completion queue, uniform snapshot arenas, recording copies and pins; and pipelines, layouts, stage libraries, job queues, the shader memory cache and the translator processes. Each subsystem reads its own counters under its lock; settings now count the replaced values they keep. Segment and scene lines say "-" for the world clock before it starts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 19:51:36 -05:00
chrg6
ecd34fb513 Under the perf trace, report frame times per route segment and take a memory snapshot after each area load.
An input script line "<tick> MARK <name>" (or "@<ms> MARK <name>") ends the running segment with one line of its frame count, mean, 1% low and worst frame, the work marks on any thread, process memory and VRAM, then the 600-frame memory lines, and starts a segment of that name. RomFS opens of an area's static placement file give a "scene:" line, and once no area has loaded for 10 s the memory snapshot is taken with a "settled" line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 19:40:02 -05:00
chrg6
66aedffcdd Name the work behind each frame on any thread under the perf trace: RomFS reads, zstd decodes, pipeline creates, guest memory maps, pool initialize and finalize, and device memory allocations.
A slow frame's line and the 60-frame window's line now say how long each kind took on any thread, how often, how many bytes and the longest single one, so a hitch outside the render thread's own waits has a named cause. While NX2NAVI_TRACE=perf is off each site pays one branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 19:27:58 -05:00
chrg6
28f78d4045 List the mesh draw switch among the settings, and count the given back pieces under the lock.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 16:46:05 -05:00
chrg6
892859cad0 Fill the records of a run of indirect mesh draws in one gap, so their pass is not cut at each of them.
An indirect mesh draw reads its workgroup count through a ring entry that a buffer copy fills, and a copy cannot be
recorded inside a pass: every such draw ended its pass, 45 times a frame in a village. The first draw of a run now
fills the entries of the mesh draws that follow it with only state, draws, counter reports and render conditions
between them, and those draws stay in the pass (mesh_ahead, default 1; 0 is the old way, ab switches in A/B blocks).
An entry is taken only by the draw it was filled for, at the same record address, while no other op was replayed
since. Counter reports count as staying in the pass because the replay closes their query inside it and writes the
value from the completion thread; the game reports one after most of these draws, and without that only 38% of the
draws were served.

Measured in the village: 42 of the 45 draws a frame are served by an entry filled ahead; gaps with these copies
41 to 3 a frame (0.22 to 0.04 ms of GPU time); passes and dispatches 248 to 208 a frame (302 to 254 in Hyrule
Field). With the piece release of the commit before, one run each: village 142.6 fps and GPU busy 3.87 ms (138.0
and 135.4 fps, 4.15 ms without this change), Hyrule Field 136.8 fps and 4.34 ms (131.5 and 131.3, 4.70 ms). In A/B
blocks of one run with the narrower rule (counter reports still ending a run) the GPU's busy time went from 4.81 to
4.64 ms.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 16:44:02 -05:00
chrg6
1241efa458 Give back only the touched piece of a lazily written range, and say in the log why ranges are written back.
With the lazy write-back on, a village and Hyrule Field still merged 20 to 29 MiB a frame back into guest memory at
the end of every submission, inside the GPU work the game's fence waits for. The new log line (mem: lazy write-back
at frame N: written back and not armed since the last line) names the reason by bytes: none of it lay in frozen
regions or on pages that could not be armed; it lay in regions held after a touch. The game's CPU reads a few words
at the start of buffers its shaders fill (12 bytes into ranges of 6.3 and 15.2 MiB among others), about once a
second. Each read gave back the whole armed range and held every 2 MiB region it reached for 1 s, doubling to 32 s,
and nothing was armed in a held region.

A touch now gives back the 64 KiB around its address, fetched alone, and holds only that piece; the rest of the
range stays armed (wb_lazy_chunk, default 1; 0 is the old rule). More than four touches in one region within a
second are taken as a reader going through the range: that range goes whole and its regions are held as before,
and the region's touches go that way at once while it keeps being touched within a minute. Each of the first eight
touches of a log window is logged under the perf trace with its thread, address, range and what was given back.

Measured on one machine (NVIDIA, uncapped, frame generation off, scripted walks, medians over the windows of 46 s
of world clock):
- village: 121.8, 124.1 and 124.4 fps before; 138.0 and 135.4 with this change; GPU busy 4.8 to 4.15 ms a frame
- Hyrule Field: 101.1 and 113.1 fps before; 131.5 and 131.3 after; GPU busy 6.5 to 4.7 ms
- merged a frame: 20 to 29 MiB before, 1.7 to 2.0 MiB after
- faults on armed pages in 1,000 frames: 14 and 16 before, 17 to 18 after in the village; 19 and 23 before, 17 to
  21 after in Hyrule Field; time spent fetching the same (4 to 5 ms in 1,000 frames)
A run with synthetic reads of four armed ranges per log window (wb_lazy_touch = 4, 769 faults) ended without an
error. The picture of every presented frame of a village walk and a field walk (thumbnails, 10,131 and 9,531
frames) had no black frame. The village walk has single frames whose brightness jumps: 14 with the change and 28
without it in about 10,000 frames each; the two looked at, one of each, are jumps of the camera.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 16:44:02 -05:00
chrg6
f17f4f9bcc Decode the game's zstd files on the host by default on 1.1.1 as well.
That build kept the game's own decoder until its sites had been through a check run. The check has now run over a village and a field: 3,072 streams and 31.7 GiB of output compared byte for byte, none different. The game's decoder took about a fifth of the running time of the game's threads in a village and about a second of each area load. zstd_native = 0 keeps the game's decoder.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:33:28 -05:00
chrg6
fc71691e91 Merge branch 'claude/amd-fight-crash-ef03a9' into test-all-fixes 2026-11-10 15:22:07 -05:00
chrg6
5151a64dbc Merge branch 'claude/pipeline-list' into test-all-fixes 2026-11-10 15:22:06 -05:00
chrg6
62e8578aa2 Turn the lazy write-back off by default on every card of AMD's own Windows driver, and let the device probe print the fatal summary line.
The default was off only on AMD's driver branch for RDNA 1 and 2, on the grounds that builds with the default on had been measured on an RX 7900 XT. Those measurements were standing in Hyrule Field and walking through a village; the report this is about is of a game that ended in fights, on an AMD card the player did not name, and fights are where the lazy path is busiest. Nobody has played one on an AMD card under our eyes on either branch, so the exception is now AMD's own driver as a whole (wb_lazy_choice.h, by VkDriverId), until a card of each branch has played fights with wb_lazy = 1 and a log to show. The cost is the open-area frame rate of 0.7.0 on every AMD card under Windows; the RX 7900 XT figures in the draft notes come from builds that had it on by default, which the notes now say. wb_lazy = 1 still turns it on.

--probe-vkdevice prints the line a fatal error or a crash adds to the log, so its wording can be read without a failing run. On an RTX 5090: "fatal: gpu NVIDIA GeForce RTX 5090, driver NVIDIA "621.01" version 621.1.0.0; memory mode mirror, lazy write-back on (the default on Windows (wb_lazy = 0 turns it off)); texture detail not chosen yet (no texture made)", and with the reporting player's nx2navi.ini beside the exe "lazy write-back off (wb_lazy = 0 is set)", after a start whose log names async_compute, vram_safe_mode and exception_handler as no settings of this version.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:20:47 -05:00
chrg6
caf50ecf1c Turn the lazy write-back off by default on AMD's driver for RX 5000 and 6000 cards, and say in the log which settings a run had.
A player on an AMD card reported crashes in fights and that a line added to nx2navi.ini cured them. That line, async_compute = 1, is no setting of the game, and neither are two more in the same file; the one line of the file that changes how memory reaches the GPU is wb_lazy = 0. A second report, an RX 6600 on 0.7.1, lost its device 3.5 minutes into play with the lazy write-back on. Neither proves it the cause: the player changed several lines at once and sent no log, the RX 6600's last pass mark is inside a draw pass and not a fetch, cards with 8 GB lost their device the same way on NVIDIA before the lazy write-back existed, and reading the fault handler, the fetch, the arming and the buffers' lifetimes found no failure path. What depends on the driver and cannot be checked without the card: pages of guest memory are inaccessible while a host-pointer import of them is bound and written by the GPU, and Vulkan calls run inside the access-fault handler.

The default is now decided once the driver is known (wb_lazy_choice.h, vk_memory.c vk_mem_lazy_decide): on under Windows as before, except on AMD's driver branch for RDNA 1 and 2, which none of us has run it on; AMD's driver for RDNA 3 and newer keeps it on, since builds with that default were measured on an RX 7900 XT. What the player sets wins everywhere, and one line at start says the choice and its reason. On those cards open areas run at the 0.7.0 frame rate again (on an RTX 5090 that was 65 fps against 90 in Hyrule Field; not measured on AMD). tests/wb_lazy_check.c checks the choice by platform, driver and setting without a GPU.

For reports of this kind the log now holds what was missing. At start: every setting nx2navi.ini sets, and in one line the keys that are no settings of this version and do nothing; the settings menu shows those in its bottom row. At a fatal error or a crash: one line with the card, its driver, the memory mode, the lazy write-back with its reason, armed ranges, fetches so far and a fetch in progress, and the texture detail (dispatch.c rt_fatal_note_add, vk_device.c fatal_note), since a report is often the end of a log alone.

--probe-cfg loads the player's file: the three lines are named, and the menu's save keeps them and every other line it does not change byte for byte. Nothing here ran on an AMD card, and no fatal error was provoked to read the new line.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:14:25 -05:00
chrg6
7b35fe7504 Let the limit on released textures follow the card's memory as well as the PC's.
The limit was a thirty-second of the installed memory, 1 GiB on a PC with 32 GB whatever its card. The game lets go of the textures of what leaves the view and takes them back when it returns: walking through a village the released ones passed 1 GiB within two or three minutes, and from then on older ones were dropped and decoded again on their way back. The limit is now also a quarter of the card's budget where the PC has an eighth of its memory to match, up to 8 GiB; the video memory and commit pressure rules still drop released textures first.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 14:57:51 -05:00
chrg6
87551c7618 An update that carries the shaders over builds the kit's current pipeline list, and a GPU without mesh shaders skips the list's mesh pipelines.
The carry path of an update does not run the shader preparation: it runs --warm-shaders, which builds the listed pipelines from the seed-pipeline-keys.bin the game folder already has. make_game.py now writes that file from the kit's list first, so a newer list reaches an updated install. Its shader variants are not translated on this path; a record whose variant the folder has neither prepared nor met is left out and counted.

On a GPU that has no mesh shaders every mesh record failed with a log line each. They are counted in the step's line instead.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 14:56:56 -05:00
chrg6
918d399d55 Add the shared pipeline list: mesh pipelines players met are built at the shader preparation, before the first start.
The preparation translates every shader and builds each one's stage library, so a pipeline a draw meets for the first time is linked in 2 to 6 ms. A mesh pipeline has no stage libraries: on an install that never drew it, it compiles whole in play, 0.2 to 1.1 s with its draws left out of the picture (18 of them on a first visit of Kokiri Forest). Whole pipelines were built ahead only from the folder's own pipeline-keys.bin.

tools/pipeline_list.py merge folds the pipeline-keys.bin and shader-answers.bin of game folders that were played into one list file: every key record once, with the answers record of each shader it names. A record names its shaders by input id, which depends on the game's code and the answers alone, and holds the game's pipeline state; what depends on the GPU is derived when the pipeline is built, so a record made on one machine is good on another with the same game version and runtime. Four folders on this PC give 5,432 records, 317 of them mesh pipelines (147 KB of keys).

tools/prepare_shaders.py reads the list (tools/data/pipeline_list.bin, data/pipeline_list.bin in a release folder, or --pipeline-list), adds its shader variants to the seed and writes its records as seed-pipeline-keys.bin. nx2navi.exe --prepare-shaders builds the mesh pipelines among them after the seed is translated and saves the pipeline cache before the stage libraries are built (NX2NAVI_PREPARE_PIPELINES=all builds the vertex pipelines too); --warm-shaders does it again after an update or a new driver, and --prepare-pipelines is that step alone. The pipelines go into the caches and are not kept or added to the folder's own key file, so they cost no start-up time or video memory until a draw meets them. A marker file keeps a driver that crashes on one of them from failing every preparation.

The list is derived from game data and stays out of the repository: check_content.py refuses files recorded from a game folder by their first bytes, and make_kit.py, make_launcher.py and make_release.py carry a list given with --pipeline-list. All three now also carry pipeline_list.py, which prepare_shaders.py imports.

Checked without the game: tests/pipeline_list_test.py, and prepare_shaders.py --no-run on the game files with a list of 317 mesh pipelines (all 266 shader variants found in this game version, 177 new to the seed). The build step in nx2navi.exe compiles and links and has not been run.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 14:52:46 -05:00
chrg6
682d9c5c50 A draw does not wait for the fast link of a mesh pipeline that is known to be compiling.
A draw whose pipeline is missing waits up to 3 ms for its fast link before it is skipped, 6 ms a frame in all. A mesh pipeline has no stage libraries and gets no fast link, so for the 0.2 to 1.1 s of its compile every frame that drew with it lost those milliseconds and skipped the draw all the same. Once such a pipeline has been building for longer than the 8 ms a cached one needs, its draws are skipped at once, as the one-shot mesh draws already are.

Not run in the game yet.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 14:52:25 -05:00
chrg6
e8f5f8e01e Merge branch 'main' of https://github.com/tangerineturtle1/NX2navi into test-all-fixes
# Conflicts:
#	runtime/cfg.c
2026-11-10 14:27:47 -05:00
chrg6
2db1ce835d Merge branch 'post-0.7.1-fixes' into test-all-fixes 2026-11-10 14:22:52 -05:00
chrg6
834b36a6d9 Stamp a retired texture as used when it is taken back.
While retired it got no use stamp, so it counted as idle the moment it came back, and a sweep over the video memory budget could evict it as idle in the same call that took it back: on the cards that are over budget, which are the ones the take-back is for.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 14:22:52 -05:00