Regions the game has not written for a while are write-protected so that the per-submission write-watch queries can skip them, and the first write thaws them. The game's loaders keep writing the buffers they stream an area into, so those regions were frozen again as soon as they had been idle a moment and thawed again moments later: running from the Kakariko Village gate across Hyrule Field froze 31,004 regions and thawed 28,918 by a write in nine minutes. While the loaders wrote and the driver locked new memory, each thaw's VirtualQuery took 15 to 20 ms: a loader thread stalled 78 to 111 ms in thaws in every one of the hitches of 125 to 140 ms of that walk, with the game's main thread waiting for it.
A write that thaws a region within a minute of its freeze now holds it thawed for 4 seconds, doubled at each such thaw up to 10 minutes; a region that stayed frozen longer starts again from nothing. On the same walk: 8,284 freezes and 6,355 thaws, no frame over 100 ms while Link ran where 29 and 52 a minute had been, the slowest 1 % of frames at 19 and 24 fps instead of 9 and 7, and at the walk's end, Link standing at the edge of Lake Hylia, 60 fps instead of 38 with the slowest 1 % at 43 instead of 20. The perf trace's write-watch line counts the holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux every driver got shadow storage (copies of guest memory beside it), which exists for amdgpu, where importing guest memory costs the write protection of its pages. NVIDIA's own driver imports without that cost, so there the copies are only the slower path: in Kokiri Forest on 0.7.1 (RTX 4090) imports ran at 51.9 frames a second and the copies at 27.4. Until now only the AppImage's play.sh knew, by setting NX2NAVI_SHADOW=0 when NVIDIA's kernel driver was present at build time, so a game started any other way got the copies, and a PC with that kernel driver whose game ran on another card got imports where it should not. The game now chooses by the driver of the device it runs on, play.sh no longer sets the variable, and NX2NAVI_SHADOW=1 or 0 still decides. The driver is asked when the device is picked, before the storage is chosen.
Windows is unchanged: the device probe here reads "mem: mode mirror" with no shadow storage line, and the shadow line with NX2NAVI_SHADOW=1. The Linux path itself was not run: there is no Linux machine here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A freeze (a VirtualQuery and a VirtualProtect of 2 MiB) takes microseconds as a rule, but in the second after an area
load each took 8 to 17 ms. The newly loaded regions all go idle together, so 32 came up in one submission: its mirror
begin took 70.5 ms and the frame 78 ms, and a 9 ms freeze followed in each of the next 20 frames. In a play session
363 regions were frozen in the second after Kakariko loaded and 70 to 120 a second for the next 20 seconds, with the
frames over 50 ms in the same windows. A pass now stops once it has spent NX2NAVI_FREEZE_BUDGET_US (default 1000;
0 as before), and a freeze over 2 ms holds the passes for 250 ms, doubled for each slow one in a row up to 2 s. On the
Kakariko route the time in slow memory calls after the load went from 1374 ms in 170 calls to 115 ms in 18, the 78 ms
frame is gone, and the regions freeze some three seconds later, once the calls are fast again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A frame some ten frames after an area load took 51.6 ms of replay with 1.2 ms of it in the replay's ops; the replaying
thread's samples sat in GetWriteWatch under vk_mem_mirror_begin and in the freeze's VirtualProtect. The lines say which
call and how much memory, under NX2NAVI_TRACE=perf.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The finalize of a used pool flushed and waited for every upload batch and then for the idle queue. During an area load
the queue also holds the load's texture uploads, which name no pool's storage: entering Kokiri Forest, one finalize
waited 418 ms and the frame took 462 ms. The game finalizes a pool once its own work with it is done, so the finalize
now waits for the submissions issued so far and returns; the pool's buffer and memory go once the upload batches handed
out by then and the submission being recorded are through. NX2NAVI_POOL_FINI_IDLE=1 keeps the old wait.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The author of pull request #353 closed it on 2026-11-11 with the words that they give no permission to use its code. Its port (3134f524) goes: the mark of a pipeline that declares the table (VkPipeInfo.gpa, vk_pipeline.c pipe_add) and of the part that draws with it (vk_mem_gpa_use in vk_exec_draw.c bind_resources and mo_emit), the sync of the table's pools in every such part's prologue (vk_mem_gpa_sync), the submission with a callback in the replay's place as it stood (vk_sync_probe_submit), and --probe-vkmem's read through the table with its shader (test_gpa_read, runtime/shaders/gpa_read.spvasm).
In its place, after pull request #551, which was written without that code: a shadow pool goes into the table and onto a list of the table's shadows (vk_memory.c shadow_table_add); the record of a binding of the table notes the guest submission (vk_exec_draw.c ssbo_info, vk_mem_shadow_table_read); and vk_mem_shadow_sync brings the CPU's writes into every shadow on the list once for that guest submission, before the part is submitted. Once is enough, because the copy-ins read the guest submission's write-watch snapshots, which a second sync would find unchanged. Measured by #551's author on Linux (RTX 4090, Kokiri Forest): 113 and 76 frames over 25 ms in 75 s against 217 and 429 with the port, 43 against 41 fps, the picture complete in both.
What this line had built on the port stays, on the new sync: the test that passes over a pool with nothing written (shadow_pool_clean), the upload of the same pages into a table pool's VRAM mirror, the write-back of stores through the table (NX2NAVI_SHADOW_TABLE_STORES, table_wb_cb, now asked once per guest submission that binds the table) and the counts of NX2NAVI_SHADOW_CHECK. The probes that record their own commands (the lost-device probe, --probe-vkmem's table parts) call vk_sync_submit_own, written anew. vk_mem_shadow_storage stays: one line that returns the mode. Finalized pools in the table were already kept for vk_mem_shutdown here.
--probe-vkmem has a check of its own for the sync (test_table_sync): the shadow of a pool that is never bound keeps its old word through a CPU write and a guest submission that does not bind the table, and holds the new one after a guest submission that does. It passes with NX2NAVI_SHADOW=1 on Windows (RTX 5090), as do the store check with NX2NAVI_SHADOW_TABLE_STORES=1 and the probe without shadow storage; --probe-devlost ends as before. Not run: the game, with or without shadow storage, and anything on Linux.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With shadow storage a write-back at a fence copies a page that has no base copy whole from the shadow into guest
memory. Every write the replay records gets base copies, so such a page means the pool the write was noted on was
finalized before the fence and another was made at its address: the new record's shadow is a copy of guest memory
from its own start, holds no byte of that write, and went over everything the CPU had written to the page since.
The page is now left alone when the record is younger than the submission whose fence it is, or when the CPU has
written the page since the shadow's last copy-in; the shadow line counts such pages.
--probe-vkmem has the case: with NX2NAVI_SHADOW=1 it fails without this change (0x11 over the CPU's 0x22, with
imports and with mirrors) and passes with it, as does the run without shadow storage.
Whether a game does finalize and remake a pool inside that window was not observed in play.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A lost device used to end the process from whichever submit or wait noticed it first, while the present only logged it
and went on and the other threads kept calling the driver. The completion thread could take the submissions' timeline
as reached and run the callbacks of submissions that never finished.
vk_device_lost_check now takes the result of every submit, present, acquire and wait. The first thread to see the loss
sets a flag, writes the report and ends the process as a fatal error does (the log written, a save commit under way
finished), with an exit code of its own; later threads stop in that call, and a watchdog ends the process after ten
seconds should the first thread hang. The replay, its submit, the uploads, the present and the lazy fetch read the flag
and return without touching the device. The completion thread asks the device with a call of its own before a new
value counts and runs no callback once the flag is set.
The executable's --probe-devlost and tests/device_lost_check.py check this without a lost device, and
device_lost_at in nx2navi.ini ends a real run the same way.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
pool_fini (vk_pool_fini) queues the record of a pool that is in the device address table and was not used, and of an unused
physical pool, for destruction at the next submission. When none follows, nothing destroyed it. Under shadow storage
that record holds an allocator buffer, and the allocator's check for allocations left over stopped the process in
vk_shutdown: --probe-vkmem with NX2NAVI_SHADOW=1 and NX2NAVI_MEMORY=import ended there before its last line. Such
records now wait on the retiring list, as mirrored pools already did, and vk_mem_shutdown destroys what is left on it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Verified by runs in the audit: the Market's periodic stall (a pool slot was listed once per refresh pass instead of once; the list fell from 160,385 entries to at most 3,192 and no frame over 20 ms remained while standing), orderly exits delete the pipeline prebuild's running marker (left behind, the next start wrote a skip file and every later start skipped the prebuild), the log and crash report name an area a save asks for that the game does not have, and the menu atlas hash no longer overflows a signed multiply. The other fixes compile and pass the regression route but their faults were never reproduced: see AUDIT.md section 13 for each.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The two sides both made the wait after a frame's first submission cover the write-backs under shadow storage; the rule is kept in one place, vk_sync_wb_split.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A pool refused for a failed allocation was never asked again and was read over the bus until the game finalized it. The shared quota of twelve log lines could be used up by budget refusals before the first allocation failure.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
NX2NAVI_SHADOW_CHECK=1 logs with the shadow line, every 60 frames: the whole copy-ins that overwrote a store a
shader made through the device address table, and the pools they fell on; the write-backs that moved bytes into
a page the CPU had written since its copy-in; the bytes both sides changed since their base copy; what the
copy-ins cost by kind, and which pools and flag sets took them. The stores are found by content: a whole
copy-in leaves a hash of the page, and the next one compares the shadow page with it first.
NX2NAVI_SHADOW_TABLE_STORES=1 keeps such stores: a page found stored into gets a base copy, so copy-ins take
only the CPU's bytes, and the fence of every submission that uses the table writes the stores back to guest
memory. --probe-vkmem checks it with a GPU write that nothing notes. It is off by default. The overwrite it
removes was the suspected cause of the flicker on Linux and is not: standing outdoors (RTX 5090 with
NX2NAVI_SHADOW=1) it happens exactly four times a frame, on ten pages in pools of 20 and 44 KiB, with the copy
that precedes the frame's stores, and no frame came out wrong with it or without it in 1,800 frames each; the
hashes cost 1.8 ms a frame of the replay (67 against 75 fps); and a count the CPU resets before the write-back
has landed would keep counting, where the whole copy-in resets it every time.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Under shadow storage the prologue of every part whose shaders use the table brings the shadow of every pool up
to date. It ran the full per-pool sync for each of them, 10,700 times a frame outdoors (3,772 pools, about
three such parts), though the write-watch snapshots of all pools are taken at once just before. A pool whose
regions hold no stamp newer than its shadow is now passed over with a compare per region: 1,500 pool syncs a
frame are left, and the copy-ins went from 1.32 to 1.01 ms a frame of the replay (RTX 5090 with
NX2NAVI_SHADOW=1, one standing run each way).
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
On an integrated Radeon 890M the ASTC scratch buffer failed to allocate and then a staging buffer ended the run. The scratch asks for 32 MiB whatever the upload needs, and the staging pool grows only in 64 MiB blocks of one memory type. When the device reports it is out of memory, the scratch is now tried again at the size the upload needs, and a staging buffer that does not fit the pool is tried again outside it with the same mapping, where a smaller block or another host-visible type may still have room. The menu's own upload flushes what it wrote, since such a buffer may lie in memory that is not coherent.
Built and run through the memory probe (96 GPU checks pass); the out-of-memory paths themselves were not reached here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Four "mem: sizes" lines give the entries and bytes of what no other line counted: the guest heap, physical maps, kernel region, lazily written ranges and the fixed kernel, setjmp, save-file, RomFS and zstd tables; pool records, mirrors, deferred destructions, upload and staging buffers, the device address table and the texture, sampler and slot caches; image views, descriptor spares, the completion queue, uniform snapshot arenas, recording copies and pins; and pipelines, layouts, stage libraries, job queues, the shader memory cache and the translator processes. Each subsystem reads its own counters under its lock; settings now count the replaced values they keep. Segment and scene lines say "-" for the world clock before it starts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An input script line "<tick> MARK <name>" (or "@<ms> MARK <name>") ends the running segment with one line of its frame count, mean, 1% low and worst frame, the work marks on any thread, process memory and VRAM, then the 600-frame memory lines, and starts a segment of that name. RomFS opens of an area's static placement file give a "scene:" line, and once no area has loaded for 10 s the memory snapshot is taken with a "settled" line.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A slow frame's line and the 60-frame window's line now say how long each kind took on any thread, how often, how many bytes and the longest single one, so a hitch outside the render thread's own waits has a named cause. While NX2NAVI_TRACE=perf is off each site pays one branch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The default was off only on AMD's driver branch for RDNA 1 and 2, on the grounds that builds with the default on had been measured on an RX 7900 XT. Those measurements were standing in Hyrule Field and walking through a village; the report this is about is of a game that ended in fights, on an AMD card the player did not name, and fights are where the lazy path is busiest. Nobody has played one on an AMD card under our eyes on either branch, so the exception is now AMD's own driver as a whole (wb_lazy_choice.h, by VkDriverId), until a card of each branch has played fights with wb_lazy = 1 and a log to show. The cost is the open-area frame rate of 0.7.0 on every AMD card under Windows; the RX 7900 XT figures in the draft notes come from builds that had it on by default, which the notes now say. wb_lazy = 1 still turns it on.
--probe-vkdevice prints the line a fatal error or a crash adds to the log, so its wording can be read without a failing run. On an RTX 5090: "fatal: gpu NVIDIA GeForce RTX 5090, driver NVIDIA "621.01" version 621.1.0.0; memory mode mirror, lazy write-back on (the default on Windows (wb_lazy = 0 turns it off)); texture detail not chosen yet (no texture made)", and with the reporting player's nx2navi.ini beside the exe "lazy write-back off (wb_lazy = 0 is set)", after a start whose log names async_compute, vram_safe_mode and exception_handler as no settings of this version.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A player on an AMD card reported crashes in fights and that a line added to nx2navi.ini cured them. That line, async_compute = 1, is no setting of the game, and neither are two more in the same file; the one line of the file that changes how memory reaches the GPU is wb_lazy = 0. A second report, an RX 6600 on 0.7.1, lost its device 3.5 minutes into play with the lazy write-back on. Neither proves it the cause: the player changed several lines at once and sent no log, the RX 6600's last pass mark is inside a draw pass and not a fetch, cards with 8 GB lost their device the same way on NVIDIA before the lazy write-back existed, and reading the fault handler, the fetch, the arming and the buffers' lifetimes found no failure path. What depends on the driver and cannot be checked without the card: pages of guest memory are inaccessible while a host-pointer import of them is bound and written by the GPU, and Vulkan calls run inside the access-fault handler.
The default is now decided once the driver is known (wb_lazy_choice.h, vk_memory.c vk_mem_lazy_decide): on under Windows as before, except on AMD's driver branch for RDNA 1 and 2, which none of us has run it on; AMD's driver for RDNA 3 and newer keeps it on, since builds with that default were measured on an RX 7900 XT. What the player sets wins everywhere, and one line at start says the choice and its reason. On those cards open areas run at the 0.7.0 frame rate again (on an RTX 5090 that was 65 fps against 90 in Hyrule Field; not measured on AMD). tests/wb_lazy_check.c checks the choice by platform, driver and setting without a GPU.
For reports of this kind the log now holds what was missing. At start: every setting nx2navi.ini sets, and in one line the keys that are no settings of this version and do nothing; the settings menu shows those in its bottom row. At a fatal error or a crash: one line with the card, its driver, the memory mode, the lazy write-back with its reason, armed ranges, fetches so far and a fetch in progress, and the texture detail (dispatch.c rt_fatal_note_add, vk_device.c fatal_note), since a report is often the end of a log alone.
--probe-cfg loads the player's file: the three lines are named, and the menu's save keeps them and every other line it does not change byte for byte. Nothing here ran on an AMD card, and no fatal error was provoked to read the new line.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Each submission's first sync of the bindless texture pool walked every registered slot, 33,000 in Kokiri Forest, to find the few textures the CPU wrote and the few slots that changed. Three parts of that walk change, each with its own switch (pool_fast, pool_presnap, pool_group; on by default, 0 and ab accepted):
- The freshness check goes through a list of the entries the slots resolved to, with a check by 2 MiB region first; slots that resolved to nothing are looked up again when the game's memory map changes and every eighth submission. The use stamps the idle eviction reads are left from a count of the slots that hold each entry.
- The check takes the write-watch snapshots of its regions first, so unchanged entries pass the region test.
- The GPU-written slots of one address bucket with one descriptor form a group, which the dirty walk leaves alone while its first member is unchanged.
Measured on an RTX 5090 at 2560x1440, standing in Kokiri Forest, in-run A/B over five runs with all three switched (three of them with the barrier change switched too): 93.6 to 100.7, 96.4 to 103.1, 95.8 to 102.6, 99.3 to 107.1 and 91.9 to 99.9 frames a second, the replay thread 0.7 to 0.8 ms a frame shorter; a control's blocks differ by 0.4 %. On a walk with NX2NAVI_POOL_GROUP_CHECK=1 no group that was left alone had missed a write, and screenshots across the A and B blocks show the same picture.
From pull request #390. Not tried: the texture eviction under memory pressure (a 9 GiB budget was never reached on the walk), AMD and Intel drivers, Linux. One of the nine runs with it on ended by itself with exit code 0 and nothing in its log two seconds after the window lost the focus; three repeats of that run did not.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A crash inside a recorder's Vulkan layer read like a fault of the game: the report now says that the module is not part of the game and what to do. A lost device now logs the video memory in use against the driver's budget, so a report from a card with little memory shows whether the memory ran out.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The user asked for this once the flicker fix was measured. The single-frame flicker that kept it off in 0.7.0 came from frames drawn before the game's signal that their data was written, which the previous commit fixes; the lazy write-back only made the frames come faster, so the signal fell into the replay more often. wb_lazy = 0 in nx2navi.ini still turns it off, and an ini that already holds the key keeps its value. Elsewhere than Windows the default stays 0, as its fetch inside the access-fault handler has never run there.
Measured on the build of the previous commit, 1,800 presented frames a run, standing in Hyrule Field by day with DLSS Performance, frame generation 4x and HDR. With the lazy write-back on and eight busy threads beside the game: 38 and 0 single-frame spikes with the fix switched off (408 and 44 frames whose signal came during the replay), 0 and 0 with it on (667 and 724 such frames). With it off and the same load: 0 spikes in four runs, two with the fix and two without, 86 to 140 such frames each. Without load, one run each: 88.5 frames a second with the fix against 93.3 without it with the lazy write-back on (GPU 7.2 against 6.8 ms a frame, 6.53 against 6.48 submissions a frame), 71.0 against 65.6 with it off, so the fix's cost is inside the run-to-run spread; every run ended cleanly without a fatal error. This build was started once with no setting anywhere: the lazy write-back was on, and the Screen tab shows no row for it.
Not measured: play by hand with the lazy write-back on and the fix (the user's sessions so far ran with it off), other areas than Hyrule Field, and more than one run for the frame rates.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
mirror_create refused a pool whose mirror did not fit under seven eighths of the VRAM budget. It first evicts idle
mirrors and asks idle textures for room, but on a 6 GiB card a loaded area's textures in use pass the budget by
themselves, so neither helps and the pools stay imported: the GPU then reads vertices, indices and storage over PCIe
at every draw. Two reports from RTX 3060 Laptop cards (6 GiB, a budget of about 4.7 GiB) measured 70 to 250 ms of GPU
time a frame and 5 to 13 fps in that state.
Now a mirror is made although the check fails while the mirrors together stay within half the budget; the textures
are still asked for the room, and past that share a pool is refused as before. A mirror the driver cannot keep in VRAM
is paged to system memory, where the imported pool already is, so it does not read slower than its pool.
Measured with the budget capped at 4,730 MiB on a 32 GiB card (1080p window, DLSS Balanced, no frame generation,
standing in Hyrule Field): before, 4 pools of 244.6 MiB stay imported, the largest pool among them (244.4 MiB), with
634 MiB mirrored; after, none stay imported and 878 MiB are mirrored, in two runs. The area still needs 5.6 to 6.3
GiB there, 3.9 GiB of it textures, so a 6 GiB card stays over its budget: this ends the buffer starvation, not the
overflow. Not run on a 6 GiB card.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Eighteen commits, one per Linux pull request that was not in yet: the build fix for a source checkout, NVIDIA's shader
disk cache only with NVIDIA's driver, tools/run_game.py for a Linux game folder, a memory limit the low-memory handling
can read, the recording thread's core on Linux, shadow storage on Linux and Proton, the Steam Deck profile and prompts,
FSR 3.1 offered only where its library loads, DLSS frame generation through NGX on Linux, the AppImage with its window
and its workflow, and the processor level from /proc/cpuinfo. Two conflicts: the settings table keeps the area-load
settings under the new comment, and the example file keeps the lazy write-back as off by default and adds that it
stays off with shadow storage.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With it on, single frames come out with the whole scene lit differently (brighter, shadows off). The flight recorder's
thumbnails of 1,800 presents standing in Hyrule Field show none with it off in every run, and with it on 43 to 181
while four other threads keep the CPU busy (10 to 97 in earlier runs, 0 on an idle machine); the cause is not found.
Two candidate fixes (arming only against a sync whose copies were handed over, and snapshots waiting for GPU writes in
flight) did not change the count, and fetching at every release made it far worse.
So wb_lazy is 0 unless set, on every platform (mem_init loses its special case), and the Screen tab has the row
Experimental FPS boost for it on Windows, with its text in the four menu languages; like every setting read at the
start it notes "after restart". The 0.7.0 notes describe it as experimental and off, say about 1.6 GiB for the video
memory a full shader cache no longer holds (2,484 to 846 MiB at the title screen in the measured run), and list the
two fixes before this one.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
On Linux the write watch protects pages, so a write-back from shadow storage into guest memory took a fault
at the first store into every page. plat_mem_prepare_write clears the protection of a range with one
UFFDIO_WRITEPROTECT call; the pages count as written, as they would after the faults. Off by default;
Windows has nothing to clear.
Pull request #327 by theurbanninja1993-sketch (a draft on GitHub). The platform function and its Linux
test are taken as they are. The call moved: the write-back it was written for is now shadow_write_back in
runtime/vk_memory.c (a file other sessions are changing), where it covers the runs of pages a submission
changed, and the setting is registered in cfg.c and described in nx2navi.ini.example, which also says
that the lazy write-back stays off with shadow storage.
The Windows files compile; the Linux code and its test are not compiled or run here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With shadow storage (Linux, and Windows builds under Wine) a CPU-visible pool is a host-visible copy with
no memory object of its own, and only pools that owned one went into the device address table: pointers
the game's shaders read from memory reached nothing. Shadow copies are now in the table. A shader that
reads through the table resolves no pool, so nothing queued the pool it reaches for the copy-in: the part
is marked where its pipeline is drawn or dispatched (VkPipeInfo.gpa, vk_mem_gpa_use), and its prologue
brings the shadow of every pool in the table up to date with the CPU's writes and queues the same pages
for those pools' VRAM mirrors (vk_memory.c vk_mem_gpa_sync). --probe-vkmem reads a word through the table
from a pool that is never bound, before and after a plain CPU write.
Pull request #353 by TophC7, ported onto the base copies of the black-world fix:
- taken as it is: the table registration, the mark in vk_exec_draw.c and vk_pipeline.c, the prologue
call and vk_sync_probe_submit in vk_sync.c (two hunks merged by hand), the probe's test;
- written again: vk_mem_gpa_sync, on top of the snapshot-based shadow_sync_pool; a lookup no longer
queues a pool for being in the table, which every shadow pool now is;
- narrowed: the sync runs only with shadow storage. Without it (Windows) a mirrored pool stays unsynced
for the table's readers as before, so the frame there is unchanged, and the probe makes the source's
mirror only with shadow storage and skips when the device has no address table;
- the probe's shader was an array of SPIR-V words in tex_probe.c; it is now the source
runtime/shaders/gpa_read.spvasm, built like the other runtime shaders;
- the exact re-upload count in the texture probe stays checked on Windows.
Files other sessions are changing: runtime/vk_memory.c (vk_pool_init's registration, mem_resolve's touch,
the new section before vk_mem_use).
The Windows files compile and the shader assembles and validates; no probe was run, and nothing was run on
Linux here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Shadow storage noted a GPU write with the whole range its binding names: the copy-in skipped that range and
the fence copied all of it back, so CPU writes to other fields of the same records never reached the GPU.
Ocarina's lighting parameter is such a field: the lighting came out NaN and the fog pass drew the scene
black. Each page a noted range covers now keeps a base copy, the copy-in moves only the bytes the CPU
changed and the write-back only those the GPU changed (vk_memory.c merge_page). The mirror's merge shader
marks the 4 KiB pages it changed, and one job per submission copies only those back (shadow_wb_cb); the
copy-in reads the CPU-side generation tables through the shared write-watch snapshots; and a caller that
reads guest memory itself resolves with vk_mem_lookup, which does not queue the pool for the copy-in.
Pull request #394 by jacc. Its changes are taken as they are; one hunk met the lazy write-back that came
later (vk_gpu_written_back) and is merged by hand. Added here: the lazy write-back stays off with shadow
storage, because its fetch copies the mirror into the pool's buffer, which is guest memory only for an
imported pool. Files other sessions are changing: runtime/vk_memory.c (the shadow section, the merge
record, vk_gpu_written_back, wb_lazy) and three one-word call changes in runtime/vk_usnap.c.
Without shadow storage (Windows outside Wine) the merge pipeline gains an unused fifth binding and two
push constants and nothing else changes. The Windows files compile; nothing was run, on Windows, Wine or
Linux.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A fetch copies the mirror over the whole range, not the merge of the bytes the GPU changed that a write-back does. The
completion thread arms a range after its write-back reached guest memory; a CPU write into those pages in between, not
yet carried up by a sync of the mirror, was then overwritten by the mirror's older bytes at the first fetch. In play
that was single frames with another frame's constants: the night lighting missing from the whole scene in Hyrule
Field every 0.1 to 0.4 s, and a black scene in Kakariko, gone with wb_lazy = 0. Now the pages are made inaccessible
first and the back end then checks the CPU-write page generations against the mirror's last sync and the write-watch
bits nobody has read yet; when either shows a write the mirror lacks, the pages are given back and the range arms at a
later write-back. The lazy line counts the arms refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A review found a gap: a part checked what it had skipped at its end, and a fault between that check and the part's
hand-over to the queue fetched without the part's writes while nothing wrote them back afterwards. The replay now
opens a window from the check to the hand-over, per part, and a fault on an armed page waits for it to close (at most
20 ms; the replay thread's own faults and thaws do not wait and are counted). What a part skipped for a range that is
no longer armed goes back through the usual snapshot and merge, taken in that part's prologue, and not as a plain
copy, which would have put the mirror's words over what the CPU had just written there. A part split off earlier
makes the same check before its prologue. wb_lazy_touch is a test that reads armed ranges on purpose.
Dev build, Hyrule Field: 71.1 fps as before; standing, a fight, pause and unpause, and the touch test made 146 to 338
faults each, none inside a window, no fault of its own.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The 2 GiB of device memory a filled cache cost was not the thousands of compute pipelines but twenty of them: the compute shaders of 0.9 to 2.0 million SPIR-V words, the game's tensor-core upscaler programs, which nothing dispatches. A pipeline of one holds about 100 MiB of device memory from the moment it exists, and the driver keeps that memory when the pipeline is destroyed, so keeping only the recently dispatched pipelines did not help: the boot pass still created the twenty from pipeline-cache.bin at every start. Measured with --probe-prewarm on a copy of an installed copy's caches (RTX 5090, NVIDIA 621.01): the pass over all 4,066 compute shaders took the device memory from 386 to 2,398 MiB and the process's commit up by 2.5 GB; the same pass before the twenty were in the cache left the device memory at 386 MiB, and all 4,046 other pipelines kept at once cost 2 MiB.
A compute shader over NX2NAVI_LIB_PREBUILD_BIG is now built ahead, from the caches or by the background builder, only when the usage record says a run dispatched it. None of the twenty is among the 138 shaders eleven runs dispatched. Should the game dispatch one for the first time, that dispatch compiles it once and later boots build it ahead.
The boot also reads the shader of a pipeline it does not keep as a copy and releases it after the build, instead of leaving 3,900 shaders loaded (240 MB of commit); a first dispatch then reads its shader from the disk, 0.04 to 0.10 ms in the probe.
With both, the same pass on the same caches leaves the device memory at 386 MiB and raises the commit by 47 MB; 256 first dispatches after it took 0.13 ms on average, 1.8 ms at most, all read from the cache file. With compute_keep = all, which builds everything as before, the pass ends at 2,528 MiB. Not yet run in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A draw whose shader or pipeline is still building is left out of its frame, and the frame was shown without it: a black or unlit picture with the interface intact when it was a lighting pass, a light going out, or ground missing for a frame. In a player's session a full-screen draw was skipped after its 50 ms and the frame came out black as a boulder hit.
The frame's present is now left out when draws were skipped (vk_backend.c frame_hold, NX2NAVI_FRAME_HOLD_MS, 120 by default, 0 off). The game runs on, so nothing waits. The hold comes from an allowance that fills again at an eighth of the clock's rate, so a long build or a run of them shows its frames as before and does not freeze the picture; a full-screen draw that would be skipped with the allowance spent waits on instead, 500 ms at most. Not with DLSS Frame Generation.
On a first walk through a village with no pipeline recorded, 21 of the 23 frames with skipped draws were held back; in the Lost Woods, 7 of 8.
The performance overlay: the VRAM line is the card's dedicated memory in use, which counts every program on it: a player read 14.5 GB as the game's while the game held 7.3 GB. The line's third cell now gives what this process holds on the device-local heaps (the driver's budget figure), in place of the GPU clock, which stays where the driver gives no budget.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Standing in Hyrule Field the GPU spent 13 ms a frame, of which 2.4 ms were the game's passes and 7 to 9 ms the copy
of 66 MiB of shader-written buffers back into guest memory at the end of every submission, in case the CPU reads
them. The write-back probe found no CPU access to them in 40 s. Now a range goes back once and its pages are made
inaccessible; while they stay so, later writes stay in the mirror. The first CPU access faults, the bytes are fetched
with a GPU copy of their own after everything submitted so far, the pages are given back, and the range is written
back every submission again for a while (1 s, doubling to 32 s while it keeps being touched). Thaws before kernel
reads and writes and before mapping changes release armed ranges the same way, and so does a mirror that goes away.
Dev build, same spot, 80 s each: 51.5 to 71.0 fps median, 1% low 43.5 to 56.3, GPU time 13.1 to 7.5 ms a frame,
fence waits 9.8 to 4.4 ms. In that run the game faulted on armed pages 226 times and 124 fetches took 51 ms in all;
a walk into a fight made 283 faults and 186 fetches, and a test that touches eight armed ranges every 60 frames
(wb_lazy_touch) 480 faults and 364 fetches, none with a fault of its own. wb_lazy = 0 is the old behaviour.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
An installed copy's warm list has 42,344 variants, 42,142 of them from shader-prepared.bin, and its 4,066 compute shaders are what the boot kept pipelines for. The comments and the settings text said the list grew with the play history; a copy with prepared shaders held the 2.3 GiB from its first boot. Comments, docs and the ini text only.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit ea984fe05b995c5dad1e0f49c5f44de9ee8f19b2)
The boot prebuild built the pipeline of every compute shader play ever recorded and kept all of them for the process. A pipeline that exists holds driver memory whether or not anything dispatches it: with a history of 4,066 compute shaders they held 2.3 GiB of device memory, and its share of the process's commit, from the title screen on (RTX 5090, NVIDIA 621.01), while a boot into the first village dispatched 95 of them.
The boot now keeps the pipelines of the 512 compute shaders dispatched most recently (compute-used.bin next to pipeline-keys.bin records which run last dispatched each; the setting is compute_keep, with "all" for the earlier behaviour). Every other one is still built at boot, so that the pipeline cache holds it after an update or a driver update, but destroyed at once, and created from the cache at its first dispatch. A dispatch that comes while such a build runs gets that build's pipeline.
Each boot stage now logs the device memory and the process commit it leaves ("mem: boot stage"), and the log says what the first dispatches cost. nx2navi.exe --probe-prewarm runs these stages without the game from the caches next to the executable and times first dispatches of pipelines the boot did not keep: 256 of them took 0.19 ms on average and 3 ms at most, all read from the pipeline cache file, and 372 pipelines built and destroyed at boot left the device memory where it was.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit 16a796d623686bca5304eaf1b29981247453a413)
The pre-snap of a submission took the pools the submission before it had synced. In Ocarina those are another submission's pools: the frame's first and fourth submissions use no pool and queried the pools of the ones before them, and the submissions after them then queried their own pools one at a time at first use. The pools are now remembered by the submission's place in its frame.
NX2NAVI_PRESNAP_PLACE=0 gives the old choice back and =ab switches it in the B blocks.
Co-Authored-By: Claude Fable 5.2 <[EMAIL]>
vk_mem_collect destroyed dead images, views and buffers while it held cmd_lock, and the replay's uploads wait for that lock. Destroying an image and its memory takes up to milliseconds. The collect now moves the objects out under the lock and destroys them after releasing it.
Co-Authored-By: Claude Fable 5.2 <[EMAIL]>
Every frame the render thread waited about 5 ms for the GPU after the frame's first submission, with the game
CPU-bound in every area. The wait is there so that the GPU has run that submission's passes before the game
rewrites what they read. But the submission also ends with the write-backs of what its shaders wrote, 60 MiB merged
into guest memory in 3 to 4 ms of GPU time, and the wait covered those too.
The write-backs now go into a part of their own. The part before them signals a second timeline, the executed
timeline, and the first-submission wait waits on that; the write-back part signals the timeline as before, so every
fence the game asks for still covers them, in the same queue order. wb_split = 0 keeps one part and the old wait.
Not measured in the game yet.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- The mesh stage's storage table starts 0x90 bytes before the end of the 2 KiB constant bank, and all sixteen slots
were written: 0x6C bytes past a stack buffer on every mesh draw. It holds the nine slots that fit.
- The prebuilt runtime is compiled without AVX, so its preemption stub saved only XMM0-15 while the lifted code of
an AVX2 processor keeps data in the upper YMM halves, which the stub's callees may clear. Both bodies are built
now and the one for the processor is chosen at the start.
- After an FSR 3.1 frame generation error the window's swapchain was still AMD's object but went to the plain Vulkan
calls. Swapchain calls follow whose swapchain it is, and a failure takes it down as a switch in the menu does.
- A movie is paced at its own 60 fps under a lower limit too: at 40 fps it ended 25 s behind its sound.
- A lost device left the wait for an upload spinning for good; it ends the run with the device's report.
- The compute pipeline key had four bytes of padding that were hashed and compared, so a prebuilt pipeline could be
missed and built again.
- A thread's TLS pointer was published before the TLS was allocated, so the slot was always 0.
- A fatal exit puts RivaTuner Statistics Server's profile back, and an ini saved as UTF-16 is read.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Per-pass GPU timestamps put the write-back of GPU-written storage buffers at 3.6 to 5.3 ms of GPU time a frame in play, against 1.8 to 2.5 ms for all of the game's own passes and dispatches: about 67 MiB a frame goes from the mirrors back into guest memory, from the same eleven pools all session, in case the CPU or a host path reads it. Nothing said whether anything does.
With wb_probe = 1 the 2 MiB regions that lie wholly inside a range just written back are made inaccessible. The first read or write of such a region faults; the handler gives the region back the protection it had (read-only when frozen, else read-write), notes the address, the instruction and the thread, and the access runs again. The bytes are in guest memory all along, so nothing changes for the game. A log block every perf window gives the regions armed, the probes a read or a write ended, and the first accesses by instruction. Every thaw ends the probe of its region first, which covers file reads into guest memory and mapping changes; a file write from guest memory thaws its buffer while the probe is on; a probed region is not frozen anew.
The platform layer gets a hook for every read or write access fault beside the write-fault hook; it is installed only with the setting. --probe-gfx-abi checks that nothing is armed without it and, run with NX2NAVI_WB_PROBE=1, that reads and writes end a probe and run again, that a frozen region thaws after it and that mem_thaw ends and holds.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A session with per-pass GPU timestamps showed the game's own render passes and compute dispatches at 1.6 to 2.3 ms of GPU time a frame while the perf line's GPU time was 7.7 to 13.5 ms: most of the frame's GPU time is in commands the replay records between the marks, and nothing said which.
The timed frames now also get a timestamp at the start and the end of each part of a submission, and every stretch between two marks (or a mark and the part's edge) is a gap. The replay names the kind of commands it records in a gap: buffer and texture copies, fills and clears, the temporal upscaler, write-back merges and copies, depth twin copies, the texture pool marks, the mesh draw counts and block-compressed target copies, each with a size where it has one. The report adds the gaps' time per frame summed by the kinds noted in them (a gap nobody noted holds barriers and state only), the parts' own time for comparison with the perf line, and each kind's count and size per frame. Nothing runs outside the timed frames.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
VMA's blocks on large heaps are 64 MiB instead of 256 (vma_block_mb): one session held 2.1 GiB free inside the
device blocks beside 8.2 GiB of allocations after a few area changes, and the driver counts a block whole until it
empties. The headroom thread keeps its spare block only for the device-local types textures and mirrors grow in; the
spare blocks it made for the host types and the PCIe BAR type at boot stayed empty for the whole run. The ASTC
decoder's scratch is 32 MiB, so it stays inside a block, and an area load that makes one per upload batch holds half
as much at once. A DLSS feature no frame used for 3600 evaluations is freed with its work images, and a model change
now frees all five sizes.
The perf log gets a "vram by use" line (textures, released textures, mirrors, physical pools, ASTC scratch, headroom
spare, free space inside the blocks, memory outside VMA and DLSS's own share), the free ranges per memory type, the
addresses whose duplicate texture entries hold the most, and a line when memory outside VMA moves by 256 MiB.
astc_bc_probe in nx2navi.ini turns on the ASTC opacity probe.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 99cfc2fa14ae208b7b23e057ff9b87986cd6a5e0)
A shader's store or atomic through the table goes to a pool's VRAM mirror and again to its guest copy, which on a
discrete GPU crosses PCIe. On PCIe 3.0 (RTX 4090, Kokiri Forest, DLSS Quality) turning the twins off took the GPU from
16 to 13.1 ms a frame; the writes that went wrong without them were foliage counts and records in pools of 20 and
44 KiB. gpa_twin_max_kb = n gives a twin only to mirrors of pools up to n KiB. A larger mirror's record names the mirror
twice, so its second write goes to the table's scratch page, its buffer fills stay in VRAM (vk_mem_guest_twin), and it
is never evicted, since guest memory lacks what the shaders wrote into it. The mirror report counts the mirrors made
without a twin. Default 0: every mirror has one, as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A device lost seen on RTX 4090, 5090 and 5070 Ti cards in 0.5.0 and 0.6.0 (#177, #61, and a 0.6.0 test run): a
"read invalid" at the start of a 128x128 scene capture pass, re-rendered at area loads. destroy_tex purged the views of
the image and deferred the image, then freed its record at once, on whichever thread released the texture. The replay
keeps &e->pub as a render target for the rest of a command buffer (vk_exec.c resolve_target) and DLSS for the frame:
a pass that began again (each indirect mesh draw ends the pass) read the freed record and cached a view of the dead
image, which nothing removed. Vulkan gives a destroyed image's handle to a later image, which then got that view, and
the pass loaded its attachments from freed memory.
- destroy_tex marks the record gone and frees it through the deferred destruction (vk_defer_free, a host kind stamped
with the submission being recorded, as the image is).
- A gone texture's attachment view is made per command buffer (vk_exec.c view_of) or deferred with it
(vk_exec_target_view, for DLSS and the contact shadows), never cached.
- A new image drops any view still cached under its handle, and the log counts it ("cached views of a destroyed image").
No rendering changes: the views and images live exactly as long as before, the records slightly longer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the local fixes package (fix 11), measured in Kokiri Forest at 4K on an RTX 4090 over PCIe 3.0: 40.5 to 62.6 fps
native and 45.6 to 74.9 fps with DLSS Quality, the mirror copies from 6.5 to 0.3 ms of GPU time a frame.
- Pages a VRAM mirror wrote back into guest memory are not copied up into it again: the mirror sync reads page stamps
of their own that a mirror's write-back leaves alone (vk_gpu_written_back, decided at record time by
vk_mem_mirrored). This replaces the lookup table NX2NAVI_WB_NO_REUP had, keeps its name, and is now on by default.
- The game's flushes of mirrored memory are not copied where the write watch covers the region, as every page the game
wrote there is stamped already (47-57 MiB a frame); NX2NAVI_FLUSH_UPLOAD=1 copies them as before.
- Written storage ranges up to NX2NAVI_WB_MERGE_MAX (now 32 MiB) are merged in 4 MiB pieces, so a range is no longer
bound to what one dispatch covers; the snapshot pool keeps its budget-scaled size with 128 slots.
- A submission's first part goes to the GPU at the first pass boundary while the GPU is idle (split_idle), long passes
are split too (split_in_pass), and the part's time limit reads the clock less often.
- A render target that needs an image of its own takes the image of a released one with the same parameters, released
two or more frames before (tex_reuse); memory requirements are kept per parameter set; a released entry another
descriptor's entry replaced at its address is kept and taken back.
- The memory check reads a page's write-watch bit and stamp under the lock that keeps them together.
- The launcher reads the gamepad only while its own window is in front, and does not start a second copy of the game.
- make_game.py says why a Python package did not load, not only that it is missing.
The fix's per-pass GPU timing is left out: NX2NAVI_GPU_PASSES does the same in main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>