The 8 ms and 16 ms tried in their place left fewer draws out in a scripted walk, but in play a frame that takes 10 to 14 ms longer felt worse to the player than one that is left out, in two sessions on the same install. The wait stays counted once for each pipeline, and a frame that has left a draw out already still waits for nothing more.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A draw that met a new pipeline waited 3 ms for its fast link, in every frame, which is just under what the link takes with the stage libraries in the driver's cache: in a first-visit session on a new install 66 of 87 such draws gave up at the end of the wait and their frame was held back. The wait is now 8 ms for each pipeline, counted from its first miss and shared by all its draws, and 16 ms for a frame, so a pipeline the driver has to compile costs one frame the wait and the frames after it nothing.
The pipeline list tool also takes the compute variants a game folder dispatched, which no pipeline record names and which were translated in the middle of play, and reads a list made before as one of its inputs.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A new install met every pipeline in play: in Hyrule Field one 600-frame stretch of a first visit had 1,769 draws wait 461 ms in all for pipelines, the longest frame 88 ms. The boot now reads seed-pipeline-keys.bin, the release's list, as recorded pipelines after the install's own (keys_open_seed, NX2NAVI_PIPE_SEED, default 1): their pairs go into the vertex output unions and the prewarm builds them as it builds the install's. The preparation builds every listed pipeline (NX2NAVI_PREPARE_PIPELINES now defaults to all), with the unions of the same list, so the first boot of a new install finds them in the caches. The list's mesh pipelines are left out at the boot on a GPU without mesh shaders, as the preparation leaves them out. Nothing of the list goes into pipeline-keys.bin.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A mesh pipeline whose fragment stage reads a location the mesh program never writes now gets a fragment module where
that input is a private variable set to (0, 0, 0, 1) at entry, or to (s, t, 0, 1) of gl_PointCoord when the mesh
program draws points. The fragment shaders recorded with two of the game's point-drawing mesh programs read location
5, which those programs never write; Vulkan leaves such an input undefined.
Checks: tests/fs_fill_inputs_check.py passes; 237 recorded fragment modules rewritten offline pass spirv-val; a
60 s Kakariko run with the change shows the area as before with no Vulkan errors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A device that reports an axis held from the start (a virtual pad nobody holds, a trigger or pedal mapped onto a stick,
a stick resting far off its middle) turned the camera or moved Link on its own, and since pad_active counted it as
pressed, it could take the gamepad role from the controller in the player's hands whenever that one was idle, and keep
it. input.c now reads every axis through pad_axis: 0 until the axis has been within a quarter of its range of rest
(a trigger's rest is 0), then the device's value. The game's sticks and triggers, the menu's stick readout and the
choice of gamepad all use it. A stick the player holds at the start counts once it has been let go.
Checked: the runtime compiles; new tests/pad_rest_check.py (pad_axis cut from input.c, SDL stood in for) passes: a
held axis reads 0 for 100 polls, a released one reads the device, and axes and gamepads keep separate states;
tests/mouse_stick_check.py still passes. Not run with a real device.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux every driver got shadow storage (copies of guest memory beside it), which exists for amdgpu, where importing guest memory costs the write protection of its pages. NVIDIA's own driver imports without that cost, so there the copies are only the slower path: in Kokiri Forest on 0.7.1 (RTX 4090) imports ran at 51.9 frames a second and the copies at 27.4. Until now only the AppImage's play.sh knew, by setting NX2NAVI_SHADOW=0 when NVIDIA's kernel driver was present at build time, so a game started any other way got the copies, and a PC with that kernel driver whose game ran on another card got imports where it should not. The game now chooses by the driver of the device it runs on, play.sh no longer sets the variable, and NX2NAVI_SHADOW=1 or 0 still decides. The driver is asked when the device is picked, before the storage is chosen.
Windows is unchanged: the device probe here reads "mem: mode mirror" with no shadow storage line, and the shadow line with NX2NAVI_SHADOW=1. The Linux path itself was not run: there is no Linux machine here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The author of the pull request it was ported from closed that request on 2026-11-11 with the words that they give no
permission to use its code. The history (NX2NAVI_VA_REPORT, runtime/vk_vaddr.c, the stack walk it used and its notes
in the docs and the example settings) goes; the lost device's report keeps the breadcrumbs, the pass marks, the video
memory and the driver's fault information.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The author of pull request #353 closed it on 2026-11-11 with the words that they give no permission to use its code. Its port (3134f524) goes: the mark of a pipeline that declares the table (VkPipeInfo.gpa, vk_pipeline.c pipe_add) and of the part that draws with it (vk_mem_gpa_use in vk_exec_draw.c bind_resources and mo_emit), the sync of the table's pools in every such part's prologue (vk_mem_gpa_sync), the submission with a callback in the replay's place as it stood (vk_sync_probe_submit), and --probe-vkmem's read through the table with its shader (test_gpa_read, runtime/shaders/gpa_read.spvasm).
In its place, after pull request #551, which was written without that code: a shadow pool goes into the table and onto a list of the table's shadows (vk_memory.c shadow_table_add); the record of a binding of the table notes the guest submission (vk_exec_draw.c ssbo_info, vk_mem_shadow_table_read); and vk_mem_shadow_sync brings the CPU's writes into every shadow on the list once for that guest submission, before the part is submitted. Once is enough, because the copy-ins read the guest submission's write-watch snapshots, which a second sync would find unchanged. Measured by #551's author on Linux (RTX 4090, Kokiri Forest): 113 and 76 frames over 25 ms in 75 s against 217 and 429 with the port, 43 against 41 fps, the picture complete in both.
What this line had built on the port stays, on the new sync: the test that passes over a pool with nothing written (shadow_pool_clean), the upload of the same pages into a table pool's VRAM mirror, the write-back of stores through the table (NX2NAVI_SHADOW_TABLE_STORES, table_wb_cb, now asked once per guest submission that binds the table) and the counts of NX2NAVI_SHADOW_CHECK. The probes that record their own commands (the lost-device probe, --probe-vkmem's table parts) call vk_sync_submit_own, written anew. vk_mem_shadow_storage stays: one line that returns the mode. Finalized pools in the table were already kept for vk_mem_shutdown here.
--probe-vkmem has a check of its own for the sync (test_table_sync): the shadow of a pool that is never bound keeps its old word through a CPU write and a guest submission that does not bind the table, and holds the new one after a guest submission that does. It passes with NX2NAVI_SHADOW=1 on Windows (RTX 5090), as do the store check with NX2NAVI_SHADOW_TABLE_STORES=1 and the probe without shadow storage; --probe-devlost ends as before. Not run: the game, with or without shadow storage, and anything on Linux.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The probe's programs are written in the shape the translator gives a task or mesh program and need no game files.
With a folder of game translations it converts those instead and builds their compute pipelines.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
tools/validate_shader_cache.py reads every module and its reflection from the pack and the loose files, says how many the game would refuse and translate again, runs spirv-val on the others and groups what it refuses by message. It writes nothing to the cache. tests/validate_shader_cache_test.py checks it on a made-up cache.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A save that found no memory for its copy, could not write its file or met a build that had just begun was not tried again until the next new pipeline was built, so a run that built none and then ended without its orderly exit lost what it had compiled since the last file. The autosave thread now repeats the try, and the same bound holds for the save owed after the play file was folded in, which before was tried at every interval for as long as it failed. tests/save_retry_check.py checks the rule.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The library creates its pipelines itself and names no pipeline cache. It is now handed pipeline creates of the runtime's in the place of the device's two, and a create without a cache is asked of the pipeline cache first and compiled into the play cache otherwise. ffx_pipe_cache = 0 turns it off. --probe-ffx-cache makes one upscaling context without a window or the game and says where its pipelines came from, and tests/ffx_cache_check.py runs it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The game held 23.9 to 24.4 GB of commit in three sessions on an RTX 5090 at 3840x2160, flat over the session, on a
machine whose commit stood at 66 of 73 GB with it. A machine that cannot give it that much pages, drops textures or
loses the device, and its log said nothing about memory until then.
mem_start_note reads the system's commit when a game starts and writes one warning line when what is free, plus what
the process already holds, is below mem_need_mb (24576 by default, 0 for no warning). The menu has no place for a
start-up notice (its only toasts are the co-op page's), so the line is in the log alone.
tests/mem_need_check.py checks the rule and, through --probe-memneed, the answer and the warning line with the
machine's own commit.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Only the large compute shaders' background builder looked at the system's commit before it started a build, and it
saw nothing of what the builds beside it were about to take. The boot's recorded pipelines (up to 64 builders), the
pool's background builds during play, the boot's compute pipelines and stage libraries and the library warm asked
nobody.
Each of them now asks the gate before a build. A build that starts reserves an estimate of the driver compiler's
working memory until it ends (build_reserve_mb, 64 MB; 4096 MB for a large compute shader), and the gate admits a
build only while the commit headroom holds the part kept free (build_free_mb, 4096 MB), the reservations and its own.
A refused boot builder ends and leaves its pipelines to the pool; the pool leaves a background build queued and asks
again every two seconds; the stage library prebuild and the library warm wait up to 30 seconds and then end, and the
warm goes on at the next start. A pipeline a draw or a dispatch waits for is never refused. The preparation of the
pipeline list runs without the gate: no game runs beside it.
tests/build_gate_check.py checks the gate's arithmetic and, through --probe-buildgate, its answers with the machine's
own commit. What the builders do when refused is not exercised by a test.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The prepared list named its shader variants and nothing about where they came from, so an update that keeps the prepared shaders (shader_carry) kept the old list whatever seed the new kit writes: variants only the new seed names, from a newer pipeline list or a change to which variants are prepared, were translated when play first drew them. The existing identities do not cover this: nx2navi_xlate.id and identities.txt name translators, which decide cache keys and what is carried over, not which variants are listed.
The list is now version 2 (runtime/prp_file.h): after the head come the sha256 of seed-programs.bin, the sha256 of seed-answers.bin and the translator identity, written by gfx_shader_prepare_seed; the three readers go through gfx_shader_prepared_read and still read a version 1 list. At an update step_shaders writes the seed first (prepare_shaders.py --no-run, about 4 s) and keeps the list only when its digests are that seed's (make_game.py seed_changed); another seed, or a list that does not say, is prepared again. Another translator alone still carries over: input ids do not depend on the translator, and preparing everything again for it is what carrying was made to avoid.
Checked with tests/prp_file_check.py (the C layout against make_game's reader), three launcher tests, and the exe's --prepare-shaders on a seed without records in a folder of its own: the digests it wrote are those of the seed files, one byte more in the seed reads as changed, and after preparing again it does not. A version 1 list of 42,142 variants and the same list as version 2 are both read whole by --probe-prewarm.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A lost device used to end the process from whichever submit or wait noticed it first, while the present only logged it
and went on and the other threads kept calling the driver. The completion thread could take the submissions' timeline
as reached and run the callbacks of submissions that never finished.
vk_device_lost_check now takes the result of every submit, present, acquire and wait. The first thread to see the loss
sets a flag, writes the report and ends the process as a fatal error does (the log written, a save commit under way
finished), with an exit code of its own; later threads stop in that call, and a watchdog ends the process after ten
seconds should the first thread hang. The replay, its submit, the uploads, the present and the lazy fetch read the flag
and return without touching the device. The completion thread asks the device with a call of its own before a new
value counts and runs no callback once the flag is set.
The executable's --probe-devlost and tests/device_lost_check.py check this without a lost device, and
device_lost_at in nx2navi.ini ends a real run the same way.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
cache_load measured pipeline-cache.bin with ftell into a long, which is 32 bits on Windows: a file past 2 GB measured as -1, so the run started with an empty cache and the log said nothing, not even that the file was over pipeline_cache_max_mb. The read now lives in runtime/vk_cache_file.h: the length comes from the 64-bit tell, the read runs in 64 MB pieces, and a refusal returns its reason and the file's size, which cache_load logs. The Windows atomic write no longer cuts a block of 4 GB or more to a 32-bit count.
tests/pipeline_cache_file_check.py reads a sparse file just over 2 GB through the header. With --probe-prewarm and such a file carrying this GPU's header, the log reads "holds 2048 MB, over NX2NAVI_PIPELINE_CACHE_MAX_MB 512" under the default limit and "2048.0 MB read" with the limit off; the exe before this change logged nothing for the same file.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
mouse_mode = direct is the default, but on 1.1.1 it did nothing: only the 1.4.3 patch table had the sites, so the mouse always went through the stick emulation, which gives no turn at slow mouse speeds and tops out at about 107 degrees a second. The 1.1.1 table now has the set mouse_direct (on by default): at main+0x37c45c, where the camera's input update holds the processed stick in s1 and s0, a host call performs the displaced store and adds the raw mouse counts to both, so the mouse meets no dead zone and no curve; at main+0x37c4f0 a second call clears the idle bit for an update the mouse moved in. The counts are divided by the frames the game steps per update and by the stick camera speed, so the same hand movement turns the same angle at every frame rate. The Controls tab gets the mouse mode and mouse sensitivity rows, and mouse_mode = stick or NX2NAVI_MOUSE_DIRECT_PATCH=0 gives the old behaviour.
Checked here: the generator lifts the 1.1.1 game with the two sites (it compares each site's original word), the lifted game compiles and links with the runtime, and tests/mouse_direct_check.py passes its 1.1.1 harness (register contract, displaced store and load, equal turn at every frame rate and stick speed, the reset pause, capture and mode). Not checked here: the game was not run, so aiming with the mouse was not tried, and the differential tests against Unicorn were not run because no Python on this PC has Unicorn. The contributor measured 0.0800 degrees per count on both axes in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit bb8fecfbe43517006eafd7be9c22e8dbe9601de7)
Each player keeps their own game and save. One player makes a room, the others join it with its secret, and story flags and, by the room's rules, items, songs, quest items, small keys (a shared pool), rupees and ammunition reach every save. The layer is a host thread with one UDP socket to a relay the players run themselves (tools/coop_relay.py, Python's standard library only); the messages between the players are sealed with a key from the room's secret (XChaCha20-Poly1305, Monocypher 4.0.2 vendored with its licence and a row in THIRD_PARTY_NOTICES.md), every change is signed by the player who made it, a player accepts a room by its members' keys and the key of its rules, the save is copied to coop-backup before anything is applied, and values are set through the game's own routines between frames. The Co-op tab makes, joins, accepts, removes and leaves, and the menu takes typed text for the name, the relay and the secret. This is the contributor's pull request #518 on this line.
Nothing reaches the network while co-op is off: the socket is opened, and the relay's name looked up, only after co-op is on, a relay is set and a room is made or joined. There is no default relay.
Left out of the pull request: the other players drawn in the world (the "puppets") with the copied engine they need. They were unfinished, added two patch sites to the game's per-frame update that could not be put through the differential tests here, and their file helpers clashed with the ones this line already has. So no position is sent: the POS message's codec stays, nothing sends it.
Changed against the pull request:
- The menu's test hook drives the Co-op tab only when it comes from the environment. From nx2navi.ini it could turn co-op on, join a room and accept its offer.
- The log's label for a room is hashed from the secret instead of being its first three characters.
- A game that has not saved yet has no save to copy, and takes nothing from the room until it has saved once; that used to count as a copy made.
- A PING's token no longer carries the PC's uptime.
- On Windows the maker's key file and its folder get an access list with the player's account alone.
- The tests that link the platform layer get the socket library, and the save test a stand-in for the title screen's hook.
Checked on Windows, where the pull request had never been compiled: the runtime builds and links; --probe-coop passes (sockets on 127.0.0.1, the codec against its byte vectors, a fuzz of every parser, two sessions through a hostile mock relay, a relay restart, the sync engine); tests/coop_relay_test.py passes its 61 tests, with the probe through the real relay on 127.0.0.1 at 0 and 30 % loss; tests/coop_page_check.py passes and its deliberate fault fails 48 checks. Not checked: two games in one room (one game runs at a time here), any real network, the tab in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The two sides both made the wait after a frame's first submission cover the write-backs under shadow storage; the rule is kept in one place, vk_sync_wb_split.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
NX2NAVI_SHADOW_CHECK=1 logs with the shadow line, every 60 frames: the whole copy-ins that overwrote a store a
shader made through the device address table, and the pools they fell on; the write-backs that moved bytes into
a page the CPU had written since its copy-in; the bytes both sides changed since their base copy; what the
copy-ins cost by kind, and which pools and flag sets took them. The stores are found by content: a whole
copy-in leaves a hash of the page, and the next one compares the shadow page with it first.
NX2NAVI_SHADOW_TABLE_STORES=1 keeps such stores: a page found stored into gets a base copy, so copy-ins take
only the CPU's bytes, and the fence of every submission that uses the table writes the stores back to guest
memory. --probe-vkmem checks it with a GPU write that nothing notes. It is off by default. The overwrite it
removes was the suspected cause of the flicker on Linux and is not: standing outdoors (RTX 5090 with
NX2NAVI_SHADOW=1) it happens exactly four times a frame, on ten pages in pools of 20 and 44 KiB, with the copy
that precedes the frame's stores, and no frame came out wrong with it or without it in 1,800 frames each; the
hashes cost 1.8 ms a frame of the replay (67 against 75 fps); and a count the CPU resets before the write-back
has landed would keep counting, where the whole copy-in resets it every time.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With shadow storage (the Linux default) what the GPU wrote reaches guest memory in the fence's completion
callback, later than an import's store on the GPU. The wait after a frame's first submission returned at the
executed timeline, before that, so the game rewrote what those passes had written and the write-back then put
the GPU's older bytes back over the rewrite. The whole 3D scene came out brighter for single frames, each of
them a frame whose main work was held at its CPU event wait and replayed late.
Standing outdoors with NX2NAVI_SHADOW=1 on an RTX 5090: 695, 624, 443, 323 and 2 flashed frames of 1,800 in
five runs with the split, none in eight runs without it. The frame rate did not drop (74.6 fps median with the
fix; runs of the build before it gave 71 to 72). NX2NAVI_WB_SPLIT=2 keeps the split with shadow storage, to
measure the fault. That this is the flicker a player reported on Linux (RX 7800 XT, RADV) is not confirmed on
their machine; NX2NAVI_WB_SPLIT=0 gives 0.7.1 the same wait there.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The action prompt at the top right of the HUD draws the four face buttons as a diamond in the console's places: A right, B bottom, X top, Y left. Under pad_layout = xbox, playstation or auto the A action is the bottom button, yet the diamond kept A on the right, so the prompt pointed at the wrong button. At start the runtime now edits the prompt's layout in the shared interface archive for such a layout (the buttons, the dots and the lit crosses move, and the bubbles of X and Y flip) and serves the result from the profile folder; with nintendo, the default, nothing is changed. Every field is found and checked before anything is written, so a layout a mod changed is left as it is, and the archive is packed again with the game's own dictionary at the same unpacked size. One SARC reader and dictionary lookup now serve this, the camera pack and the menu's assets. When the layout changes, the menu's Restart counts the diamond.
tests/action_prompt_check.py passes without game files and on the game's own archive (4 copies, 208 fields, 13 ms at start).
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The preparation translates every shader and builds each one's stage library, so a pipeline a draw meets for the first time is linked in 2 to 6 ms. A mesh pipeline has no stage libraries: on an install that never drew it, it compiles whole in play, 0.2 to 1.1 s with its draws left out of the picture (18 of them on a first visit of Kokiri Forest). Whole pipelines were built ahead only from the folder's own pipeline-keys.bin.
tools/pipeline_list.py merge folds the pipeline-keys.bin and shader-answers.bin of game folders that were played into one list file: every key record once, with the answers record of each shader it names. A record names its shaders by input id, which depends on the game's code and the answers alone, and holds the game's pipeline state; what depends on the GPU is derived when the pipeline is built, so a record made on one machine is good on another with the same game version and runtime. Four folders on this PC give 5,432 records, 317 of them mesh pipelines (147 KB of keys).
tools/prepare_shaders.py reads the list (tools/data/pipeline_list.bin, data/pipeline_list.bin in a release folder, or --pipeline-list), adds its shader variants to the seed and writes its records as seed-pipeline-keys.bin. nx2navi.exe --prepare-shaders builds the mesh pipelines among them after the seed is translated and saves the pipeline cache before the stage libraries are built (NX2NAVI_PREPARE_PIPELINES=all builds the vertex pipelines too); --warm-shaders does it again after an update or a new driver, and --prepare-pipelines is that step alone. The pipelines go into the caches and are not kept or added to the folder's own key file, so they cost no start-up time or video memory until a draw meets them. A marker file keeps a driver that crashes on one of them from failing every preparation.
The list is derived from game data and stays out of the repository: check_content.py refuses files recorded from a game folder by their first bytes, and make_kit.py, make_launcher.py and make_release.py carry a list given with --pipeline-list. All three now also carry pipeline_list.py, which prepare_shaders.py imports.
Checked without the game: tests/pipeline_list_test.py, and prepare_shaders.py --no-run on the game files with a list of 317 mesh pipelines (all 266 shader variants found in this game version, 177 new to the seed). The build step in nx2navi.exe compiles and links and has not been run.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
After a lost device the driver's fault report gives an address and nothing says what was there. With va_report = 1 (NX2NAVI_VA_REPORT, off by default) the game keeps every address range the driver binds to a buffer or image and every range it unbinds (VK_EXT_device_address_binding_report, runtime/vk_vaddr.c). The report of a lost device then names the ranges that covered each faulting page, or the nearest ones when none did, and vaddr-history.txt beside the log holds the whole history, also when the driver gives no fault report. With 2 the call stack behind each range of 1 MiB or more is kept too. A player sees nothing of this unless the setting is on: without it no extension is enabled and no callback runs.
From pull request #349. Its second part, which stops a frame dump from turning on mesh query statistics, is left out: it changes what a dump writes for everyone who uses one. Built for Windows; no device was lost to try it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With the mouse camera in stick mode and a camera speed above 1, the mouse could never turn faster than a full stick does at camera speed 1, however high the mouse sensitivity was set. The mouse's counts were limited to full tilt first and divided by the camera speed afterwards, so at camera speed 1.5 the most the mouse could send was two thirds of full tilt, which the game's own factor brought back to the old top speed. The division now comes before the limit (runtime/input_map.c input_mouse_stick): a slow movement turns as far as before, and a fast flick reaches full tilt, which is the game's top speed with the camera speed in it. Direct mouse mode, the default, does not use this code. A camera speed of 1 or less behaves as before.
From pull request #424. tests/mouse_stick_check.py is new here: it builds input_map.c alone and checks the deflection at each camera speed without the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The developer build stopped at undefined dav1d symbols since the AV1 movie decoder (runtime/ov_movie2.c) came in: tools/make_game.py linked dav1d, tools/build_game.py did not. Windows links libdav1d.a statically, as libvpx and zstd are, so the exe imports no libdav1d DLL; Linux links -ldav1d. The llvm-mingw link list and the Linux release's static list get dav1d too, and docs/building.md lists the package for MSYS2 and Arch.
From pull request #439, with the two other link lists added.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A player's AppImage build of 0.7.1 stopped in its compile step after eight minutes of RomFS work: four
files did not compile, because the kit builds with the system's compiler and headers and theirs were
older than the runtime needs. vk_pipeline.c and vk_device.c name VK_KHR_pipeline_binary,
VK_KHR_dynamic_rendering_local_read and VK_KHR_shader_maximal_reconvergence, which their Vulkan headers
did not declare, and ui/menu.cpp takes its font in with #embed, which clang has from version 19.
- Vulkan headers: tools/vulkan_pin.py holds the release the other builds use (1.4.363, the archive and
hash tools/make_release_linux.py already pinned, which now reads them from there). The AppImage carries
its C headers in toolchain/vulkan-headers and builds against them whatever the system has; a kit or a
source checkout uses the system's when they are at least that release and fetches the pinned ones
otherwise (build_game.py vulkan_include). Their folder goes first on the include path of the runtime's
sources only, so the lifted code's objects keep their keys, and a runtime object compiled before
against the system's headers is taken over instead of compiled again. AppRun no longer asks for the
system's Vulkan headers.
- #embed: tools/build_shaders.py also writes the font's bytes as runtime/ui/roboto_ttf.h (ignored by git,
carried by a kit), and menu.cpp includes that where the compiler has no __has_embed. With #embed
nothing changes.
- Before its long steps a Linux build that will compile now checks every runtime source that needs no
header of the install with the compile step's own command (clang -fsyntax-only) and links an empty
program against the game's libraries (make_game.py preflight): an old compiler, an old header or a
missing development package stops it in the first seconds with the compiler's own errors. A failed
compile names every file that failed, not only the three whose errors it prints.
Checked on Windows: tests/launcher_test.py passes with four new tests (117); the header of bytes and
#embed give the same 162,588 bytes, and both branches of menu.cpp compile; the early check passes on the
Windows file set with clang 23 and stops on files made to fail; the 34 local headers of release 363 are
the upstream tag's files by git blob id. Not run: anything on Linux, the pinned archive's download, a
compiler older than 23.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A draw whose pipeline is still building is skipped when its pass is rendered every frame and waits otherwise, so that a lookup table or a volume generated once is not left empty. "Every frame" meant "in the last four frames", so after a loading screen or a full-screen menu every pass of the first frame read as new: each draw with a new pipeline waited on the replay thread, up to 30 s each, and a mesh draw, whose pipeline has no fast link, for its whole compile. A player's 0.7.1 log shows it at three area changes: frames of 954, 1015 and 684 ms with 853, 960 and 663 ms of pipeline wait, six mesh draws waiting 52 to 660 ms in passes whose targets had been rendered in thousands of frames.
The target history (now runtime/vk_target_frames.h) also counts the frames each target was rendered in, for as long as it is the same guest texture. From eight frames on its pass is per-frame however long ago the last one was, and its draws are skipped and built by the workers like any other (pipe_skip_steady, on by default). A target rendered for the first time still waits, now from one pool of 50 ms a presented frame that the full-screen draws share (pipe_frame_wait_ms; oneshot_budget = 0 gives the old wait back), a mesh draw 8 ms at most, and the wait for a one-shot draw's shader comes from the same pool (gfx_shader_await takes a limit). A full-screen draw with no picture left to hold waits 40 ms more instead of 500. One frame so waits under 100 ms for pipelines in all.
Every 600 frames with a miss the log says how many draws found a shader or pipeline missing, how long they held the replay thread in all and in the worst frame, by the kind of wait, and how many were skipped at its end.
tests/target_frames_check.py replays the area change of that log against the header. Not yet run in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A thread that found the queue's lock taken for 4,000 spins wrote its line itself, ahead of the lines it had queued
earlier. On a busy machine the lock's holder loses the processor inside the copy and the lock stays taken for a time
slice, so one thread's lines reached the log out of order. A line now waits for the lock, and a line that finds no
room, or is larger than the queue, is written after everything queued before it.
tests/logq_check.py failed on this in 8 of 30 runs beside four busy processes and passes 100 of 100 now. It gained a
check that holds the queue's locks itself and fails on the old code every time, and it no longer depends on time: it
waits for the queue to empty, keeps the line about slow writes back except in the step that checks it, and counts the
lines that waited instead of timing the slowest.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
One conflict, in the build kit's notes, which keep both the shader cache item and the Texture detail text.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The probe takes the level from NX2NAVI_TEXTURE_DETAIL. It covers a 1024x1024 texture, a 256x256 array of three layers, and a 128x128 and an unmipped texture that stay whole, with the images as RGBA8 or BC3, and the sampler LOD clamp of a reduced texture. It passes at full, reduced and low on an RTX 5090, with NX2NAVI_ASTC_BC 0 and 1.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The driver keeps the prepared shaders' libraries in its shader cache, and under Windows that cache has a size limit
set in NVIDIA's settings. A limit smaller than the set drops the oldest libraries as new ones come in (a 2048 MB cache
could not hold them on the reporter's RTX 5070 Ti), so every start builds them again in the background and play
stutters until that is done. The cold lines of the library warm now say so on NVIDIA under Windows, most plainly when
the cache is cold right after a warm that finished on the same driver; the build kit's notes and the example file say
it too. tests/warm_hint_check.py checks which hint each case gets.
From pull request #403 by tarballz; two conflicts resolved (an include and the build kit's notes).
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Every log line was two writes on the thread that logged it, to stderr and to the log file. Such a write can stall for
tens or hundreds of milliseconds (a busy disk, a virus scanner, a console or a pipe that is slow to take it), and the
render thread logs too: a frame it spent in a write was a hitch with nothing else to show for it.
In a game run the lines now go into a queue that a log thread writes out in order (runtime/logq.c). A fatal error,
the crash report and the end of the process write the queue out on their own thread first and write their own lines
directly, so nothing is lost to an exit. A thread that finds the queue's lock held for long, or the queue full, writes
its line itself. NX2NAVI_LOG_ASYNC=0 writes every line where it is logged, as before.
Every write is timed in both modes: a "log: ... writes of the log took over 2 ms" line counts the slow ones, gives
the longest and says whose time they took.
tests/logq_check.py checks order and completeness with eight threads, a slow destination, a stopped destination with
the queue full, and the synchronous flush.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A signalled kernel event, or a thread that ended, woke every thread blocked in WaitSynchronization. Each woken thread
takes a virtual core before it looks at its handles, asking a running thread of lower priority for its core when all
are held, finds nothing signalled and blocks again. The audio renderer signals its event 200 times a second, so every
thread that waits for a message, a timer or another thread's end did this 200 times a second at the cost of the
threads that were working.
A wait now records its handle list, and a signal wakes the waits that hold a handle of the signalled event or thread.
NX2NAVI_SYNC_WAKE_ALL=1 (or ab) wakes them all as before, and the sched line counts the waits woken and left asleep.
tests/sync_wake_check.py builds kernel.c with the rest of the runtime stubbed and checks the wakes, the timeouts, the
cancel and that none of 4,000 signals to eight looping waiters is lost, in both modes.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With shadow storage (Linux, and Windows builds under Wine) a CPU-visible pool is a host-visible copy with
no memory object of its own, and only pools that owned one went into the device address table: pointers
the game's shaders read from memory reached nothing. Shadow copies are now in the table. A shader that
reads through the table resolves no pool, so nothing queued the pool it reaches for the copy-in: the part
is marked where its pipeline is drawn or dispatched (VkPipeInfo.gpa, vk_mem_gpa_use), and its prologue
brings the shadow of every pool in the table up to date with the CPU's writes and queues the same pages
for those pools' VRAM mirrors (vk_memory.c vk_mem_gpa_sync). --probe-vkmem reads a word through the table
from a pool that is never bound, before and after a plain CPU write.
Pull request #353 by TophC7, ported onto the base copies of the black-world fix:
- taken as it is: the table registration, the mark in vk_exec_draw.c and vk_pipeline.c, the prologue
call and vk_sync_probe_submit in vk_sync.c (two hunks merged by hand), the probe's test;
- written again: vk_mem_gpa_sync, on top of the snapshot-based shadow_sync_pool; a lookup no longer
queues a pool for being in the table, which every shadow pool now is;
- narrowed: the sync runs only with shadow storage. Without it (Windows) a mirrored pool stays unsynced
for the table's readers as before, so the frame there is unchanged, and the probe makes the source's
mirror only with shadow storage and skips when the device has no address table;
- the probe's shader was an array of SPIR-V words in tex_probe.c; it is now the source
runtime/shaders/gpa_read.spvasm, built like the other runtime shaders;
- the exact re-upload count in the texture probe stays checked on Windows.
Files other sessions are changing: runtime/vk_memory.c (vk_pool_init's registration, mem_resolve's touch,
the new section before vk_mem_use).
The Windows files compile and the shader assembles and validates; no probe was run, and nothing was run on
Linux here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Shadow storage noted a GPU write with the whole range its binding names: the copy-in skipped that range and
the fence copied all of it back, so CPU writes to other fields of the same records never reached the GPU.
Ocarina's lighting parameter is such a field: the lighting came out NaN and the fog pass drew the scene
black. Each page a noted range covers now keeps a base copy, the copy-in moves only the bytes the CPU
changed and the write-back only those the GPU changed (vk_memory.c merge_page). The mirror's merge shader
marks the 4 KiB pages it changed, and one job per submission copies only those back (shadow_wb_cb); the
copy-in reads the CPU-side generation tables through the shared write-watch snapshots; and a caller that
reads guest memory itself resolves with vk_mem_lookup, which does not queue the pool for the copy-in.
Pull request #394 by jacc. Its changes are taken as they are; one hunk met the lazy write-back that came
later (vk_gpu_written_back) and is merged by hand. Added here: the lazy write-back stays off with shadow
storage, because its fetch copies the mirror into the pool's buffer, which is guest memory only for an
imported pool. Files other sessions are changing: runtime/vk_memory.c (the shadow section, the merge
record, vk_gpu_written_back, wb_lazy) and three one-word call changes in runtime/vk_usnap.c.
Without shadow storage (Windows outside Wine) the merge pipeline gains an unused fifth binding and two
push constants and nothing else changes. The Windows files compile; nothing was run, on Windows, Wine or
Linux.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The first part of this pull request (the cache beside the exe with an 8 GiB limit) was already in. This
is the rest: nothing is set without NVIDIA's kernel driver, neither variable is set when the folder cannot
be made or written (the driver caches nothing in such a folder), the log says which folder and limit hold,
and a game run sets the drivers' cache variables in main before any of its threads can load a Vulkan
driver (vk_device.c vk_driver_cache_env); the other modes still set them in create_instance. On Windows
that only moves the MESA_SHADER_CACHE_DIR setenv earlier.
Pull request #369 by mooki000, taken as it is. The Linux part is not compiled or run here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The 2 GiB of device memory a filled cache cost was not the thousands of compute pipelines but twenty of them: the compute shaders of 0.9 to 2.0 million SPIR-V words, the game's tensor-core upscaler programs, which nothing dispatches. A pipeline of one holds about 100 MiB of device memory from the moment it exists, and the driver keeps that memory when the pipeline is destroyed, so keeping only the recently dispatched pipelines did not help: the boot pass still created the twenty from pipeline-cache.bin at every start. Measured with --probe-prewarm on a copy of an installed copy's caches (RTX 5090, NVIDIA 621.01): the pass over all 4,066 compute shaders took the device memory from 386 to 2,398 MiB and the process's commit up by 2.5 GB; the same pass before the twenty were in the cache left the device memory at 386 MiB, and all 4,046 other pipelines kept at once cost 2 MiB.
A compute shader over NX2NAVI_LIB_PREBUILD_BIG is now built ahead, from the caches or by the background builder, only when the usage record says a run dispatched it. None of the twenty is among the 138 shaders eleven runs dispatched. Should the game dispatch one for the first time, that dispatch compiles it once and later boots build it ahead.
The boot also reads the shader of a pipeline it does not keep as a copy and releases it after the build, instead of leaving 3,900 shaders loaded (240 MB of commit); a first dispatch then reads its shader from the disk, 0.04 to 0.10 ms in the probe.
With both, the same pass on the same caches leaves the device memory at 386 MiB and raises the commit by 47 MB; 256 first dispatches after it took 0.13 ms on average, 1.8 ms at most, all read from the cache file. With compute_keep = all, which builds everything as before, the pass ends at 2,528 MiB. Not yet run in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A draw whose shader or pipeline is still building is left out of its frame, and the frame was shown without it: a black or unlit picture with the interface intact when it was a lighting pass, a light going out, or ground missing for a frame. In a player's session a full-screen draw was skipped after its 50 ms and the frame came out black as a boulder hit.
The frame's present is now left out when draws were skipped (vk_backend.c frame_hold, NX2NAVI_FRAME_HOLD_MS, 120 by default, 0 off). The game runs on, so nothing waits. The hold comes from an allowance that fills again at an eighth of the clock's rate, so a long build or a run of them shows its frames as before and does not freeze the picture; a full-screen draw that would be skipped with the allowance spent waits on instead, 500 ms at most. Not with DLSS Frame Generation.
On a first walk through a village with no pipeline recorded, 21 of the 23 frames with skipped draws were held back; in the Lost Woods, 7 of 8.
The performance overlay: the VRAM line is the card's dedicated memory in use, which counts every program on it: a player read 14.5 GB as the game's while the game held 7.3 GB. The line's third cell now gives what this process holds on the device-local heaps (the driver's budget figure), in place of the GPU clock, which stays where the driver gives no budget.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The boot prebuild built the pipeline of every compute shader play ever recorded and kept all of them for the process. A pipeline that exists holds driver memory whether or not anything dispatches it: with a history of 4,066 compute shaders they held 2.3 GiB of device memory, and its share of the process's commit, from the title screen on (RTX 5090, NVIDIA 621.01), while a boot into the first village dispatched 95 of them.
The boot now keeps the pipelines of the 512 compute shaders dispatched most recently (compute-used.bin next to pipeline-keys.bin records which run last dispatched each; the setting is compute_keep, with "all" for the earlier behaviour). Every other one is still built at boot, so that the pipeline cache holds it after an update or a driver update, but destroyed at once, and created from the cache at its first dispatch. A dispatch that comes while such a build runs gets that build's pipeline.
Each boot stage now logs the device memory and the process commit it leaves ("mem: boot stage"), and the log says what the first dispatches cost. nx2navi.exe --probe-prewarm runs these stages without the game from the caches next to the executable and times first dispatches of pipelines the boot did not keep: 256 of them took 0.19 ms on average and 3 ms at most, all read from the pipeline cache file, and 372 pipelines built and destroyed at boot left the device memory where it was.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit 16a796d623686bca5304eaf1b29981247453a413)