Commit Graph

114 Commits

Author SHA1 Message Date
chrg6
f3856c8dca Ask for a reused compute request before its shader module is made, and report the reuse with the prebuild.
A compute warm request the same as one built before still made its shader module, with the SPIR-V fixes, pool rebind
and marks, before the reuse table answered; the table is now asked first, all of its key being known by then. The
table is cleared under its lock, the count of reused requests is in the prebuild's and the preparation's report
instead of a line at every autosave, and NX2NAVI_COMPUTE_WARM_REUSE is a known setting, so nx2navi.ini can set it
without an "unknown key" line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 22:54:02 -05:00
automata.
9e23aec717 Reuse equivalent shader translation and compute preparation work 2026-11-12 22:53:00 -05:00
chrg6
6590f68f6e Check the driver's cache of stage libraries at normal priority, into the driver's cache only, and go on with a warm from where it stopped.
Taken from pull request 760 and redone on the line's check: the sample was built at below-normal priority as the
guest started beside the boot's background builds, and on an RTX 5080 (0.9.2) it read a warm cache as cold, a median
of 136 ms, after which 42,231 libraries were built again in 873 s although the driver had them. The sample is now built
at normal priority and into the driver's cache only, as the warm's workers build (a library our own pipeline cache
holds read fast whatever the driver had), and a sample that says cold is taken again 10 s later, except while the start
waits for the whole warm after a driver change, where cold is the expected answer. The thread goes back to below
normal afterwards only when the game runs beside it.

Two parts of the pull request are not taken. The third fastest of twelve called a cache warm when three libraries
were in it, which is the partly evicted cache the warm is for; the cache counts as warm when three quarters of the
sample are fast. A resumed warm was sampled and stamped done when it looked fast, which a warm cut off past a quarter
of the list would pass with the rest never built; it is not sampled, and goes on from the variant the earlier run
reached, which shader-warm.txt now keeps every 8,000 variants, instead of from the first variant again: the cause of
the warm at every start the pull request saw with sessions shorter than a warm.

Co-Authored-By: Leyukaka <45659323+Leyukaka@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 22:52:54 -05:00
chrg6
de3f10fe82 Keep the pipeline cache limit at 1024 MB at least, and fold the play file only with room left for the fold's save.
The RAM-based default gave 509 MB on a 16 GB machine, below the about 550 MB a full cache of the pipeline list reaches
on an NVIDIA card: with the limit kept on the write side the files would stop there and every pipeline past it would
compile again at every start, on the most common RAM size. The default is now RAM / 32 between 1024 and 4096 MB.

A fold was taken when the merged cache fitted the limit at the boot, but pipeline-cache.bin is written at the first
autosave, after the boot's builds have added to it. Past the limit that save was refused while the play file was
written over with the new run's pipelines, and what had been folded from it in memory was then in neither file. The
fold now needs a play file's worth of room (pipeline_cache_play_mb) under the limit, and the play file is not written
while a fold is not yet in pipeline-cache.bin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 22:51:08 -05:00
Leyukaka
fe5bff8328 Keep the pipeline cache files under their limit when writing them, and set the limit from the RAM.
The limit was only checked when a file was read, all or nothing, so a run could write a file the next start then
dropped whole. On an RTX 5080 (0.9.2) pipeline-cache-play.bin reached 524.9 MB in one session and the next start began
empty; pipeline-cache.bin grows 14-28 MB at each fold and crosses 512 MB within a few sessions. Now a fold is made only
when its result fits, and a cache past the limit is not saved again, so the file keeps its last save under the limit.
Unset, the limit is the RAM / 32 between 512 and 4096 MB (512 on 16 GB as before, 2048 on 64 GB), which on that PC
took boot hits from 14 to 84 % and left no pipeline to background builds.

Taken into the 0.10.0 line with its conflicts resolved: the line had raised the limit to 1024 MB and says why in
the comment above cache_data_read, in nx2navi.ini.example and in AUDIT.md; those texts keep that reason and give the
RAM-based default, whose floor the next commit raises to match.
2026-11-12 22:50:53 -05:00
Leyukaka
e0271c0298 Treat pool arrays of the old 32768 length as pool arrays, for translations carried over from the translator before.
After an update the game keeps the previous translator's translations until a background pass translates them again. Those still declare texture pool arrays of 32768, which spirv_pool_rebind no longer took for pool arrays once VK_POOL_TEX grew, so their textures were never bound and areas loaded with empty images. They now bind to the first 32768 entries of the larger binding.
2026-11-12 22:45:06 -05:00
chrg6
f2a5800c1e Merge the stability pass's fixes for the points its review found.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

# Conflicts:
#	runtime/cfg.c
2026-11-12 22:32:48 -05:00
chrg6
1a206434e0 Build the mesh pipelines' stage libraries ahead of their draws, at the preparation and at boot.
The preparation and the boot build the stage libraries of single shaders. A mesh pipeline's pre-rasterization
library needs its mesh stage and its task stage together, and its fragment shader library depends on what that mesh
stage leaves unwritten, so only a pipeline record names what they are made of.

Every listed mesh pipeline the preparation builds and every recorded one the boot queues now notes its two shader
libraries, each key once. The preparation builds them on every core after the listed pipelines and before the
pipeline cache is saved, inside the marker that covers a driver dying on a listed pipeline. The boot builds them on
one thread at idle priority beside the game, behind the memory gate, with the rest of the library prebuild
(lib_prebuild = 0 leaves it out). Each is destroyed once built, which leaves it in the pipeline caches and the
driver's own, as for the single shaders' libraries.

A mesh pipeline of known stages that a draw meets with other targets, another blend state or sample count then finds
its libraries in the caches and is linked, where it was compiled whole. The recorded and listed mesh pipelines are
still built whole ahead of play, also on a machine that leaves the rest of the shared list to its first draw.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 22:31:13 -05:00
chrg6
17aee6ea73 Link mesh pipelines from stage libraries like the vertex pipelines.
A mesh pipeline (task and mesh stages: the foliage, the grass, the terrain) had no stage libraries. Each new one was
a whole build, 0.2 to 1.1 s the first time foliage of a kind appeared, and its draws waited or were left out of the
picture until it was done.

Where the graphics pipeline library is on, a mesh pipeline is now linked at its first use from three libraries and
its whole build follows behind, as for a vertex pipeline. Its pre-rasterization library holds the task and mesh
stages and is keyed by both shaders. No vertex input library is linked: a mesh pipeline has no such state. Its layout
is a stage-independent pair, the mesh and task stages' bindings in set 0 and the fragment stage's in set 1, so the
fragment shader library can be the one a vertex pipeline uses. That library's key now also carries what a mesh
pipeline's fragment module gets filled in and whether the shading rate is dynamic state, which the pre-rasterization
and the fragment shader state both hold. The fragment output library is shared as it is.

The fast link on the library workers, the wait of a draw for its libraries and the pipeline hints take mesh pipelines
in. The summary counts them by themselves: fast links and their time, how long the ones a draw missed took to become
usable, and the libraries made for them.

NX2NAVI_GPL_MESH=0 (gpl_mesh in nx2navi.ini) keeps the whole build. It is also kept without the pipeline library, on
a device that draws no mesh pipeline, and where mesh pipelines alone are built without the driver's optimiser; the
start logs which is in use and why. The mesh emulation of the low-end GPU mode is not touched.

A mesh pipeline with libraries is another pipeline to the driver than the one built before, so an install's recorded
mesh pipelines are compiled once more at the first start with this change.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 22:31:11 -05:00
chrg6
b1f1ee3d48 Say in the log that the whole warm waits for the recorded pipelines only as long as it may take.
With the warm's time limit the line that promised a wait for all of them was wrong when the limit ended it first.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 22:30:56 -05:00
chrg6
4a6ba66115 Tell a start that was closed or stopped during the boot's builds from a driver crash, and limit the whole warm.
A marker or a journal left by a start that died in the boot's builds was always read as a driver crash. Since the
whole warm those builds run for minutes, so a start ended from outside in that time (the task manager, a program that
stops the game, the power) switched the pipeline prebuild off for that driver for good, or put up to 32 variants on
the skip list that the driver had merely been building. Closing the window was already an orderly end and still is.

The crash handler now leaves a note while the boot's builds run. With the note the next start does what it did: it
skips the prebuild, or the variants the journal names. Without it the end is taken for a stopped start: the pipelines
are built again and the journal's variants are blamed on nothing. Three such ends in a row still skip the prebuild,
for a driver that ends the process without a fault the handler sees. The boot's journal has a file of its own, so the
preparation's keeps its meaning.

The whole warm waits prewarm_full_s seconds at most (600): when they are up the game starts, the builders step down
to the background's thread counts and priority, and the file says "done", so the boot screen has one turn per driver.

tests\boot_warm_check.py covers what a leftover marker means with and without the note.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 22:24:22 -05:00
chrg6
cd0b1a387d Do the whole warm on the boot screen at the first start after a driver change.
A start that finds the pipeline cache written by another device or driver has nothing built, and everything was
compiled beside the game: an 8-core integrated GPU drew fewer than 600 frames in its first 11 minutes. Such a start
now waits on the "Preparing shaders" screen, with counts, for every recorded pipeline, for the compute pipelines on
every core at normal priority, and, when a sample says the driver's own cache lacks the prepared stage libraries, for
all of them on every core. The memory gate still holds for the builds, and the library builds write the preparation's
journal while the start waits, so a variant the driver's compiler dies on is skipped from the next start on.

boot-warm-owed.txt beside the caches names the GPU and driver: a start closed during the warm is followed by one that
goes on with it, three at most, and "done" keeps later starts ordinary while the cache file still carries the earlier
driver's header. When the boot stops waiting (prewarm_wait_ms, or a minute without progress) the builders step down
to the background's thread counts and idle priority. prewarm_full = 0 turns the warm off.

With nothing in the driver's cache the probe took 425 s on an RTX 5090 with 16 CPU threads: 87 s for 5,775 recorded
pipelines, 19 s for 4,075 compute pipelines and 317 s for 42,231 prepared variants.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 20:03:12 -05:00
chrg6
016c90fafb Wait at the start for every recorded pipeline when the pipeline cache was written by another device or driver.
After a driver update pipeline-cache.bin is dropped and every recorded pipeline is built from nothing. With the pipeline library the boot waited 20 s and left the rest to be built beside the game. In a player's log of an 8-core integrated Radeon that was 2,141 of 8,122 pipelines, finished 296 s after the start, with the 42,231 prepared variants being built behind them: the game drew fewer than 600 frames in its first 11 minutes, which reads as a freeze. A start that finds such a file now waits for all recorded pipelines, with their count on the boot screen, as a start without the library does; NX2NAVI_PREWARM_FULL=0 keeps the 20 s. The prepared variants still follow in the background. Checked with --probe-prewarm on a cache copy whose header names another device: the line is logged and all 5,775 pipelines are built before the stage ends.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 15:31:56 -05:00
chrg6
bc256496cb Build task and mesh shader pipelines without the driver's optimiser on AMD's driver for RDNA 1 and 2, unless vk_noopt says otherwise.
That driver loses the device in play ("No fault detected", a timeout) on RX 6600 to RX 6950 XT cards in every version from 0.6.0 to 0.9.2, with the pipeline library on or off and the lazy write-back on or off (#591, #602, #659, #683, #684, #689, #635, #719, #279, and fourteen of twenty sessions in a player's launcher log of an RX 6800). Where the pass marks name a pass the GPU was in, twelve logs of five reporters, its first draw is a mesh pipeline: mesh shader ba4302f1 with fragment shader f64a96dd in nine, the one-draw pass of 1bbdb17e with 6cc3a476 in three. The five logs of 0.9.1 and 0.9.2 stop between the compute dispatch before such a pass and the pass. vk_noopt = mesh was there as a test switch for this suspicion and no reporter had run with it.

It is now what an unset vk_noopt means on that driver branch; every other driver keeps 0, and vk_noopt = 0 gives the optimised pipelines back. The picture is the same; the mesh and task shaders run slower, by how much is not measured, and those pipelines are built once more because the flag is part of the driver's cache key. That the optimiser is the cause is not shown: no AMD card ran this.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 15:31:56 -05:00
chrg6
823e0cdcd4 Raise the pipeline cache's size limit to 1024 MB, and leave never-drawn list pipelines to their first draw on a machine short of memory.
The limit was 512 MB, and a file over it starts the run empty until a play file replaces it. With the pipeline list of 0.9.1 one run builds more than that: on an RTX 2080 Ti (#722) pipeline-cache.bin held 551 MB and the play file the run then wrote 541 MB, so both were dropped at every start, 7,247 of 8,531 pipelines were built again each time (17 s of the boot) and 541 MB were written in every session. Seven of the players' logs read here have a file over 512 MB, five of them between 517 and 691 MB. At 1024 MB those are read, the play file is folded in once and the loop ends; a start that still finds both files over the limit now says so.

pipe_seed_boot (auto, 1, 0) decides whether the boot builds the pipelines of the shared list that the install has never drawn. Each is a pipeline the driver holds for the whole run: the boot's pipeline stage went from about 1.6 GB of commit on 0.8.0 to 3.9-4.4 GB on NVIDIA cards with the list (1.5-1.8 GB on AMD's driver), and 0.3-0.6 GiB of video memory. auto, the default, builds them as before except on a machine that starts with less commit free than the game takes in play, the start's own memory warning: there they are made at their first draw from the pipeline cache and recorded as the install's own. Mesh pipelines are built at boot either way. Compiled; not run in the game.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 15:13:40 -05:00
chrg6
2324a74602 Turn the settings for older graphics cards on with one key, low_end_gpu = 1, and leave them off without it.
Mesh shaders as compute, the texture that gets memory of its own, staged mirror uploads and the shorter wait for the GPU, widened write-watch queries, the video memory limits and reserve, early upload batches, the asynchronous replay, the hot-pool rule and the recording thread on AMD cards without mesh shaders were measured on a Radeon RX 580 and run on no other card. Each keeps what 0.9.1 did unless the key or its own setting says otherwise; the key gives the defaults the way the device profile does. The notes describe them as that option, hold main's two new entries in one Changed section, and list the white edges in towns as a known issue.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 13:00:13 -05:00
chrg6
a464ffafbc Merge Tobim6's work for older graphics cards: mesh shaders emulated with compute, video memory kept within the heaps' budgets, staged mirror uploads and the submit wait that goes with them.
Conflicts: the settings key lists take both sides; frame generation keeps the camera-cut handling and logs its counters every 600 frames with the cuts among them; the pass-time mark moves into draw_viewport; mo_desc keeps the pose depth and notes the reader.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 12:14:58 -05:00
Tobim6
83eade69d1 Prewarm: emulated mesh pipelines are built ahead of play
On a GPU without mesh shaders the boot prewarm dropped every recorded mesh pipeline, also with the mesh
emulation, which builds them as a vertex pipeline plus compute stages (pipe_miss: layout_get_emu). Each was
then built at its first draw, 113-830 ms on an RX 580 (22 of the 48 pipelines missed entering Kokiri Forest on
0.9.1), and the draws waiting for it were skipped: frames held back or shown without them. With
NX2NAVI_MESH_EMULATION on they are built by the prewarm like the others.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 16:23:24 +01:00
Tobim6
4027229930 Mesh emulation: its compute pipelines are built outside the lock, and looked up without one
A draw of an emulated mesh pipeline looked its entry up under the lock the build held across the driver's compile of
both stages: on a first run every such draw waited for whatever entry was being built, 2.5 s of the replay thread in
Kokiri Forest on an RX 580. Entries are now published complete at the head of one of 256 buckets and read without a lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 16:23:10 +01:00
Tobim6
0b4fdac38d Defaults from the RX 580 runs: asynchronous replay, mesh emulation, mirror_hot 25, freeze_idle 1200, native zstd, game code at -O2; freeze and zstd and replay rows in the Lama patches tab.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 16:22:57 +01:00
Tobim6
93e68d57f1 Pool marks: a fragment shader stores its marks where it ends, not between a sample's coordinates and the sample.
On AMD's driver for Polaris (RX 580, 23.10) a store between a pool texture's access chain and its sample left the
quad's helper lanes out of the code in between: derivatives at a triangle's edge were garbage and edge pixels came out
dark, the dotted lines along the paths' triangles since 0.7.0 (frame dump f1714: the colour pass covers every pixel,
the depth test is not what drops them). Each access now notes its slot in a Private variable; before each return of
the entry function and before a kill, terminate or demote every noted slot is marked, an unset one going to a spare
slot past the pool. Accesses in loops or outside the entry function note theirs in a ring of 8, marked in a loop at
the way out. Checked with spirv-val on the 45 marked shaders of the dump.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 16:22:34 +01:00
Tobim6
bb654470b2 Crash report: the log names what the crashing thread was building, and a library link names its case.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 16:22:24 +01:00
Tobim6
1768d5f607 Mesh emulation: the vertex shader declares the fragment shader's inputs one for one.
The generated vertex shader wrote one vector per location. A fragment shader that packs two inputs into one location
(Component decorations: a vec2 at component 0 and another at component 2) then had nothing matching its second input,
whose value is undefined: wrong colours on the emulated foliage. The vertex shader now declares an output for every
input variable of the fragment shader, with its Location, Component, type and Flat, filled from those components of
the mesh stage's vector at that location (zero where the mesh stage writes none).

(cherry picked from commit c38eb31889314ebfb164ff9dffad5796eba46ed4)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 16:22:08 +01:00
Tobim6
8b78e64539 Mesh emulation: point meshes too, and a warning for draws from a nonzero workgroup.
The 8 point-list mesh programs of Ocarina 1.1.1 (32 points, position and point size) were left unemulated. Their
indices are single uints and the point size goes to an extra vertex slot, which the generated vertex shader writes to
PointSize; the draw asks for a point list (the pipeline key's topology class follows the draw's primitive for mesh
draws), chosen from the mesh stage's OutputPoints. A mesh draw from a nonzero first workgroup is drawn from 0 as
before, now with a warning in the emulated path too. All 1016 programs convert and pass spirv-val.

(cherry picked from commit 8900aeaffa3264e4cc655ffd487cb2d2f3e3dd17)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 16:22:07 +01:00
Tobim6
aec5b5a840 Emulate mesh shaders with compute on GPUs without VK_EXT_mesh_shader (NX2NAVI_MESH_EMULATION, off by default).
Without the extension every mesh draw was skipped ("mesh draws need VK_EXT_mesh_shader: skipped"): on an RX 580 the
game's grass and foliage were missing. The translator already builds each task and mesh program as compute code that
MeshSpirv.cs wraps into TaskEXT / MeshEXT modules; runtime/mesh_emu_spirv.c turns those modules into GLCompute ones:
outputs and payload move to workgroup memory, SetMeshOutputsEXT's counts are kept, and the entry function's end
reserves vertex slots and indices in an arena with atomics and copies the vertices and triangles out (task: the payload
and the mesh group count go to a task record, and the mesh dispatch to the draw record). A generated vertex shader
reads the vertices back for the game's fragment shader. Checked with spirv-val (Vulkan 1.3) on the 1016 task and mesh
programs of Ocarina of Time 1.1.1: 1008 convert and validate; the 8 point-list meshes stay unemulated.

vk_pipeline.c gives an emulated mesh pipeline a layout whose task and mesh bindings are visible to compute, with push
constants and a compute-side descriptor template, builds the generated vertex shader's graphics pipeline and the two
compute pipelines; vk_exec_draw.c prepares each mesh draw as before, then outside the pass zeroes the arena counters
(first emulated draw of a submission), fills a draw record, pushes the same descriptors to the compute bind point,
dispatches the task and mesh stages (direct, or from the indirect ring entry) and draws DrawIndexedIndirect from the
record. mesh_emulation = 1 turns it on; mesh_emu_mb sizes the arena (160 MiB).

(cherry picked from commit efc43b177694d8f8ce5700fcf28d80e889a45589)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 16:22:03 +01:00
chrg6
518d55ed07 Draw lines as wide as the game sets them.
The game's line width was dropped at the call and every line drawn 1 pixel wide; the game draws two lines a frame at 4.5 pixels into a 144x144 mask. The width is now recorded with the draw state and set as dynamic state for draws of the line topology class, held to the device's range. Only pipelines of that class list the new dynamic state, so the others keep their entries in the pipeline caches and the driver's. line_width = 0 draws every line 1 pixel wide as before.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 09:31:04 -05:00
chrg6
d31c3a1036 Make the water corrections in the runtime, on by default, instead of loading replacement modules from a folder.
The coordinate pair and the absorption guard needed modules made on each PC by tools/projective_recipe.py and a folder named in NX2NAVI_SPV_OVERRIDE, so no player had them. runtime/spirv_water.c makes the recipe's edits on the translation in memory, in the recipe's order: on the NVIDIA profile its three modules are byte for byte the recipe's (RTX 5090, game 1.1.1). vk_pipeline.c keeps the edited modules in a small table, made at first use, and builds a pair from them only when both of its stages were made; a module of another shape is built as it is, with a log line. NX2NAVI_WATER_PROJECTIVE=0 turns the corrections off. tests/water_projective_check.py compares the runtime's edits with the recipe's on made-up modules.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 07:55:57 -05:00
chrg6
a13af7a8c2 Replace a water pair's vertex module before the vertex output union's additions, keep pairs with dual-source blending original, and log each replacement.
The paired modules were looked up by the hash of the module as built, so a vertex module that had outputs added for its union (which differs from one install to the next) or a fragment module changed for dual-source blending ended the run. The vertex replacement now stands for the translation itself and takes the added outputs as any module does; a pair with dual-source factors is built from its original modules in both stages.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 06:37:04 -05:00
ChurchOfIlias
37e8e6dbda Guard water absorption distance alongside paired coordinates
Add the exact-material nonnegative distance guard and the recorded NVIDIA translation profiles. Keep generated shaders and the private native patcher out of the repository. Document reported cross-vendor testing and the unresolved origin of invalid absorption distance.
2026-11-12 06:34:27 -05:00
ChurchOfIlias
d0d0112c3a Add an opt-in paired projective coordinate correction for water
Carry clip coordinates through perspective interpolation for the identified water shader pair. Keep generated modules private and document the observed results and remaining portability limits.
2026-11-12 06:34:27 -05:00
chrg6
6ccd1e224d Build the shared pipeline list before play on every start, and build every listed pipeline at install.
A new install met every pipeline in play: in Hyrule Field one 600-frame stretch of a first visit had 1,769 draws wait 461 ms in all for pipelines, the longest frame 88 ms. The boot now reads seed-pipeline-keys.bin, the release's list, as recorded pipelines after the install's own (keys_open_seed, NX2NAVI_PIPE_SEED, default 1): their pairs go into the vertex output unions and the prewarm builds them as it builds the install's. The preparation builds every listed pipeline (NX2NAVI_PREPARE_PIPELINES now defaults to all), with the unions of the same list, so the first boot of a new install finds them in the caches. The list's mesh pipelines are left out at the boot on a GPU without mesh shaders, as the preparation leaves them out. Nothing of the list goes into pipeline-keys.bin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 06:19:39 -05:00
chrg6
677a69378f Fill the fragment inputs a mesh program does not write, with the point sprite's coordinate for point meshes.
A mesh pipeline whose fragment stage reads a location the mesh program never writes now gets a fragment module where
that input is a private variable set to (0, 0, 0, 1) at entry, or to (s, t, 0, 1) of gl_PointCoord when the mesh
program draws points. The fragment shaders recorded with two of the game's point-drawing mesh programs read location
5, which those programs never write; Vulkan leaves such an input undefined.

Checks: tests/fs_fill_inputs_check.py passes; 237 recorded fragment modules rewritten offline pass spirv-val; a
60 s Kakariko run with the change shows the area as before with no Vulkan errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 19:38:18 -05:00
chrg6
ef64f341a0 Retry a graphics pipeline the driver refuses with VK_ERROR_UNKNOWN without its optimiser, and add VK_NOOPT=mesh.
AMD's Windows driver answers VK_ERROR_UNKNOWN for some pipelines its compiler gives up on, and the draws that need
such a pipeline were left out for good. create_graphics now tries once more with VK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BIT
(slower shaders for that pipeline instead of a missing draw) and logs the first such retry. NX2NAVI_VK_NOOPT gains a
value, mesh: only pipelines with task and mesh stages, the foliage's, are built without the optimiser, whose
optimisation is a suspect in device losses some minutes into a forest on AMD GPUs; with it a report can be narrowed
without slowing every pipeline.

Checked: the runtime compiles. A 60 s run with VK_NOOPT=mesh on an RTX GPU logs the mesh-only line, has no error and
the same GPU time as the default (4.09 ms a frame against 4.09), 134.5 fps. The retry cannot be triggered on this
GPU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 19:38:18 -05:00
chrg6
8c353cdf21 Build the boot's first recorded pipeline alone before its builder threads start.
The boot's prewarm and the shared list's preparation started a builder on every core at once, so the driver began
that many graphics pipelines at the same moment, its first ones of the run among them. NVIDIA's 621.14 was reported to
fault in a sizable share of starts that way; a driver's first graphics pipeline sets up its compiler. warm_builder
now takes a one-job mode, and both callers build one queued pipeline on the starting thread before they start the
others.

Checked on an RTX GPU with --probe-prewarm in a copy of a dev folder (its prewarm skip marker removed), twice each with
the baseline and this build, alternating: no fault in four runs. With the 20 s budget the first pass built 2,203 of
3,226 recorded pipelines before and 2,135 after; the second pass built all 3,226 in both. A fault rate of a few
percent cannot be shown in four runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 19:38:18 -05:00
chrg6
276c886143 Prepare every pipeline of the shared list where the driver has no graphics pipeline library.
NX2NAVI_PREPARE_PIPELINES=mesh, the default, builds only the list's mesh pipelines, because a vertex pipeline is
fast-linked from its stage libraries in a few milliseconds and its vertex module depends on the output union this
install has recorded. Neither holds where the library is off (vk_caps_gpl: AMD's Windows driver for RDNA 2 and older,
or gpl = 0): a vertex pipeline a draw misses is compiled whole while that draw is left out, and its vertex outputs come
from its own fragment shader, since independent layouts need the library, so the listed pipeline is the one the draw
asks for. There mesh now builds every record, and the summary line says so.

Checked: all runtime files compile. Not run: no install on this PC holds the shared list, and the case is a GPU without
the library (NX2NAVI_GPL=0 would show it on any GPU).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 19:38:18 -05:00
chrg6
15bd64f4b8 Hold new pipeline builds back during the save at exit on every driver, instead of copying the cache while NVIDIA's builds run.
The save at exit now copies the cache only once the builds already in the driver have finished, with new ones held back meanwhile, as it already did on other drivers. Intel's driver crashed copying a cache while builds ran into it, and when the window closes the game's save is written after the pipeline cache, so a crash in that copy would lose it. The case the save at exit missed, a window closed while the boot's prewarm workers built pipelines back to back, is covered by holding new builds back: a build from the file takes about half a millisecond. The prebuild's comment keeps why its default is two threads, with the measurements of one thread for the rest of the list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 18:12:47 -05:00
automata.
27f93a314a Save completed pipeline cache data during NVIDIA background builds 2026-11-11 18:10:38 -05:00
automata.
60ac5f5982 Prioritize recent compute pipelines before reducing background concurrency 2026-11-11 18:10:37 -05:00
chrg6
295922c224 Sync the device address table's shadows once per guest submission that binds the table, and take out the port of pull request #353.
The author of pull request #353 closed it on 2026-11-11 with the words that they give no permission to use its code. Its port (3134f524) goes: the mark of a pipeline that declares the table (VkPipeInfo.gpa, vk_pipeline.c pipe_add) and of the part that draws with it (vk_mem_gpa_use in vk_exec_draw.c bind_resources and mo_emit), the sync of the table's pools in every such part's prologue (vk_mem_gpa_sync), the submission with a callback in the replay's place as it stood (vk_sync_probe_submit), and --probe-vkmem's read through the table with its shader (test_gpa_read, runtime/shaders/gpa_read.spvasm).

In its place, after pull request #551, which was written without that code: a shadow pool goes into the table and onto a list of the table's shadows (vk_memory.c shadow_table_add); the record of a binding of the table notes the guest submission (vk_exec_draw.c ssbo_info, vk_mem_shadow_table_read); and vk_mem_shadow_sync brings the CPU's writes into every shadow on the list once for that guest submission, before the part is submitted. Once is enough, because the copy-ins read the guest submission's write-watch snapshots, which a second sync would find unchanged. Measured by #551's author on Linux (RTX 4090, Kokiri Forest): 113 and 76 frames over 25 ms in 75 s against 217 and 429 with the port, 43 against 41 fps, the picture complete in both.

What this line had built on the port stays, on the new sync: the test that passes over a pool with nothing written (shadow_pool_clean), the upload of the same pages into a table pool's VRAM mirror, the write-back of stores through the table (NX2NAVI_SHADOW_TABLE_STORES, table_wb_cb, now asked once per guest submission that binds the table) and the counts of NX2NAVI_SHADOW_CHECK. The probes that record their own commands (the lost-device probe, --probe-vkmem's table parts) call vk_sync_submit_own, written anew. vk_mem_shadow_storage stays: one line that returns the mode. Finalized pools in the table were already kept for vk_mem_shutdown here.

--probe-vkmem has a check of its own for the sync (test_table_sync): the shadow of a pool that is never bound keeps its old word through a CPU write and a guest submission that does not bind the table, and holds the new one after a guest submission that does. It passes with NX2NAVI_SHADOW=1 on Windows (RTX 5090), as do the store check with NX2NAVI_SHADOW_TABLE_STORES=1 and the probe without shadow storage; --probe-devlost ends as before. Not run: the game, with or without shadow storage, and anything on Linux.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 06:10:09 -05:00
chrg6
31e0a02901 Merge the mesh shader fallback: task and mesh programs drawn through compute passes and an ordinary draw where the card has no mesh shaders.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 06:01:39 -05:00
chrg6
33d605fb13 Merge the texture origin diagnostic and the fill of a second descriptor's image from its storage, and let the play file replace a pipeline cache that is over the limit.
A pipeline cache over the size limit was dropped at every start and, since the file is no longer written in play, never replaced: every start began empty. The play file is now folded in at the next start whatever its size when the file beside it is over the limit.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 05:58:56 -05:00
chrg6
a8c6e992ee Draw task and mesh programs through compute passes when the mesh fallback is on.
A mesh draw becomes its record (the workgroup count, copied on the GPU for an indirect draw), the task dispatch, a
scan that lists the mesh workgroups the tasks started, the mesh dispatch, and an instanced indirect draw with a
generated vertex stage and the program's fragment stage. The compute modules bind the resources of the task and mesh
stages, and their pipelines are made on a thread of their own while the program's draws are skipped.

The indirect mesh ring and the texture pool marks are released at a probe's shutdown.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 05:58:42 -05:00
chrg6
f9219180b9 Try a pipeline cache save again when it did not happen, up to six times at the save interval.
A save that found no memory for its copy, could not write its file or met a build that had just begun was not tried again until the next new pipeline was built, so a run that built none and then ended without its orderly exit lost what it had compiled since the last file. The autosave thread now repeats the try, and the same bound holds for the save owed after the play file was folded in, which before was tried at every interval for as long as it failed. tests/save_retry_check.py checks the rule.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 05:25:38 -05:00
chrg6
d073d80e70 Create FSR 3.1's pipelines through the pipeline caches kept across runs, so a context is not compiled again at every start the driver's own cache does not serve.
The library creates its pipelines itself and names no pipeline cache. It is now handed pipeline creates of the runtime's in the place of the device's two, and a create without a cache is asked of the pipeline cache first and compiled into the play cache otherwise. ffx_pipe_cache = 0 turns it off. --probe-ffx-cache makes one upscaling context without a window or the game and says where its pipelines came from, and tests/ffx_cache_check.py runs it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 05:22:56 -05:00
chrg6
2e28ba12df Merge the stutter fixes: a pipeline cache past 2 GB is read, the prepared shader list knows its seed, the pipeline cache is written only when it changed with a small file for what play adds, and a queued build a draw waits for goes first.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 04:59:05 -05:00
chrg6
70afa1584b Start a whole pipeline build a draw waits for ahead of the other queued builds.
A draw that waits for a pipeline with no fast link (a mesh pipeline, one without stage libraries, or one whose link failed) needs the whole build, and its job stood where it was queued: the draw queue is first in, first out, so on a burst of new pipelines the build a draw stood waiting for started after the fast links and builds of the draws before it, and the draw was skipped at the end of its wait all the same. graphics now hands the job to job_wanted: while no worker has taken it, it goes to the head of the draw queue and a worker is woken. A wait longer than 500 ms, which is no frame budget (a one-shot draw with oneshot_budget = 0), builds the job on the waiting thread at once when no worker is idle, as a stage library no worker took already is. A wait inside a frame's budget is never turned into a build on the replay thread: the budget is there so that thread is not held for a compile. The worker's build and bookkeeping moved into job_build for both to use; the log's statistics count both cases.

The other half of this item was there: the boot's prebuild asks the caches for a large compute shader's pipeline (compute_create, cached only) before it queues the background build (lb_compute), and with the play cache that question now reaches both caches.

Checked: it builds, --probe-cfg is ok, and --probe-prewarm builds the 3,212 recorded pipelines through the moved code as before. The queue jump and the build on the waiting thread need a draw that waits, which no probe makes: not run.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 04:57:21 -05:00
chrg6
719ee94e67 Write a pipeline cache only when its data grew, and give what a run compiles a cache file of its own.
pipeline-cache.bin was copied out of the driver and written whole at every save after any pipeline was created, also one the cache had held: tonight's logs show three saves of 358.1 MB in a run that added 0.3 MB, and 302 to 306 MB written in each of seven sessions that added about 0.5 MB each. The library warm in play built its 37,572 stage libraries into the same cache, which grew it by 1.6 GB: a save of 2026.7 MB in play, a file over pipeline_cache_max_mb, and the next start dropped it whole and compiled the pipelines it had held.

cache_save now asks a cache for the size of its data first and neither copies nor writes one whose size is what its file holds. What a run compiles goes into a second cache with its own file, pipeline-cache-play.bin (pipeline_cache_play, default 1; needs pipelineCreationCacheControl): pipe_create_in asks the first cache with VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED_BIT and builds into the play cache only when that says a compile is needed, and in play only the play file is saved. At a start the play file is read back and merged into the first cache in memory; once it reached pipeline_cache_play_mb (64) it is folded in for good: pipeline-cache.bin is written once, before play, and the play file removed. The preparation modes fold it at any size and build into the one file as before. The library warm's threads create with no cache of ours, so their libraries reach the driver's own cache alone.

Checked with --probe-prewarm on a copy of a dev folder's caches (3,212 recorded pipelines, 333 compute shaders), no game run: with a filled pipeline-cache.bin every pipeline came from it and neither file was written; with none and a cold driver cache the compiles went into pipeline-cache-play.bin (75.9 MB written, pipeline-cache.bin not created); the next start merged it and wrote pipeline-cache.bin once (191.1 MB) and removed it; kept below the fold size, a play file of 68.8 MB gave 1,445 pipelines at 0.3 ms from the first cache where 470 hit without the merge, and was not written again. Not checked in play: the saves during a session, the library warm without a cache, and drivers other than NVIDIA's.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 04:56:26 -05:00
chrg6
39fa0be78c Make every pipeline built ahead of its first use pass a memory gate that counts the builds already running.
Only the large compute shaders' background builder looked at the system's commit before it started a build, and it
saw nothing of what the builds beside it were about to take. The boot's recorded pipelines (up to 64 builders), the
pool's background builds during play, the boot's compute pipelines and stage libraries and the library warm asked
nobody.

Each of them now asks the gate before a build. A build that starts reserves an estimate of the driver compiler's
working memory until it ends (build_reserve_mb, 64 MB; 4096 MB for a large compute shader), and the gate admits a
build only while the commit headroom holds the part kept free (build_free_mb, 4096 MB), the reservations and its own.
A refused boot builder ends and leaves its pipelines to the pool; the pool leaves a background build queued and asks
again every two seconds; the stage library prebuild and the library warm wait up to 30 seconds and then end, and the
warm goes on at the next start. A pipeline a draw or a dispatch waits for is never refused. The preparation of the
pipeline list runs without the gate: no game runs beside it.

tests/build_gate_check.py checks the gate's arithmetic and, through --probe-buildgate, its answers with the machine's
own commit. What the builders do when refused is not exercised by a test.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 04:42:07 -05:00
chrg6
cba95f8d0a Record in shader-prepared.bin which seed it was prepared from, and prepare again at an update when the seed changed.
The prepared list named its shader variants and nothing about where they came from, so an update that keeps the prepared shaders (shader_carry) kept the old list whatever seed the new kit writes: variants only the new seed names, from a newer pipeline list or a change to which variants are prepared, were translated when play first drew them. The existing identities do not cover this: nx2navi_xlate.id and identities.txt name translators, which decide cache keys and what is carried over, not which variants are listed.

The list is now version 2 (runtime/prp_file.h): after the head come the sha256 of seed-programs.bin, the sha256 of seed-answers.bin and the translator identity, written by gfx_shader_prepare_seed; the three readers go through gfx_shader_prepared_read and still read a version 1 list. At an update step_shaders writes the seed first (prepare_shaders.py --no-run, about 4 s) and keeps the list only when its digests are that seed's (make_game.py seed_changed); another seed, or a list that does not say, is prepared again. Another translator alone still carries over: input ids do not depend on the translator, and preparing everything again for it is what carrying was made to avoid.

Checked with tests/prp_file_check.py (the C layout against make_game's reader), three launcher tests, and the exe's --prepare-shaders on a seed without records in a folder of its own: the digests it wrote are those of the seed files, one byte more in the seed reads as changed, and after preparing again it does not. A version 1 list of 42,142 variants and the same list as version 2 are both read whole by --probe-prewarm.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 04:36:33 -05:00
chrg6
871b06e2e7 Read a pipeline cache larger than 2 GB on Windows, and log every refused file with its size.
cache_load measured pipeline-cache.bin with ftell into a long, which is 32 bits on Windows: a file past 2 GB measured as -1, so the run started with an empty cache and the log said nothing, not even that the file was over pipeline_cache_max_mb. The read now lives in runtime/vk_cache_file.h: the length comes from the 64-bit tell, the read runs in 64 MB pieces, and a refusal returns its reason and the file's size, which cache_load logs. The Windows atomic write no longer cuts a block of 4 GB or more to a 32-bit count.

tests/pipeline_cache_file_check.py reads a sparse file just over 2 GB through the header. With --probe-prewarm and such a file carrying this GPU's header, the log reads "holds 2048 MB, over NX2NAVI_PIPELINE_CACHE_MAX_MB 512" under the default limit and "2048.0 MB read" with the limit off; the exe before this change logged nothing for the same file.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 04:29:31 -05:00