Commit Graph

320 Commits

Author SHA1 Message Date
epicmeme546
e10ef60fd5 Keep the driver's cache of stage libraries warm, link a missing pipeline instead of skipping its draw, and count the texture cache's waits in time.
The preparation builds every stage library once so the driver's disk cache holds it and a library a draw needs later takes under a millisecond. Turning dynamic vertex input off for AMD's RDNA 3 and newer driver changed the dynamic states every library lists, which gave each one another entry in the driver's cache, and nothing built them again: an update carried the prepared shaders over without the driver step. On an RX 7900 XT (26.10.1) all 42,142 prepared libraries were cold, 93 ms a vertex and 242 ms a fragment library on average. With no pipeline recorded, a walk through a village skipped 11,180 draws while their pipelines built and made 94 draws wait 5.5 s in all, four of them 1.0 to 2.0 s; a player's session log shows up to 114,834 draws skipped in 600 frames and 181 waits of 251 ms on average. Skipped ground and walls appear and disappear, and a skipped lighting pass is a black frame with the interface intact.

- The game checks the driver's cache at every start (vk_pipeline.c lw_start): a sample of the prepared fragment libraries is built, and when their median time says the cache is cold the whole prepared list is built into it again on idle-priority threads while the game runs. shader-warm.txt records a warm in progress, so the next start goes on with one a run ended, and stops the tries on a driver that does not keep the libraries. This also covers a driver update and a cache the driver emptied.
- An update that carries the prepared shaders over runs the game's --warm-shaders (tools/make_game.py warm_driver_cache): 11 s with the libraries there, about 8 minutes with all of them cold on that card.
- A draw whose pipeline is missing waits for its fast link from a budget of 6 ms a presented frame, 3 ms at most each (NX2NAVI_PIPE_LINK_MS), before it is skipped: with warm libraries the link is done within that.
- Full-screen draws wait from their 50 ms budget again instead of being skipped (NX2NAVI_PIPE_SKIP = any is the default; fullscreen stays selectable). With warm libraries the wait is a millisecond.

The same walk afterwards: libraries 0.3 and 0.7 ms on average, 1,605 draws skipped, 1,473 of them in the scene's first 600 frames and 8 to 80 in each 600 after, and 19 waits of 60 ms in all. First visits to four other saves (a mountain pass, the Lost Woods, Kokiri Forest, a river) waited 12 to 84 ms in all except the Lost Woods, where mesh shader pipelines, which have no stage libraries, and vertex libraries that need added outputs still compiled once: 23 waits, 2.1 s in all, the longest 776 ms.

The texture cache counted its waits in presents chosen at 60 a second: at 165 presents a second the 1800 that stood for thirty seconds were 11 s, so a released render target went sooner than meant and a texture out of view for 11 s was retired and showed its stand-in for some frames when it came back. The counts are now taken times the present rate over 60 (presents_for), and a texture is retired after two minutes without a sample instead of thirty seconds.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 01:36:29 -05:00
epicmeme546
8d91667c28 Keep only the game's select and back sounds in the launcher, and make none of its own.
The cursor's sound played at every move of the pointer and cut the other two short, as one sound plays at a time: it is gone, and moving the cursor is silent. A choice, a changed switch, Play and the end of a build play the game's decide sound; going back and a failure play its cancel sound. The chimes the launcher made from sine tones are removed too, so every sound it plays is decoded from the player's own game into data/cache/sounds and it is silent until the game has been read. No sound file is in the repository, the release or the setup.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 00:44:36 -05:00
epicmeme546
158ba315d5 Play the game's own cursor, select and back sounds in the launcher.
The launcher's sound effects were chimes made from sine tones. They are now the game's interface sounds, the ones its menus make for the cursor, a choice and going back: SE_Sys_Cursor00, SE_Sys_Do00 and SE_Sys_Cancel00, the files the in-game menu plays. The launcher decodes them from the player's built game or dump into data/cache/sounds (gamefiles.extract_sounds, a BWAV reader for PCM16 and DSP-ADPCM), beside the font it already reads that way, so no sound ships. The chimes stand in until the game has been read. The cursor's sound is a little under the others, as in the game's menu, and is left out for a quarter of a second after a choice, since one sound plays at a time and the pointer landing on a row of the next screen would cut the choice's sound short. Sounds stay off until the player picks a loudness.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 00:21:31 -05:00
epicmeme546
41eb0e635b Apply the camera's stick sensitivity and the speaker layout during a run.
NX2NAVI_STICK_SENS was served to the game as a multiplied camera pack at start only. A change during the run is now written into the game's memory: the StickSensitivity array as the game holds it, at the factor in effect, is looked for in the guest's heap and every copy is overwritten at the new factor. The game reads the array each time it steps the camera, so the speed follows at once. The copies found are remembered, so a later step of the menu's slider takes no search, and a search runs again a moment after the game opens the pack anew. On Ocarina 1.1.1 the first change of a run found 4 copies in about 2 s on its own thread, and the same stick push for a third of a second turned the camera about 30 degrees at 1, 90 at 3, 10 at 0.3 and 30 again at 1, all in one run.

NX2NAVI_AUDIO_CHANNELS was decided once. A change that gives another channel count now signals the device change events the game holds, as a console does when its output moves: the game asks GetActiveChannelCount again and mixes for the new count, and the mixer makes its SDL stream again with that count at its next render. On Ocarina 1.1.1 the centre input of the device sink went from 0.0006 rms to 0.013 within the 5 s after stereo became 5.1 and back to 0.0006 after stereo was set again, with no underrun. NX2NAVI_TRACE=audio logs the six inputs' levels with its level line.

Both rows lose their "after restart" note. The graphics card, widescreen, the resolution, the game's language, the button prompts and the game's native HDR still take a restart: the game or the Vulkan device reads them once while starting.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 00:21:31 -05:00
epicmeme546
44d4095094 Add Customize Launcher: colour schemes, a background picture of the player's own, animations and sound effects.
The launcher's home screen has a Customize Launcher row. Color Scheme steps through six schemes (Kokiri Forest, the colours so far, Death Mountain, Zora's Domain, Gerudo Valley, Temple of Time and Shadow Temple): each is the first scheme with its woods, accents, backdrop and glow turned to other hues, so all keep the same light and dark steps and the same text. Background takes a picture file, cuts it to the window's shape and keeps a fitted copy in data/background.npy; Background Dimming darkens it and the panels turn a little see-through over it. Animations (on by default) make the cursor glide between rows, a new screen slide in, sparks drift over the backdrop and Navi bob in the title band; they move by the time that passed and stand still while the window is hidden. Sounds are short chimes made from sine tones, off until the player picks a loudness.

Redrawing in another scheme takes about a third of a second: the blurred masks and the noise, which no colour changes, are kept, and compositing onto a solid picture is a plain mix.

The selftest's screen capture passed window handles as plain ints, which failed for a handle above 2^31 and left some screens without their picture; its calls are typed now.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 23:27:02 -05:00
epicmeme546
9ae7764ab5 Change a setting with the mouse only on the two framed arrows beside its value, and switch HDR without a restart.
In the settings menu a click anywhere on a row stepped its setting to the next value, and the left arrow stood left of the value's text, so it moved whenever a value of another width came up: clicking the left arrow through a list landed on the row at the second click and stepped the other way. Each stepper now has its arrows in white frames at fixed places, the left one the width of the row's widest value from the right one, and the mouse changes the value on those frames only; a click on the rest of the row selects it. On and off rows have the same arrows. An arrow at the end of its values is drawn faint and does nothing, and a button held on an arrow repeats as a held key does. A, Enter and left and right work as before.

NX2NAVI_MENU_TEST_CLICK clicks rows by their setting's name during a menu test run. On an RX 7900 XT at 3232x1352 the arrows stepped the performance overlay one value a click and stopped at both ends, and clicks on the body of four rows changed nothing.

HDR between off and the tone mapper now follows the menu. The HDR path is set up at every start (vk_hdr_armed): the swapchain colour space and metadata extensions, the tonemap splice and the settings in the support buffer, with the spliced tonemap returning the game's own SDR result while the swapchain is SDR. A change of the setting makes the swapchain again in the other format. On the same card a run that started with HDR off went to an HDR10 swapchain and back, and one that started with HDR on went to SDR and back, with the picture right in each state. The game's native HDR still waits for a restart, as the game asks about the display once, and the row says so. NX2NAVI_HDR_LIVE = 0 leaves the HDR path out of a run that starts with HDR off.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 23:27:02 -05:00
chrg6
78b3bc5a80 Clear a texture pool slot's left-this-frame note when it is filled, so a skipped slot only ever holds the stand-ins.
pool_refill notes a slot it could not fill (pool_defer), and once the frame's creation time is spent it skips slots
noted in that frame. A fill did not clear the note. So a slot left and then filled in the same frame was skipped later
in that frame with the filled texture's view still in the set, also after that texture was destroyed: a released
texture dropped past the size limit, or one evicted. A shader sampling the slot would then read freed memory. A left
slot holds the stand-ins (its lookup resolved to nothing), so with the note cleared on a fill, a skipped slot can only
hold those.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 8d04e17a954e640a9ff06baedc3e7614dc978798)
2026-11-09 21:34:53 -05:00
chrg6
49dd6639f1 Count a texture pool take only when its sync left no slot behind, so a retired texture is not freed while a set still holds its view.
Once the frame's creation time is spent, pool_refill skips a slot it left earlier in the frame, and pool_sync_entries
may return before the walk. Such a slot keeps its old view in the current set with nothing in the change log, so the
takes after it do not carry the change into the other sets. vk_pool_takes counted those takes all the same, and at
about six submissions a frame, more than VK_POOL_SETS of them could pass within one frame while a retired texture's slot
was skipped every time. retire_sweep would then free the image while every set still named it. A take now counts only
when its sync left no slot behind (PL.retry clear): after it, every slot change made before it is in that set and in
the log, and the takes after it bring it into the others. A sealed second take inside one submission still goes
uncounted, which only delays the free.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 5cd431f792e00ca3b9a03657cd4aab64fc38dae7)
2026-11-09 21:34:53 -05:00
chrg6
a87f6e1bd6 Merge the test line's texture retirement fix into the combined branch.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 21:27:08 -05:00
chrg6
f63b6796ee Decode Ocarina 1.1.1's loading streams on the host decoder, behind the same zstd_native = 1 switch.
The loading threads feed each chunk they read to the game's stream decoder, which is where most of the
decompression time goes. That call now reaches a host stream that writes straight into the game's output
buffer and keeps the game's state fields and results. A stream is taken from its first chunk or left to
the game; zstd_check = 1 runs both and compares the outputs. --probe-zstd checks the host streams on
frames it makes itself.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 21:21:48 -05:00
chrg6
524cd23266 Merge the save export and import for bug reports into the test line.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 21:11:18 -05:00
chrg6
22103a57bf Merge main into the test line: the pull requests taken there and two render target fixes.
Main brought the fixes of eleven pull requests (the texture pool set sealed when an early replay part is submitted, every location reflected in interface arrays, RGB9E5 texels recognised for integer storage views, the upscaler a frame started with kept, GPU pass timestamps without vendor breadcrumbs, the Linux build, window and NGX fixes), render targets released when drawn over at another address, the vertex layout kept in the pipeline on newer AMD drivers, and the upscaler choice read once per evaluation.

Four hunks conflicted and take both sides: pool_sync returns for a sealed set before the per-frame creation budget's clock (vk_exec_draw.c); make_module keeps the test line's early_ok argument and main's module hash (vk_pipeline.c); a texture taken back counts as sampled again and records how long it was away, and the texture line logs the disuse counters and the overwritten render targets (vk_texture.c).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 21:10:23 -05:00
chrg6
f1127f8be9 Retire textures evicted by disuse before freeing them, take them back on any use, and add tex_disuse = 2 to tell the freeing from the marks.
0.6.1.14 faulted the GPU (a read of an invalid address in a pass) about 30 seconds after its last disuse evictions, and
no path by which an evicted image is still read was found in the code. An evicted texture is now first retired: the
pool refresh gives its slots the stand-in while the image stays whole. It is destroyed only once the texture pool has
taken every set of its ring since and the GPU has finished the submissions recorded until then, so no pool set can
still be bound with its view. A mark on one of its slots or any lookup outside the pool refresh takes it back as it is,
and the perf line counts those, which shows whether the marks miss a real use. tex_disuse = 2 retires and never frees,
so the next fault can be put on the freeing or on the marks and stand-ins.

The registration shortcut also read a slot's cached entry without checking that the resolution was current, so it could
read the record of a destroyed texture; it now checks the slot's generation first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 4c5b2ae2e030ab00a910d8f071c030e610fd78e1)
2026-11-09 21:09:33 -05:00
chrg6
57cdfa1520 Add wb_probe = 1, which finds out who reads what the GPU wrote back into guest memory.
Per-pass GPU timestamps put the write-back of GPU-written storage buffers at 3.6 to 5.3 ms of GPU time a frame in play, against 1.8 to 2.5 ms for all of the game's own passes and dispatches: about 67 MiB a frame goes from the mirrors back into guest memory, from the same eleven pools all session, in case the CPU or a host path reads it. Nothing said whether anything does.

With wb_probe = 1 the 2 MiB regions that lie wholly inside a range just written back are made inaccessible. The first read or write of such a region faults; the handler gives the region back the protection it had (read-only when frozen, else read-write), notes the address, the instruction and the thread, and the access runs again. The bytes are in guest memory all along, so nothing changes for the game. A log block every perf window gives the regions armed, the probes a read or a write ended, and the first accesses by instruction. Every thaw ends the probe of its region first, which covers file reads into guest memory and mapping changes; a file write from guest memory thaws its buffer while the probe is on; a probed region is not frozen anew.

The platform layer gets a hook for every read or write access fault beside the write-fault hook; it is installed only with the setting. --probe-gfx-abi checks that nothing is armed without it and, run with NX2NAVI_WB_PROBE=1, that reads and writes end a probe and run again, that a frozen region thaws after it and that mem_thaw ends and holds.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 21:01:47 -05:00
tangerineturtle1
6dcb11f93c Read the upscaler choice once in the DLAA evaluation, so a change from the present's thread cannot split it between FSR 3.1 and DLSS.
vk_dlaa_frame runs with the game's present and vk_dlaa_pass with the replay, which can be another thread. evaluate read
the choice three times; when the menu's change from FSR 3.1 to a DLSS mode landed between two of the reads, the rest of
the function took DLSS's path with NGX never brought up.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 20:59:48 -05:00
chrg6
c32b9c40a8 Send Ocarina 1.1.1's one-shot zstd decodes to the host decoder, switched on by zstd_native = 1 until a check run.
A profile of a play session showed the game's own zstd decoder, lifted from ARM64, taking 16 to 19% of all running CPU time on the loading threads: the patch set that sends decodes to the host's libzstd had sites for 1.4.3 only. recomp/patches-v111.txt now has the three calls of ZSTD_decompressMultiFrame (main+0xeb380) of this build, checked against its main by the generator; a lift with them changes three chunks. ov_zstd.c takes the guest function's address by game version.

The sites have not been through a check run on this build, and 1.4.3 has a caller that relies on the game decoder's own result, so on 1.1.1 the host function passes every call on to the game's decoder unless zstd_native = 1 is set or zstd_check = 1 runs both and logs what differs. The loading threads' stream feed (main+0xf5c80), which decodes most of the data, is not replaced yet.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 20:52:48 -05:00
Chris Toph
47d9a2d502 Seal the texture pool set when an early replay part is submitted.
A replay part handed to the queue early kept reading the bindless pool set while later parts of the same submission wrote new entries into it. That updates descriptors a pending command buffer reads, and on RADV a texture whose memory was mapped after the part was submitted is absent from its page tables. This caused a gfxhub read fault in the 128x128 pass of the first world frame in Kokiri Forest.

Seal the set when a part goes, take another ring set at the next sync that has entries to write, and keep recording the same part while every other set belongs to the submission. Document the descriptor lifetime contract in maxwell.md 4.9.

Fixes #257.
2026-11-09 20:52:40 -05:00
Michael Schienbein
a85f5c15d3 Report where a Wayland fullscreen window ended up once the compositor placed it.
SDL_SyncWindow returns before the compositor moves the fullscreen window to the chosen output, so the check right after it logged the output the window opened on, though the game then ran on the chosen monitor. On Wayland the report now waits until SDL sees the window on the chosen monitor, which it learns from a display, scale or mode change (Hyprland 0.56 sends only the latter two).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 20:51:09 -05:00
Michael Schienbein
a103673341 Make the SDL window honour monitor, --monitor and --list-monitors.
On Linux the window ignored NX2NAVI_MONITOR (the log said it had no effect) and --list-monitors only printed that monitor selection is Win32-only. The SDL window now picks a monitor with the same rules as the Win32 one: auto (the system places it), primary, a 1-based index in listing order, or part of the monitor's name, ignoring case, the first match in list order winning. The monitors are SDL's displays; --list-monitors prints their index, name (the make and model, with the connector on Wayland and X11), pixel size and position, primary flag and display scale, then what NX2NAVI_MONITOR would pick, and the log says the same when the window opens.

The window is created centred on the chosen monitor and centred again at its final size, and that position also names the display the fullscreen setting covers, which is how a Wayland window, which a client cannot move, opens fullscreen on the chosen output. On Wayland a windowed window opens where the compositor puts it; the log says so once. With a chosen monitor, the size widescreen.c reads for fullscreen before the window exists is that monitor's.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 20:51:09 -05:00
Michael Schienbein
4d41188d8b Size the lift's workers by free memory, hold off sleep and key the compile by the CPU on Linux.
On Linux the three host probes of the build did nothing. available_ram returned None, so the lift started one worker per CPU whatever the memory and could run the machine out of it; keep_awake did nothing, so the machine could sleep in the middle of a build of 10 to 25 minutes; cpu_tag returned 'any', so the compile step's key ignored the processor and a game folder moved to a CPU without AVX2 kept its x86-64-v3 objects.

available_ram now reads MemAvailable from /proc/meminfo and, when the process runs under a cgroup v2 memory limit (its own or a parent's), caps it by memory.max less memory.current. keep_awake starts systemd-inhibit with a lock on sleep (not idle: the screen may still blank and lock, as with ES_SYSTEM_REQUIRED on Windows) around a cat that reads a pipe this process holds open, so the lock ends with the process even when it is killed; without systemd-inhibit it does nothing. cpu_tag reads the flags of /proc/cpuinfo and returns the digits the Windows branch returns for SSE4.2, AVX2 and AVX-512F, counting a feature only when every logical processor has it.

The compile key now holds the real tag instead of 'any', so an existing Linux build compiles once more. The compile --jobs defaults are unchanged. tests/host_probes_test.py covers the three probes over fake /proc and cgroup files and a fake systemd-inhibit.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 20:51:08 -05:00
Chris Toph
6dd69eadbf Build the runtime library and refresh installed tools correctly on Linux.
Use the native toolchain and archiver for runtime libraries, accept VMA_INCLUDE, and report missing Linux tools before compiling. Read the host CPU through compiler wrappers without changing the environment used by real compiles.

Compare copied tools by content rather than normalized timestamps, then replace them atomically so read-only POSIX installs can update without changing executable modes. Cover equal-metadata copies and translator refresh during shader preparation, and correct the patch-application comment.
2026-11-09 20:51:07 -05:00
Michael Schienbein
d8ea169fcb Fix CPU-backed texture fixtures in the Vulkan memory probe 2026-11-09 20:51:07 -05:00
Michael Schienbein
033bf85625 Release Linux NGX before destroying the Vulkan device 2026-11-09 20:51:06 -05:00
Michael Schienbein
4cb1a56e6c Give --boot and --seconds the call sites of game 1.1.1.
On 1.1.1 the SDK's nn::oe::InitializeApplet calls SetFocusHandlingMode itself, before DT_INIT, so the first call reached the boot checkpoint from sdk+0x598840 and --boot and --seconds stopped with a FATAL before the game started. Only a call from main is the checkpoint now. 1.1.1 also gets its own return addresses (nnMain's SetFocusHandlingMode at main+0x1202d44, the first nvnBootstrapLoader at main+0x346bf8c, read from x30 in the two host functions during a normal start) and its init_array counts, so a 1.1.1 run is no longer checked against the 1.2.1 values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 20:51:05 -05:00
Chris Toph
b88439e2e0 Reflect every location in vertex and fragment interface arrays. 2026-11-09 20:51:04 -05:00
peigongdsd
3cdb3b810b Enable GPU pass timestamp labels without vendor breadcrumbs 2026-11-09 20:51:03 -05:00
Leyukaka
4d330dd76e Keep the upscaler a frame started with, so choosing a DLSS mode while FSR 3.1 runs no longer calls NGX before it is loaded.
vk_dlaa_frame decides at each present whether DLAA runs, with FSR 3.1 when NX2NAVI_TAA is fsr3 and NGX is not up. evaluate read NX2NAVI_TAA again. Choosing a DLSS mode in the menu sets it to dlss, which only takes effect after a restart when the run started with FSR 3.1; when that happened between the two reads, evaluate took DLSS's path and called N.alloc_params, which is null without NGX (issue #325). The choice is now kept from vk_dlaa_frame (fsr_frame), and evaluate returns when neither upscaler can run, until the next present turns DLAA off.
2026-11-09 20:51:03 -05:00
ajcohn716
e3e2d999c0 Recognize RGB9E5 texels when selecting integer storage views 2026-11-09 20:51:02 -05:00
chrg6
14969fc671 Keep DLSS Frame Generation's resources while it is suspended, which stops the 100 to 150 ms frames after menus and cutscenes.
Generation is switched off for every menu, cutscene and frame without depth and motion, and on again after it. Off freed the generator's resources and on allocated them again inside the present, which holds the queue lock for its whole flip, so the submitting thread waited there: four of five slow frames profiled in a session with frame generation on had the log line "frame generation on" right before them and nothing else blocked. nx2navi_dlssg.dll now sets Streamline's retain flag, so off only suspends; whether frame generation runs is fixed for the run, and Streamline frees the resources at shutdown. The DLL has to be rebuilt (tools/build_dlssg.py).

The wait for the queue lock in vk_sync.c queue_submit is counted as a present wait, so a frame held up by a slow present no longer shows in the slow-frame line with every blocked time at zero.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit 94a0da9f7694cfe4d65777f9efe8eff837ba2bc8)
2026-11-09 20:48:10 -05:00
chrg6
bab143e1a2 Wake the recorder once per sleep, stop its idle spinning, and skip per-draw work in the replay that repeats.
From the replay review of a profiled session:
- vk_rec.c: the producer set the wake event for every packet until the recorder had cleared its sleeping flag, a system call per packet on the submit thread for as long as the recorder took to wake; it now takes the flag back itself and sets the event once. A sleep that ended with nothing new was followed by another 4096-pause spin, every 2 ms for as long as the game was idle or paused; the recorder now sleeps again at once. head, tail and the producer's words are on cache lines of their own, and the producer reads tail only when the ring looks full by the value it last saw.
- vk_exec_draw.c c0_info: a vertex or fragment bank depends on the storage slots and the base instance alone, so those are compared directly instead of clearing, filling and comparing a 1.8 KiB record on every draw.
- vk_exec_draw.c bind_vertex_buffers: a draw that changes no vertex stream returns before the 17 streams are rebuilt and compared.
- vk_exec_draw.c bind_resources: the texture lock is taken at the first texture descriptor that has to be resolved, not for every texture binding whose descriptors are all kept.
- vk_usnap.c vk_usnap_find: the region the last lookup hit is tried before the binary search.
None of this changes what is recorded.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit 8eac567e2a9a353f894f3c66e3fc20fd2c0d295d)
2026-11-09 20:48:10 -05:00
chrg6
745d48bc8c Free released recording copies outside their lock, skip the yield's lock and scans when nobody waits for a core, and wait instead of spinning before a paced frame.
Found by profiling a play session and reading the kernel and the command recording:
- nvn_api_cmd.c: a released recording copy had its pins uncounted (a walk over every op) and was freed while copy_lock was held exclusively; the main-loop thread waited for that lock 2.8% of its time. The copies are now chained under the lock and freed after it. The walks over a recording's records test a mask of the five op kinds that name pins before calling op_pins.
- kernel.c: SleepThread(0) took the kernel lock and computed the caller's inherited priority, a scan over every blocked thread, also when no thread waited for a core, which is nearly always. It now yields at once then. A wait without a timeout no longer reads the clock.
- nvn_api_obj.c sleep_until: the wait for a paced frame's slot slept coarsely and left the last 1 to 2 ms to a yield loop; the high-resolution wait now takes all but its own margin.
- input.c: the keyboard scan (a system call per key) is reused for reads within a millisecond.
- nvn_api_core.c: the read-lock observer's commit-point address is 1.2.1's and is now only compared there.
- kernel_probe.c: the probe expected three virtual cores and wait type 3 to be refused; the kernel has six cores and implements the 64-bit wait. It now checks both as they are, counts a core number past the cores as a clash without unbalancing core 0, and --probe-kernel passes again.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit a3dbd63878d7d6de096ceb4c25988192b4bce738)
2026-11-09 20:48:10 -05:00
chrg6
0703cfa944 Mark the texture pool in SPIR-V 1.4 to 1.6 shaders too, and name in the log each shader the marks cannot take.
The translator's mesh shaders are SPIR-V 1.4, and the marks took only 1.3, so the first mesh shader that indexed the
pool turned eviction by disuse off for the whole run. From 1.4 on an entry point lists every global variable it uses,
so each entry point now gets the marks variable too. Over the 44,255 shaders of a test install's cache, 4,728 are
marked, none is refused, and every marked one passes spirv-val for Vulkan 1.3; 11 of them are 1.4 mesh shaders. The
log names up to eight refused modules by stage, SPIR-V version, size, hash and reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit af50808a1cb56f450f99938c9d420e1f5deb333b)
2026-11-09 20:45:04 -05:00
chrg6
06f4aeb694 Time what a submission spends on the GPU outside the game's passes and dispatches, with gpu_passes = 1.
A session with per-pass GPU timestamps showed the game's own render passes and compute dispatches at 1.6 to 2.3 ms of GPU time a frame while the perf line's GPU time was 7.7 to 13.5 ms: most of the frame's GPU time is in commands the replay records between the marks, and nothing said which.

The timed frames now also get a timestamp at the start and the end of each part of a submission, and every stretch between two marks (or a mark and the part's edge) is a gap. The replay names the kind of commands it records in a gap: buffer and texture copies, fills and clears, the temporal upscaler, write-back merges and copies, depth twin copies, the texture pool marks, the mesh draw counts and block-compressed target copies, each with a size where it has one. The report adds the gaps' time per frame summed by the kinds noted in them (a gap nobody noted holds barriers and state only), the parts' own time for comparison with the perf line, and each kind's count and size per frame. Nothing runs outside the timed frames.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 18:49:12 -05:00
epicmeme546
76c3661aff Keep the vertex layout in the pipeline on AMD's RDNA 3 and newer driver, and save the pipeline cache while a large compute shader builds.
AMD's driver for RDNA 3 and newer compiles a pipeline that has dynamic vertex input again at every boot: neither pipeline-cache.bin nor the driver's own disk cache gives it back. On an RX 7900 XT (26.10.1) a player's key file of 3,979 records used the whole 20 s boot budget at every boot. Dynamic vertex input is now off by default on that driver (vk_caps.amd_llpc); with the vertex layout in the pipeline the third boot of that file built 4,273 pipelines in 0.9 s, all from the file. NX2NAVI_VK_DYN_VERTEX=1 turns it back on. The RDNA 1 and 2 driver branch is not measured and keeps it.

The background builder of large compute shaders now compiles into a pipeline cache of its own and merges it into the main one when the build is done. A create into the main cache holds the cache's lock for its whole length and the cache is only saved while nobody holds it, so a build of minutes kept pipeline-cache.bin from being written at all: on the same card one compute shader of 4,874,502 SPIR-V words was still compiling 590 s after the boot, and nothing built in that session reached the file.

NX2NAVI_PIPE_FEEDBACK=2 also logs every graphics pipeline build with hashes of its modules and its fixed state, which is how the recompiles were found.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-09 18:17:12 -05:00
epicmeme546
b303c2aa90 Release render targets that were drawn over at another address, and keep released ones the game still samples.
Ocarina lays its per-frame render targets out anew every frame in one GPU-only pool. When an early target changes its size, every target after it moves, and each move made a new entry with an image of its own while the old entry stayed live for as long as a pool descriptor named it. After 6,000 frames of walking through Kakariko Village 1,043 such entries held 6.1 GiB of device memory against 233 MiB in the targets still drawn. On a driver that charges device memory to the system commit this kept the commit low, and the idle textures of the area were evicted and uploaded again while walking.

A GPU write in the GPU-only window now marks the entries at other addresses whose storage it runs over as overwritten and releases them. Their images go to the next texture made with the same parameters, or away after 600 presents. On the same walk the process's private memory ends at 23.2 GB instead of 29.3 GB and device memory in use at 8.5 GiB.

A released render target whose content exists only in its image is no longer dropped after 120 presents under commit pressure: the game went on sampling such targets and took them back empty, which drew the scene black for one frame every two to three seconds. Such an entry now waits 1,800 presents, or eight times the longest it stayed released before the game took it back, and none goes while lookups still reach it.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-09 17:12:21 -05:00
chrg6
b1a9e9683c Name the exe's functions in the profile report for a GNU ld map too, take nm output for static functions, and list where running threads are.
The flat list resolved an exe address without the image base a GNU ld map's addresses include, so with the launcher's build-cache/ocarina.map every sample inside the exe came out as "?"; it now resolves as the call chains do. In the map's place the report takes the output of nm -n of the exe, which names the static functions a map leaves out (their time landed on the global function before them). A new table gives the self time of the running call chains: the innermost exe function of every chain that does not end in a wait.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 15:27:02 -05:00
chrg6
8e656b85a4 Hold less video memory in empty space: smaller VMA blocks, spare blocks only where textures grow, a smaller ASTC scratch, and DLSS features freed once unused, with a VRAM breakdown in the perf log.
VMA's blocks on large heaps are 64 MiB instead of 256 (vma_block_mb): one session held 2.1 GiB free inside the
device blocks beside 8.2 GiB of allocations after a few area changes, and the driver counts a block whole until it
empties. The headroom thread keeps its spare block only for the device-local types textures and mirrors grow in; the
spare blocks it made for the host types and the PCIe BAR type at boot stayed empty for the whole run. The ASTC
decoder's scratch is 32 MiB, so it stays inside a block, and an area load that makes one per upload batch holds half
as much at once. A DLSS feature no frame used for 3600 evaluations is freed with its work images, and a model change
now frees all five sizes.

The perf log gets a "vram by use" line (textures, released textures, mirrors, physical pools, ASTC scratch, headroom
spare, free space inside the blocks, memory outside VMA and DLSS's own share), the free ranges per memory type, the
addresses whose duplicate texture entries hold the most, and a line when memory outside VMA moves by 256 MiB.
astc_bc_probe in nx2navi.ini turns on the ASTC opacity probe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 99cfc2fa14ae208b7b23e057ff9b87986cd6a5e0)
2026-11-09 14:56:17 -05:00
chrg6
860bc9cf0b Spread the texture pool's entry creation over frames by a per-frame time budget, and stop looking up the slots it left.
The pool refresh runs before every draw that binds the pools, so its count budget of 16 creations held per draw, not
per submission. An area load in the 0.6.1.12 test made 4,925 entries (1.7 GB of uploads) in one 430 ms frame, and
2,018 in a 195 ms one. While slots were left over, every draw walked the whole table and looked each of them up again,
about 100 ms of the 430 ms frame in key hashing alone.

Once a frame's refreshes have taken pool_create_ms (default 6 ms; 0: no limit), they create nothing more until the next
frame, and skip the slots left earlier in the frame. The first refresh of each frame keeps its count budget. The slots
left keep their stand-in for a few frames during a large load.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 14:51:29 -05:00
chrg6
853449fd8d Merge main into the test line: the setup game-file check and the log of game files that keep failing to load.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 14:42:00 -05:00
chrg6
52cf976857 Merge the 0.6.1 fixes from the 0.6.1.x test builds.
This brings in the test branch up to the shader reads, not the texture eviction by disuse still in progress there:
- black frames: a released or taken-back texture keeps its content, and a new texture over a live one of another size gets the region both cover;
- the menu font atlas uploaded through the runtime's checked upload batch;
- the game held to 240 frames a second, which stops the quick menu's crash;
- the FSR 3.1 contexts evicted instead of turning FSR off after a few render scales;
- AMD frame generation kept off while NVIDIA's holds the device;
- less device memory held that only costs the system's commit;
- shaders read from the disk cache off the render thread past a small budget;
- the exit code 70 text;
- pull requests #260, #261, #270 and #271.
The tool lists in make_kit.py and make_launcher.py and the setting names in cfg.c take both sides: romfs_check.py and pack_shader_cache.py, FOG_CULL, SCENE_SHARE and GPA_TWIN_MAX_KB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 14:39:51 -05:00
chrg6
dc9f164afc Evict textures no shader sampled for 30 seconds while video memory or the system's commit runs low.
The texture pool refresh looks up every registered slot, so the textures of every area the game keeps loaded never
counted as idle and their memory only grew. Marked shaders now store into a buffer with one word per pool slot. Every
four submissions that recorded work, the replay copies it out and clears it, and the texture side notes when each
texture was last sampled. Under VRAM pressure or low commit, once idle textures are gone, live textures that come back
intact and were not sampled for 1800 presents go too. Their keys are held, so the pool refresh keeps the stand-in until
a shader samples that slot again, and the next refresh makes the texture again.

Fragment shaders that neither discard nor write depth, sample mask, stencil reference or memory keep early depth tests
(EarlyFragmentTests), so the marks do not take away early depth rejection. Alpha to coverage leaves the dynamic state
while marks are on, so each pipeline knows whether early tests are safe. Disuse eviction waits until every module that
indexes the pool was marked and two reads arrived. NX2NAVI_TEX_DISUSE=0 (tex_disuse = 0) turns it off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 1a4c896150a748f9275c3024e96af3055db94af4)
2026-11-09 14:39:07 -05:00
chrg6
d016a6dc86 Mark the texture pool slots that shaders index, for eviction by disuse (work in progress: not built; the runtime side follows).
vk_pipeline.c spirv_pool_mark gives each module that indexes the texture pool a storage buffer at set 2, binding
VK_POOL_MARK_BINDING, and a store of 1 into the slot after every access chain into a pool texture array; modules that
cannot be marked are counted in vk_pool_marks_missed. NX2NAVI_SPV_MARK_TEST=<list> runs the rewrite over files for
spirv-val under --selftest. Still to do: the marks buffer and its read-back (vk_exec_draw.c), the pool lookup context,
the holds and the eviction by disuse (vk_texture.c).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 7ae86070f166d935a9949e95dae7a95e2cf3272b)
2026-11-09 14:39:07 -05:00
chrg6
87f51f94a9 Read shaders from the disk cache off the render thread past a small budget, and pack the cache on carried updates.
A profiled traversal session had 9 of its 16 worst hitches (100 to 500 ms) inside the render thread's shader lookup,
reading <key>.res.json and <key>.spv of shaders met for the first time: opening a loose file costs about 3 ms the first
time on Windows. get_async_one now reads in place while those reads stay within 1/8 of the wall time (2 ms at once);
past that three loader threads read them and the draws wait as for a translation. The perf line counts both.

The launcher's updates carry the shaders over and never packed the cache, which held 44,000 loose translations in the
test install. An update now packs it once it holds 1,000 loose ones, and the pack tool reads the loose files on 16
threads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 14:34:45 -05:00
epicmeme546
44cecb5500 Log a game file the game keeps opening and cannot load.
A file its loader cannot use is opened again and again until the game does without it, and what needs it stays out of the world. The log now names such a file after 24 opens in a row, which tells a player's report of missing geometry apart from a rendering bug. It is one count and one timestamp per file, touched only when a file is opened.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 14:16:19 -05:00
epicmeme546
91003c3227 Check every player's game files during setup, and stand in for damaged textures.
A dump can hold files whose first pages are zeros, with a container checksum that passes because the damage was packed along with them. The game then opens such a file over and over and gives up on whatever needs it: one damaged normal map left the Great Fairy Fountain without walls.

The game build's new last step, tools/romfs_check.py, decompresses every file of the game data once, in worker threads and never while the game runs, and writes romfs-check.json beside the game. A damaged texture gets a stand-in, an undamaged texture of the same kind renamed to it, in an ordinary mod folder the Mods menu can turn off, so the game can load what needs it. Other damaged files are reported, since only a clean dump restores them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 14:16:19 -05:00
chrg6
fbb6783664 Hold less device memory that only costs the system's commit: a smaller released-texture cache, no spare blocks while commit is low, large images in memory of their own.
The driver charges device memory to the system's commit. A 32 GB PC lost the device at an area load with the
system at 61 of 65.5 GB committed, the game at 28 GB: 12.35 GB of VRAM in use, of it up to 1.3 GB in released
textures kept for reuse, 0.5 to 1.8 GB free inside VMA's blocks, and a spare 256 MiB block per memory type.
- Released textures keep a thirty-second of the installed memory (512 MiB to 2 GiB) instead of a sixteenth (1 to 4
  GiB); only those released STALE_FRAMES presents ago go, so the transient ones the game takes back stay.
- The headroom thread keeps no spare block while the system's commit is low (vk_texture_commit_low), as it already
  keeps none under VRAM pressure.
- A texture image of 32 MiB or more gets memory of its own (DEDICATED_MIN), so freeing it gives the memory back at
  once instead of leaving a hole in a shared block that the driver counts in full until the block empties.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 14:00:12 -05:00
Michael Schienbein
7a15c063ca Mark the pre-commit hook and tools/fetch_refs.sh executable, so they run on Linux.
Every file was committed with mode 100644. Git on Linux and macOS ignores a hook without the executable bit ("The '.githooks/pre-commit' hook was ignored because it's not set as executable"), so the content check that CONTRIBUTING.md turns on with core.hooksPath never ran before a commit there. tools/fetch_refs.sh, which AGENTS.md runs as a command, stopped with "Permission denied". Windows does not look at the bit, so nothing changes there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 13:49:52 -05:00
Michael Schienbein
b6a5cc302c Key the shaders step by the translator the build actually uses.
The shaders step key hashed only the kit's bin/nx2navi_xlate.id, but the build takes the translator from the kit's bin/ or, in a source checkout, from build/xlate/ (xlate_source). From a source checkout, main() worked out the shaders key right after the compile step and stopped with a FileNotFoundError on bin/nx2navi_xlate.id before any shader was prepared.

The key now hashes the identity file beside the translator xlate_source finds, so a source checkout is keyed by build/xlate/nx2navi_xlate.id and a kit's key is unchanged. A build without a translator stops with the existing message that names tools/build_xlate.py. The prompts test kit gets a translator stand-in beside its identity file, and a new test covers a kit that has the translator only under build/xlate/.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 13:49:43 -05:00
chrg6
6910723aed Let a released texture give its content to a new one at its address, and log only the empty starts that can show.
The 0.6.1.9 test still had one black flash, two presents long with FSR 3.1 and its frame generation: a new texture
made over a live one that holds no GPU content started empty (perf counters: one "left uninitialised" at that frame).
A released entry at the same address, kept since 0.6.1 for its descriptor to take back, was never asked for its
content, though on the console the memory still holds what it drew. start_at now takes the newest released entry with
GPU content and a compatible image as the source when no live texture there holds GPU content. Depth buffers that
start empty over another target's memory at every area load are no longer logged, so the 24 lines go to the others,
and the perf line counts copies from released textures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 13:27:45 -05:00
Sebastian Choperena
6589476872 Pack the shader cache after the launcher prepares the shaders, as the installer did.
The launcher builds through tools/make_game.py, whose shaders step never ran tools/pack_shader_cache.py, so a
launcher install keeps about 175,000 loose files (4.9 GB) in shader-cache/. The game looks them up one by one at
every start and reads the use masks of 42,000 of them for 20 s after it. make_game.py now packs the cache after the
preparation; a failure is reported and the build goes on, since the game still reads loose files. The kit and the
launcher ship the tool.
2026-11-09 13:15:47 -05:00