Commit Graph

69 Commits

Author SHA1 Message Date
Tobim6
ad288e3bc1 Staged mirror uploads: the replay copies guest bytes for the VRAM mirrors while it records, and the first-submission wait ends once the replay has submitted; 8 texture pool sets.
On 0.9.1 this is the release's own mirror capture (NX2NAVI_MIRROR_CAPTURE_KB) turned on: with
NX2NAVI_MIRROR_STAGE=1 (the default) the pages the CPU wrote are copied out of guest memory, up to 64 MiB a
part, when the part is handed to the queue. A submission whose prologue copied everything that way reads no
guest memory late, so the game's wait after a frame's first submission ends once the replay has submitted it
(vk_sync_recorded_wait). One with a range left to the GPU-time copy (a pool with a write-back in flight or a
twin) or a draw that reads a pool without a mirror is noted (vk_mem_note_live), and its wait is the GPU's.
Also 8 texture pool sets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 16:22:47 +01:00
chrg6
7dcfb72b70 Keep the time of day at the game's pace under the real-time step, its fraction carried from frame to frame.
The day's update (main+0x3cdf70) adds trunc(2 x frames per update) game seconds a frame. With the set ostep's step,
the display interval in 60 Hz frames, that sum lost its fraction every frame: 1 a frame from 61 to 119 fps, 0 above
120 (the clock stood still), and at 60 fps a step a little under one frame dropped a second now and then. Its convert
now calls the patch page's +0xf60 (runtime/frame_rate.c frame_rate_day_host), which keeps the fraction per counter and
hands out the whole seconds. NX2NAVI_TRACE=perf logs, every 10 s, the seconds given beside those the cut would have
given.

Measured in Kakariko, 10 s windows: at a 60 fps limit 998 game seconds against 881 from the cut alone; at 144 fps
1,201 against 219 to 972 (the steady 1,200 a 10 s window should have).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 03:54:21 -05:00
chrg6
0c343e04a4 Record the game's time step at every present in the flight recorder: the patch page's step, the time manager's and the game's present count.
frames.tsv gains step_frames, game_step_frames and guest_presents. The game writes its step at its own present, a frame or two before the replay presents that frame, so the present count is there to line the two up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 02:54:32 -05:00
chrg6
10dd325dbe Copy the pages the CPU wrote into a VRAM mirror when their part is handed to the queue, behind NX2NAVI_MIRROR_CAPTURE_KB.
The prologue of a part copies the pages the CPU wrote since a mirror's last sync from guest memory into the mirror when
the GPU runs it, a frame or more after the game's submit when the GPU is behind: the GPU then reads what the game
wrote there meanwhile, values meant for a later frame. With NX2NAVI_MIRROR_CAPTURE_KB=n the CPU copies up to n KiB of
those pages per part into a host buffer of the submission when the part is handed over, and the prologue copies them
from there. A pool keeps the GPU-time copy while a write-back of its mirror into guest memory is still on the GPU (its
fence, now kept per pool), when it has a twin (the device address table's writes reach guest memory unrecorded), and
for a range with a page that cannot be read without a fault (memory.c mem_copy_readable, under the freeze lock, so a
lazily armed range stays with the GPU). With the default of every mirror twinned (NX2NAVI_GPA_TWIN_MAX_KB=0) it
copies nothing; it is a test switch for the replay's late reads, default 0.

Measured on an RTX 5090 with NX2NAVI_GPA_TWIN_MAX_KB=64 (mirrors of pools over 64 KiB without a twin), 75 s runs
with warm caches, NX2NAVI_MIRROR_CAPTURE_KB 0 against 8192: Kakariko 0.95 MiB a frame copied in 149 ranges for
0.21 ms of the replay thread, 116.7 against 118.4 fps (interval median 6.26 against 6.46 ms); Hyrule Field walk
1.38 MiB in 224 ranges, 0.37 ms, 110.2 against 107.2 fps, 1% low 83.3 against 78.1. 0.08 MiB a frame stayed with the
GPU-time copy. A first version that took the freeze lock and asked the system for each range's pages cost 2.9 to 4.3
ms a frame. A Kakariko run with NX2NAVI_REPLAY_ASYNC=1 and the copy on played through without errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 19:38:18 -05:00
chrg6
2cccdb8823 Keep the game on the largest-L3 die only when that die has 8 cores, and reserve no recorder core there.
cache_die moved the process to the logical processors of the largest L3 cache whenever the L3 sizes differed. That is
right for a two-die X3D desktop part with 8 cores a die, but it also held the game to the 4 large cores of a mobile
part whose 8 small cores share a smaller L3, and to the 6 cores of a 12-core X3D part, where the recording thread's
reserved core then left the game 5. The die is now used only when it has 8 or more physical cores (perf.c
cpu_placement logs when one is passed over). And while the process runs on that die alone, RECORDER_CORE=auto
reserves no core for the recording thread, since the game's threads already have the die to themselves; =1 still
reserves one.

Checked: all runtime files compile. This PC has a single-die processor (one L3), where neither rule applies: a run
still reserves the recording core (logical processors 14 and 15 of 16) and logs no cache die.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 19:38:18 -05:00
chrg6
d3e50328bf Keep movies with their sound at any frame rate: one movie frame per update, and the pacer keeps the movie's slots through a late frame.
The game's movie player adds the time manager's step to an accumulator at each update and advances the movie by whole
pictures of it, at most the player's limit per update, which is one for the Triforce movie. While a movie plays the
game is held at 60 updates a second, but every late frame cost an update, the pacer started its slots again from the
late one, and the steady step's payback of the lost time came as steps over one frame, which the player cannot turn
into more than one picture. So each late frame put the picture further behind the sound, which plays by the clock, and
it never caught up.

While a movie plays the step is now exactly one movie frame, and the pacer keeps the movie's slots through a lateness
of up to a second, presenting the frames after a late one at once until the game is back on them. The updates then
average the movie's rate, and the picture, the event's update count and the game's clock all stay with the sound.
movie_clock = 0 gives the earlier behaviour. The movie log now measures the picture on screen against real time where
the game releases it (the game fetches pictures ahead into a list of its own) and reports it at the end of each movie.
test_hitch = <every>:<ms> makes a frame that much late every so many presents, for test runs.

Measured on the Triforce movie (event Dm_Kokiri_0095, 7,040 pictures, 117 s), one exe with movie_clock = 0 against 1,
and a 70 ms late frame every two seconds (test_hitch = 120:70), as a run with natural hitches had shown a drift of 40 ms
every ten seconds. Drift of the picture on screen behind real time, from the movie's first 10 s to its end:

  fps_limit   before (updates/s, drift at the end)   after (updates/s, drift over the movie)
  unlimited   58.86, 2,187 ms behind                  60.22, flat at 53 to 55 ms
  144         58.86, 2,196 ms behind                  60.04, flat at -11 to -15 ms
  120         58.98, 2,257 ms behind                  60.20, flat at -1 to 0 ms
  60          58.88, 2,220 ms behind                  59.98, flat at -21 to -22 ms

Without the late frames both kept time (unlimited: before flat at 21 ms, after flat at -13 ms). The selftest,
--probe-cfg, --probe-kernel, --probe-movie and tests/frame_step_check.py pass; the check now plays a movie started at
240 fps with late and short frames and needs exactly one frame every update.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 15:09:16 -05:00
chrg6
5d51933969 Arm a written-back range only while its mirror is still the pool's, and give back an armed range of another mirror instead of marking it stale.
A write-back finishing on the completion thread could arm its range between a mirror's eviction and the drop of its
buffer, naming the freed buffer; a later write into the new mirror then marked that range stale and its next touch
copied from the freed buffer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 15:09:15 -05:00
epicmeme546
760a7aaaf9 Hold a memory region thawed after a write thaws it soon after its freeze, so the game's streaming buffers stop going back and forth between frozen and thawed.
Regions the game has not written for a while are write-protected so that the per-submission write-watch queries can skip them, and the first write thaws them. The game's loaders keep writing the buffers they stream an area into, so those regions were frozen again as soon as they had been idle a moment and thawed again moments later: running from the Kakariko Village gate across Hyrule Field froze 31,004 regions and thawed 28,918 by a write in nine minutes. While the loaders wrote and the driver locked new memory, each thaw's VirtualQuery took 15 to 20 ms: a loader thread stalled 78 to 111 ms in thaws in every one of the hitches of 125 to 140 ms of that walk, with the game's main thread waiting for it.

A write that thaws a region within a minute of its freeze now holds it thawed for 4 seconds, doubled at each such thaw up to 10 minutes; a region that stayed frozen longer starts again from nothing. On the same walk: 8,284 freezes and 6,355 thaws, no frame over 100 ms while Link ran where 29 and 52 a minute had been, the slowest 1 % of frames at 19 and 24 fps instead of 9 and 7, and at the walk's end, Link standing at the edge of Lake Hylia, 60 fps instead of 38 with the slowest 1 % at 43 instead of 20. The perf trace's write-watch line counts the holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:06:16 -05:00
chrg6
31e0a02901 Merge the mesh shader fallback: task and mesh programs drawn through compute passes and an ordinary draw where the card has no mesh shaders.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 06:01:39 -05:00
chrg6
b1408ee371 Add --probe-meshfb, which draws mesh programs through the mesh pipeline and through the fallback and compares the pictures.
The probe's programs are written in the shape the translator gives a task or mesh program and need no game files.
With a folder of game translations it converts those instead and builds their compute pipelines.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 05:58:43 -05:00
chrg6
d073d80e70 Create FSR 3.1's pipelines through the pipeline caches kept across runs, so a context is not compiled again at every start the driver's own cache does not serve.
The library creates its pipelines itself and names no pipeline cache. It is now handed pipeline creates of the runtime's in the place of the device's two, and a create without a cache is asked of the pipeline cache first and compiled into the play cache otherwise. ffx_pipe_cache = 0 turns it off. --probe-ffx-cache makes one upscaling context without a window or the game and says where its pipelines came from, and tests/ffx_cache_check.py runs it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 05:22:56 -05:00
chrg6
147e8a02e3 Warn in the log at the start when the machine has less memory free than the game takes in play.
The game held 23.9 to 24.4 GB of commit in three sessions on an RTX 5090 at 3840x2160, flat over the session, on a
machine whose commit stood at 66 of 73 GB with it. A machine that cannot give it that much pages, drops textures or
loses the device, and its log said nothing about memory until then.

mem_start_note reads the system's commit when a game starts and writes one warning line when what is free, plus what
the process already holds, is below mem_need_mb (24576 by default, 0 for no warning). The menu has no place for a
start-up notice (its only toasts are the co-op page's), so the line is in the log alone.

tests/mem_need_check.py checks the rule and, through --probe-memneed, the answer and the warning line with the
machine's own commit.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 04:48:36 -05:00
chrg6
39fa0be78c Make every pipeline built ahead of its first use pass a memory gate that counts the builds already running.
Only the large compute shaders' background builder looked at the system's commit before it started a build, and it
saw nothing of what the builds beside it were about to take. The boot's recorded pipelines (up to 64 builders), the
pool's background builds during play, the boot's compute pipelines and stage libraries and the library warm asked
nobody.

Each of them now asks the gate before a build. A build that starts reserves an estimate of the driver compiler's
working memory until it ends (build_reserve_mb, 64 MB; 4096 MB for a large compute shader), and the gate admits a
build only while the commit headroom holds the part kept free (build_free_mb, 4096 MB), the reservations and its own.
A refused boot builder ends and leaves its pipelines to the pool; the pool leaves a background build queued and asks
again every two seconds; the stage library prebuild and the library warm wait up to 30 seconds and then end, and the
warm goes on at the next start. A pipeline a draw or a dispatch waits for is never refused. The preparation of the
pipeline list runs without the gate: no game runs beside it.

tests/build_gate_check.py checks the gate's arithmetic and, through --probe-buildgate, its answers with the machine's
own commit. What the builders do when refused is not exercised by a test.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 04:42:07 -05:00
chrg6
534591be04 End a run whose GPU device is lost in one place: one report, no callbacks of unfinished submissions, and exit code 72.
A lost device used to end the process from whichever submit or wait noticed it first, while the present only logged it
and went on and the other threads kept calling the driver. The completion thread could take the submissions' timeline
as reached and run the callbacks of submissions that never finished.

vk_device_lost_check now takes the result of every submit, present, acquire and wait. The first thread to see the loss
sets a flag, writes the report and ends the process as a fatal error does (the log written, a save commit under way
finished), with an exit code of its own; later threads stop in that call, and a watchdog ends the process after ten
seconds should the first thread hang. The replay, its submit, the uploads, the present and the lazy fetch read the flag
and return without touching the device. The completion thread asks the device with a call of its own before a new
value counts and runs no callback once the flag is set.

The executable's --probe-devlost and tests/device_lost_check.py check this without a lost device, and
device_lost_at in nx2navi.ini ends a real run the same way.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 04:35:30 -05:00
chrg6
861357b2a4 Make the direct mouse camera work on game 1.1.1.
mouse_mode = direct is the default, but on 1.1.1 it did nothing: only the 1.4.3 patch table had the sites, so the mouse always went through the stick emulation, which gives no turn at slow mouse speeds and tops out at about 107 degrees a second. The 1.1.1 table now has the set mouse_direct (on by default): at main+0x37c45c, where the camera's input update holds the processed stick in s1 and s0, a host call performs the displaced store and adds the raw mouse counts to both, so the mouse meets no dead zone and no curve; at main+0x37c4f0 a second call clears the idle bit for an update the mouse moved in. The counts are divided by the frames the game steps per update and by the stick camera speed, so the same hand movement turns the same angle at every frame rate. The Controls tab gets the mouse mode and mouse sensitivity rows, and mouse_mode = stick or NX2NAVI_MOUSE_DIRECT_PATCH=0 gives the old behaviour.

Checked here: the generator lifts the 1.1.1 game with the two sites (it compares each site's original word), the lifted game compiles and links with the runtime, and tests/mouse_direct_check.py passes its 1.1.1 harness (register contract, displaced store and load, equal turn at every frame rate and stick speed, the reset pause, capture and mode). Not checked here: the game was not run, so aiming with the mouse was not tried, and the differential tests against Unicorn were not run because no Python on this PC has Unicorn. The contributor measured 0.0800 degrees per count on both axes in the game.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit bb8fecfbe43517006eafd7be9c22e8dbe9601de7)
2026-11-11 02:36:54 -05:00
chrg6
acc1ae25bf Merge co-op, off by default: rooms through a relay, progress sync between the players' own saves, and a Co-op tab.
Without the other players drawn in the world, which are unfinished. No relay address is built in, and with co-op off nothing opens a socket. Built on Windows and tested against a relay on this PC only; two games in one room and any real network have not been tried.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 00:40:20 -05:00
chrg6
1b89c0227b Let --probe-movie test this title's AV1 movies, which it rejected before testing anything.
The probe checked for VP9 first and stopped on anything else, and every movie of this game is AV1, so the decoder the game actually uses had no test short of a game run. For an AV1 movie the probe now drives the host's movie::VideoDecoder as the game does: it feeds the samples, takes the pictures at 30 and at 60 updates a second, and checks that every picture is delivered once, in time-stamp order, with the end of the stream after the last one, and how far behind real time the pictures are. It starts no window and no device. A run that never reaches the end of the stream fails after a bounded number of updates instead of hanging.

On the game's files here: before, "FAILED: WebM: codec V_AV1"; now 240 of 240 pictures at 30 updates a second and 180 of 180 at 60, none late, the last 1 ms behind real time at 60, "probe-movie: passed". It does not look at the pictures' content.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit 2543b07117c638e4e3bf7df01e16f6801f9cbcfb)
2026-11-11 00:39:58 -05:00
chrg6
420aef041a Correct what a review of this intake's commits found.
A second reading of the commits taken in from pull requests found these, each traced in the code:

- heap_lazy: the camera speed's search read the whole heap, which with the setting on committed every page and undid what it saves; a scan now steps over the pages the lazy heap never committed (mem_untouched_run). tests/frame_step_check.py and tests/save_share_check.py no longer linked (frame_rate.c and fs_host.c call the new memory functions; the second also missed mem_probing before); both pass again. The settings text says that with the setting on a PC out of commit ends the game in play, not at start, and the census comment is back over its own function.
- Pipeline libraries on AMD's older Windows driver: the condition is the driver for RDNA 2 and older (RX 400 to RX 6000), not RX 5000 and RX 6000 only, and the texts named a price the code does not have. With libraries off a new pipeline is built by a worker while its draw is left out, and the start waits for every recorded pipeline instead of 20 s at most. The texts say so, and that the finding is one player's on one card. The menu strings nothing shows are as they were.
- Page file hint: it comes only when Windows sizes every page file, and it names the initial size and the automatic management that must be unticked first.
- Frame generation note: it keeps "needs an RTX 40 or newer" for the players it is right for and points the others to the log; the bridge's line says it is Streamline's general support check.
- Comments and settings texts that still described the old behaviour: the process commit bound, the wait with shadow storage, the scratch's least size, the action prompt under auto.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit 19f03aa01b26d6e17fb5fc70ad987aa8c68e8225)
2026-11-11 00:39:50 -05:00
chrg6
fc1a50f0db Merge the audit: its map and ledger, the perf-trace markers and size lines, and the fixes of its Phase 4.
Verified by runs in the audit: the Market's periodic stall (a pool slot was listed once per refresh pass instead of once; the list fell from 160,385 entries to at most 3,192 and no frame over 20 ms remained while standing), orderly exits delete the pipeline prebuild's running marker (left behind, the next start wrote a skip file and every later start skipped the prebuild), the log and crash report name an area a save asks for that the game does not have, and the menu atlas hash no longer overflows a signed multiply. The other fixes compile and pass the regression route but their faults were never reproduced: see AUDIT.md section 13 for each.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 00:39:08 -05:00
chrg6
7e07e13bac Add co-op, off by default: rooms through a relay, progress sync between the players' own saves, and a Co-op tab.
Each player keeps their own game and save. One player makes a room, the others join it with its secret, and story flags and, by the room's rules, items, songs, quest items, small keys (a shared pool), rupees and ammunition reach every save. The layer is a host thread with one UDP socket to a relay the players run themselves (tools/coop_relay.py, Python's standard library only); the messages between the players are sealed with a key from the room's secret (XChaCha20-Poly1305, Monocypher 4.0.2 vendored with its licence and a row in THIRD_PARTY_NOTICES.md), every change is signed by the player who made it, a player accepts a room by its members' keys and the key of its rules, the save is copied to coop-backup before anything is applied, and values are set through the game's own routines between frames. The Co-op tab makes, joins, accepts, removes and leaves, and the menu takes typed text for the name, the relay and the secret. This is the contributor's pull request #518 on this line.

Nothing reaches the network while co-op is off: the socket is opened, and the relay's name looked up, only after co-op is on, a relay is set and a room is made or joined. There is no default relay.

Left out of the pull request: the other players drawn in the world (the "puppets") with the copied engine they need. They were unfinished, added two patch sites to the game's per-frame update that could not be put through the differential tests here, and their file helpers clashed with the ones this line already has. So no position is sent: the POS message's codec stays, nothing sends it.

Changed against the pull request:
- The menu's test hook drives the Co-op tab only when it comes from the environment. From nx2navi.ini it could turn co-op on, join a room and accept its offer.
- The log's label for a room is hashed from the secret instead of being its first three characters.
- A game that has not saved yet has no save to copy, and takes nothing from the room until it has saved once; that used to count as a copy made.
- A PING's token no longer carries the PC's uptime.
- On Windows the maker's key file and its folder get an access list with the player's account alone.
- The tests that link the platform layer get the socket library, and the save test a stand-in for the title screen's hook.

Checked on Windows, where the pull request had never been compiled: the runtime builds and links; --probe-coop passes (sockets on 127.0.0.1, the codec against its byte vectors, a fuzz of every parser, two sessions through a hostile mock relay, a relay restart, the sync engine); tests/coop_relay_test.py passes its 61 tests, with the probe through the real relay on 127.0.0.1 at 0 and 30 % loss; tests/coop_page_check.py passes and its deliberate fault fails 48 checks. Not checked: two games in one room (one game runs at a time here), any real network, the tab in the game.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 00:11:41 -05:00
chrg6
7ccb958c47 Let a save Commit that is under way finish before the crash report or a fatal error ends the process, for at most two seconds.
The game's Commit returns once the commit is queued, so the game may already have reported the save as done while its thread still copied; ending then left an unfinished copy that the next start rolled back.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:26:01 -05:00
chrg6
dda12051f5 Add heap_lazy (off by default): on Windows the game's heap is committed where it is first used.
Windows charges every committed page to the commit limit whether it is used or not, and the game sizes its whole heap up front: in Castle Town after Hyrule Field the heap held 6.8 GiB of commit of which 3.4 GiB had ever been used. With heap_lazy = 1 a change of the heap's size only moves its end, an access to a reserved heap page commits the reserved runs of the 64 KiB around it in the access-fault hook and runs again, and the paths that do not fault commit their range first: mem_thaw (file reads, the host-pointer import), mem_kernel_access (file writes, the frame-rate fix's read) and mem_host_readable (the host's pointer checks). A page another path committed is left as it is, so a frozen, probed or armed page keeps its protection. With trace = perf a line per window gives the faults, the bytes committed and the time they took.

The contributor's runs on a 16 GB PC: 4.67 instead of 6.77 GiB of heap committed and 29.3 instead of 32.9 GiB of process commit in Castle Town, no "system commit low", 12,858 faults in 440 ms over four minutes, the longest 41 ms. With the setting off nothing is hooked and the heap is committed as before.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 23:14:00 -05:00
chrg6
78223225f4 Mark DLSS and DLSS frame generation feature creates as work under the perf trace, with the device idle wait of releasing the kept features before them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 22:34:54 -05:00
chrg6
ded13958de Log the sizes of the caches, pools and tables under the perf trace, every 600 presents and with each memory snapshot.
Four "mem: sizes" lines give the entries and bytes of what no other line counted: the guest heap, physical maps, kernel region, lazily written ranges and the fixed kernel, setjmp, save-file, RomFS and zstd tables; pool records, mirrors, deferred destructions, upload and staging buffers, the device address table and the texture, sampler and slot caches; image views, descriptor spares, the completion queue, uniform snapshot arenas, recording copies and pins; and pipelines, layouts, stage libraries, job queues, the shader memory cache and the translator processes. Each subsystem reads its own counters under its lock; settings now count the replaced values they keep. Segment and scene lines say "-" for the world clock before it starts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 19:51:36 -05:00
chrg6
ecd34fb513 Under the perf trace, report frame times per route segment and take a memory snapshot after each area load.
An input script line "<tick> MARK <name>" (or "@<ms> MARK <name>") ends the running segment with one line of its frame count, mean, 1% low and worst frame, the work marks on any thread, process memory and VRAM, then the 600-frame memory lines, and starts a segment of that name. RomFS opens of an area's static placement file give a "scene:" line, and once no area has loaded for 10 s the memory snapshot is taken with a "settled" line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 19:40:02 -05:00
chrg6
66aedffcdd Name the work behind each frame on any thread under the perf trace: RomFS reads, zstd decodes, pipeline creates, guest memory maps, pool initialize and finalize, and device memory allocations.
A slow frame's line and the 60-frame window's line now say how long each kind took on any thread, how often, how many bytes and the longest single one, so a hitch outside the render thread's own waits has a named cause. While NX2NAVI_TRACE=perf is off each site pays one branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 19:27:58 -05:00
chrg6
fc71691e91 Merge branch 'claude/amd-fight-crash-ef03a9' into test-all-fixes 2026-11-10 15:22:07 -05:00
chrg6
caf50ecf1c Turn the lazy write-back off by default on AMD's driver for RX 5000 and 6000 cards, and say in the log which settings a run had.
A player on an AMD card reported crashes in fights and that a line added to nx2navi.ini cured them. That line, async_compute = 1, is no setting of the game, and neither are two more in the same file; the one line of the file that changes how memory reaches the GPU is wb_lazy = 0. A second report, an RX 6600 on 0.7.1, lost its device 3.5 minutes into play with the lazy write-back on. Neither proves it the cause: the player changed several lines at once and sent no log, the RX 6600's last pass mark is inside a draw pass and not a fetch, cards with 8 GB lost their device the same way on NVIDIA before the lazy write-back existed, and reading the fault handler, the fetch, the arming and the buffers' lifetimes found no failure path. What depends on the driver and cannot be checked without the card: pages of guest memory are inaccessible while a host-pointer import of them is bound and written by the GPU, and Vulkan calls run inside the access-fault handler.

The default is now decided once the driver is known (wb_lazy_choice.h, vk_memory.c vk_mem_lazy_decide): on under Windows as before, except on AMD's driver branch for RDNA 1 and 2, which none of us has run it on; AMD's driver for RDNA 3 and newer keeps it on, since builds with that default were measured on an RX 7900 XT. What the player sets wins everywhere, and one line at start says the choice and its reason. On those cards open areas run at the 0.7.0 frame rate again (on an RTX 5090 that was 65 fps against 90 in Hyrule Field; not measured on AMD). tests/wb_lazy_check.c checks the choice by platform, driver and setting without a GPU.

For reports of this kind the log now holds what was missing. At start: every setting nx2navi.ini sets, and in one line the keys that are no settings of this version and do nothing; the settings menu shows those in its bottom row. At a fatal error or a crash: one line with the card, its driver, the memory mode, the lazy write-back with its reason, armed ranges, fetches so far and a fetch in progress, and the texture detail (dispatch.c rt_fatal_note_add, vk_device.c fatal_note), since a report is often the end of a log alone.

--probe-cfg loads the player's file: the three lines are named, and the menu's save keeps them and every other line it does not change byte for byte. Nothing here ran on an AMD card, and no fatal error was provoked to read the new line.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:14:25 -05:00
chrg6
918d399d55 Add the shared pipeline list: mesh pipelines players met are built at the shader preparation, before the first start.
The preparation translates every shader and builds each one's stage library, so a pipeline a draw meets for the first time is linked in 2 to 6 ms. A mesh pipeline has no stage libraries: on an install that never drew it, it compiles whole in play, 0.2 to 1.1 s with its draws left out of the picture (18 of them on a first visit of Kokiri Forest). Whole pipelines were built ahead only from the folder's own pipeline-keys.bin.

tools/pipeline_list.py merge folds the pipeline-keys.bin and shader-answers.bin of game folders that were played into one list file: every key record once, with the answers record of each shader it names. A record names its shaders by input id, which depends on the game's code and the answers alone, and holds the game's pipeline state; what depends on the GPU is derived when the pipeline is built, so a record made on one machine is good on another with the same game version and runtime. Four folders on this PC give 5,432 records, 317 of them mesh pipelines (147 KB of keys).

tools/prepare_shaders.py reads the list (tools/data/pipeline_list.bin, data/pipeline_list.bin in a release folder, or --pipeline-list), adds its shader variants to the seed and writes its records as seed-pipeline-keys.bin. nx2navi.exe --prepare-shaders builds the mesh pipelines among them after the seed is translated and saves the pipeline cache before the stage libraries are built (NX2NAVI_PREPARE_PIPELINES=all builds the vertex pipelines too); --warm-shaders does it again after an update or a new driver, and --prepare-pipelines is that step alone. The pipelines go into the caches and are not kept or added to the folder's own key file, so they cost no start-up time or video memory until a draw meets them. A marker file keeps a driver that crashes on one of them from failing every preparation.

The list is derived from game data and stays out of the repository: check_content.py refuses files recorded from a game folder by their first bytes, and make_kit.py, make_launcher.py and make_release.py carry a list given with --pipeline-list. All three now also carry pipeline_list.py, which prepare_shaders.py imports.

Checked without the game: tests/pipeline_list_test.py, and prepare_shaders.py --no-run on the game files with a list of 317 mesh pipelines (all 266 shader variants found in this game version, 177 new to the seed). The build step in nx2navi.exe compiles and links and has not been run.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 14:52:46 -05:00
ordinarybob
4a954a1c6d Refresh the texture pool from a list of its entries instead of walking every slot, for about 7 % more frames a second where the replay thread is the limit.
Each submission's first sync of the bindless texture pool walked every registered slot, 33,000 in Kokiri Forest, to find the few textures the CPU wrote and the few slots that changed. Three parts of that walk change, each with its own switch (pool_fast, pool_presnap, pool_group; on by default, 0 and ab accepted):

- The freshness check goes through a list of the entries the slots resolved to, with a check by 2 MiB region first; slots that resolved to nothing are looked up again when the game's memory map changes and every eighth submission. The use stamps the idle eviction reads are left from a count of the slots that hold each entry.
- The check takes the write-watch snapshots of its regions first, so unchanged entries pass the region test.
- The GPU-written slots of one address bucket with one descriptor form a group, which the dirty walk leaves alone while its first member is unchanged.

Measured on an RTX 5090 at 2560x1440, standing in Kokiri Forest, in-run A/B over five runs with all three switched (three of them with the barrier change switched too): 93.6 to 100.7, 96.4 to 103.1, 95.8 to 102.6, 99.3 to 107.1 and 91.9 to 99.9 frames a second, the replay thread 0.7 to 0.8 ms a frame shorter; a control's blocks differ by 0.4 %. On a walk with NX2NAVI_POOL_GROUP_CHECK=1 no group that was left alone had missed a write, and screenshots across the A and B blocks show the same picture.

From pull request #390. Not tried: the texture eviction under memory pressure (a 9 GiB budget was never reached on the walk), AMD and Intel drivers, Linux. One of the nine runs with it on ended by itself with exit code 0 and nothing in its log two seconds after the window lost the focus; three repeats of that run did not.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 14:09:21 -05:00
chrg6
63c7d3cb74 Make the frame limit count the frames the game draws, with frame generation on top.
The limit counted the frames on screen, so the game was paced at the limit over frame generation's multiplier: 60 at 4x drew 15 frames a second, and each step up of the multiplier took drawn frames away. Now a limit of 60 at 4x is 60 drawn and 240 shown. Only the display choice still counts the screen's frames, since a frame past the monitor's refresh is never seen, and with no limit set the default is 60 drawn rather than the monitor's rate.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 11:48:24 -05:00
chrg6
d0b15e4578 Merge the two performance changes into the 0.7.0 release line: a signalled event wakes only its own waiters, and a thread of its own writes the log.
One conflict, the list of setting names, which keeps both sides' additions.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 07:32:34 -05:00
chrg6
a03e059d2b Write the log from a thread of its own, so a stalled write no longer holds the thread that logged.
Every log line was two writes on the thread that logged it, to stderr and to the log file. Such a write can stall for
tens or hundreds of milliseconds (a busy disk, a virus scanner, a console or a pipe that is slow to take it), and the
render thread logs too: a frame it spent in a write was a hitch with nothing else to show for it.

In a game run the lines now go into a queue that a log thread writes out in order (runtime/logq.c). A fatal error,
the crash report and the end of the process write the queue out on their own thread first and write their own lines
directly, so nothing is lost to an exit. A thread that finds the queue's lock held for long, or the queue full, writes
its line itself. NX2NAVI_LOG_ASYNC=0 writes every line where it is logged, as before.

Every write is timed in both modes: a "log: ... writes of the log took over 2 ms" line counts the slow ones, gives
the longest and says whose time they took.

tests/logq_check.py checks order and completeness with eight threads, a slow destination, a stopped destination with
the queue full, and the synchronous flush.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 07:22:41 -05:00
chrg6
c117cd8999 Set NVIDIA's shader disk cache on Linux only with NVIDIA's driver, in a folder that could be made and written, before any thread of the game starts.
The first part of this pull request (the cache beside the exe with an 8 GiB limit) was already in. This
is the rest: nothing is set without NVIDIA's kernel driver, neither variable is set when the folder cannot
be made or written (the driver caches nothing in such a folder), the log says which folder and limit hold,
and a game run sets the drivers' cache variables in main before any of its threads can load a Vulkan
driver (vk_device.c vk_driver_cache_env); the other modes still set them in create_instance. On Windows
that only moves the MESA_SHADER_CACHE_DIR setenv earlier.

Pull request #369 by mooki000, taken as it is. The Linux part is not compiled or run here.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 06:17:13 -05:00
chrg6
e473dd8826 Arm a lazily written range only when its mirror holds every CPU write to it, so a fetch no longer copies older mirror bytes over what the CPU wrote.
A fetch copies the mirror over the whole range, not the merge of the bytes the GPU changed that a write-back does. The
completion thread arms a range after its write-back reached guest memory; a CPU write into those pages in between, not
yet carried up by a sync of the mirror, was then overwritten by the mirror's older bytes at the first fetch. In play
that was single frames with another frame's constants: the night lighting missing from the whole scene in Hyrule
Field every 0.1 to 0.4 s, and a black scene in Kakariko, gone with wb_lazy = 0. Now the pages are made inaccessible
first and the back end then checks the CPU-write page generations against the mirror's last sync and the write-watch
bits nobody has read yet; when either shows a write the mirror lacks, the pages are given back and the range arms at a
later write-back. The lazy line counts the arms refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 05:37:14 -05:00
chrg6
bd9b48ff04 Keep a fault from releasing a lazily written range unseen while a part is handed over, merge what a part skipped and must write back after all, and leave the lazy write-back off on Linux unless set.
A review found a gap: a part checked what it had skipped at its end, and a fault between that check and the part's
hand-over to the queue fetched without the part's writes while nothing wrote them back afterwards. The replay now
opens a window from the check to the hand-over, per part, and a fault on an armed page waits for it to close (at most
20 ms; the replay thread's own faults and thaws do not wait and are counted). What a part skipped for a range that is
no longer armed goes back through the usual snapshot and merge, taken in that part's prologue, and not as a plain
copy, which would have put the mirror's words over what the CPU had just written there. A part split off earlier
makes the same check before its prologue. wb_lazy_touch is a test that reads armed ranges on purpose.

Dev build, Hyrule Field: 71.1 fps as before; standing, a fight, pause and unpause, and the touch test made 146 to 338
faults each, none inside a window, no fault of its own.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 05:23:43 -05:00
chrg6
17c13e49ff Copy what shaders write into the game's memory back only when the CPU touches it, which takes Hyrule Field from 51 to 71 fps.
Standing in Hyrule Field the GPU spent 13 ms a frame, of which 2.4 ms were the game's passes and 7 to 9 ms the copy
of 66 MiB of shader-written buffers back into guest memory at the end of every submission, in case the CPU reads
them. The write-back probe found no CPU access to them in 40 s. Now a range goes back once and its pages are made
inaccessible; while they stay so, later writes stay in the mirror. The first CPU access faults, the bytes are fetched
with a GPU copy of their own after everything submitted so far, the pages are given back, and the range is written
back every submission again for a while (1 s, doubling to 32 s while it keeps being touched). Thaws before kernel
reads and writes and before mapping changes release armed ranges the same way, and so does a mirror that goes away.

Dev build, same spot, 80 s each: 51.5 to 71.0 fps median, 1% low 43.5 to 56.3, GPU time 13.1 to 7.5 ms a frame,
fence waits 9.8 to 4.4 ms. In that run the game faulted on armed pages 226 times and 124 fetches took 51 ms in all;
a walk into a fight made 283 faults and 186 fetches, and a test that touches eight armed ranges every 60 frames
(wb_lazy_touch) 480 faults and 364 fetches, none with a fault of its own. wb_lazy = 0 is the old behaviour.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 04:36:46 -05:00
chrg6
61a13c62b9 Keep only the compute pipelines of recently dispatched shaders from the boot, and build the others into the pipeline cache without keeping them.
The boot prebuild built the pipeline of every compute shader play ever recorded and kept all of them for the process. A pipeline that exists holds driver memory whether or not anything dispatches it: with a history of 4,066 compute shaders they held 2.3 GiB of device memory, and its share of the process's commit, from the title screen on (RTX 5090, NVIDIA 621.01), while a boot into the first village dispatched 95 of them.

The boot now keeps the pipelines of the 512 compute shaders dispatched most recently (compute-used.bin next to pipeline-keys.bin records which run last dispatched each; the setting is compute_keep, with "all" for the earlier behaviour). Every other one is still built at boot, so that the pipeline cache holds it after an update or a driver update, but destroyed at once, and created from the cache at its first dispatch. A dispatch that comes while such a build runs gets that build's pipeline.

Each boot stage now logs the device memory and the process commit it leaves ("mem: boot stage"), and the log says what the first dispatches cost. nx2navi.exe --probe-prewarm runs these stages without the game from the caches next to the executable and times first dispatches of pipelines the boot did not keep: 256 of them took 0.19 ms on average and 3 ms at most, all read from the pipeline cache file, and 372 pipelines built and destroyed at boot left the device memory where it was.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit 16a796d623686bca5304eaf1b29981247453a413)
2026-11-10 03:34:03 -05:00
chrg6
fa29f3fea5 Merge the second PR intake pass: anisotropic filtering in the Screen tab, the X3D die choice, the NVIDIA shader cache limit, the NGX core search and other fixes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 03:32:46 -05:00
chrg6
2a9efee4c5 Merge GitHub main into the test line: the launcher's customisation, the settings and sound changes, and the warm stage-library cache.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 03:32:30 -05:00
ordinarybob
662a6e88ee Run the game on the die with the larger L3 cache on processors whose dies differ (AMD's two-die X3D processors).
Windows places a game on the V-Cache die only when it recognizes the exe as a game, and it does not know this one, so the threads were spread over both dies by frequency preference. cache_die = 1 (the default) narrows the process to the logical processors that share the largest L3; it never widens the set the process was given, and does nothing where every L3 has the same size. The recorder's reserved core is then chosen inside that set.

Co-Authored-By: Claude Fable 5.2 <[EMAIL]>
2026-11-10 02:01:35 -05:00
epicmeme546
41eb0e635b Apply the camera's stick sensitivity and the speaker layout during a run.
NX2NAVI_STICK_SENS was served to the game as a multiplied camera pack at start only. A change during the run is now written into the game's memory: the StickSensitivity array as the game holds it, at the factor in effect, is looked for in the guest's heap and every copy is overwritten at the new factor. The game reads the array each time it steps the camera, so the speed follows at once. The copies found are remembered, so a later step of the menu's slider takes no search, and a search runs again a moment after the game opens the pack anew. On Ocarina 1.1.1 the first change of a run found 4 copies in about 2 s on its own thread, and the same stick push for a third of a second turned the camera about 30 degrees at 1, 90 at 3, 10 at 0.3 and 30 again at 1, all in one run.

NX2NAVI_AUDIO_CHANNELS was decided once. A change that gives another channel count now signals the device change events the game holds, as a console does when its output moves: the game asks GetActiveChannelCount again and mixes for the new count, and the mixer makes its SDL stream again with that count at its next render. On Ocarina 1.1.1 the centre input of the device sink went from 0.0006 rms to 0.013 within the 5 s after stereo became 5.1 and back to 0.0006 after stereo was set again, with no underrun. NX2NAVI_TRACE=audio logs the six inputs' levels with its level line.

Both rows lose their "after restart" note. The graphics card, widescreen, the resolution, the game's language, the button prompts and the game's native HDR still take a restart: the game or the Vulkan device reads them once while starting.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 00:21:31 -05:00
chrg6
202fc247d0 Name the gamepad probe probe_gamepad, so it does not clash with ov_probe.c's own static probe_input.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 23:54:29 -05:00
chrg6
11898757d1 Name the gamepad probe probe_gamepad, so it does not clash with ov_probe.c's own static probe_input.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 23:53:29 -05:00
chrg6
be8c2620e5 Run the gamepad update on a thread of its own, so a controller that comes or goes no longer stalls the game.
When a device came or went, SDL's gamepad update first found every device again, 100 ms and more, on the game's
polling thread with the input lock held: a controller disconnecting gave five frames of 120 to 240 ms in a row, with
the game's other pollers queued on the lock. The pad thread now makes SDL's initialisation and every update. A poll
asks for an update and waits at most 2 ms for it; an ordinary one is done in microseconds and the poll reads fresh
state as before, and during a device search the poll goes on with the state it read last. Rumble, the motion samples
and the menu's stick reading skip a beat then instead of waiting.
--probe-input checks it on a virtual gamepad, with an update made to take 300 ms.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 23:51:58 -05:00
chrg6
f63b6796ee Decode Ocarina 1.1.1's loading streams on the host decoder, behind the same zstd_native = 1 switch.
The loading threads feed each chunk they read to the game's stream decoder, which is where most of the
decompression time goes. That call now reaches a host stream that writes straight into the game's output
buffer and keeps the game's state fields and results. A stream is taken from its first chunk or left to
the game; zstd_check = 1 runs both and compares the outputs. --probe-zstd checks the host streams on
frames it makes itself.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 21:21:48 -05:00
chrg6
22103a57bf Merge main into the test line: the pull requests taken there and two render target fixes.
Main brought the fixes of eleven pull requests (the texture pool set sealed when an early replay part is submitted, every location reflected in interface arrays, RGB9E5 texels recognised for integer storage views, the upscaler a frame started with kept, GPU pass timestamps without vendor breadcrumbs, the Linux build, window and NGX fixes), render targets released when drawn over at another address, the vertex layout kept in the pipeline on newer AMD drivers, and the upscaler choice read once per evaluation.

Four hunks conflicted and take both sides: pool_sync returns for a sealed set before the per-frame creation budget's clock (vk_exec_draw.c); make_module keeps the test line's early_ok argument and main's module hash (vk_pipeline.c); a texture taken back counts as sampled again and records how long it was away, and the texture line logs the disuse counters and the overwritten render targets (vk_texture.c).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 21:10:23 -05:00
chrg6
57cdfa1520 Add wb_probe = 1, which finds out who reads what the GPU wrote back into guest memory.
Per-pass GPU timestamps put the write-back of GPU-written storage buffers at 3.6 to 5.3 ms of GPU time a frame in play, against 1.8 to 2.5 ms for all of the game's own passes and dispatches: about 67 MiB a frame goes from the mirrors back into guest memory, from the same eleven pools all session, in case the CPU or a host path reads it. Nothing said whether anything does.

With wb_probe = 1 the 2 MiB regions that lie wholly inside a range just written back are made inaccessible. The first read or write of such a region faults; the handler gives the region back the protection it had (read-only when frozen, else read-write), notes the address, the instruction and the thread, and the access runs again. The bytes are in guest memory all along, so nothing changes for the game. A log block every perf window gives the regions armed, the probes a read or a write ended, and the first accesses by instruction. Every thaw ends the probe of its region first, which covers file reads into guest memory and mapping changes; a file write from guest memory thaws its buffer while the probe is on; a probed region is not frozen anew.

The platform layer gets a hook for every read or write access fault beside the write-fault hook; it is installed only with the setting. --probe-gfx-abi checks that nothing is armed without it and, run with NX2NAVI_WB_PROBE=1, that reads and writes end a probe and run again, that a frozen region thaws after it and that mem_thaw ends and holds.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 21:01:47 -05:00
Michael Schienbein
4cb1a56e6c Give --boot and --seconds the call sites of game 1.1.1.
On 1.1.1 the SDK's nn::oe::InitializeApplet calls SetFocusHandlingMode itself, before DT_INIT, so the first call reached the boot checkpoint from sdk+0x598840 and --boot and --seconds stopped with a FATAL before the game started. Only a call from main is the checkpoint now. 1.1.1 also gets its own return addresses (nnMain's SetFocusHandlingMode at main+0x1202d44, the first nvnBootstrapLoader at main+0x346bf8c, read from x30 in the two host functions during a normal start) and its init_array counts, so a 1.1.1 run is no longer checked against the 1.2.1 values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 20:51:05 -05:00
chrg6
cb234fade1 With frame generation and no frame limit chosen, limit to the display's refresh rate.
Since the pacer divides the limit by frame generation's factor, the default limit of 60 held the game to 30, 20 or 15 rendered frames a second at 2x, 3x or 4x on any monitor, 15 at DLSS 4x on a 165 Hz one. While AMD's or NVIDIA's frame generation swapchain is in use and neither fps_limit nor the older fps60 = 0 is set, the limit is now the monitor's rate, as with fps_limit = display: 4x on 165 Hz draws about 41 and shows 165. A limit the player set is kept, and the menu's Frame rate limit row shows the display choice while this applies.

The mouse camera divided its input by the limit's frame ratio, which counts generated frames, so frame generation turned it faster by its factor; it now uses the rate the game draws at (frame_step_game_hz).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 12:18:17 -05:00