With the page file left to Windows the commit limit stays near the installed memory plus what the file held at the start, and on a 16 GB PC with a 16 GB card it ran low in Hyrule Field at 32 GB, where a fixed page file of 32 GB or more gave 49 GB and no low commit. The first "system commit low" line of a run is now followed, once, by a line that names the cause and the Windows setting, and the build kit's guide gives the same advice for 16 GB PCs. plat_page_file_managed reads the PagingFiles value once; on this PC (page file sized by Windows) it returns 1.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
After a lost device the driver's fault report gives an address and nothing says what was there. With va_report = 1 (NX2NAVI_VA_REPORT, off by default) the game keeps every address range the driver binds to a buffer or image and every range it unbinds (VK_EXT_device_address_binding_report, runtime/vk_vaddr.c). The report of a lost device then names the ranges that covered each faulting page, or the nearest ones when none did, and vaddr-history.txt beside the log holds the whole history, also when the driver gives no fault report. With 2 the call stack behind each range of 1 MiB or more is kept too. A player sees nothing of this unless the setting is on: without it no extension is enabled and no callback runs.
From pull request #349. Its second part, which stops a frame dump from turning on mesh query statistics, is left out: it changes what a dump writes for everyone who uses one. Built for Windows; no device was lost to try it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Every log line was two writes on the thread that logged it, to stderr and to the log file. Such a write can stall for
tens or hundreds of milliseconds (a busy disk, a virus scanner, a console or a pipe that is slow to take it), and the
render thread logs too: a frame it spent in a write was a hitch with nothing else to show for it.
In a game run the lines now go into a queue that a log thread writes out in order (runtime/logq.c). A fatal error,
the crash report and the end of the process write the queue out on their own thread first and write their own lines
directly, so nothing is lost to an exit. A thread that finds the queue's lock held for long, or the queue full, writes
its line itself. NX2NAVI_LOG_ASYNC=0 writes every line where it is logged, as before.
Every write is timed in both modes: a "log: ... writes of the log took over 2 ms" line counts the slow ones, gives
the longest and says whose time they took.
tests/logq_check.py checks order and completeness with eight threads, a slow destination, a stopped destination with
the queue full, and the synchronous flush.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
On Linux the write watch protects pages, so a write-back from shadow storage into guest memory took a fault
at the first store into every page. plat_mem_prepare_write clears the protection of a range with one
UFFDIO_WRITEPROTECT call; the pages count as written, as they would after the faults. Off by default;
Windows has nothing to clear.
Pull request #327 by theurbanninja1993-sketch (a draft on GitHub). The platform function and its Linux
test are taken as they are. The call moved: the write-back it was written for is now shadow_write_back in
runtime/vk_memory.c (a file other sessions are changing), where it covers the runs of pages a submission
changed, and the setting is registered in cfg.c and described in nx2navi.ini.example, which also says
that the lazy write-back stays off with shadow storage.
The Windows files compile; the Linux code and its test are not compiled or run here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
On Linux recorder_core reserved the last logical processor of the mask (an efficiency core on a hybrid
CPU) without its sibling, for the first thread to call vk_exec_record, and the first process-wide affinity
call took it back. plat_cpu_reserve there now takes the fast core that clocks highest with both of its
logical processors (none when fewer than four cores are fast), threads the holder starts get the other
processors, plat_process_affinity leaves the core with its holder, and a sweep every 300 presents moves
threads that inherited the core and gives it back once the holder has ended.
Shared code, so Windows changes too: the priority and the core go to the first thread that has spent half
a second in vk_exec_record (vk_exec.c rec_claim) instead of the first to call it, and the log names the
logical processors.
Pull request #358 by mooki000, taken as it is. The Windows files compile; nothing was run, and the Linux
code is not compiled here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Linux overcommits, so plat_commit_enforced was 0 there and nothing the runtime does when memory runs short
ever ran: no stale-texture drops, no idle or disuse eviction. plat_commit_info on Linux now reports the
limit with the least headroom of the cgroup v2 limits the process lives under (memory.max, memory.high,
against memory.current less its file cache), the machine's RAM (MemAvailable against MemTotal) and, with
vm.overcommit_memory = 2, the commit limit; a read is reused for 250 ms and the first one is logged. The
spare VRAM blocks stand down under low memory only where device memory counts against that limit
(plat_commit_counts_vram: Windows, and any integrated GPU). Windows behaves as before.
Pull request #378 by mooki000, taken as it is. The Windows files compile; the Linux code is not compiled
or run here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Windows places a game on the V-Cache die only when it recognizes the exe as a game, and it does not know this one, so the threads were spread over both dies by frequency preference. cache_die = 1 (the default) narrows the process to the logical processors that share the largest L3; it never widens the set the process was given, and does nothing where every L3 has the same size. The recorder's reserved core is then chosen inside that set.
Co-Authored-By: Claude Fable 5.2 <[EMAIL]>
Per-pass GPU timestamps put the write-back of GPU-written storage buffers at 3.6 to 5.3 ms of GPU time a frame in play, against 1.8 to 2.5 ms for all of the game's own passes and dispatches: about 67 MiB a frame goes from the mirrors back into guest memory, from the same eleven pools all session, in case the CPU or a host path reads it. Nothing said whether anything does.
With wb_probe = 1 the 2 MiB regions that lie wholly inside a range just written back are made inaccessible. The first read or write of such a region faults; the handler gives the region back the protection it had (read-only when frozen, else read-write), notes the address, the instruction and the thread, and the access runs again. The bytes are in guest memory all along, so nothing changes for the game. A log block every perf window gives the regions armed, the probes a read or a write ended, and the first accesses by instruction. Every thaw ends the probe of its region first, which covers file reads into guest memory and mapping changes; a file write from guest memory thaws its buffer while the probe is on; a probed region is not frozen anew.
The platform layer gets a hook for every read or write access fault beside the write-fault hook; it is installed only with the setting. --probe-gfx-abi checks that nothing is armed without it and, run with NX2NAVI_WB_PROBE=1, that reads and writes end a probe and run again, that a frozen region thaws after it and that mem_thaw ends and holds.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The last-chance exception filter is set at the start, before the GPU driver and the other DLLs load. A DLL that sets
its own filter later takes its place, and a fault in that DLL then ends the game with no FATAL line in the log. Two
0.6.0 logs from an RX 7800 XT end that way on the title screen. Once a second the filter is now put back and the
module that replaced it is logged (NX2NAVI_CRASH_FILTER_GUARD, default 1). When the game ends with a nonzero code,
the launcher adds that code, in hex with its meaning, to the end of logs\nx2navi.log, so a fail-fast exit that no
handler in the process can see still shows in the log a player sends.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every shader cache key names the translator, so a release with a new translator had no cached translation at all:
the launcher re-translated about 42,000 programs before play, and play waited for any program not prepared.
The cache now lists the translator identities it holds translations of in shader-cache/identities.txt, newest first.
A shader the current translator has no translation of uses a previous one's, looked up with that identity mixed with
this machine's host options and read with the usual checks; it keeps that translation for the rest of the session, so
no pipeline is built twice. After the boot prewarm a background pass translates every stored input that has a previous
translator's entry again under the current one, on its own translator child at idle priority that waits while the
game translates. A program the new translator refuses keeps its old translation and is logged. Once the pass has gone
through every such input, the old entries of what it covered are dropped: loose files are deleted, and a pruned copy
of shader-cache.pack replaces the pack at the next start when the pack is still the one it was copied from.
--prepare-shaders still translates everything under the current translator. NX2NAVI_SHADER_CARRY=0 keeps the earlier
behaviour.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each translation flushed its .spv and .res.json to the disk, wrote its .use mask with a flushed, written-through
rename and renamed its outputs written through: about six waits for the drive per shader. An NVMe drive does about
1,100 flushed files a second, a SATA SSD or a hard drive far fewer, so the translators sat mostly idle. Every read
already checks these files (spirv_complete, reflection_valid, the .use magic and size) and a torn one is translated
again, so they are now written whole but not flushed (plat_file_write_cached, plat_file_replace without durable).
The .in files stay flushed: they hold the guest's code and only their header and size are checked.
4,683 shaders of the install's preparation on a Ryzen 7 7700X and a Samsung 990 PRO: translation 36.4 and 37.2 s
before, 18.4 and 18.9 s after; every .spv, .res.json, .use and .in file and shader-prepared.bin are the same.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lock_time (NX2NAVI_LOCK_TIME) makes the replay threads time each lock they have to wait for, kept per asking site (platform.h plat_lock_timing, plat_mutex_lock_at and plat_rmutex_lock_at carry the site), and the frame-time summary lists the sites with the most wait, the recorder's full-ring and drain waits, and the texture lock's holds over 1 ms with the code that took it. replay_time now also keeps each frame's replay ops: a frame over 25 ms charged to the replay says how many draws, binds and transfers it replayed, their time, its slowest op, and how much of the replay's wall time the thread was on the CPU. NX2NAVI_PROFILE_REPLAY=1 samples only the replay thread without call chains, and the profiler now follows the thread that replays instead of the one that started the back end.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Performance: draws reuse the bindings that did not change and record only the dynamic states that differ from what
the command buffer holds; small uploads come from a staging arena; finished submissions are held briefly and handed to
the queue in batches; the recorder thread can keep a CPU core of its own; ASTC block-linear textures are deswizzled on
the GPU; dynamic resolution steps the upscaler mode to hold a frame-time target. Each has a setting in nx2navi.ini
(bind_reuse, dyn_delta, stg_arena, submit_batch, recorder_core, recorder_prio, astc_gpu_deswizzle, dyn_res) and the
staging cache now frees what it holds at shutdown.
Capture: NX2NAVI_DUMP=key saves the frame after each press of Insert, with mesh draws listed by task, mesh and fragment
shader, depth test and face culling, and mesh_prims.txt giving each mesh draw's primitive, clipping and fragment counts.
All of it is off unless NX2NAVI_DUMP is set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NX2NAVI_TRACE=audio logs voices starting, stopping and sharing a wave buffer; NX2NAVI_DUMP_MOVIE dumps movie frames; NX2NAVI_LOG_CS_BINDS says why the stand-in for an untranslated compute program skipped a dispatch; and NX2NAVI_TRACE=perf adds heap and commit census lines on their own thread, with VMA maps when write-combined memory grows by 2 GiB. From a tester's fixes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After an hour at 2160p the 2048 released entries held 5-7 GiB of render targets and decoded textures, charged to Windows' commit as well. NX2NAVI_TEX_RELEASED_MB caps them by size (default 1/16 of RAM, 1-4 GiB), and with less commit free than NX2NAVI_TEX_COMMIT_FREE_MB they are dropped to 256 MiB. Only entries released 120 frames ago or more go, at most 32 a check, oldest first. On Linux the commit check applies only with vm.overcommit_memory = 2 (plat_commit_enforced). From a tester's fixes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing kept a second copy of the game off the same profile (a double start, or the restart going ahead after 30 s
while the old process still ran). Its start took the first copy's commit in progress for an interrupted one and
deleted it, removed the first copy's working copy under its open files, and then both wrote into one save-tx/work,
so saves were lost or mixed. The save storage now takes <profile>/instance.lock (plat_lock_file: a file no other
process can open on Windows, flock on Linux) and a second copy stops with "another copy of the game is running".
--probe-fs gives the lock back before deleting its temporary profile and passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Opt the process out of Windows power throttling (execution speed and
timer resolution) at startup; NX2NAVI_POWER_FULL=0 turns it off.
- Fence completion thread wakes waiters before collecting freed memory,
so collection no longer delays them.
- Add a by-region barrier when a draw or clear writes an attachment that
an earlier draw in the same pass sampled; count it and inline uniform
updates in the per-frame stats.
- Hash small texture contents at upload and skip re-uploads whose bytes
are unchanged (removes ~420 redundant uploads per frame in Kokiri
Forest); NX2NAVI_TEX_SAME=0 turns it off.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Crashes: the render-target heap (HeapDef DynamicTextureAllocatorMem and its parent) grows with the widescreen
picture, so the pause menu no longer reads a null block at 5120x2160; texture uploads keep the game's memory mapped
while they copy it, so UnmapPhysicalMemory no longer pulls it away mid-copy; DLSS Frame Generation no longer crashes
in Streamline's set-up (physical devices and the window surface go through its proxies, its swapchain images are
counted first, resources are tagged per frame); a stale core-wait target is cleared before its thread is freed.
Freezes and races: a resize during a swapchain rebuild is kept, a failed frame generation swapchain is retried
instead of replaced by a plain one, three acquire timeouts in a row rebuild the swapchain, device waits take the
queue lock, a translator child that stops answering times out, an in-flight compute pipeline builds at normal
priority, a save Commit started before the window closes finishes and none starts after, a settings change just
before closing is saved, and a movie sample taken before a flush is dropped.
HDR: Enhanced HDR extends the game's own tonemap so highlights go above the picture's white (the hue slider,
which no longer applies, is gone from the menu); frame generation gets a HUD-less picture, the HDR flag and the HDR10
metadata under HDR too.
Memory: ASTC textures are encoded to BC3 unless the PC has 20 GiB of VRAM and 48 GiB of RAM, as Windows charges video
memory to the commit charge (a long session held 17 GiB of RGBA8 textures and 45.7 GiB of commit on a 32 GiB PC).
Shaders: the translator lowers spill-mode programs with more than eight uniform register groups in a wider spill
layout, and local loads and stores with a uniform-register address (36029 of the game's 36031 variants translate,
20 failed before).
The launcher release ships the shader tools and AMD's FidelityFX library (make_launcher.py --ffx); windres there gets
the toolchain on PATH.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README takes the name NX2navi as on GitHub and lists the new upscalers, frame generation, the HDR modes, the
higher resolutions and the shader preparation, and points to the build kit for installing. A crash calling address 0
now logs the return address on top of the stack and its module (it was inside sl.common.dll for Streamline).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each file that contains copied code names its copyright holder and license and points at THIRD_PARTY_NOTICES.md; zlib code keeps its full notice and the mark that it was altered. Comments and probe text only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The NVN2 front end and the SM86 to Maxwell shader lowering (tools/xlate/AmpereLowering.cs, mesh and task stages,
bindless texture pools, BRX jump tables, the NVN2 compute c0 layout) with four translator patches (0017-0020) bring the
opening of the game to 58-60 fps on the Vulkan renderer.
This round fixes the haze: SM86 IPA is lowered as a plain Maxwell IPA.PASS (texture coordinates were corrected for
perspective twice); LDSM, TXD, both F2IP byte forms, HMMA, MOVM and HFMA2.RELU are lowered and spill mode has eight
uniform register groups; conditional rendering keeps the newest counter report per address and treats a report as
pending until its value is in guest memory, so occlusion tests and the depth clears around the depth copy behave; and
the upscaler stand-in decodes the post pass's sRGB picture before writing the linear target. Tensor-core programs keep
the stand-in unless NX2NAVI_NATIVE_TENSOR=1, because their lowering does not match the hardware yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>