The table never removes entries and a session keeps meeting new buffer addresses: 12 to 15 more with every reload of a save. A longjmp to a replaced buffer ends the process as one to an unknown buffer always did.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A save that names such an area (a child save with HyruleCastle_Child on 1.1.1) makes the game read through a null area object right after the missing files; the report now says which area it was and that the save cannot be loaded.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The marker was deleted only when the boot's builds ended. A window closed before that left it, the next start read it as a start that died in the driver's compiler, and every later start on that GPU and driver skipped the prebuild and built each pipeline at its first draw.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A slot that resolved kept its place in the list until the next walk, and when it stopped resolving before that walk it was listed again. The list reached 160,000 entries for the 32,768 slots after a minute in the adult Market, and each walk over it held the texture lock for 15 to 30 ms.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Eleven findings are confirmed by a run or by tool output: the save that names an area this game version lacks, three device losses at a save load, the Market's stall from a pool list that gathers duplicates, the DLSS feature created on the replaying thread, and a pipeline prebuild that stays skipped after the game is closed during boot, among them. The other 44 are suspected from reading the code. Phase 3 was stopped before its last runs; section 12 lists them.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The synchronous replay's front end takes no lock of its own and relies on one such call at a time; a run now shows whether that holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The route reaches Kakariko Village, Hyrule Field by day and night, the Market, Death Mountain Trail and the graveyard edge from the available saves, every pause subscreen, and a save, return to the title and reload in one process; Kokiri Forest, a dungeon and a boss are not reachable. The ledger gains a confirmed crash on loading the test install's Kokiri Forest save and a confirmed UBSan report in the menu's grain texture; ASan cannot start because its shadow covers the guest's fixed windows, and TSan has no Windows target.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Four "mem: sizes" lines give the entries and bytes of what no other line counted: the guest heap, physical maps, kernel region, lazily written ranges and the fixed kernel, setjmp, save-file, RomFS and zstd tables; pool records, mirrors, deferred destructions, upload and staging buffers, the device address table and the texture, sampler and slot caches; image views, descriptor spares, the completion queue, uniform snapshot arenas, recording copies and pins; and pipelines, layouts, stage libraries, job queues, the shader memory cache and the translator processes. Each subsystem reads its own counters under its lock; settings now count the replaced values they keep. Segment and scene lines say "-" for the world clock before it starts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An input script line "<tick> MARK <name>" (or "@<ms> MARK <name>") ends the running segment with one line of its frame count, mean, 1% low and worst frame, the work marks on any thread, process memory and VRAM, then the 600-frame memory lines, and starts a segment of that name. RomFS opens of an area's static placement file give a "scene:" line, and once no area has loaded for 10 s the memory snapshot is taken with a "settled" line.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A slow frame's line and the 60-frame window's line now say how long each kind took on any thread, how often, how many bytes and the longest single one, so a hitch outside the render thread's own waits has a named cause. While NX2NAVI_TRACE=perf is off each site pays one branch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An indirect mesh draw reads its workgroup count through a ring entry that a buffer copy fills, and a copy cannot be
recorded inside a pass: every such draw ended its pass, 45 times a frame in a village. The first draw of a run now
fills the entries of the mesh draws that follow it with only state, draws, counter reports and render conditions
between them, and those draws stay in the pass (mesh_ahead, default 1; 0 is the old way, ab switches in A/B blocks).
An entry is taken only by the draw it was filled for, at the same record address, while no other op was replayed
since. Counter reports count as staying in the pass because the replay closes their query inside it and writes the
value from the completion thread; the game reports one after most of these draws, and without that only 38% of the
draws were served.
Measured in the village: 42 of the 45 draws a frame are served by an entry filled ahead; gaps with these copies
41 to 3 a frame (0.22 to 0.04 ms of GPU time); passes and dispatches 248 to 208 a frame (302 to 254 in Hyrule
Field). With the piece release of the commit before, one run each: village 142.6 fps and GPU busy 3.87 ms (138.0
and 135.4 fps, 4.15 ms without this change), Hyrule Field 136.8 fps and 4.34 ms (131.5 and 131.3, 4.70 ms). In A/B
blocks of one run with the narrower rule (counter reports still ending a run) the GPU's busy time went from 4.81 to
4.64 ms.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With the lazy write-back on, a village and Hyrule Field still merged 20 to 29 MiB a frame back into guest memory at
the end of every submission, inside the GPU work the game's fence waits for. The new log line (mem: lazy write-back
at frame N: written back and not armed since the last line) names the reason by bytes: none of it lay in frozen
regions or on pages that could not be armed; it lay in regions held after a touch. The game's CPU reads a few words
at the start of buffers its shaders fill (12 bytes into ranges of 6.3 and 15.2 MiB among others), about once a
second. Each read gave back the whole armed range and held every 2 MiB region it reached for 1 s, doubling to 32 s,
and nothing was armed in a held region.
A touch now gives back the 64 KiB around its address, fetched alone, and holds only that piece; the rest of the
range stays armed (wb_lazy_chunk, default 1; 0 is the old rule). More than four touches in one region within a
second are taken as a reader going through the range: that range goes whole and its regions are held as before,
and the region's touches go that way at once while it keeps being touched within a minute. Each of the first eight
touches of a log window is logged under the perf trace with its thread, address, range and what was given back.
Measured on one machine (NVIDIA, uncapped, frame generation off, scripted walks, medians over the windows of 46 s
of world clock):
- village: 121.8, 124.1 and 124.4 fps before; 138.0 and 135.4 with this change; GPU busy 4.8 to 4.15 ms a frame
- Hyrule Field: 101.1 and 113.1 fps before; 131.5 and 131.3 after; GPU busy 6.5 to 4.7 ms
- merged a frame: 20 to 29 MiB before, 1.7 to 2.0 MiB after
- faults on armed pages in 1,000 frames: 14 and 16 before, 17 to 18 after in the village; 19 and 23 before, 17 to
21 after in Hyrule Field; time spent fetching the same (4 to 5 ms in 1,000 frames)
A run with synthetic reads of four armed ranges per log window (wb_lazy_touch = 4, 769 faults) ended without an
error. The picture of every presented frame of a village walk and a field walk (thumbnails, 10,131 and 9,531
frames) had no black frame. The village walk has single frames whose brightness jumps: 14 with the change and 28
without it in about 10,000 frames each; the two looked at, one of each, are jumps of the camera.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The launcher asked for 31 GB where the build's own check now asks for about 34, so a disk between the two passed the first check and was refused seconds later. Both say 34 GB now. The build no longer asks for the whole shader preparation's space when the game folder already holds translations. The texts said 10 to 25 minutes for an install that took players 27 to 41, and 5 GB for a build cache that measures 1.6. A build error's message goes into the launcher's log as one line.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The test of a game folder outside the Basic Multilingual Plane compiled
the whole generated file with the first C compiler it found. The
runtime's headers need clang, so with gcc as that compiler the test
failed on them, not on the paths. It now compiles the four strings alone
with whatever compiler there is, and the whole file when that is clang.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
pad_prompts.py gave each packed game file to the decompressor in one
call, which returns one bytes object grown by copying. On made-up data
of the size of the interface texture archive, 69 MB packed and 244 MB
unpacked, that takes 21 s on a fast PC; players' builds waited 44 to 52 s
before the step's first folder, most of its minute. In 64 KiB pieces
joined once the same bytes take 0.3 s.
The prompts step's key held the digest of the three scripts' bytes, so
this edit would have drawn every install's prompts again, as two of the
last four releases did. It now holds a constant changed by hand, whose
value is the scripts' digest in 0.7.0 and 0.7.1, and a test fails when
the scripts' logic changes, to ask whether the constant must change.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Only the RomFS image's space was looked at, and only when that step
began: a disk that held the image but not the build cache or the shaders
stopped the build in the compile or, half an hour in, in the shader
preparation. The build now adds up what the steps it will run write into
the game folder (the image and 1 GiB, 2 GiB for the build cache, 4 GiB
for a full shader preparation, the least each is known to need) and
stops at once when the disk has less, naming each part and how much is
missing.
On Windows a build whose compile step will run first compiles and links
an empty program with the installed tools, against the libraries the
game links. Tools that are incomplete or blocked stopped the build only
in its compile step, after the image and the lift; the message says what
failed and how to get the tools back.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
romfs_check.py gave the decompressor 4 MiB of a file at a time. Each call
returns its output as one bytes object grown by copying, under the
interpreter's lock: on 32 MiB frames that ran at about 120 MB/s of packed
data on twelve threads, and at about 2,500 MB/s with 64 KiB pieces; on a
1 MiB file at 490 against 3,000. Files smaller than a piece are as fast
as before. The check now stops at the frame's end, so what it says of a
file does not depend on the piece size; a test compares both sizes and
the check as it was on damaged, cut and whole frames.
The scan step's key held the sha256 of the script's bytes, so this edit
would have made every install check its 26 GB again. The key now holds a
constant changed by hand, whose value is the sha256 the script had in
0.7.0 and 0.7.1: the key of every install that passed stays what it is.
A test fails when the script's logic changes, to ask whether the constant
must change.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
modules_c wrote the paths of the game folder as C strings with JSON's
default escapes, which turn a character outside the Basic Multilingual
Plane into two surrogate escapes. C refuses those ("invalid universal
character"), so a game folder with an emoji or a rare Han character in
its path failed at the end of the compile. The characters now stand as
themselves in the UTF-8 source, and a test compiles the generated file.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Every selection of the Updates row started a request to GitHub, which
allows an address 60 an hour: a player who selected it 25 times in 90
seconds had every check of the next hour refused. The row and F5 now
start no check within a minute of the last one.
A refused check said "try again in an hour" whatever the answer held. It
now names the time from the answer's X-RateLimit-Reset or Retry-After.
A failed check wrote two tracebacks, 22 lines, into launcher.log each
time; a FeedError is now one line with its message and its cause.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
build.Run read the build's output on a thread that ended on the first
exception of a line or event handler, such as the launcher's log on a
full disk. Nobody read the pipe after that, the build stopped at its next
line once the pipe was full, and the job waited for it for good. The
thread now counts the line as dropped and reads on, and Context.log loses
a line it cannot write instead of raising.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
That build kept the game's own decoder until its sites had been through a check run. The check has now run over a village and a field: 3,072 streams and 31.7 GiB of output compared byte for byte, none different. The game's decoder took about a fifth of the running time of the game's threads in a village and about a second of each area load. zstd_native = 0 keeps the game's decoder.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The build compiles on every processor for ten minutes and more while the
job screen invites the player to keep using the PC. build.Run now starts
make_game.py with Windows' below-normal priority class, which the
compilers and the other processes the build starts take over from it, so
the build gives way to what the player does and loses nothing on an idle
PC.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
make_game.py ran one compile per logical processor whatever the memory.
Without --jobs it now runs as many as 80 % of the free memory holds at
768 MiB a compile, the peak build_game.py records for one, never more
than the processors and never fewer than one, on Windows and Linux alike.
The build's first lines say the number and what decided it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The game exits with 70 after a FATAL line of its own and with 71 after
its crash report, and the launcher said only that the line near the end
of the game's log names the error. It now reads the last FATAL line of
the run from logs\nx2navi.log and puts it into the home screen's message
and into launcher.log, so a launcher log a player sends says what the
error was.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The check printed its counts into the build's lines only, so a player
whose build ended with "4 damaged, 3 given a stand-in, 1 left as they
are" saw nothing of it. A job now reads romfs-check.json after the build:
the count goes into the step's row, and when the check ran in that job, a
sentence goes onto the home screen and into launcher.log.
The same four files are flagged on every copy of the game, and three of
them get a stand-in. For those the sentence says so and that nothing
needs doing; other files are counted and romfs-check.json is named.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
An exception make_game.py did not report itself only printed a traceback,
and the launcher showed "the build stopped (exit code 1)". main now sends
every such error through fail with the exception's message (the system's
text and the file for a file error, the full disk named as such), and the
traceback still goes to the log.
A dump whose data does not match its own checksums ended the RomFS step
that way. The step now names the damaged file and says to copy or dump
the game again, and it removes the partial image of a failed run.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The Updates row promised that an update rebuilds the game "in a few
minutes". The code cannot keep that: a version that changes the lifter
compiles the game's code again, and one that changes what the shaders are
built with builds all of them into the driver's cache again. Players'
updates took 24, 34 and 45 minutes. The row now says it takes a few
minutes when little changed and half an hour or more otherwise.
The warm-up of the driver's cache after an update only builds, but its
count was drawn in the second half of the bar as in a full preparation,
so a 20 minute run read 50 % in its first second. Its count now fills the
whole bar, and the row says so while the shader cache is packed before it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The default was off only on AMD's driver branch for RDNA 1 and 2, on the grounds that builds with the default on had been measured on an RX 7900 XT. Those measurements were standing in Hyrule Field and walking through a village; the report this is about is of a game that ended in fights, on an AMD card the player did not name, and fights are where the lazy path is busiest. Nobody has played one on an AMD card under our eyes on either branch, so the exception is now AMD's own driver as a whole (wb_lazy_choice.h, by VkDriverId), until a card of each branch has played fights with wb_lazy = 1 and a log to show. The cost is the open-area frame rate of 0.7.0 on every AMD card under Windows; the RX 7900 XT figures in the draft notes come from builds that had it on by default, which the notes now say. wb_lazy = 1 still turns it on.
--probe-vkdevice prints the line a fatal error or a crash adds to the log, so its wording can be read without a failing run. On an RTX 5090: "fatal: gpu NVIDIA GeForce RTX 5090, driver NVIDIA "621.01" version 621.1.0.0; memory mode mirror, lazy write-back on (the default on Windows (wb_lazy = 0 turns it off)); texture detail not chosen yet (no texture made)", and with the reporting player's nx2navi.ini beside the exe "lazy write-back off (wb_lazy = 0 is set)", after a start whose log names async_compute, vram_safe_mode and exception_handler as no settings of this version.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A step that runs drops the record of every step after it, and the RomFS
step comes before the lift although the lift reads only the unpacked
ExeFS. An install that was cut short while the image was still being
packed, the lift beside it already done, therefore lifted the whole game
again at the next try. The RomFS step now leaves the lift's record; the
compile and the shaders, which read the image, still follow it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
compile_all wrote the object cache only when every file was done, so
Cancel or a crash in the middle of the compile lost the record of every
object and the next run compiled the whole game again. The cache now goes
to disk every 25 objects, at most every two seconds, moved into place
whole. Results are taken as the compiles end instead of in the order they
started: the largest file starts first, and in that order nothing was
recorded until it had finished.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A RomFS step that was cut short (Cancel, a full disk, a power cut) leaves
romfs.bin.part, up to 26 GB. The next run writes the image anew, but
make_game.py's step_romfs and the launcher's Begin Installation measured
the free space first, with that file counted as used: on a nearly full
disk the retry was refused although it needs no more room than the first
try. Both remove the file before they measure, and the first-run screen
counts its size as free.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
tests/wb_lazy_check.py took the compiler from tests/diff/common.py, which imports Unicorn and Capstone before anything else: on a PC without them the check stopped at the import, though it needs only a C compiler. It now picks the compiler as tests/gpa_check.py does.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A player wrote that the settings file is overwritten on updates and that a line has to be added again each time. Nothing an update of the launcher's install does writes the file: the updater works in the launcher's own folders, and the build only reads it (the bind_ lines for the button prompts, shader_carry for the shaders step). The tests now hold that with the player's file as reported: settings the menu wrote, three lines that are no settings of the game, a comment, CRLF line ends and no line end after the last line. It comes through the launcher's part of an update (staging, the switch, the cleanup; BackgroundUpdateTest), through the rebuild under a newer kit, a rebuild with nothing to do and a build from the game files again (MakeGameTest, with the check, the ExeFS, RomFS, shaders and prompts steps real and the lift and compile as stand-ins), and through the launcher's own settings switch, which adds its one line (PlayerIniTest).
One writer did write the whole file again: installer/ini.py, the older installer's last step at every install and update. It kept every line, but turned CRLF into LF, added a line end after the last line and dropped a byte order mark, and a file saved as UTF-16 stopped the step with an error. It now changes the lang line only and keeps every other byte; a UTF-16 file is read as that and written as UTF-8, as the game's menu writes it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A player on an AMD card reported crashes in fights and that a line added to nx2navi.ini cured them. That line, async_compute = 1, is no setting of the game, and neither are two more in the same file; the one line of the file that changes how memory reaches the GPU is wb_lazy = 0. A second report, an RX 6600 on 0.7.1, lost its device 3.5 minutes into play with the lazy write-back on. Neither proves it the cause: the player changed several lines at once and sent no log, the RX 6600's last pass mark is inside a draw pass and not a fetch, cards with 8 GB lost their device the same way on NVIDIA before the lazy write-back existed, and reading the fault handler, the fetch, the arming and the buffers' lifetimes found no failure path. What depends on the driver and cannot be checked without the card: pages of guest memory are inaccessible while a host-pointer import of them is bound and written by the GPU, and Vulkan calls run inside the access-fault handler.
The default is now decided once the driver is known (wb_lazy_choice.h, vk_memory.c vk_mem_lazy_decide): on under Windows as before, except on AMD's driver branch for RDNA 1 and 2, which none of us has run it on; AMD's driver for RDNA 3 and newer keeps it on, since builds with that default were measured on an RX 7900 XT. What the player sets wins everywhere, and one line at start says the choice and its reason. On those cards open areas run at the 0.7.0 frame rate again (on an RTX 5090 that was 65 fps against 90 in Hyrule Field; not measured on AMD). tests/wb_lazy_check.c checks the choice by platform, driver and setting without a GPU.
For reports of this kind the log now holds what was missing. At start: every setting nx2navi.ini sets, and in one line the keys that are no settings of this version and do nothing; the settings menu shows those in its bottom row. At a fatal error or a crash: one line with the card, its driver, the memory mode, the lazy write-back with its reason, armed ranges, fetches so far and a fetch in progress, and the texture detail (dispatch.c rt_fatal_note_add, vk_device.c fatal_note), since a report is often the end of a log alone.
--probe-cfg loads the player's file: the three lines are named, and the menu's save keeps them and every other line it does not change byte for byte. Nothing here ran on an AMD card, and no fatal error was provoked to read the new line.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
start.pyw showed the import failure in a message box before it ended with
code 86, and the stub waits six seconds for that code: unless the player
closed the box in time, the broken version stayed the current one. When
app/previous.txt names another complete release, the failure now goes to
crash.log only and the process ends at once, so the stub rolls back and
the previous version says on its home screen which version did not start.
The box stays when there is nothing to go back to, when Steam started the
launcher without the stub, and when the stub's wait is already over.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
An update whose lift ran again drops the compile step's record before the
compile runs, so a compile that then stopped left state.json without it,
build.built_version said '' and the launcher's next start showed the
first-run screen and asked for the dump. build.installed now also counts a
folder that holds the exe of the build before and game files a rebuild can
start from: the window opens on home, where Rebuild the Game reads "Not
Finished" and Play says which build it starts, and an update's rebuild and
the command line's go on with the game. Begin Installation into a folder
whose game files are already unpacked builds on from them instead of
reading the dump again.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>