tools/make_game.py, the build players run through the launcher, read no profile: only tools/build_game.py did, which
does not build 1.1.1. A kit with data/pgo-v111.profdata (build/pgo/ in a checkout) now compiles the lifted code at -O2
with it and the runtime too, the prebuilt runtime library included. The profile is left out, and every command and
key is what it was, when the kit has none, when its record names another recomp/ than the kit's, when a compiler of
the build refuses the file in one trial compile before the others start, and with NX2NAVI_PGO=off. The record holds
recomp/ by content, which a kit without git can compare. The object keys hold the profile's content and not the kit's
folder, so a launcher update that moves the kit keeps the objects.
--pgo gen (NX2NAVI_PGO=gen from a launcher) builds the instrumented game into the game folder: it counts into
<out>/pgo, records what it was built from for the merge, and stops at once when the build tools lack the profile
runtime. tools/pgo.py merge takes that record, refuses a game built from another recomp/ and finds llvm-profdata in an
MSYS2 folder too. tools/check_content.py refuses profiles in a commit.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The settings had a tab of their own; they are rows of the Game tab now, listed under the switch once the mode is on after a restart, and the tab is gone. With the mode off every one of them reads as unset, whatever nx2navi.ini or the environment says, so nothing of the mode acts unless a player turned it on; the build reads guest_opt the same way. The notes describe that, and take the white edges in towns and the dropped textures from the known issues to the fixes.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The three change what every card does: the asynchronous replay can show a dark or flickering frame now and then, the hot-pool rule moves pools into system memory on every discrete card, and -O2 makes each rebuild of the game several times longer. Their rows stay in the settings tab. The USB joystick mapping is described by the device's own name, and the tab's introduction says what its settings are for.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Conflicts: the settings key lists take both sides; frame generation keeps the camera-cut handling and logs its counters every 600 frames with the cuts among them; the pass-time mark moves into draw_viewport; mo_desc keeps the pose depth and notes the reader.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
After Keep Build Cache removed the generated sources and the kit was updated, the lift ran because layout.json was
gone while its key still matched. An interruption after layout.json was back left the saved state calling the lift
complete, and the next run compiled unfinished sources. The pull request's two commits (one a merge of main) are
applied here as their net change.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Released textures kept, top texture levels left out, hot pools in system memory, the heap limit, the upload batch
limit, the kept ASTC buffer, mesh emulation and its arena and task records (greyed where the GPU has mesh shaders),
the overwritten-texture fix, the vector index fix, -O2 game code, pass times and the frame dump key. Each key gets a
row in the settings table; make_game.py reads guest_opt from nx2navi.ini when the environment does not set it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the lifted code built for the CPU's level, a profile on an RX 580 PC (Haswell-class CPU) has two game threads
running it nearly all the time and the replay thread idle a quarter of the time: the game's own code bounds the frame.
The lifted code is built at -O1 for build time; the environment variable NX2NAVI_GUEST_OPT=2 builds it at -O2 for a
player who will wait for the longer build. Part of each object's key, so changing it rebuilds the lifted code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A NX2NAVI_PROFILE profile names Rips; tools/prof_report.py needs the link map to name them, and the launcher's build
wrote none. The link now writes the map and keeps its non-lifted lines (the runtime, libraries; the f_<hex> game
functions are hundreds of MiB of it) as nx2navi.map in the game folder. The link key changes once, so the next build
links again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mods, on the home screen, opens the game's mods folder, adds a downloaded mod from its .zip, .7z or .rar file or its
folder, switches each mod on or off, and sets the load order (mods_disabled and mods_order in nx2navi.ini, written as
the game writes them). Each mod's description says what it changes, which other mod changes the same files and which
of the two wins, and why the game leaves it off when it does.
Before the game starts, Play puts the mods folder in order (launcher/mods.py prepare): an archive in it is unpacked into
a folder of its name and kept, a mod whose files sit a folder or more down has them moved up, and a romfs, exefs or
cheats folder put straight into the mods folder becomes a mod of its own. An archive that would write outside its
folder is refused, and nothing is deleted.
The launcher's build (tools/make_game.py) left the mods' code cheats and ExeFS patch files out, so those mods stayed
off. The lift now turns them into patch sets first, as the setup does, with the files in the lift's key and
build-cache/mod-code.json recording what it took up. Play builds the game again before it starts when an enabled mod
brings code no build has seen. The shaders' key names the lift and the compile without the mods' code, so such a build
compiles only what the patches change and leaves the shaders prepared. tools/mod_files.py is the one place the tools,
the build and the launcher find a mod's parts.
Tested: a zipped mod holding its own folder, put into a mods folder, was unpacked and moved up by the launcher's code,
and the game served its file in place of the game's. The lifter took a code cheat and a pchtxt patch made for the game's
main as patch sets. The launcher's and the setup's tests pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brought onto the line with FSR 4, its frame generation and the debug menu. Where both sides had added to the same
place, both stay: the patch table keeps the pickup count and the free camera, the settings list keeps both sides'
names, the menu includes the co-op page and the debug menu, and the launcher builder takes the pipeline list and the
FSR 4 folder. Both sides had added the menu's typed-character event; one definition stays, with the window code that
passes characters outside the basic plane.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
runtime/vk_fg4.cpp makes the frame generation swapchain and context with the libraries FSR 4 comes with
(amd_fidelityfx_loader_vk.dll and amd_fidelityfx_framegeneration_vk.dll beside the executable). vk_fg.cpp takes
that path only with fsr4_fg = 1 while FSR 4 is the upscaler; the default stays the FSR 3.1 library. The loader
picks the version for the card: the machine-learned FSR Frame Generation 4 on Radeon RX 9000 cards with Windows 11,
which is untested, and 3.1.6 elsewhere. On an RX 7900 XT (driver 26.10.1) it lists frame generation 3.1.6 and the
swapchains 4.0.0, which the driver brings and the loader chooses, and 3.1.7. The swapchain in place is wanted only
while its library is the one to use, so a change of the setting or of the upscaler, or a failure of this path,
takes it down as switching frame generation off does and the next present makes it with the other library.
The preparation gives this library the camera's position and axes, which FSR Frame Generation 4 requires, and a
scene scale of one metre a unit: with the 0.01 the FSR 3.1 path uses, version 3.1.6 drew grey translucent shapes
into the sky of its frames while the camera turned (160 captures of the screen during a camera sweep; clean with 1).
Both paths now generate a frame only when the picture without the HUD is there for it: generated from the
presented picture alone, the HUD would be warped with the scene behind it. In four configurations (21:9 with either
library, a 16:9 picture in a 21:9 window, a 1080 picture without HDR) every generated frame had it, and the log
counts the frames without. On screen, of the 6,404 pixels of the HUD's item button that stayed fixed through a
camera sweep with the FSR 3.1 library, 6,162 stayed fixed with this one.
Standing in Hyrule Field at 3440x1440 with FSR 4 Quality the game draws 43.0 frames a second with this path and
42.2 with the FSR 3.1 library, each shown twice over. One run changed the present mode, HDR, fullscreen, frame
generation, FSR 4, this setting and the upscaler in turn: 216 screenshots without a bad frame, and no failure.
The SDK's frame generation headers join runtime/third_party/ffx_sdk, unmodified. tools/make_kit.py and
tools/make_launcher.py take the third library from the --ffx4 folder when it is there, and tools/make_game.py
copies it beside the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The prepared list named its shader variants and nothing about where they came from, so an update that keeps the prepared shaders (shader_carry) kept the old list whatever seed the new kit writes: variants only the new seed names, from a newer pipeline list or a change to which variants are prepared, were translated when play first drew them. The existing identities do not cover this: nx2navi_xlate.id and identities.txt name translators, which decide cache keys and what is carried over, not which variants are listed.
The list is now version 2 (runtime/prp_file.h): after the head come the sha256 of seed-programs.bin, the sha256 of seed-answers.bin and the translator identity, written by gfx_shader_prepare_seed; the three readers go through gfx_shader_prepared_read and still read a version 1 list. At an update step_shaders writes the seed first (prepare_shaders.py --no-run, about 4 s) and keeps the list only when its digests are that seed's (make_game.py seed_changed); another seed, or a list that does not say, is prepared again. Another translator alone still carries over: input ids do not depend on the translator, and preparing everything again for it is what carrying was made to avoid.
Checked with tests/prp_file_check.py (the C layout against make_game's reader), three launcher tests, and the exe's --prepare-shaders on a seed without records in a folder of its own: the digests it wrote are those of the seed files, one byte more in the seed reads as changed, and after preparing again it does not. A version 1 list of 42,142 variants and the same list as version 2 are both read whole by --probe-prewarm.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
runtime/vk_ffx4.cpp loads amd_fidelityfx_loader_vk.dll from beside the executable and asks it which upscaler
versions the device can run. Where a 4.x version is listed (Radeon RX 6000, RX 7000 and RX 9000 cards with AMD's
driver 26.10.1 or newer) the menu's Upscaling row gains AMD FSR 4 Native AA, Quality, Balanced, Performance and
Ultra Performance, and fsr4 = 1 runs it in FSR 3.1's place with the same inputs. Its contexts are made with that
version asked for by its id and the provider is asked afterwards which version it runs, so no other upscaler can
be shown under the name. When a context or a dispatch fails, FSR 3.1 makes the frame, and after three failures
FSR 4 is off for the run. Each upscaler starts its history anew on the frame after a change from one to the other.
The setting is off by default and the Linux build compiles stubs.
On an RX 7900 XT (driver 26.10.1, 3440x1440, HDR) the library runs FSR 4.1.2: a context takes 54 ms to make and
217 MiB of video memory. Standing in Hyrule Field: Quality 47.8 fps where FSR 3.1 gives 51.5, Native AA 41.4.
Screenshots of 601 consecutive frames standing, 801 during a camera sweep, 701 each in Kakariko and the
graveyard, and 1,301 across changes between FSR 3.1 and FSR 4, their modes and off show no flicker that FSR 3.1
does not have, and it runs together with FSR 3.1 frame generation.
The SDK's API headers are in runtime/third_party/ffx_sdk, unmodified, with the licence. tools/make_kit.py and
tools/make_launcher.py take AMD's two signed libraries with --ffx4, and tools/make_game.py copies them beside the
game when the kit has both.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Each player keeps their own game and save. One player makes a room, the others join it with its secret, and story flags and, by the room's rules, items, songs, quest items, small keys (a shared pool), rupees and ammunition reach every save. The layer is a host thread with one UDP socket to a relay the players run themselves (tools/coop_relay.py, Python's standard library only); the messages between the players are sealed with a key from the room's secret (XChaCha20-Poly1305, Monocypher 4.0.2 vendored with its licence and a row in THIRD_PARTY_NOTICES.md), every change is signed by the player who made it, a player accepts a room by its members' keys and the key of its rules, the save is copied to coop-backup before anything is applied, and values are set through the game's own routines between frames. The Co-op tab makes, joins, accepts, removes and leaves, and the menu takes typed text for the name, the relay and the secret. This is the contributor's pull request #518 on this line.
Nothing reaches the network while co-op is off: the socket is opened, and the relay's name looked up, only after co-op is on, a relay is set and a room is made or joined. There is no default relay.
Left out of the pull request: the other players drawn in the world (the "puppets") with the copied engine they need. They were unfinished, added two patch sites to the game's per-frame update that could not be put through the differential tests here, and their file helpers clashed with the ones this line already has. So no position is sent: the POS message's codec stays, nothing sends it.
Changed against the pull request:
- The menu's test hook drives the Co-op tab only when it comes from the environment. From nx2navi.ini it could turn co-op on, join a room and accept its offer.
- The log's label for a room is hashed from the secret instead of being its first three characters.
- A game that has not saved yet has no save to copy, and takes nothing from the room until it has saved once; that used to count as a copy made.
- A PING's token no longer carries the PC's uptime.
- On Windows the maker's key file and its folder get an access list with the player's account alone.
- The tests that link the platform layer get the socket library, and the save test a stand-in for the title screen's hook.
Checked on Windows, where the pull request had never been compiled: the runtime builds and links; --probe-coop passes (sockets on 127.0.0.1, the codec against its byte vectors, a fuzz of every parser, two sessions through a hostile mock relay, a relay restart, the sync engine); tests/coop_relay_test.py passes its 61 tests, with the probe through the real relay on 127.0.0.1 at 0 and 30 % loss; tests/coop_page_check.py passes and its deliberate fault fails 48 checks. Not checked: two games in one room (one game runs at a time here), any real network, the tab in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
make_game.py passed the map as -Wl,--Map=<game folder>/build-cache/ocarina.map, and -Wl, cuts its argument at every comma: with a comma in the folder's path the part after it reached the linker as an input file and an update ended with "the link failed" (issue #470). -Xlinker hands the path over as one argument. build_game.py already did so on Windows and now does on Linux too.
With MSYS2's clang and ld here, a link into a folder named "a, b" fails with the -Wl, form ("cannot find b/t.map") and writes the map with -Xlinker.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The shader step's progress event rounded its share to four decimals, and the launcher shows the whole percent below it: 42,140 of 42,142 shaders built read as 100% while a long driver compile was still running. The event now carries the share as computed, so the bar stays at 99% until the count is complete. The test feeds the last counts of a 100,000-shader build and fails with the rounding.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The launcher asked for 31 GB where the build's own check now asks for about 34, so a disk between the two passed the first check and was refused seconds later. Both say 34 GB now. The build no longer asks for the whole shader preparation's space when the game folder already holds translations. The texts said 10 to 25 minutes for an install that took players 27 to 41, and 5 GB for a build cache that measures 1.6. A build error's message goes into the launcher's log as one line.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
pad_prompts.py gave each packed game file to the decompressor in one
call, which returns one bytes object grown by copying. On made-up data
of the size of the interface texture archive, 69 MB packed and 244 MB
unpacked, that takes 21 s on a fast PC; players' builds waited 44 to 52 s
before the step's first folder, most of its minute. In 64 KiB pieces
joined once the same bytes take 0.3 s.
The prompts step's key held the digest of the three scripts' bytes, so
this edit would have drawn every install's prompts again, as two of the
last four releases did. It now holds a constant changed by hand, whose
value is the scripts' digest in 0.7.0 and 0.7.1, and a test fails when
the scripts' logic changes, to ask whether the constant must change.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Only the RomFS image's space was looked at, and only when that step
began: a disk that held the image but not the build cache or the shaders
stopped the build in the compile or, half an hour in, in the shader
preparation. The build now adds up what the steps it will run write into
the game folder (the image and 1 GiB, 2 GiB for the build cache, 4 GiB
for a full shader preparation, the least each is known to need) and
stops at once when the disk has less, naming each part and how much is
missing.
On Windows a build whose compile step will run first compiles and links
an empty program with the installed tools, against the libraries the
game links. Tools that are incomplete or blocked stopped the build only
in its compile step, after the image and the lift; the message says what
failed and how to get the tools back.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
romfs_check.py gave the decompressor 4 MiB of a file at a time. Each call
returns its output as one bytes object grown by copying, under the
interpreter's lock: on 32 MiB frames that ran at about 120 MB/s of packed
data on twelve threads, and at about 2,500 MB/s with 64 KiB pieces; on a
1 MiB file at 490 against 3,000. Files smaller than a piece are as fast
as before. The check now stops at the frame's end, so what it says of a
file does not depend on the piece size; a test compares both sizes and
the check as it was on damaged, cut and whole frames.
The scan step's key held the sha256 of the script's bytes, so this edit
would have made every install check its 26 GB again. The key now holds a
constant changed by hand, whose value is the sha256 the script had in
0.7.0 and 0.7.1: the key of every install that passed stays what it is.
A test fails when the script's logic changes, to ask whether the constant
must change.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
modules_c wrote the paths of the game folder as C strings with JSON's
default escapes, which turn a character outside the Basic Multilingual
Plane into two surrogate escapes. C refuses those ("invalid universal
character"), so a game folder with an emoji or a rare Han character in
its path failed at the end of the compile. The characters now stand as
themselves in the UTF-8 source, and a test compiles the generated file.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
make_game.py ran one compile per logical processor whatever the memory.
Without --jobs it now runs as many as 80 % of the free memory holds at
768 MiB a compile, the peak build_game.py records for one, never more
than the processors and never fewer than one, on Windows and Linux alike.
The build's first lines say the number and what decided it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
An exception make_game.py did not report itself only printed a traceback,
and the launcher showed "the build stopped (exit code 1)". main now sends
every such error through fail with the exception's message (the system's
text and the file for a file error, the full disk named as such), and the
traceback still goes to the log.
A dump whose data does not match its own checksums ended the RomFS step
that way. The step now names the damaged file and says to copy or dump
the game again, and it removes the partial image of a failed run.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The Updates row promised that an update rebuilds the game "in a few
minutes". The code cannot keep that: a version that changes the lifter
compiles the game's code again, and one that changes what the shaders are
built with builds all of them into the driver's cache again. Players'
updates took 24, 34 and 45 minutes. The row now says it takes a few
minutes when little changed and half an hour or more otherwise.
The warm-up of the driver's cache after an update only builds, but its
count was drawn in the second half of the bar as in a full preparation,
so a 20 minute run read 50 % in its first second. Its count now fills the
whole bar, and the row says so while the shader cache is packed before it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A step that runs drops the record of every step after it, and the RomFS
step comes before the lift although the lift reads only the unpacked
ExeFS. An install that was cut short while the image was still being
packed, the lift beside it already done, therefore lifted the whole game
again at the next try. The RomFS step now leaves the lift's record; the
compile and the shaders, which read the image, still follow it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
compile_all wrote the object cache only when every file was done, so
Cancel or a crash in the middle of the compile lost the record of every
object and the next run compiled the whole game again. The cache now goes
to disk every 25 objects, at most every two seconds, moved into place
whole. Results are taken as the compiles end instead of in the order they
started: the largest file starts first, and in that order nothing was
recorded until it had finished.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A RomFS step that was cut short (Cancel, a full disk, a power cut) leaves
romfs.bin.part, up to 26 GB. The next run writes the image anew, but
make_game.py's step_romfs and the launcher's Begin Installation measured
the free space first, with that file counted as used: on a nearly full
disk the retry was refused although it needs no more room than the first
try. Both remove the file before they measure, and the first-run screen
counts its size as free.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The carry path of an update does not run the shader preparation: it runs --warm-shaders, which builds the listed pipelines from the seed-pipeline-keys.bin the game folder already has. make_game.py now writes that file from the kit's list first, so a newer list reaches an updated install. Its shader variants are not translated on this path; a record whose variant the folder has neither prepared nor met is left out and counted.
On a GPU that has no mesh shaders every mesh record failed with a log line each. They are counted in the step's line instead.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The preparation translates every shader and builds each one's stage library, so a pipeline a draw meets for the first time is linked in 2 to 6 ms. A mesh pipeline has no stage libraries: on an install that never drew it, it compiles whole in play, 0.2 to 1.1 s with its draws left out of the picture (18 of them on a first visit of Kokiri Forest). Whole pipelines were built ahead only from the folder's own pipeline-keys.bin.
tools/pipeline_list.py merge folds the pipeline-keys.bin and shader-answers.bin of game folders that were played into one list file: every key record once, with the answers record of each shader it names. A record names its shaders by input id, which depends on the game's code and the answers alone, and holds the game's pipeline state; what depends on the GPU is derived when the pipeline is built, so a record made on one machine is good on another with the same game version and runtime. Four folders on this PC give 5,432 records, 317 of them mesh pipelines (147 KB of keys).
tools/prepare_shaders.py reads the list (tools/data/pipeline_list.bin, data/pipeline_list.bin in a release folder, or --pipeline-list), adds its shader variants to the seed and writes its records as seed-pipeline-keys.bin. nx2navi.exe --prepare-shaders builds the mesh pipelines among them after the seed is translated and saves the pipeline cache before the stage libraries are built (NX2NAVI_PREPARE_PIPELINES=all builds the vertex pipelines too); --warm-shaders does it again after an update or a new driver, and --prepare-pipelines is that step alone. The pipelines go into the caches and are not kept or added to the folder's own key file, so they cost no start-up time or video memory until a draw meets them. A marker file keeps a driver that crashes on one of them from failing every preparation.
The list is derived from game data and stays out of the repository: check_content.py refuses files recorded from a game folder by their first bytes, and make_kit.py, make_launcher.py and make_release.py carry a list given with --pipeline-list. All three now also carry pipeline_list.py, which prepare_shaders.py imports.
Checked without the game: tests/pipeline_list_test.py, and prepare_shaders.py --no-run on the game files with a list of 317 mesh pipelines (all 266 shader variants found in this game version, 177 new to the seed). The build step in nx2navi.exe compiles and links and has not been run.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A player who keeps the game on a mapped network drive could not finish a build. Three things change in tools/make_game.py:
- The output folder keeps its drive letter on Windows. It was resolved to the share's UNC name, which is a second connection to the same server, and some servers drop that one under the compilers' load while the mapped drive stays up.
- When a source's size cannot be read because its folder stopped answering, the build says so and asks again for up to a minute before it stops with a message that names the drive. It used to end in a traceback. A file that is gone from a folder that answers still ends the build at once.
- A finished file is put in place through a temporary file beside it instead of a file in a temporary folder. Such a folder is made for its owner alone, and some shares refuse that while they take ordinary files; the build then reported a permissions fault in the output folder after everything had compiled.
From pull request #419: the first and third change are the contributor's, the wait is written again so that a file that is really gone does not wait a minute, and the launcher tests cover all three.
Co-Authored-By: Matthias <dev@sleepyyui.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A player's AppImage build of 0.7.1 stopped in its compile step after eight minutes of RomFS work: four
files did not compile, because the kit builds with the system's compiler and headers and theirs were
older than the runtime needs. vk_pipeline.c and vk_device.c name VK_KHR_pipeline_binary,
VK_KHR_dynamic_rendering_local_read and VK_KHR_shader_maximal_reconvergence, which their Vulkan headers
did not declare, and ui/menu.cpp takes its font in with #embed, which clang has from version 19.
- Vulkan headers: tools/vulkan_pin.py holds the release the other builds use (1.4.363, the archive and
hash tools/make_release_linux.py already pinned, which now reads them from there). The AppImage carries
its C headers in toolchain/vulkan-headers and builds against them whatever the system has; a kit or a
source checkout uses the system's when they are at least that release and fetches the pinned ones
otherwise (build_game.py vulkan_include). Their folder goes first on the include path of the runtime's
sources only, so the lifted code's objects keep their keys, and a runtime object compiled before
against the system's headers is taken over instead of compiled again. AppRun no longer asks for the
system's Vulkan headers.
- #embed: tools/build_shaders.py also writes the font's bytes as runtime/ui/roboto_ttf.h (ignored by git,
carried by a kit), and menu.cpp includes that where the compiler has no __has_embed. With #embed
nothing changes.
- Before its long steps a Linux build that will compile now checks every runtime source that needs no
header of the install with the compile step's own command (clang -fsyntax-only) and links an empty
program against the game's libraries (make_game.py preflight): an old compiler, an old header or a
missing development package stops it in the first seconds with the compiler's own errors. A failed
compile names every file that failed, not only the three whose errors it prints.
Checked on Windows: tests/launcher_test.py passes with four new tests (117); the header of bytes and
#embed give the same 162,588 bytes, and both branches of menu.cpp compile; the early check passes on the
Windows file set with clang 23 and stops on files made to fail; the 34 local headers of release 363 are
the upstream tag's files by git blob id. Not run: anything on Linux, the pinned archive's download, a
compiler older than 23.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The launcher release carries the runtime's shader headers built and none of the tools that build them, and
make_game.py's compile step imported build_shaders unconditionally: every update to the 0.7.0 build stopped at
"Compile Changed Files" with ModuleNotFoundError. The import is now tried, and without it the step that rebuilds stale
headers is left out, as its comment intended. A launcher test checks that every tool make_game.py imports is in the
launcher release unless the import is guarded or only runs on Linux.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Controls tab's Button prompts row and nx2navi.ini take steamdeck: the Deck's own labels in the game's
text, action prompt and ocarina notes, which are A B X Y where an Xbox pad has them, L1 R1 L2 R2 and L3 R3,
Menu and View. The style is drawn by tools/pad_prompts.py like the others, into romfs-patches-steamdeck
and romfs-patches-steamdeck-label, and every build with button art writes them.
Pull request #107 by loteran asked for PlayStation and Steam Deck prompts; it was closed, and the tree
took the PlayStation half in another form, drawn from button art instead of redrawn font glyphs. This is
the Steam Deck half in that form, written here: the art has no Deck style, so its entry in
tools/prompt_art.py names the Xbox pictures for the face buttons, Menu, View, the d-pad and the sticks
and the PlayStation pictures for the shoulders and stick clicks. It needs four pictures of the Xbox set
that no other style used (the white letters on a light button, for the ocarina's notes); a release made
with --prompt-art copies them with the rest.
The Windows files compile and tests/launcher_test.py passes, with the prompts step's test following the
number of folders (seven now); every picture the style names is in the art used for releases. The style
was not drawn here (that needs the game's font), and --probe-cfg, which has a new check, was not run.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Where a sandbox or a compiler wrapper hides the processor from clang -march=native, tools/make_game.py found
no x86-64 level and built the lifted code as generic x86-64. It now falls back to the features every
logical processor lists in /proc/cpuinfo and takes a level only when they prove it. cpu_tag reads the same
list through the new cpu_flags.
From pull request #295 by repparw, whose three build fixes came with its FSR 3.1 work. The other two were
in already (the processor in the compile key, the shaders step keyed by the translator in use), and the
wrapper case that prompted this one is also handled by the probe's own environment; this is the fallback
for what that does not reach. The pull request's main part, FSR 3.1 upscaling on Linux from vendored
sources, is not taken: 366 of its 494 vendored files are compiled SPIR-V shaders as C headers, which this
repository does not hold.
Checked on Windows with made-up cpuinfo files (v3 on every core, one core at v2, v2, no flags line);
tests/launcher_test.py passes.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
On Linux the game looks for libnvidia-ngx-dlss.so.<version> beside the executable, and the driver brings
only NGX's core, so the DLSS modes did nothing in every Linux build. tools/dlss_pin.py pins the Linux
library of the DLSS version it pins for Windows, with its own size and SHA-256; tools/make_appimage.py
puts it in the kit's bin/ with its notice (downloaded from NVIDIA's DLSS SDK repository and checked, or
--dlss; --no-dlss leaves it out), and tools/make_game.py copies it beside the game on Linux as it copies
the DLL on Windows, keeping a library the player put there.
Pull request #297 by mooki000, taken as it is; the builder's description is merged with the button art
option that came with the release workflow. On Windows place_dlss does what it did. No NVIDIA file is in
the repository.
tests/launcher_test.py passes with its two new tests (112 tests, Python 3.12 on Windows); the AppImage
was not built here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
tools/make_game.py has its own compile step and did not build runtime/*_spv.h, which the repository
no longer holds: from a source checkout (the Linux path of docs/README-BuildKit.md) every runtime file
that includes one failed to compile. A kit carries the headers built, so nothing is stale there and no
shader compiler is looked for.
Pull request #405 by mooki000, taken as it is. Not run on Linux here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The preparation builds every stage library once so the driver's disk cache holds it and a library a draw needs later takes under a millisecond. Turning dynamic vertex input off for AMD's RDNA 3 and newer driver changed the dynamic states every library lists, which gave each one another entry in the driver's cache, and nothing built them again: an update carried the prepared shaders over without the driver step. On an RX 7900 XT (26.10.1) all 42,142 prepared libraries were cold, 93 ms a vertex and 242 ms a fragment library on average. With no pipeline recorded, a walk through a village skipped 11,180 draws while their pipelines built and made 94 draws wait 5.5 s in all, four of them 1.0 to 2.0 s; a player's session log shows up to 114,834 draws skipped in 600 frames and 181 waits of 251 ms on average. Skipped ground and walls appear and disappear, and a skipped lighting pass is a black frame with the interface intact.
- The game checks the driver's cache at every start (vk_pipeline.c lw_start): a sample of the prepared fragment libraries is built, and when their median time says the cache is cold the whole prepared list is built into it again on idle-priority threads while the game runs. shader-warm.txt records a warm in progress, so the next start goes on with one a run ended, and stops the tries on a driver that does not keep the libraries. This also covers a driver update and a cache the driver emptied.
- An update that carries the prepared shaders over runs the game's --warm-shaders (tools/make_game.py warm_driver_cache): 11 s with the libraries there, about 8 minutes with all of them cold on that card.
- A draw whose pipeline is missing waits for its fast link from a budget of 6 ms a presented frame, 3 ms at most each (NX2NAVI_PIPE_LINK_MS), before it is skipped: with warm libraries the link is done within that.
- Full-screen draws wait from their 50 ms budget again instead of being skipped (NX2NAVI_PIPE_SKIP = any is the default; fullscreen stays selectable). With warm libraries the wait is a millisecond.
The same walk afterwards: libraries 0.3 and 0.7 ms on average, 1,605 draws skipped, 1,473 of them in the scene's first 600 frames and 8 to 80 in each 600 after, and 19 waits of 60 ms in all. First visits to four other saves (a mountain pass, the Lost Woods, Kokiri Forest, a river) waited 12 to 84 ms in all except the Lost Woods, where mesh shader pipelines, which have no stage libraries, and vertex libraries that need added outputs still compiled once: 23 waits, 2.1 s in all, the longest 776 ms.
The texture cache counted its waits in presents chosen at 60 a second: at 165 presents a second the 1800 that stood for thirty seconds were 11 s, so a released render target went sooner than meant and a texture out of view for 11 s was retired and showed its stand-in for some frames when it came back. The counts are now taken times the present rate over 60 (presents_for), and a texture is retired after two minutes without a sample instead of thirty seconds.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
An update decided the lifted code was 0.5.0's generic build from any old-format entry in objects.json, and 0.5.0's
runtime objects stay there for good, so 0.5.0 to 0.6.0 to 0.6.1 recompiled the whole game for generic x86-64. Only
the lifted objects' entries decide now, and entries of objects a build no longer makes are dropped. Rebuild the Game
could not undo it: the compile step's key names no level, so the step read as done before; it now runs when the
recorded level is not this processor's. With Keep Build Cache off the lift is deleted but its key still matched, and
the next compile found nothing to compile: the lift runs again when the compile step will. The two reads of the
compiler's version no longer fail on bytes outside the system's code page.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
On Linux the three host probes of the build did nothing. available_ram returned None, so the lift started one worker per CPU whatever the memory and could run the machine out of it; keep_awake did nothing, so the machine could sleep in the middle of a build of 10 to 25 minutes; cpu_tag returned 'any', so the compile step's key ignored the processor and a game folder moved to a CPU without AVX2 kept its x86-64-v3 objects.
available_ram now reads MemAvailable from /proc/meminfo and, when the process runs under a cgroup v2 memory limit (its own or a parent's), caps it by memory.max less memory.current. keep_awake starts systemd-inhibit with a lock on sleep (not idle: the screen may still blank and lock, as with ES_SYSTEM_REQUIRED on Windows) around a cat that reads a pipe this process holds open, so the lock ends with the process even when it is killed; without systemd-inhibit it does nothing. cpu_tag reads the flags of /proc/cpuinfo and returns the digits the Windows branch returns for SSE4.2, AVX2 and AVX-512F, counting a feature only when every logical processor has it.
The compile key now holds the real tag instead of 'any', so an existing Linux build compiles once more. The compile --jobs defaults are unchanged. tests/host_probes_test.py covers the three probes over fake /proc and cgroup files and a fake systemd-inhibit.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Use the native toolchain and archiver for runtime libraries, accept VMA_INCLUDE, and report missing Linux tools before compiling. Read the host CPU through compiler wrappers without changing the environment used by real compiles.
Compare copied tools by content rather than normalized timestamps, then replace them atomically so read-only POSIX installs can update without changing executable modes. Cover equal-metadata copies and translator refresh during shader preparation, and correct the patch-application comment.
This brings in the test branch up to the shader reads, not the texture eviction by disuse still in progress there:
- black frames: a released or taken-back texture keeps its content, and a new texture over a live one of another size gets the region both cover;
- the menu font atlas uploaded through the runtime's checked upload batch;
- the game held to 240 frames a second, which stops the quick menu's crash;
- the FSR 3.1 contexts evicted instead of turning FSR off after a few render scales;
- AMD frame generation kept off while NVIDIA's holds the device;
- less device memory held that only costs the system's commit;
- shaders read from the disk cache off the render thread past a small budget;
- the exit code 70 text;
- pull requests #260, #261, #270 and #271.
The tool lists in make_kit.py and make_launcher.py and the setting names in cfg.c take both sides: romfs_check.py and pack_shader_cache.py, FOG_CULL, SCENE_SHARE and GPA_TWIN_MAX_KB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A profiled traversal session had 9 of its 16 worst hitches (100 to 500 ms) inside the render thread's shader lookup,
reading <key>.res.json and <key>.spv of shaders met for the first time: opening a loose file costs about 3 ms the first
time on Windows. get_async_one now reads in place while those reads stay within 1/8 of the wall time (2 ms at once);
past that three loader threads read them and the draws wait as for a translation. The perf line counts both.
The launcher's updates carry the shaders over and never packed the cache, which held 44,000 loose translations in the
test install. An update now packs it once it holds 1,000 loose ones, and the pack tool reads the loose files on 16
threads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dump can hold files whose first pages are zeros, with a container checksum that passes because the damage was packed along with them. The game then opens such a file over and over and gives up on whatever needs it: one damaged normal map left the Great Fairy Fountain without walls.
The game build's new last step, tools/romfs_check.py, decompresses every file of the game data once, in worker threads and never while the game runs, and writes romfs-check.json beside the game. A damaged texture gets a stand-in, an undamaged texture of the same kind renamed to it, in an ordinary mod folder the Mods menu can turn off, so the game can load what needs it. Other damaged files are reported, since only a clean dump restores them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
