Commit Graph

51 Commits

Author SHA1 Message Date
chrg6
e6fdfde7c6 Report the build's unexpected errors with their message.
An exception make_game.py did not report itself only printed a traceback,
and the launcher showed "the build stopped (exit code 1)". main now sends
every such error through fail with the exception's message (the system's
text and the file for a file error, the full disk named as such), and the
traceback still goes to the log.

A dump whose data does not match its own checksums ended the RomFS step
that way. The step now names the damaged file and says to copy or dump
the game again, and it removes the partial image of a failed run.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:25:37 -05:00
chrg6
752c172ccf Say how long an update really takes, and fill the warm-up's bar from zero.
The Updates row promised that an update rebuilds the game "in a few
minutes". The code cannot keep that: a version that changes the lifter
compiles the game's code again, and one that changes what the shaders are
built with builds all of them into the driver's cache again. Players'
updates took 24, 34 and 45 minutes. The row now says it takes a few
minutes when little changed and half an hour or more otherwise.

The warm-up of the driver's cache after an update only builds, but its
count was drawn in the second half of the bar as in a full preparation,
so a 20 minute run read 50 % in its first second. Its count now fills the
whole bar, and the row says so while the shader cache is packed before it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:22:48 -05:00
chrg6
d27654ad9b Do not lift again because the RomFS step ran.
A step that runs drops the record of every step after it, and the RomFS
step comes before the lift although the lift reads only the unpacked
ExeFS. An install that was cut short while the image was still being
packed, the lift beside it already done, therefore lifted the whole game
again at the next try. The RomFS step now leaves the lift's record; the
compile and the shaders, which read the image, still follow it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:19:37 -05:00
chrg6
e732173bea Keep the finished objects when the compile is cut short.
compile_all wrote the object cache only when every file was done, so
Cancel or a crash in the middle of the compile lost the record of every
object and the next run compiled the whole game again. The cache now goes
to disk every 25 objects, at most every two seconds, moved into place
whole. Results are taken as the compiles end instead of in the order they
started: the largest file starts first, and in that order nothing was
recorded until it had finished.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:18:39 -05:00
chrg6
6e38458f77 Remove a cut RomFS image before measuring the free space.
A RomFS step that was cut short (Cancel, a full disk, a power cut) leaves
romfs.bin.part, up to 26 GB. The next run writes the image anew, but
make_game.py's step_romfs and the launcher's Begin Installation measured
the free space first, with that file counted as used: on a nearly full
disk the retry was refused although it needs no more room than the first
try. Both remove the file before they measure, and the first-run screen
counts its size as free.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 15:15:35 -05:00
chrg6
0e5e40b620 Merge branch 'claude/linux-old-headers' into test-all-fixes 2026-11-10 14:18:29 -05:00
chrg6
86c0fe4199 Let a game build on a network share get past a short dropout and a share that refuses private folders.
A player who keeps the game on a mapped network drive could not finish a build. Three things change in tools/make_game.py:

- The output folder keeps its drive letter on Windows. It was resolved to the share's UNC name, which is a second connection to the same server, and some servers drop that one under the compilers' load while the mapped drive stays up.
- When a source's size cannot be read because its folder stopped answering, the build says so and asks again for up to a minute before it stops with a message that names the drive. It used to end in a traceback. A file that is gone from a folder that answers still ends the build at once.
- A finished file is put in place through a temporary file beside it instead of a file in a temporary folder. Such a folder is made for its owner alone, and some shares refuse that while they take ordinary files; the build then reported a permissions fault in the output folder after everything had compiled.

From pull request #419: the first and third change are the contributor's, the wait is written again so that a file that is really gone does not wait a minute, and the launcher tests cover all three.

Co-Authored-By: Matthias <dev@sleepyyui.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 12:47:22 -05:00
chrg6
61cd4784f8 Build the Linux kit against pinned Vulkan headers and without #embed, and try the compiler before the long steps.
A player's AppImage build of 0.7.1 stopped in its compile step after eight minutes of RomFS work: four
files did not compile, because the kit builds with the system's compiler and headers and theirs were
older than the runtime needs. vk_pipeline.c and vk_device.c name VK_KHR_pipeline_binary,
VK_KHR_dynamic_rendering_local_read and VK_KHR_shader_maximal_reconvergence, which their Vulkan headers
did not declare, and ui/menu.cpp takes its font in with #embed, which clang has from version 19.

- Vulkan headers: tools/vulkan_pin.py holds the release the other builds use (1.4.363, the archive and
  hash tools/make_release_linux.py already pinned, which now reads them from there). The AppImage carries
  its C headers in toolchain/vulkan-headers and builds against them whatever the system has; a kit or a
  source checkout uses the system's when they are at least that release and fetches the pinned ones
  otherwise (build_game.py vulkan_include). Their folder goes first on the include path of the runtime's
  sources only, so the lifted code's objects keep their keys, and a runtime object compiled before
  against the system's headers is taken over instead of compiled again. AppRun no longer asks for the
  system's Vulkan headers.
- #embed: tools/build_shaders.py also writes the font's bytes as runtime/ui/roboto_ttf.h (ignored by git,
  carried by a kit), and menu.cpp includes that where the compiler has no __has_embed. With #embed
  nothing changes.
- Before its long steps a Linux build that will compile now checks every runtime source that needs no
  header of the install with the compile step's own command (clang -fsyntax-only) and links an empty
  program against the game's libraries (make_game.py preflight): an old compiler, an old header or a
  missing development package stops it in the first seconds with the compiler's own errors. A failed
  compile names every file that failed, not only the three whose errors it prints.

Checked on Windows: tests/launcher_test.py passes with four new tests (117); the header of bytes and
#embed give the same 162,588 bytes, and both branches of menu.cpp compile; the early check passes on the
Windows file set with clang 23 and stops on files made to fail; the 34 local headers of release 363 are
the upstream tag's files by git blob id. Not run: anything on Linux, the pinned archive's download, a
compiler older than 23.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 12:42:17 -05:00
chrg6
f747c4e43d Skip the runtime shader step in make_game.py when the release does not carry build_shaders.py, so a player's update compiles again.
The launcher release carries the runtime's shader headers built and none of the tools that build them, and
make_game.py's compile step imported build_shaders unconditionally: every update to the 0.7.0 build stopped at
"Compile Changed Files" with ModuleNotFoundError. The import is now tried, and without it the step that rebuilds stale
headers is left out, as its comment intended. A launcher test checks that every tool make_game.py imports is in the
launcher release unless the import is guarded or only runs on Linux.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 07:44:54 -05:00
chrg6
75b0dca907 Add Steam Deck button prompts beside the Xbox and PlayStation ones (prompts = steamdeck).
The Controls tab's Button prompts row and nx2navi.ini take steamdeck: the Deck's own labels in the game's
text, action prompt and ocarina notes, which are A B X Y where an Xbox pad has them, L1 R1 L2 R2 and L3 R3,
Menu and View. The style is drawn by tools/pad_prompts.py like the others, into romfs-patches-steamdeck
and romfs-patches-steamdeck-label, and every build with button art writes them.

Pull request #107 by loteran asked for PlayStation and Steam Deck prompts; it was closed, and the tree
took the PlayStation half in another form, drawn from button art instead of redrawn font glyphs. This is
the Steam Deck half in that form, written here: the art has no Deck style, so its entry in
tools/prompt_art.py names the Xbox pictures for the face buttons, Menu, View, the d-pad and the sticks
and the PlayStation pictures for the shoulders and stick clicks. It needs four pictures of the Xbox set
that no other style used (the white letters on a light button, for the ocarina's notes); a release made
with --prompt-art copies them with the rest.

The Windows files compile and tests/launcher_test.py passes, with the prompts step's test following the
number of folders (seven now); every picture the style names is in the art used for releases. The style
was not drawn here (that needs the game's font), and --probe-cfg, which has a new check, was not run.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 06:53:30 -05:00
chrg6
bc4ea7574f Read the processor's level from /proc/cpuinfo on Linux when clang's own detection finds none.
Where a sandbox or a compiler wrapper hides the processor from clang -march=native, tools/make_game.py found
no x86-64 level and built the lifted code as generic x86-64. It now falls back to the features every
logical processor lists in /proc/cpuinfo and takes a level only when they prove it. cpu_tag reads the same
list through the new cpu_flags.

From pull request #295 by repparw, whose three build fixes came with its FSR 3.1 work. The other two were
in already (the processor in the compile key, the shaders step keyed by the translator in use), and the
wrapper case that prompted this one is also handled by the probe's own environment; this is the fallback
for what that does not reach. The pull request's main part, FSR 3.1 upscaling on Linux from vendored
sources, is not taken: 366 of its 494 vendored files are compiled SPIR-V shaders as C headers, which this
repository does not hold.

Checked on Windows with made-up cpuinfo files (v3 on every core, one core at v2, v2, no flags line);
tests/launcher_test.py passes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 06:48:09 -05:00
chrg6
32ee021905 Include NVIDIA's pinned DLSS library for Linux in the AppImage, and place it beside the built game.
On Linux the game looks for libnvidia-ngx-dlss.so.<version> beside the executable, and the driver brings
only NGX's core, so the DLSS modes did nothing in every Linux build. tools/dlss_pin.py pins the Linux
library of the DLSS version it pins for Windows, with its own size and SHA-256; tools/make_appimage.py
puts it in the kit's bin/ with its notice (downloaded from NVIDIA's DLSS SDK repository and checked, or
--dlss; --no-dlss leaves it out), and tools/make_game.py copies it beside the game on Linux as it copies
the DLL on Windows, keeping a library the player put there.

Pull request #297 by mooki000, taken as it is; the builder's description is merged with the button art
option that came with the release workflow. On Windows place_dlss does what it did. No NVIDIA file is in
the repository.

tests/launcher_test.py passes with its two new tests (112 tests, Python 3.12 on Windows); the AppImage
was not built here.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 06:38:02 -05:00
chrg6
32901bb337 Build the runtime's shader headers before make_game.py compiles, so a source checkout builds again.
tools/make_game.py has its own compile step and did not build runtime/*_spv.h, which the repository
no longer holds: from a source checkout (the Linux path of docs/README-BuildKit.md) every runtime file
that includes one failed to compile. A kit carries the headers built, so nothing is stale there and no
shader compiler is looked for.

Pull request #405 by mooki000, taken as it is. Not run on Linux here.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 06:16:58 -05:00
chrg6
fa29f3fea5 Merge the second PR intake pass: anisotropic filtering in the Screen tab, the X3D die choice, the NVIDIA shader cache limit, the NGX core search and other fixes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 03:32:46 -05:00
chrg6
2a9efee4c5 Merge GitHub main into the test line: the launcher's customisation, the settings and sound changes, and the warm stage-library cache.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 03:32:30 -05:00
DeepDaddyTTV
b4a898d47b Bound the RomFS packer’s parallel reads by file size. 2026-11-10 01:55:30 -05:00
epicmeme546
e10ef60fd5 Keep the driver's cache of stage libraries warm, link a missing pipeline instead of skipping its draw, and count the texture cache's waits in time.
The preparation builds every stage library once so the driver's disk cache holds it and a library a draw needs later takes under a millisecond. Turning dynamic vertex input off for AMD's RDNA 3 and newer driver changed the dynamic states every library lists, which gave each one another entry in the driver's cache, and nothing built them again: an update carried the prepared shaders over without the driver step. On an RX 7900 XT (26.10.1) all 42,142 prepared libraries were cold, 93 ms a vertex and 242 ms a fragment library on average. With no pipeline recorded, a walk through a village skipped 11,180 draws while their pipelines built and made 94 draws wait 5.5 s in all, four of them 1.0 to 2.0 s; a player's session log shows up to 114,834 draws skipped in 600 frames and 181 waits of 251 ms on average. Skipped ground and walls appear and disappear, and a skipped lighting pass is a black frame with the interface intact.

- The game checks the driver's cache at every start (vk_pipeline.c lw_start): a sample of the prepared fragment libraries is built, and when their median time says the cache is cold the whole prepared list is built into it again on idle-priority threads while the game runs. shader-warm.txt records a warm in progress, so the next start goes on with one a run ended, and stops the tries on a driver that does not keep the libraries. This also covers a driver update and a cache the driver emptied.
- An update that carries the prepared shaders over runs the game's --warm-shaders (tools/make_game.py warm_driver_cache): 11 s with the libraries there, about 8 minutes with all of them cold on that card.
- A draw whose pipeline is missing waits for its fast link from a budget of 6 ms a presented frame, 3 ms at most each (NX2NAVI_PIPE_LINK_MS), before it is skipped: with warm libraries the link is done within that.
- Full-screen draws wait from their 50 ms budget again instead of being skipped (NX2NAVI_PIPE_SKIP = any is the default; fullscreen stays selectable). With warm libraries the wait is a millisecond.

The same walk afterwards: libraries 0.3 and 0.7 ms on average, 1,605 draws skipped, 1,473 of them in the scene's first 600 frames and 8 to 80 in each 600 after, and 19 waits of 60 ms in all. First visits to four other saves (a mountain pass, the Lost Woods, Kokiri Forest, a river) waited 12 to 84 ms in all except the Lost Woods, where mesh shader pipelines, which have no stage libraries, and vertex libraries that need added outputs still compiled once: 23 waits, 2.1 s in all, the longest 776 ms.

The texture cache counted its waits in presents chosen at 60 a second: at 165 presents a second the 1800 that stood for thirty seconds were 11 s, so a released render target went sooner than meant and a texture out of view for 11 s was retired and showed its stand-in for some frames when it came back. The counts are now taken times the present rate over 60 (presents_for), and a texture is retired after two minutes without a sample instead of thirty seconds.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-10 01:36:29 -05:00
chrg6
ed3db71485 Keep the processor's level on an update over 0.5.0's leftovers, let Rebuild the Game retune, and lift again after the build cache was dropped.
An update decided the lifted code was 0.5.0's generic build from any old-format entry in objects.json, and 0.5.0's
runtime objects stay there for good, so 0.5.0 to 0.6.0 to 0.6.1 recompiled the whole game for generic x86-64. Only
the lifted objects' entries decide now, and entries of objects a build no longer makes are dropped. Rebuild the Game
could not undo it: the compile step's key names no level, so the step read as done before; it now runs when the
recorded level is not this processor's. With Keep Build Cache off the lift is deleted but its key still matched, and
the next compile found nothing to compile: the lift runs again when the compile step will. The two reads of the
compiler's version no longer fail on bytes outside the system's code page.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-09 21:54:31 -05:00
Michael Schienbein
4d41188d8b Size the lift's workers by free memory, hold off sleep and key the compile by the CPU on Linux.
On Linux the three host probes of the build did nothing. available_ram returned None, so the lift started one worker per CPU whatever the memory and could run the machine out of it; keep_awake did nothing, so the machine could sleep in the middle of a build of 10 to 25 minutes; cpu_tag returned 'any', so the compile step's key ignored the processor and a game folder moved to a CPU without AVX2 kept its x86-64-v3 objects.

available_ram now reads MemAvailable from /proc/meminfo and, when the process runs under a cgroup v2 memory limit (its own or a parent's), caps it by memory.max less memory.current. keep_awake starts systemd-inhibit with a lock on sleep (not idle: the screen may still blank and lock, as with ES_SYSTEM_REQUIRED on Windows) around a cat that reads a pipe this process holds open, so the lock ends with the process even when it is killed; without systemd-inhibit it does nothing. cpu_tag reads the flags of /proc/cpuinfo and returns the digits the Windows branch returns for SSE4.2, AVX2 and AVX-512F, counting a feature only when every logical processor has it.

The compile key now holds the real tag instead of 'any', so an existing Linux build compiles once more. The compile --jobs defaults are unchanged. tests/host_probes_test.py covers the three probes over fake /proc and cgroup files and a fake systemd-inhibit.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 20:51:08 -05:00
Chris Toph
6dd69eadbf Build the runtime library and refresh installed tools correctly on Linux.
Use the native toolchain and archiver for runtime libraries, accept VMA_INCLUDE, and report missing Linux tools before compiling. Read the host CPU through compiler wrappers without changing the environment used by real compiles.

Compare copied tools by content rather than normalized timestamps, then replace them atomically so read-only POSIX installs can update without changing executable modes. Cover equal-metadata copies and translator refresh during shader preparation, and correct the patch-application comment.
2026-11-09 20:51:07 -05:00
chrg6
52cf976857 Merge the 0.6.1 fixes from the 0.6.1.x test builds.
This brings in the test branch up to the shader reads, not the texture eviction by disuse still in progress there:
- black frames: a released or taken-back texture keeps its content, and a new texture over a live one of another size gets the region both cover;
- the menu font atlas uploaded through the runtime's checked upload batch;
- the game held to 240 frames a second, which stops the quick menu's crash;
- the FSR 3.1 contexts evicted instead of turning FSR off after a few render scales;
- AMD frame generation kept off while NVIDIA's holds the device;
- less device memory held that only costs the system's commit;
- shaders read from the disk cache off the render thread past a small budget;
- the exit code 70 text;
- pull requests #260, #261, #270 and #271.
The tool lists in make_kit.py and make_launcher.py and the setting names in cfg.c take both sides: romfs_check.py and pack_shader_cache.py, FOG_CULL, SCENE_SHARE and GPA_TWIN_MAX_KB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 14:39:51 -05:00
chrg6
87f51f94a9 Read shaders from the disk cache off the render thread past a small budget, and pack the cache on carried updates.
A profiled traversal session had 9 of its 16 worst hitches (100 to 500 ms) inside the render thread's shader lookup,
reading <key>.res.json and <key>.spv of shaders met for the first time: opening a loose file costs about 3 ms the first
time on Windows. get_async_one now reads in place while those reads stay within 1/8 of the wall time (2 ms at once);
past that three loader threads read them and the draws wait as for a translation. The perf line counts both.

The launcher's updates carry the shaders over and never packed the cache, which held 44,000 loose translations in the
test install. An update now packs it once it holds 1,000 loose ones, and the pack tool reads the loose files on 16
threads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 14:34:45 -05:00
epicmeme546
91003c3227 Check every player's game files during setup, and stand in for damaged textures.
A dump can hold files whose first pages are zeros, with a container checksum that passes because the damage was packed along with them. The game then opens such a file over and over and gives up on whatever needs it: one damaged normal map left the Great Fairy Fountain without walls.

The game build's new last step, tools/romfs_check.py, decompresses every file of the game data once, in worker threads and never while the game runs, and writes romfs-check.json beside the game. A damaged texture gets a stand-in, an undamaged texture of the same kind renamed to it, in an ordinary mod folder the Mods menu can turn off, so the game can load what needs it. Other damaged files are reported, since only a clean dump restores them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 14:16:19 -05:00
Michael Schienbein
b6a5cc302c Key the shaders step by the translator the build actually uses.
The shaders step key hashed only the kit's bin/nx2navi_xlate.id, but the build takes the translator from the kit's bin/ or, in a source checkout, from build/xlate/ (xlate_source). From a source checkout, main() worked out the shaders key right after the compile step and stopped with a FileNotFoundError on bin/nx2navi_xlate.id before any shader was prepared.

The key now hashes the identity file beside the translator xlate_source finds, so a source checkout is keyed by build/xlate/nx2navi_xlate.id and a kit's key is unchanged. A build without a translator stops with the existing message that names tools/build_xlate.py. The prompts test kit gets a translator stand-in beside its identity file, and a new test covers a kit that has the translator only under build/xlate/.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-09 13:49:43 -05:00
Sebastian Choperena
6589476872 Pack the shader cache after the launcher prepares the shaders, as the installer did.
The launcher builds through tools/make_game.py, whose shaders step never ran tools/pack_shader_cache.py, so a
launcher install keeps about 175,000 loose files (4.9 GB) in shader-cache/. The game looks them up one by one at
every start and reads the use masks of 42,000 of them for 20 s after it. make_game.py now packs the cache after the
preparation; a failure is reported and the build goes on, since the game still reads loose files. The kit and the
launcher ship the tool.
2026-11-09 13:15:47 -05:00
itaskaev2
b4a876200b Link the game's manifest and icon as a .res written in Python when lld links, so no build path goes through windres.
windres (binutils) opens its files and starts its preprocessor through the system's ANSI code page. A build or user
folder whose name holds a character outside that code page (Victor with an accent on a Cyrillic Windows, a Japanese
user name, a folder named with a full-width colon) failed the Compile step with "Invalid argument" or "Cannot create
temporary file". make_game.py now writes runtime/nx2navi.rc's two resources as a .res file itself (resource_file) and
hands it to lld, which builds the resource section from it and opens paths as Unicode. The resources come out as
windres made them: the manifest as 24/1, the icon's images as RT_ICON 1..7 and their group as RT_GROUP_ICON 1, US
English. Without lld the windres path stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 15:45:50 +01:00
chrg6
4223e0aa5c Name the translator a shader cache was made with before an update replaces it, so the first update carries too.
A shader cache made before the game carried translations over has no identities.txt, so after the update that brings
the carrying the game could carry nothing, yet the update skipped the shader step whenever shaders had been prepared.
Before a new translator goes beside the game (the compile step, and the shader step a translator-only kit runs alone),
note_cache_translator writes identities.txt naming the translator still there, in the form runtime/gfx_shader.c
reads; a file already there is left alone. The shader step is skipped only when the cache names a translator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 10:42:36 -05:00
chrg6
2ce8451fe9 Keep the compiled game across updates: stable compile keys, objects of earlier versions taken over, shaders carried.
An update recompiled every file of the game's code whenever anything in its keys changed, though the compile was the
same. The compiler's id held the file time of clang.exe, so a tool update that put the same compiler back made every
object stale; it is now the version text and the size. The fast CLANG64 compiler, whose objects are UCRT64's byte for
byte, was part of the key through its id and --sysroot; it no longer is.
So that this change itself costs nothing, an object whose entry has the key an earlier version gave the same compile
is taken over (ObjectCache.adopt) when its sources and headers read as they did: 0.6.0's form, and the form of 0.5.0
and earlier, which held the kit's own folder (app/<version>) in its paths; a header in that folder is read in this kit.
Checked read-only against two real game folders: all 551 lifted objects of a 0.6.0 install and of a 0.3.10 install
match their form and would be taken over.
The objects of 0.5.0 and earlier were compiled for generic x86-64, so an update keeps compiling the lifted code for the
level the objects were compiled for (build-cache/obj/lifted-build.json, lifted_level); --retune, for Rebuild the
Game, compiles it for this processor's level. An update also leaves the shaders prepared before in use and lets the
game translate them again in the background (shader_carry, NX2NAVI_SHADER_CARRY), instead of a full preparation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 10:42:36 -05:00
chrg6
0101a87079 Raise the frame rate: fewer copies into the VRAM mirrors, earlier GPU starts, render target images reused.
From the local fixes package (fix 11), measured in Kokiri Forest at 4K on an RTX 4090 over PCIe 3.0: 40.5 to 62.6 fps
native and 45.6 to 74.9 fps with DLSS Quality, the mirror copies from 6.5 to 0.3 ms of GPU time a frame.
- Pages a VRAM mirror wrote back into guest memory are not copied up into it again: the mirror sync reads page stamps
  of their own that a mirror's write-back leaves alone (vk_gpu_written_back, decided at record time by
  vk_mem_mirrored). This replaces the lookup table NX2NAVI_WB_NO_REUP had, keeps its name, and is now on by default.
- The game's flushes of mirrored memory are not copied where the write watch covers the region, as every page the game
  wrote there is stamped already (47-57 MiB a frame); NX2NAVI_FLUSH_UPLOAD=1 copies them as before.
- Written storage ranges up to NX2NAVI_WB_MERGE_MAX (now 32 MiB) are merged in 4 MiB pieces, so a range is no longer
  bound to what one dispatch covers; the snapshot pool keeps its budget-scaled size with 128 slots.
- A submission's first part goes to the GPU at the first pass boundary while the GPU is idle (split_idle), long passes
  are split too (split_in_pass), and the part's time limit reads the clock less often.
- A render target that needs an image of its own takes the image of a released one with the same parameters, released
  two or more frames before (tex_reuse); memory requirements are kept per parameter set; a released entry another
  descriptor's entry replaced at its address is kept and taken back.
- The memory check reads a page's write-watch bit and stamp under the lock that keeps them together.
- The launcher reads the gamepad only while its own window is in front, and does not start a second copy of the game.
- make_game.py says why a Python package did not load, not only that it is missing.
The fix's per-pass GPU timing is left out: NX2NAVI_GPU_PASSES does the same in main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 09:55:25 -05:00
chrg6
f1779ec9e4 Refresh the translator beside the game before the shader step prepares shaders.
The compile step copies the kit's translator next to the game, but a kit that changes only the translator runs the
shader step alone (its step key holds the translator's identity), which then prepared with the old copy. From the
local fixes package (fix 9); its translator build guard is in main already.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 09:48:29 -05:00
chrg6
dfc5b90b59 Fix what the 0.6.0 review found, and ship DLSS Frame Generation in the launcher release.
The commit census of the perf trace walked the address space on the presenting thread outside mem_census, which
brought back the hitch mem_census was added to remove; it now runs only with mem_census = 1. DLSS's output pass at
the mode's share fell back to no pass at all when the scene was not at that share (the first presents, a dynamic
resolution step): after such a frame the first pass wider than the scene is taken again for 120 frames. The launcher's
nx2navi.ini editor no longer rewrites a file it cannot read, keeps the byte order mark and bytes that are not UTF-8,
reads an empty line as the game does, and leaves no temporary file behind. The build kit carries the two modules
build_game.py imports on Linux, the Linux steps list the right packages and Python, and stale comments and notes are
corrected (Discord on by default, two prebuild threads, Enhanced HDR gone).

DLSS Frame Generation needs nx2navi_dlssg.dll, NVIDIA Streamline's runtime and nvngx_dlssg.dll beside the game, which
no release carried, so the menu offered it and the game turned it off. tools/make_launcher.py --dlssg and --streamline
put them with their licences and a notice in the release's bin/dlssg/, and tools/make_game.py place_dlssg copies the
whole set beside the game. THIRD_PARTY_NOTICES.md has the row; the notes and README say RTX 40 and 50.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 09:10:36 -05:00
chrg6
7530ca5044 Build on Linux without libunwind, and ship build_game.py in the kit for the Linux build's VMA header.
The Linux build linked libunwind for a frame walk the runtime no longer uses: platform/linux.c walks the stack with
libgcc's unwinder. make_game.py's Linux toolchain imports build_game.vma_include, so the kit carries that file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 08:22:06 -05:00
loteran
f3daf7e2b4 Build Ocarina natively on Linux with tools/make_game.py, and pack the kit as an AppImage.
make_game.py only knew MSYS2. On Linux it now does what tools/build_game.py's Linux path does:
- clang, clang++ and lld from PATH (checked at start), the system's headers and libraries, vk_mem_alloc.h from the
  system or the pinned copy build_game.py fetches into toolchain/vma;
- the runtime's *_sdl.* files and platform/linux.c in place of the Win32 ones; no shim.c (ui/menu_sdl.cpp defines
  menu_sdl_event), no windres, no DLLs to copy;
- -fdeclspec for C++, as third_party/ffx_api/ffx_api.h declares __declspec(dllexport) unconditionally;
- build_game.py's Linux link flags (-no-pie, noexecstack, rpath $ORIGIN, libstdc++ static) and libraries, plus
  dav1d (the AV1 movies), libatomic and libunwind (the previous commit's plat_frame_capture);
- ocarina and nx2navi_xlate without .exe; the translator from the kit's bin/ or a checkout's build/xlate.
Windows is unchanged: every difference is under os.name.

tools/make_appimage.py packs the kit (tools, recomp, runtime, the built translator, VMA) with a relocatable CPython 3.14
and numpy, lz4, zstandard and fonttools into NX2navi-Linux-<version>-x86_64.AppImage. Its AppRun checks for the
compiler and headers and names the packages for Arch, Fedora and Debian/Ubuntu, installs zbic (GPL-2.0, not bundled)
with pip into ~/.local/share/nx2navi/pydeps on the first run, picks --jobs from the memory, builds and writes play.sh
beside the game. README-BuildKit.md has a Linux section.

Tested on CachyOS (kernel 7.3, clang 23, RTX 3080, NVIDIA 620.70.06, 16 GB): make_game.py --nup ... --jobs 4 builds
ocarina in 9.9 minutes from a 1.1.1 dump, which passes --probe-cfg; a build with the same sources and flags
(compiled by hand before this script could) starts, loads a save and plays. With one compile per thread on 16 GB
the build filled memory and swap and the kernel stopped desktop processes, hence the --jobs advice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 08:21:36 -05:00
Tyler
a0bba31248 Write the linker map in the launcher's build so sampling profiles can be symbolised
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtayeqEG2x5woYzWuEQBGV
2026-11-09 08:16:43 -05:00
chrg6
8e30ad7927 Carry the button art in releases and draw the button prompts during the game build.
make_launcher.py and make_kit.py take --prompt-art (folders holding XGamepad, P5Gamepad and Keyboard_Mouse, or those
folders) and copy only the pictures pad_prompts.py reads into the release's prompt-art folder; tools/prompt_art.py
holds the art tables and that list, with no imaging packages. make_game.py's new last step, prompts, runs
pad_prompts.py on that art into the game folder's romfs-patches-* folders. Its key names everything it reads (the
RomFS, the art, the tools, the bind_ lines of nx2navi.ini, the folders being there), so it runs again only when one of
those changed, and it makes no other step run. A kit without art skips it. The launcher's Python gets fonttools and
pillow, and its jobs show a Draw the Button Prompts row when the version has art.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 08:09:26 -05:00
chrg6
7f25329e72 Stop saying that shader preparation keeps play from ever stopping to compile.
The preparation translates the game's shaders in their default bind state and builds their stage libraries and
compute pipelines into the driver's own cache. Variants met in play with another bind state are still translated then,
pipelines are still linked in play, and the driver cache is kept per executable and is lost when the driver clears
it. The launcher's step text, the README, make_game.py and docs/launcher.md now say it makes such stops far rarer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 08:09:26 -05:00
chrg6
ef08fed45f Ship NVIDIA's DLSS library in the setup and the build kit, and put it beside the game.
The release carried AMD's FSR 3.1 library but not nvngx_dlss.dll, so DLSS ran only where the NVIDIA driver supplied the model itself; elsewhere players had to fetch the DLL by hand. tools/make_launcher.py and tools/make_kit.py now download NVIDIA's own build from the DLSS SDK repository at a pinned tag (tools/dlss_pin.py: 310.9.1, checked against its size and SHA-256, cached), and put it in the release's bin/ with a notice naming the NVIDIA RTX SDKs License that covers it. --dlss FILE takes a copy on disk instead, --no-dlss leaves it out.

tools/make_game.py copies the DLL and its notice beside the game. A different nvngx_dlss.dll the player put there is kept; the one an earlier build placed (named by the notice's sha256 line) is replaced, so a newer release can bring a newer DLSS. The compile step's key covers make_game.py and the kit version, so existing installs get the DLL with their next update.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 08:09:26 -05:00
Claude
b1421faaf9 Show Prepare the Shaders moving in the launcher: the game prints its counts and the step sends them on.
The step sat at 0 % until it was done, an hour on some PCs, because nothing it ran reported progress to the launcher. nx2navi.exe --prepare-shaders now prints how many shaders it has translated and how many it has built into the driver's cache every 2 s. make_game.py reads them as they come and fills the bar, the translation the first half and the build the second, with the counts in the step's text. An event every 10 s keeps the text current while nothing is printed, and after 3 minutes without a count moving it says so, and that the step is still running.

The selftest draws the step as a tenth screen. Its made-up update was version 0.3.11, so from 0.4.0 on the update-ready screen showed Up to Date and was no longer drawn; it is now a minor version past the launcher running it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BSHQpLnScF9sNv4LNAaaS3
2026-11-09 08:32:13 +00:00
chrg6
215c423d84 Write make_game.py's reset state at once when the game files changed.
A new input reset the steps in memory only; the next save came after the first step finished, while the exefs step
rewrites <out>/exefs in place. A run stopped there left the old state.json saying the old game's steps were done,
and --rebuild then trusted the half-replaced files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 15:51:48 -05:00
chrg6
35ff999958 Rebuild from extracted game folders whose files changed, not only when their count did.
With --exefs and --romfs folders, the identity the exefs and romfs steps are keyed on was the two paths and the
number of RomFS files: a RomFS extracted again or edited with the same number of files, or a replaced sdk or
subsdk0, printed "done before" and the build kept the old unpacked files. Each file's size and modification time
is now part of it. A build from such folders unpacks once more after this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 15:51:48 -05:00
chrg6
a24a21bff5 Write make_game.py's state.json whole or not at all.
A write cut by a full disk or by Cancel ending the build left a state.json that loads as {}: the launcher then took
the built game for none, showed the first run and needed the dump again for a rebuild. It is now written to a
temporary file and moved into place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 15:51:48 -05:00
chrg6
f57a327a86 Keep saves safe in Import Saves and Quit, draw no traced game art, and recompile the game for a new processor.
Import Saves: the saves folder picked in the dialog imports what waits in it (it had made the folder's README the
save); the save backed up is the one the game would load, in runtime/fs_host.c save_recover's order, so an interrupted
commit's only copy is kept, and the rest of the commit journal moves into the backup instead of being deleted; the
imported save goes into place before the old one is deleted, which comes back if that fails; another NX2navi install's
finished commit counts as its save. Tests cover each.

Quit waits while the game writes a save, as Restart does: the game runs on under the menu, and closing the window
finishes only a commit that has begun. The VSync row says "in use: Immediate" when VSync On falls back to tearing, and
its default follows the device profile (cfg_default). Without the game's textures the menu's divider is a plain line
and diamond of its own, not a tracing of the game's ornament.

make_game.py: the compile key holds the processor's features (cpu_tag), so a game folder moved to a PC without AVX2
compiles again instead of crashing; the main thread gets 16 MiB of stack with GNU ld too; any error waits for the
RomFS image, so its .part file is not cut short. The launcher tests use a neutral title id; tools/menu_preview.cpp
has cfg_default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 03:57:02 -05:00
chrg6
8daedbd806 Redraw the in-game menu with the game's own interface textures, and bring in the launcher's save import and faster builds.
In-game menu: the settings window, its tabs, rows, buttons and cursor, the performance overlay and the Preparing
shaders box are drawn with the game's interface textures where the RomFS holds them (runtime/ui/menu_skin.cpp,
perf_panel.cpp; plain shapes until they are decoded). Restart joins Continue and Quit, each behind a confirmation;
VSync is an on/off switch; DLSS is listed only on NVIDIA GPUs; the menu sounds are quieter; Picture shape is gone and
the setting descriptions are shorter; edge cases of input, layout and fonts are fixed. tools/menu_preview.py draws the
menu without the game.

Launcher: Import Saves takes a save from the game folder's saves/ folder or one the player picks; downloads fall back
to Windows' own certificate check when Python's refuses a server; updates are faster; the title bar is drawn in the
window and Navi is every icon. tools/make_game.py builds with CLANG64's clang where the launcher installed it, packs
the RomFS beside the lift and the compile, takes a prebuilt runtime, and compiles the game's resources without cmd.exe,
so a user folder with a space in its name builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 03:57:01 -05:00
chrg6
96b6e04937 Compile the resources with windres when the install folder's path holds a space.
GNU windres runs its preprocessor through cmd.exe and, left to find it, puts the full path of the gcc beside it on that
command line. When that path and the .rc's both hold a space (a Windows user name such as "Joe Budson"), cmd.exe drops
the outer quotes and splits the command at the first space, so 0.5.0's Compile step failed with "'C:\Users\Joe' is not
recognized". make_game.py, build_game.py and make_launcher.py now name gcc by bare name (the toolchain's bin is first
on PATH) with windres's own preprocessor arguments, and make_game.py and make_launcher.py run windres in the .rc's
folder with relative names. The objects are byte-identical to before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 00:34:29 -05:00
Claude
c403547c8b Add the NX2navi launcher: a one-click setup that builds the game from the player's dump, starts it and keeps it up to date.
NX2navi-Setup.exe installs the launcher into %LOCALAPPDATA%\Programs\NX2navi without administrator rights, with
shortcuts and an Apps entry. The launcher finds the player's .nup, downloads the build tools itself (the MSYS2 UCRT64
packages toolchain.lock.json pins, checked by sha256 and unpacked without pacman, and zbic from PyPI) and runs
tools/make_game.py, whose steps it shows with their progress. Its window follows the game's options menu: colours
sampled from it, the menu's filigree, the white glowing cursor, the Triforce in the title band and as the programs'
icon, and all text in the game's Cetra Display, decoded from the player's own dump. Mouse, keyboard and an XInput
pad drive it.

Updates come from GitHub releases (NX2navi-update.json): the new version is downloaded, checked and staged beside
the running one, the stub restarts into it and falls back to the previous version when it fails to start, and the
game is rebuilt without the dump. make_game.py now keys its exefs and romfs steps on the game files alone, has
--rebuild and --events, compiles only the files whose source, headers, flags or compiler changed, and replaces
ocarina.exe only once the new one links. tools/make_launcher.py builds the release on Windows or Linux.

Tested: tests/launcher_test.py (19 tests), the real toolchain download and install, and under Wine 11 the setup,
the stub, the window on the bundled Windows Python, an update from a local feed by clicking Updates, the rollback
of a version that fails to start, and the downloaded clang compiling and linking a program against the game's
libraries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Merged onto the current port: make_game.py keeps the launcher's step keys, cached link and in-place swap, and its
own runtime manifest (windres), AMD's FidelityFX library and the shader preparation, which becomes a step of its own
("shaders") that the launcher shows in its install, rebuild and update jobs. make_launcher.py ships the shader tools
and, with --ffx, AMD's library.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 14:50:52 -05:00
chrg6
a42648e27a Add the game's native HDR, render at the monitor's real size, and refresh movie frames on the GPU.
HDR is now a menu row: Off, Enhanced (the runtime's own HDR tonemap) or Native. Native reports an HDR10 display to
the game with the settings' peak and paper white; the game then renders its own HDR picture and presents it as PQ in
A2B10G10R10, which goes to the HDR10 swapchain as it is. The peak, game, HUD and (for Enhanced) hue rows follow it.

The build kit's link left out runtime/nx2navi.rc, so the game had no manifest and no icon: without per-monitor DPI
awareness, a 5120x2160 monitor at 150 % gave a 3413x1440 window that Windows stretched. It is compiled with windres
and linked again.

The movie decoder's frames are copied into the game's buffer by the host; the back end is now told of that write, so
the GPU's copy of the buffer does not keep older or empty bytes (green stripes in NV12).

A longjmp out of the game's JPEG decoder now logs the guest functions that called it, to trace a later crash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 13:31:07 -05:00
chrg6
ca5e5eb5a4 Add AMD FSR 3.1 upscaling, and run DLSS's smaller modes on a 16:9 picture too.
The Upscaling choice gains FSR 3.1 Native AA, Quality, Balanced, Performance and Ultra Performance. FSR 3.1 runs
through AMD's signed amd_fidelityfx_vk.dll (loaded at run time; the build kit ships it) in DLSS's place, on the same
scene light, depth and camera motion vectors and at the same render scales. Its Vulkan back end needs
VK_KHR_get_memory_requirements2, which the device now enables when FSR 3.1 is chosen.

A half-size effect pass with three targets and a half-size depth was taken for a G-buffer, which switched every later
frame to the deferred frame shape: there the output pass of a smaller scene was never found, so DLSS Quality,
Balanced, Performance and Ultra Performance did nothing on a 16:9 picture. Such a pass smaller than the scene is now
left alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 13:31:06 -05:00
chrg6
fc6d84aa63 Prepare every one of the game's shaders when the build kit builds the game, so play does not stop to compile them.
The shader reader now reads Switch 2 (NVN2) programs, whose 0xe0-byte control sections and task and mesh stages the
runtime keys as it does, and runs on Python 3.12 through the zstandard package. The kit's new last step writes the
36,031 shaders of the player's game files as a seed and runs ocarina.exe --prepare-shaders, which translates them and
has the GPU driver build each one's library, about 5 minutes. On a cold walk into play this took the stalls from
14 seconds of pipeline builds and 9 of translations to none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 13:31:06 -05:00
chrg6
b48b05dacf Build kit: take each module from the .nup package that holds it whole, and keep the PC awake while it builds.
A .nup with the base game and its update has the update tried first; a module the update leaves out, or holds as
something other than an NSO, comes from the base. The build holds off sleep while it runs, since it runs for many
minutes without input.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 10:59:59 -05:00
chrg6
a4fde2090a Put DLSS back in the settings menu, and build the kit's runtime at -O2 without strict aliasing.
The Screen tab gets the DLSS switch and its model choice after the picture rows. There is no quality-mode row: this
build cannot change the game's scene size, so every mode would render the same. The description says that DLSS works
on each frame on its own for now and takes effect after a restart; the model row is greyed out while DLSS is off.

tools/make_game.py compiles with tools/build_game.py's aliasing rule: -fno-strict-aliasing everywhere, since the lifted
code reaches guest memory through casts of one address to several types, and the runtime at -O2 (the lifted code
stays at -O1). x86-64-v3 is left out so that the kit runs on CPUs without AVX2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 09:26:51 -05:00