Add the exact-material nonnegative distance guard and the recorded NVIDIA translation profiles. Keep generated shaders and the private native patcher out of the repository. Document reported cross-vendor testing and the unresolved origin of invalid absorption distance.
Carry clip coordinates through perspective interpolation for the identified water shader pair. Keep generated modules private and document the observed results and remaining portability limits.
A new install met every pipeline in play: in Hyrule Field one 600-frame stretch of a first visit had 1,769 draws wait 461 ms in all for pipelines, the longest frame 88 ms. The boot now reads seed-pipeline-keys.bin, the release's list, as recorded pipelines after the install's own (keys_open_seed, NX2NAVI_PIPE_SEED, default 1): their pairs go into the vertex output unions and the prewarm builds them as it builds the install's. The preparation builds every listed pipeline (NX2NAVI_PREPARE_PIPELINES now defaults to all), with the unions of the same list, so the first boot of a new install finds them in the caches. The list's mesh pipelines are left out at the boot on a GPU without mesh shaders, as the preparation leaves them out. Nothing of the list goes into pipeline-keys.bin.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SM86 lowering keeps the uniform registers in the general registers after the highest one the program uses
(HighestGpr), with one spare so pairs stay aligned. The scan read the register fields as single registers, so a
128-bit load or store whose upper registers the program names nowhere else (LDG, LDS, LDL and STG, STS, STL with size
6) left Rd+2 and Rd+3 to UR0 and UR1, and the load overwrote them. The scan now counts Rd..Rd+3 (Rd..Rd+1 for 64 bits)
of these forms. An overestimate only moves the uniform registers up.
Checked: no translation changes in the game's 520 task and mesh programs, its 4,238 NVN2 compute programs, or the
2,450 vertex and fragment programs whose stored input id starts with 0, all from an installed shader cache, and all
pass spirv-val (vulkan1.3): the case does not occur in what this game version has shown so far.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The loops the translator writes for narrow storage buffer stores and float atomic adds end only when their
compare-and-swap succeeds. With robust buffer access on, an atomic on a word outside its buffer may return anything,
so the compare can fail for ever and the GPU hangs until the device is lost, as reported right after the first
cutscene on an Intel Arc GPU (#610). New translator patch 0041 counts the attempts in a private word (a local cannot
count round a loop in that pass, which runs on SSA form) and leaves the loop after 4,096; contention on a word needs
about as many attempts as invocations race for it. Shared memory loops stay as they are: a shared word in range is
always settled, and the mesh programs run that loop for every primitive index byte.
Checked: all 4,238 NVN2 compute programs an installed shader cache stores translate and pass spirv-val (vulkan1.3); 45
contain a compare-and-swap, 11 of them in a storage loop, which grew by 256 bytes each and now compare the count with
4096; the other 34 are unchanged. tools/xlate_smoke.py passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An NVN2 compute program's control section gives its local memory in bytes a thread at +0x68, beside the shared memory
size at +0x64 that the job already carried. Without it the translator declared its default of 0x1000 bytes: one of the
game's compute programs needs 4,752 bytes a thread, and its translation declared a 1,024-word local array that it
indexed up to word 1,187, outside the array. The job now carries "local_mem" from +0x68, and the lowering starts a
compute program's spill area after its own local memory (at least 0x1000 bytes) instead of always at 0x1000, where a
larger program's own memory would have overlapped it.
Checked: that program's translation now declares 1,188 words (indices up to 1,187) and passes spirv-val; two programs
of 3,456 bytes a thread translate as before. All 266 compute programs whose stored input id starts with 0 in an
installed shader cache translate and pass spirv-val (vulkan1.3), and tools/xlate_smoke.py passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A mesh program's header lists the components of each output vector it has; the fragment stage may read the others.
The native mesh entry function never wrote them, so they read as whatever the output held, and the fallback's vertex
stage wrote 0 for all of them, w included. Maxwell gives an unwritten component (0, 0, 0, 1), as the pipeline code
already pads a missing vertex output. Both paths now write that default: MeshSpirv.cs after copying each vertex's lines,
mesh_fb.c meshfb_vs with w as the bits of 1.0f whatever the type.
Checked: all 520 task and mesh programs of an installed shader cache translate and pass spirv-val (vulkan1.3); 500
mesh modules gained about 730 bytes each on average, so many of the game's mesh programs have partly listed vectors.
--probe-meshfb passes (fallback pictures equal the mesh pipeline's), and its module part converts all 520 new
translations with 0 failures, all 1,020 modules pass spirv-val and their compute pipelines build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Vulkan leaves workgroup memory undefined, and a mesh program need not write every vertex it has room for or every
index byte. The mesh entry function copies all of them into the outputs, so what an earlier workgroup left in shared
memory became vertices and primitives: on AMD, long thin triangles across the foliage's shadow map. The entry function
now sets the vertex lines, the primitive count and the index bytes to zero in a loop over the workgroup's threads
before the payload copy and the barrier. Task programs keep clearing only their mesh count.
Checked by translating all 520 task and mesh programs an installed shader cache stores with the translator before and
after the change: every one translates and passes spirv-val (vulkan1.3) both times; each mesh module grows by about
88 bytes, task modules are unchanged. tools/xlate_smoke.py passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ocarina anti-aliases every frame with its own SMAA in three compute passes after the bloom composite. The new aa
setting (nx2navi.ini aa, the Screen tab's Anti-aliasing row) keeps that by default. off, fxaa and smaa leave out the
game's edge and weight passes and replace its composition with a pass of our own that writes the same square-rooted
RGB9E5 words into the scene target: a plain copy, an edge smoothing written here, or the SMAA reference implementation
with its area and search textures (MIT). taa is DLSS or FSR 3.1 at the scene's size, which the row sets through the
upscaler; while either runs, the game's SMAA stays whatever aa says. The game's passes are known by a hash of their
guest code. The shader build now also rebuilds a header when a file its source includes changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The HDR row lists Off, RenoDX (the game's picture tone mapped to the display, the one fixed look) and Native (the game's own HDR, taken up at the next start), with the peak, game and HUD whites under either.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HDR on is the game's scene through the observer-response mapper, anchored to the game's own mid grey and tone slope, with half of the hue bend of the game's clamp: the defaults the menu had. The tone mapper choice, the mid grey, the tone slope and its match, the hue clip, exposure, highlights, shadows, contrast, saturation, highlight saturation, blowout, flare, hue, grade and gamma are gone from the menu, the settings table and the example ini; a file that still sets them works as before, the keys read by nothing and logging nothing. The menu's HDR row is Off and On, with the peak, game and HUD whites under it. The game's own HDR is still there for a file that sets hdr = native, and the row lists it only then. The luminance mapper and its grading code, which no setting reaches any more, are removed from the tonemap library, and the support buffer carries one vec4 for it.
The dark tones of a dark scene sat far above black: the present showed the game's sRGB-encoded picture as that encode decodes, linear near black, while the game is graded for a display of gamma about 2.2, and since the observer-response mapper became the default the 2.2 correction ran with the luminance mapper only. A scene value of 0.003 came out at 0.29 nits at a white of 350 nits, where a 2.2 display shows the game at 0.015. The present now always shows the picture as a 2.2 display would, and the tonemap aims its peak lower by that correction's inverse, so the brightest light still reaches the display's peak; the same value comes out at 0.017 nits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux every driver got shadow storage (copies of guest memory beside it), which exists for amdgpu, where importing guest memory costs the write protection of its pages. NVIDIA's own driver imports without that cost, so there the copies are only the slower path: in Kokiri Forest on 0.7.1 (RTX 4090) imports ran at 51.9 frames a second and the copies at 27.4. Until now only the AppImage's play.sh knew, by setting NX2NAVI_SHADOW=0 when NVIDIA's kernel driver was present at build time, so a game started any other way got the copies, and a PC with that kernel driver whose game ran on another card got imports where it should not. The game now chooses by the driver of the device it runs on, play.sh no longer sets the variable, and NX2NAVI_SHADOW=1 or 0 still decides. The driver is asked when the device is picked, before the storage is chosen.
Windows is unchanged: the device probe here reads "mem: mode mirror" with no shadow storage line, and the shadow line with NX2NAVI_SHADOW=1. The Linux path itself was not run: there is no Linux machine here.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Brought onto the line with FSR 4, its frame generation and the debug menu. Where both sides had added to the same
place, both stay: the patch table keeps the pickup count and the free camera, the settings list keeps both sides'
names, the menu includes the co-op page and the debug menu, and the launcher builder takes the pipeline list and the
FSR 4 folder. Both sides had added the menu's typed-character event; one definition stays, with the window code that
passes characters outside the basic plane.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
With it on, the game draws from a camera of the menu's instead of its own, in play and in cutscenes. The left stick moves it along its view and sideways, the right stick turns it, ZR and ZL raise and lower it, R makes it faster and L slower, and Link stands meanwhile; or the camera stays where it is and the pad plays the game.
It works inside the game's own camera update: a new patch site sends the instruction at which the frame's position, target and up vector are final to a host function, which replaces them for the camera that follows Link and the one the events direct, so the game draws, culls and loads detail for what the free camera sees. The camera of the interface's 3D models and the one the sound follows stay the game's. The set is off unless the menu switches it on. A teleport's arrival starts the free camera again from the game's view.
Tested in Hyrule Field with scripted stick input (turn, rise, strafe, with Link's position unchanged), with the pad playing while the camera stays, across a teleport to Kakariko Village, during the Kakariko arrival cutscene, and switched on by a click in the menu.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The probe's programs are written in the shape the translator gives a task or mesh program and need no game files.
With a folder of game translations it converts those instead and builds their compute pipelines.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
tools/validate_shader_cache.py reads every module and its reflection from the pack and the loose files, says how many the game would refuse and translate again, runs spirv-val on the others and groups what it refuses by message. It writes nothing to the cache. tests/validate_shader_cache_test.py checks it on a made-up cache.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
pipeline-cache.bin was copied out of the driver and written whole at every save after any pipeline was created, also one the cache had held: tonight's logs show three saves of 358.1 MB in a run that added 0.3 MB, and 302 to 306 MB written in each of seven sessions that added about 0.5 MB each. The library warm in play built its 37,572 stage libraries into the same cache, which grew it by 1.6 GB: a save of 2026.7 MB in play, a file over pipeline_cache_max_mb, and the next start dropped it whole and compiled the pipelines it had held.
cache_save now asks a cache for the size of its data first and neither copies nor writes one whose size is what its file holds. What a run compiles goes into a second cache with its own file, pipeline-cache-play.bin (pipeline_cache_play, default 1; needs pipelineCreationCacheControl): pipe_create_in asks the first cache with VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED_BIT and builds into the play cache only when that says a compile is needed, and in play only the play file is saved. At a start the play file is read back and merged into the first cache in memory; once it reached pipeline_cache_play_mb (64) it is folded in for good: pipeline-cache.bin is written once, before play, and the play file removed. The preparation modes fold it at any size and build into the one file as before. The library warm's threads create with no cache of ours, so their libraries reach the driver's own cache alone.
Checked with --probe-prewarm on a copy of a dev folder's caches (3,212 recorded pipelines, 333 compute shaders), no game run: with a filled pipeline-cache.bin every pipeline came from it and neither file was written; with none and a cold driver cache the compiles went into pipeline-cache-play.bin (75.9 MB written, pipeline-cache.bin not created); the next start merged it and wrote pipeline-cache.bin once (191.1 MB) and removed it; kept below the fold size, a play file of 68.8 MB gave 1,445 pipelines at 0.3 ms from the first cache where 470 hit without the merge, and was not written again. Not checked in play: the saves during a session, the library warm without a cache, and drivers other than NVIDIA's.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
runtime/vk_fg4.cpp makes the frame generation swapchain and context with the libraries FSR 4 comes with
(amd_fidelityfx_loader_vk.dll and amd_fidelityfx_framegeneration_vk.dll beside the executable). vk_fg.cpp takes
that path only with fsr4_fg = 1 while FSR 4 is the upscaler; the default stays the FSR 3.1 library. The loader
picks the version for the card: the machine-learned FSR Frame Generation 4 on Radeon RX 9000 cards with Windows 11,
which is untested, and 3.1.6 elsewhere. On an RX 7900 XT (driver 26.10.1) it lists frame generation 3.1.6 and the
swapchains 4.0.0, which the driver brings and the loader chooses, and 3.1.7. The swapchain in place is wanted only
while its library is the one to use, so a change of the setting or of the upscaler, or a failure of this path,
takes it down as switching frame generation off does and the next present makes it with the other library.
The preparation gives this library the camera's position and axes, which FSR Frame Generation 4 requires, and a
scene scale of one metre a unit: with the 0.01 the FSR 3.1 path uses, version 3.1.6 drew grey translucent shapes
into the sky of its frames while the camera turned (160 captures of the screen during a camera sweep; clean with 1).
Both paths now generate a frame only when the picture without the HUD is there for it: generated from the
presented picture alone, the HUD would be warped with the scene behind it. In four configurations (21:9 with either
library, a 16:9 picture in a 21:9 window, a 1080 picture without HDR) every generated frame had it, and the log
counts the frames without. On screen, of the 6,404 pixels of the HUD's item button that stayed fixed through a
camera sweep with the FSR 3.1 library, 6,162 stayed fixed with this one.
Standing in Hyrule Field at 3440x1440 with FSR 4 Quality the game draws 43.0 frames a second with this path and
42.2 with the FSR 3.1 library, each shown twice over. One run changed the present mode, HDR, fullscreen, frame
generation, FSR 4, this setting and the upscaler in turn: 216 screenshots without a bad frame, and no failure.
The SDK's frame generation headers join runtime/third_party/ffx_sdk, unmodified. tools/make_kit.py and
tools/make_launcher.py take the third library from the --ffx4 folder when it is there, and tools/make_game.py
copies it beside the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The prepared list named its shader variants and nothing about where they came from, so an update that keeps the prepared shaders (shader_carry) kept the old list whatever seed the new kit writes: variants only the new seed names, from a newer pipeline list or a change to which variants are prepared, were translated when play first drew them. The existing identities do not cover this: nx2navi_xlate.id and identities.txt name translators, which decide cache keys and what is carried over, not which variants are listed.
The list is now version 2 (runtime/prp_file.h): after the head come the sha256 of seed-programs.bin, the sha256 of seed-answers.bin and the translator identity, written by gfx_shader_prepare_seed; the three readers go through gfx_shader_prepared_read and still read a version 1 list. At an update step_shaders writes the seed first (prepare_shaders.py --no-run, about 4 s) and keeps the list only when its digests are that seed's (make_game.py seed_changed); another seed, or a list that does not say, is prepared again. Another translator alone still carries over: input ids do not depend on the translator, and preparing everything again for it is what carrying was made to avoid.
Checked with tests/prp_file_check.py (the C layout against make_game's reader), three launcher tests, and the exe's --prepare-shaders on a seed without records in a folder of its own: the digests it wrote are those of the seed files, one byte more in the seed reads as changed, and after preparing again it does not. A version 1 list of 42,142 variants and the same list as version 2 are both read whole by --probe-prewarm.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
runtime/vk_ffx4.cpp loads amd_fidelityfx_loader_vk.dll from beside the executable and asks it which upscaler
versions the device can run. Where a 4.x version is listed (Radeon RX 6000, RX 7000 and RX 9000 cards with AMD's
driver 26.10.1 or newer) the menu's Upscaling row gains AMD FSR 4 Native AA, Quality, Balanced, Performance and
Ultra Performance, and fsr4 = 1 runs it in FSR 3.1's place with the same inputs. Its contexts are made with that
version asked for by its id and the provider is asked afterwards which version it runs, so no other upscaler can
be shown under the name. When a context or a dispatch fails, FSR 3.1 makes the frame, and after three failures
FSR 4 is off for the run. Each upscaler starts its history anew on the frame after a change from one to the other.
The setting is off by default and the Linux build compiles stubs.
On an RX 7900 XT (driver 26.10.1, 3440x1440, HDR) the library runs FSR 4.1.2: a context takes 54 ms to make and
217 MiB of video memory. Standing in Hyrule Field: Quality 47.8 fps where FSR 3.1 gives 51.5, Native AA 41.4.
Screenshots of 601 consecutive frames standing, 801 during a camera sweep, 701 each in Kakariko and the
graveyard, and 1,301 across changes between FSR 3.1 and FSR 4, their modes and off show no flicker that FSR 3.1
does not have, and it runs together with FSR 3.1 frame generation.
The SDK's API headers are in runtime/third_party/ffx_sdk, unmodified, with the licence. tools/make_kit.py and
tools/make_launcher.py take AMD's two signed libraries with --ffx4, and tools/make_game.py copies them beside the
game when the kit has both.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Without the other players drawn in the world, which are unfinished. No relay address is built in, and with co-op off nothing opens a socket. Built on Windows and tested against a relay on this PC only; two games in one room and any real network have not been tried.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A program that only asks a combined-handle texture for its size got that texture declared as plain 2D, the runtime bound a one-layer view, and the layer count read as 1 where the console answers the texture's real count. The game's particle vertex shaders read exactly that count, of a texture their pixel shaders sample as a 2D array, and use it as the frame count of a flipbook, so such particles stayed on their first frame. A size query that reads the component after a 1D or 2D type's dimensions now makes the declaration an array; a plain texture bound through an array view still reports one layer. 74 of the stored shader variants change (the contributor's sweep, all valid; a reader here found the same 74 in a shader cache on this PC), and the contributor's before-and-after pictures differ in the one scene tried, the Wallmaster's warning disc.
Goes in with the image read capability so that installs translate their shaders again once. Built here; tools/xlate_smoke.py's new case reports the texture as plain 2D with the translator before the patch and as a 2D array with it, and all three modules validate. Not played with the new translator.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit e4fac75190b4baf3811feef45b3302a9216cb8b2)
The translator declares every float storage image without a format and turns a load of it into OpImageRead, which SPIR-V allows only in a module that declares StorageImageReadWithoutFormat; only the write-side capability was declared. Of the 44,206 shader variants a 0.7.1 install stores, 117 compute shaders were invalid SPIR-V for that reason (the contributor's sweep with spirv-val, for NVIDIA's, AMD's and Intel's host options: 117 before, 0 after). The drivers tried accept them, so nothing crashed, but an invalid module is whatever a driver makes of it. The read now declares the capability, once a module.
The translator's identity changes, so an updated install translates its shaders again (the carry pass serves the old translations meanwhile). Built here from the local package cache; tools/xlate_smoke.py's new compute case fails with the translator before the patch and passes with it, and both modules validate. Not played with the new translator.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
(cherry picked from commit 62e5c345a2ec3e2cc70be49214997ff76e691309)
Each player keeps their own game and save. One player makes a room, the others join it with its secret, and story flags and, by the room's rules, items, songs, quest items, small keys (a shared pool), rupees and ammunition reach every save. The layer is a host thread with one UDP socket to a relay the players run themselves (tools/coop_relay.py, Python's standard library only); the messages between the players are sealed with a key from the room's secret (XChaCha20-Poly1305, Monocypher 4.0.2 vendored with its licence and a row in THIRD_PARTY_NOTICES.md), every change is signed by the player who made it, a player accepts a room by its members' keys and the key of its rules, the save is copied to coop-backup before anything is applied, and values are set through the game's own routines between frames. The Co-op tab makes, joins, accepts, removes and leaves, and the menu takes typed text for the name, the relay and the secret. This is the contributor's pull request #518 on this line.
Nothing reaches the network while co-op is off: the socket is opened, and the relay's name looked up, only after co-op is on, a relay is set and a room is made or joined. There is no default relay.
Left out of the pull request: the other players drawn in the world (the "puppets") with the copied engine they need. They were unfinished, added two patch sites to the game's per-frame update that could not be put through the differential tests here, and their file helpers clashed with the ones this line already has. So no position is sent: the POS message's codec stays, nothing sends it.
Changed against the pull request:
- The menu's test hook drives the Co-op tab only when it comes from the environment. From nx2navi.ini it could turn co-op on, join a room and accept its offer.
- The log's label for a room is hashed from the secret instead of being its first three characters.
- A game that has not saved yet has no save to copy, and takes nothing from the room until it has saved once; that used to count as a copy made.
- A PING's token no longer carries the PC's uptime.
- On Windows the maker's key file and its folder get an access list with the player's account alone.
- The tests that link the platform layer get the socket library, and the save test a stand-in for the title screen's hook.
Checked on Windows, where the pull request had never been compiled: the runtime builds and links; --probe-coop passes (sockets on 127.0.0.1, the codec against its byte vectors, a fuzz of every parser, two sessions through a hostile mock relay, a relay restart, the sync engine); tests/coop_relay_test.py passes its 61 tests, with the probe through the real relay on 127.0.0.1 at 0 and 30 % loss; tests/coop_page_check.py passes and its deliberate fault fails 48 checks. Not checked: two games in one room (one game runs at a time here), any real network, the tab in the game.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
make_game.py passed the map as -Wl,--Map=<game folder>/build-cache/ocarina.map, and -Wl, cuts its argument at every comma: with a comma in the folder's path the part after it reached the linker as an input file and an update ended with "the link failed" (issue #470). -Xlinker hands the path over as one argument. build_game.py already did so on Windows and now does on Linux too.
With MSYS2's clang and ld here, a link into a folder named "a, b" fails with the -Wl, form ("cannot find b/t.map") and writes the map with -Xlinker.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The prompt overlays for the pad layouts repeat the same note pictures, and every one was fitted and encoded to ASTC again for each layout. The encoded bytes are now kept for the build under the picture and its box. With stand-in pictures, 42 note textures over seven layouts came out byte for byte the same from 12 encodes instead of 42, in 0.66 s instead of 2.30 s.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The shader step's progress event rounded its share to four decimals, and the launcher shows the whole percent below it: 42,140 of 42,142 shaders built read as 100% while a long driver compile was still running. The event now carries the share as computed, so the bar stays at 99% until the count is complete. The test feeds the last counts of a 100,000-shader build and fails with the rounding.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The launcher asked for 31 GB where the build's own check now asks for about 34, so a disk between the two passed the first check and was refused seconds later. Both say 34 GB now. The build no longer asks for the whole shader preparation's space when the game folder already holds translations. The texts said 10 to 25 minutes for an install that took players 27 to 41, and 5 GB for a build cache that measures 1.6. A build error's message goes into the launcher's log as one line.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
pad_prompts.py gave each packed game file to the decompressor in one
call, which returns one bytes object grown by copying. On made-up data
of the size of the interface texture archive, 69 MB packed and 244 MB
unpacked, that takes 21 s on a fast PC; players' builds waited 44 to 52 s
before the step's first folder, most of its minute. In 64 KiB pieces
joined once the same bytes take 0.3 s.
The prompts step's key held the digest of the three scripts' bytes, so
this edit would have drawn every install's prompts again, as two of the
last four releases did. It now holds a constant changed by hand, whose
value is the scripts' digest in 0.7.0 and 0.7.1, and a test fails when
the scripts' logic changes, to ask whether the constant must change.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Only the RomFS image's space was looked at, and only when that step
began: a disk that held the image but not the build cache or the shaders
stopped the build in the compile or, half an hour in, in the shader
preparation. The build now adds up what the steps it will run write into
the game folder (the image and 1 GiB, 2 GiB for the build cache, 4 GiB
for a full shader preparation, the least each is known to need) and
stops at once when the disk has less, naming each part and how much is
missing.
On Windows a build whose compile step will run first compiles and links
an empty program with the installed tools, against the libraries the
game links. Tools that are incomplete or blocked stopped the build only
in its compile step, after the image and the lift; the message says what
failed and how to get the tools back.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
romfs_check.py gave the decompressor 4 MiB of a file at a time. Each call
returns its output as one bytes object grown by copying, under the
interpreter's lock: on 32 MiB frames that ran at about 120 MB/s of packed
data on twelve threads, and at about 2,500 MB/s with 64 KiB pieces; on a
1 MiB file at 490 against 3,000. Files smaller than a piece are as fast
as before. The check now stops at the frame's end, so what it says of a
file does not depend on the piece size; a test compares both sizes and
the check as it was on damaged, cut and whole frames.
The scan step's key held the sha256 of the script's bytes, so this edit
would have made every install check its 26 GB again. The key now holds a
constant changed by hand, whose value is the sha256 the script had in
0.7.0 and 0.7.1: the key of every install that passed stays what it is.
A test fails when the script's logic changes, to ask whether the constant
must change.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
modules_c wrote the paths of the game folder as C strings with JSON's
default escapes, which turn a character outside the Basic Multilingual
Plane into two surrogate escapes. C refuses those ("invalid universal
character"), so a game folder with an emoji or a rare Han character in
its path failed at the end of the compile. The characters now stand as
themselves in the UTF-8 source, and a test compiles the generated file.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
make_game.py ran one compile per logical processor whatever the memory.
Without --jobs it now runs as many as 80 % of the free memory holds at
768 MiB a compile, the peak build_game.py records for one, never more
than the processors and never fewer than one, on Windows and Linux alike.
The build's first lines say the number and what decided it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
An exception make_game.py did not report itself only printed a traceback,
and the launcher showed "the build stopped (exit code 1)". main now sends
every such error through fail with the exception's message (the system's
text and the file for a file error, the full disk named as such), and the
traceback still goes to the log.
A dump whose data does not match its own checksums ended the RomFS step
that way. The step now names the damaged file and says to copy or dump
the game again, and it removes the partial image of a failed run.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The Updates row promised that an update rebuilds the game "in a few
minutes". The code cannot keep that: a version that changes the lifter
compiles the game's code again, and one that changes what the shaders are
built with builds all of them into the driver's cache again. Players'
updates took 24, 34 and 45 minutes. The row now says it takes a few
minutes when little changed and half an hour or more otherwise.
The warm-up of the driver's cache after an update only builds, but its
count was drawn in the second half of the bar as in a full preparation,
so a 20 minute run read 50 % in its first second. Its count now fills the
whole bar, and the row says so while the shader cache is packed before it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A step that runs drops the record of every step after it, and the RomFS
step comes before the lift although the lift reads only the unpacked
ExeFS. An install that was cut short while the image was still being
packed, the lift beside it already done, therefore lifted the whole game
again at the next try. The RomFS step now leaves the lift's record; the
compile and the shaders, which read the image, still follow it.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
compile_all wrote the object cache only when every file was done, so
Cancel or a crash in the middle of the compile lost the record of every
object and the next run compiled the whole game again. The cache now goes
to disk every 25 objects, at most every two seconds, moved into place
whole. Results are taken as the compiles end instead of in the order they
started: the largest file starts first, and in that order nothing was
recorded until it had finished.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A RomFS step that was cut short (Cancel, a full disk, a power cut) leaves
romfs.bin.part, up to 26 GB. The next run writes the image anew, but
make_game.py's step_romfs and the launcher's Begin Installation measured
the free space first, with that file counted as used: on a nearly full
disk the retry was refused although it needs no more room than the first
try. Both remove the file before they measure, and the first-run screen
counts its size as free.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The carry path of an update does not run the shader preparation: it runs --warm-shaders, which builds the listed pipelines from the seed-pipeline-keys.bin the game folder already has. make_game.py now writes that file from the kit's list first, so a newer list reaches an updated install. Its shader variants are not translated on this path; a record whose variant the folder has neither prepared nor met is left out and counted.
On a GPU that has no mesh shaders every mesh record failed with a log line each. They are counted in the step's line instead.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The preparation translates every shader and builds each one's stage library, so a pipeline a draw meets for the first time is linked in 2 to 6 ms. A mesh pipeline has no stage libraries: on an install that never drew it, it compiles whole in play, 0.2 to 1.1 s with its draws left out of the picture (18 of them on a first visit of Kokiri Forest). Whole pipelines were built ahead only from the folder's own pipeline-keys.bin.
tools/pipeline_list.py merge folds the pipeline-keys.bin and shader-answers.bin of game folders that were played into one list file: every key record once, with the answers record of each shader it names. A record names its shaders by input id, which depends on the game's code and the answers alone, and holds the game's pipeline state; what depends on the GPU is derived when the pipeline is built, so a record made on one machine is good on another with the same game version and runtime. Four folders on this PC give 5,432 records, 317 of them mesh pipelines (147 KB of keys).
tools/prepare_shaders.py reads the list (tools/data/pipeline_list.bin, data/pipeline_list.bin in a release folder, or --pipeline-list), adds its shader variants to the seed and writes its records as seed-pipeline-keys.bin. nx2navi.exe --prepare-shaders builds the mesh pipelines among them after the seed is translated and saves the pipeline cache before the stage libraries are built (NX2NAVI_PREPARE_PIPELINES=all builds the vertex pipelines too); --warm-shaders does it again after an update or a new driver, and --prepare-pipelines is that step alone. The pipelines go into the caches and are not kept or added to the folder's own key file, so they cost no start-up time or video memory until a draw meets them. A marker file keeps a driver that crashes on one of them from failing every preparation.
The list is derived from game data and stays out of the repository: check_content.py refuses files recorded from a game folder by their first bytes, and make_kit.py, make_launcher.py and make_release.py carry a list given with --pipeline-list. All three now also carry pipeline_list.py, which prepare_shaders.py imports.
Checked without the game: tests/pipeline_list_test.py, and prepare_shaders.py --no-run on the game files with a list of 317 mesh pipelines (all 266 shader variants found in this game version, 177 new to the seed). The build step in nx2navi.exe compiles and links and has not been run.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A player who keeps the game on a mapped network drive could not finish a build. Three things change in tools/make_game.py:
- The output folder keeps its drive letter on Windows. It was resolved to the share's UNC name, which is a second connection to the same server, and some servers drop that one under the compilers' load while the mapped drive stays up.
- When a source's size cannot be read because its folder stopped answering, the build says so and asks again for up to a minute before it stops with a message that names the drive. It used to end in a traceback. A file that is gone from a folder that answers still ends the build at once.
- A finished file is put in place through a temporary file beside it instead of a file in a temporary folder. Such a folder is made for its owner alone, and some shares refuse that while they take ordinary files; the build then reported a permissions fault in the output folder after everything had compiled.
From pull request #419: the first and third change are the contributor's, the wait is written again so that a file that is really gone does not wait a minute, and the launcher tests cover all three.
Co-Authored-By: Matthias <dev@sleepyyui.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The AppImage tested for zbic with PYTHONPATH and the user's site-packages in view, then ran the build with Python's isolated mode, which sees neither. A player with zbic installed for their own Python of the same minor version passed the test, never got it installed where the build looks, and was told to run the pip command that had put it in the place the build ignores. The test now uses the same isolation and the same search path as the build, so zbic is installed into the AppImage's own folder whenever the build could not import it.
From pull request #436.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The developer build stopped at undefined dav1d symbols since the AV1 movie decoder (runtime/ov_movie2.c) came in: tools/make_game.py linked dav1d, tools/build_game.py did not. Windows links libdav1d.a statically, as libvpx and zstd are, so the exe imports no libdav1d DLL; Linux links -ldav1d. The llvm-mingw link list and the Linux release's static list get dav1d too, and docs/building.md lists the package for MSYS2 and Arch.
From pull request #439, with the two other link lists added.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A player's AppImage build of 0.7.1 stopped in its compile step after eight minutes of RomFS work: four
files did not compile, because the kit builds with the system's compiler and headers and theirs were
older than the runtime needs. vk_pipeline.c and vk_device.c name VK_KHR_pipeline_binary,
VK_KHR_dynamic_rendering_local_read and VK_KHR_shader_maximal_reconvergence, which their Vulkan headers
did not declare, and ui/menu.cpp takes its font in with #embed, which clang has from version 19.
- Vulkan headers: tools/vulkan_pin.py holds the release the other builds use (1.4.363, the archive and
hash tools/make_release_linux.py already pinned, which now reads them from there). The AppImage carries
its C headers in toolchain/vulkan-headers and builds against them whatever the system has; a kit or a
source checkout uses the system's when they are at least that release and fetches the pinned ones
otherwise (build_game.py vulkan_include). Their folder goes first on the include path of the runtime's
sources only, so the lifted code's objects keep their keys, and a runtime object compiled before
against the system's headers is taken over instead of compiled again. AppRun no longer asks for the
system's Vulkan headers.
- #embed: tools/build_shaders.py also writes the font's bytes as runtime/ui/roboto_ttf.h (ignored by git,
carried by a kit), and menu.cpp includes that where the compiler has no __has_embed. With #embed
nothing changes.
- Before its long steps a Linux build that will compile now checks every runtime source that needs no
header of the install with the compile step's own command (clang -fsyntax-only) and links an empty
program against the game's libraries (make_game.py preflight): an old compiler, an old header or a
missing development package stops it in the first seconds with the compiler's own errors. A failed
compile names every file that failed, not only the three whose errors it prints.
Checked on Windows: tests/launcher_test.py passes with four new tests (117); the header of bytes and
#embed give the same 162,588 bytes, and both branches of menu.cpp compile; the early check passes on the
Windows file set with clang 23 and stops on files made to fail; the 34 local headers of release 363 are
the upstream tag's files by git blob id. Not run: anything on Linux, the pinned archive's download, a
compiler older than 23.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A setting read at the start showed "after restart" under its value always. The values the game started with are now
kept (cfg.c start_values), a row shows the note only while its value differs from them, and the Restart button counts
the settings that wait, the upscaler, frame generation and HDR among them. A value the environment sets never waits.
--probe-cfg checks the count.
From pull request #377 by mooki000. Three conflicts in the menu resolved against the frame generation rows that came
in since: the upscaling row keeps its description by what can load, and the frame generation restart test knows both
ways NVIDIA's runs.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The launcher release carries the runtime's shader headers built and none of the tools that build them, and
make_game.py's compile step imported build_shaders unconditionally: every update to the 0.7.0 build stopped at
"Compile Changed Files" with ModuleNotFoundError. The import is now tried, and without it the step that rebuilds stale
headers is left out, as its comment intended. A launcher test checks that every tool make_game.py imports is in the
launcher release unless the import is guarded or only runs on Linux.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Controls tab's Button prompts row and nx2navi.ini take steamdeck: the Deck's own labels in the game's
text, action prompt and ocarina notes, which are A B X Y where an Xbox pad has them, L1 R1 L2 R2 and L3 R3,
Menu and View. The style is drawn by tools/pad_prompts.py like the others, into romfs-patches-steamdeck
and romfs-patches-steamdeck-label, and every build with button art writes them.
Pull request #107 by loteran asked for PlayStation and Steam Deck prompts; it was closed, and the tree
took the PlayStation half in another form, drawn from button art instead of redrawn font glyphs. This is
the Steam Deck half in that form, written here: the art has no Deck style, so its entry in
tools/prompt_art.py names the Xbox pictures for the face buttons, Menu, View, the d-pad and the sticks
and the PlayStation pictures for the shoulders and stick clicks. It needs four pictures of the Xbox set
that no other style used (the white letters on a light button, for the ocarina's notes); a release made
with --prompt-art copies them with the rest.
The Windows files compile and tests/launcher_test.py passes, with the prompts step's test following the
number of folders (seven now); every picture the style names is in the art used for releases. The style
was not drawn here (that needs the game's font), and --probe-cfg, which has a new check, was not run.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>