Add the shared pipeline list: mesh pipelines players met are built at the shader preparation, before the first start.

The preparation translates every shader and builds each one's stage library, so a pipeline a draw meets for the first time is linked in 2 to 6 ms. A mesh pipeline has no stage libraries: on an install that never drew it, it compiles whole in play, 0.2 to 1.1 s with its draws left out of the picture (18 of them on a first visit of Kokiri Forest). Whole pipelines were built ahead only from the folder's own pipeline-keys.bin.

tools/pipeline_list.py merge folds the pipeline-keys.bin and shader-answers.bin of game folders that were played into one list file: every key record once, with the answers record of each shader it names. A record names its shaders by input id, which depends on the game's code and the answers alone, and holds the game's pipeline state; what depends on the GPU is derived when the pipeline is built, so a record made on one machine is good on another with the same game version and runtime. Four folders on this PC give 5,432 records, 317 of them mesh pipelines (147 KB of keys).

tools/prepare_shaders.py reads the list (tools/data/pipeline_list.bin, data/pipeline_list.bin in a release folder, or --pipeline-list), adds its shader variants to the seed and writes its records as seed-pipeline-keys.bin. nx2navi.exe --prepare-shaders builds the mesh pipelines among them after the seed is translated and saves the pipeline cache before the stage libraries are built (NX2NAVI_PREPARE_PIPELINES=all builds the vertex pipelines too); --warm-shaders does it again after an update or a new driver, and --prepare-pipelines is that step alone. The pipelines go into the caches and are not kept or added to the folder's own key file, so they cost no start-up time or video memory until a draw meets them. A marker file keeps a driver that crashes on one of them from failing every preparation.

The list is derived from game data and stays out of the repository: check_content.py refuses files recorded from a game folder by their first bytes, and make_kit.py, make_launcher.py and make_release.py carry a list given with --pipeline-list. All three now also carry pipeline_list.py, which prepare_shaders.py imports.

Checked without the game: tests/pipeline_list_test.py, and prepare_shaders.py --no-run on the game files with a list of 317 mesh pipelines (all 266 shader variants found in this game version, 177 new to the seed). The build step in nx2navi.exe compiles and links and has not been run.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
chrg6
2026-11-10 14:52:46 -05:00
parent 682d9c5c50
commit 918d399d55
15 changed files with 627 additions and 14 deletions

View File

@@ -158,6 +158,22 @@ of them.
version; they build at their first draw instead, another driver version builds them again, and so does deleting the
file.
What this step leaves for play, and the shared pipeline list. The step translates every shader's default variant
and the shader list's variants and builds each one's stage library or compute pipeline. A whole graphics pipeline
is built ahead only at the start of a game folder that has drawn it before (`pipeline-keys.bin`); a pipeline met
for the first time is linked from its stage libraries in 2 to 6 ms, except a mesh pipeline, which has no libraries
and compiles whole (0.2 to 1.1 s, its draws left out of the picture meanwhile). `tools/pipeline_list.py merge`
folds the `pipeline-keys.bin` and `shader-answers.bin` of game folders that were played into a pipeline list;
`prepare_shaders.py` reads it from `tools\data\pipeline_list.bin` (`data\pipeline_list.bin` in a release folder, or
`--pipeline-list`), adds the shader variants it names to the seed and writes its records next to the game as
`seed-pipeline-keys.bin`, and `--prepare-shaders` builds the mesh pipelines among them into the pipeline cache and
the driver's (`runtime/vk_pipeline.c` `seed_pipelines`; `NX2NAVI_PREPARE_PIPELINES=all` builds the vertex pipelines
too). `--warm-shaders` builds them again, and `nx2navi.exe --prepare-pipelines` does that step alone. The list is
derived from game data: it is not in the repository (`tools/check_content.py` refuses it), and a kit, a launcher
release or an installer release takes one with `--pipeline-list` (`tools/make_kit.py`, `make_launcher.py`,
`make_release.py`). A list belongs to the record layout of the runtime that wrote it; another runtime says so and
builds nothing from it. `python tests\pipeline_list_test.py` checks the tool on made-up folders.
7. Run. The game keeps running when its window loses the focus (`assume_focus = 1` in `nx2navi.ini`, the default; the
Game tab switches it). With 0 it pauses, as it does on the Switch when Home is pressed.

View File

@@ -148,7 +148,8 @@ llvm/ llvm-mingw, x86_64-only subset (Apache-2.0 with LLVM excep
runtime/ libnx2navirt.a, libnx2navirt-v2.a and the headers the generated code and modules.c/romfs.c/patches.c
include
bin/ nx2navi_xlate.exe, hactool.exe, SDL3.dll, libopus-0.dll, ninja.exe
data/ shader_list.tsv (MIT, THIRD_PARTY_NOTICES.md); pgo-<ver>.profdata + .json, the execution profile an
data/ shader_list.tsv (MIT, THIRD_PARTY_NOTICES.md); pipeline_list.bin when the release was made with
one (building.md, step 6); pgo-<ver>.profdata + .json, the execution profile an
update's code and the runtime libraries compile with (building.md, "Profile-guided build")
THIRD_PARTY_NOTICES.md
nx2navi.ini.example, VERSION the default ini; the release string (the repository's VERSION plus the commit)