Redraw the in-game menu with the game's own interface textures, and bring in the launcher's save import and faster builds.

In-game menu: the settings window, its tabs, rows, buttons and cursor, the performance overlay and the Preparing
shaders box are drawn with the game's interface textures where the RomFS holds them (runtime/ui/menu_skin.cpp,
perf_panel.cpp; plain shapes until they are decoded). Restart joins Continue and Quit, each behind a confirmation;
VSync is an on/off switch; DLSS is listed only on NVIDIA GPUs; the menu sounds are quieter; Picture shape is gone and
the setting descriptions are shorter; edge cases of input, layout and fonts are fixed. tools/menu_preview.py draws the
menu without the game.

Launcher: Import Saves takes a save from the game folder's saves/ folder or one the player picks; downloads fall back
to Windows' own certificate check when Python's refuses a server; updates are faster; the title bar is drawn in the
window and Navi is every icon. tools/make_game.py builds with CLANG64's clang where the launcher installed it, packs
the RomFS beside the lift and the compile, takes a prebuilt runtime, and compiles the game's resources without cmd.exe,
so a user folder with a space in its name builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
chrg6
2026-11-08 03:57:01 -05:00
parent 5b28da3a06
commit 8daedbd806
42 changed files with 4138 additions and 972 deletions

View File

@@ -49,9 +49,9 @@ With extracted folders instead, use `--exefs "C:\path\exefs" --romfs "C:\path\ro
check that your dump is the right version, add `--check` (it builds nothing).
The build checks the version, unpacks the program files, packs the 26 GB RomFS image (every file is checked against
the dump's own checksums), turns the game's code into C and compiles it (about 6 minutes on a 16-thread Ryzen 7; longer on fewer cores),
then prepares every one of the game's shaders for your graphics card (about 5 more minutes), so the game does not stop
to compile shaders while you play.
the dump's own checksums) while it turns the game's code into C and compiles it (about 6 minutes on a 16-thread
Ryzen 7; longer on fewer cores), then prepares every one of the game's shaders for your graphics card (about 5 more
minutes), so the game does not stop to compile shaders while you play. The game it builds is for this PC's processor.
You can use your PC meanwhile. If it stops,
run the same command again: finished steps are skipped.
@@ -62,7 +62,9 @@ When it says `finished`, start `C:\Games\Ocarina\ocarina.exe`. You can delete `C
- **F1** opens the settings menu. Settings are saved in `nx2navi.ini` beside the game; saves go to `profile\`.
- Every shader was prepared during the build. After an update of your graphics driver, the next start shows
"Preparing shaders" once while the driver rebuilds them.
"Preparing shaders" once while the driver rebuilds them. With a driver that has Vulkan's graphics pipeline library
it lasts at most about 20 seconds once the shaders are translated, and the remaining pipelines are built in the
background while you play.
- **Screen** in the menu has Upscaling (DLSS on NVIDIA RTX cards, FSR 3.1 on any card), Frame generation (FSR 3.1; it
needs DLSS or FSR 3.1 upscaling on), HDR (Native is the game's own HDR; turn HDR on in Windows first) and the
resolution. Those marked "after restart" take effect when the game starts again.

View File

@@ -85,9 +85,13 @@ keeps per-core data without locks and relies on that, so the host models it (`ru
- Shaders: Ampere (SM86) code is lowered to Maxwell (`tools/xlate/AmpereLowering.cs`) and translated to SPIR-V by the
pinned shader translator (`tools/vendor/ryujinx`, 1.3.2) with our patches (`tools/xlate/patches/`), built with .NET 10 NativeAOT as
`nx2navi_xlate.exe` and run as separate processes. Translations are cached on disk; shaders and pipelines seen in
play are prepared at the next boot. The game waits for all of it (`runtime/boot.c` `boot_wait`): about half a second
play are prepared at the next boot. The game waits for it (`runtime/boot.c` `boot_wait`): about half a second
when warm; after a translator, GPU or driver change a "Preparing shaders" screen shows the progress (`ui/menu.cpp`
`menu_boot_frame`). `NX2NAVI_PREWARM_WAIT_MS` caps the wait, after which the rest finishes in the background.
`menu_boot_frame`). With the graphics pipeline library the boot builds recorded pipelines in record order for at
most 20 s after the shaders are ready and leaves the rest to the pipeline workers below normal priority; a draw
that meets one before it is built gets a fast-linked pipeline (`vk_pipeline.c` `PREWARM_PIPE_MS`).
`NX2NAVI_PREWARM_FULL=1`, and drivers without the library, wait for every pipeline. `NX2NAVI_PREWARM_WAIT_MS` caps
the whole wait, after which the rest finishes in the background.
- Mesh and task shaders run as `VK_EXT_mesh_shader`; indirect mesh draws read NVN's `{count, first}` record through a
ring of Vulkan `{x, y, z}` records filled on the GPU.
- Pipelines: `VK_EXT_graphics_pipeline_library` where the driver has it: stage libraries compiled ahead, a fast link

View File

@@ -228,6 +228,13 @@ Before a commit:
`git config core.hooksPath .githooks` runs it before each local commit.
- `build\<ver>\nx2navi.exe --selftest`: layout, dispatch table and host-function registry, without game files.
A change to the in-game menu's look (`runtime/ui/menu_skin.cpp`, `option_rows.cpp`, `perf_panel.cpp`) can be looked at
without game files: `python tools/menu_preview.py` builds the menu's own drawing code with the host's clang++ and
writes the Screen tab and the full performance overlay to `build/menu-preview/` (a stand-in font, and the window's
plain border in place of the game's ornamented frame). With `--romfs <extracted RomFS folder>` it draws the game's
interface textures as the game does; `python tools/ui_textures.py <folder>` lists the textures there and dumps those
matching `--match` to `build/ui-textures/`.
A change to the button prompts (`tools/make_romfs_patches.py --prompts`, `tools/button_icons.py`) passes
`.venv-py314\Scripts\python tests\romfs_prompts_test.py`, which runs the tool on a RomFS it makes itself (no game
files); a change to the settings passes `nx2navi.exe --probe-cfg`.

View File

@@ -23,20 +23,23 @@ game. Saves and settings are never touched.
## Look
The window follows the game's options menu, with colours sampled from it: a wooden title band with the Triforce and
the name, bronze label cells, dark value cells, orange On/Off choices, a white glowing cursor around the selected
The window follows the game's options menu, with colours sampled from it: a wooden band across the whole top with
Navi and the name, which is also the window's title bar (no system title bar: drag the band to move the window,
minimise and close sit at its right end, Alt+F4 closes), bronze label cells, dark value cells, orange On/Off choices, a white glowing cursor around the selected
row, the menu's filigree dividers and a description box for the selected row. All text is in the game's own Cetra
Display: the launcher decodes it from the player's dump or built game into `data/cache/fonts` (as
`runtime/menu_assets.c` does for the in-game menu) and uses the system's interface font until then. Every picture is
drawn in code (`launcher/ui/art.py`); no texture, font or sound of the game is in the repository or the release.
The Triforce is also the icon of `NX2navi.exe`, `NX2navi-Setup.exe`, the window and the taskbar
(`art.triforce_ico`, written into the release by `tools/make_launcher.py`).
Navi (`art.NAVI`, 22 x 21 pixels, drawn in whole pixels at every size) is also the icon of `NX2navi.exe`,
`NX2navi-Setup.exe`, the window and its taskbar button (`art.navi_ico`, written into the release by
`tools/make_launcher.py`). The window sets that icon on itself, and the setup's shortcuts carry the window's taskbar
identity (`NX2navi.Launcher`), so the taskbar groups and pins it under NX2navi rather than Python.
Screens (`launcher/ui/app.py`): first run (dump, install folder, Desktop Shortcut, Check for Updates at Start,
Update Channel, Begin Installation); the steps of an install, update or rebuild with their progress, the build log's
last lines and Cancel, and after a failure Try Again and Copy Report; home (Play, Updates, Game Folder, Rebuild the
Game, Settings); settings (update check, Hide Launcher While Playing, Keep Build Cache, Update Channel, the folders,
last lines and Cancel, and after a failure Try Again and Copy Report; home (Play, Updates, Game Folder, Import Saves,
Rebuild the Game, Settings); settings (update check, Hide Launcher While Playing, Keep Build Cache, Update Channel, the folders,
Open Logs Folder, Uninstall, Back); uninstall (keep or remove the built game).
Controls: the mouse (hover moves the cursor, a click selects); the keyboard (arrows, Enter or Space, Esc, F5 checks
@@ -48,7 +51,7 @@ for updates); an XInput gamepad (d-pad or left stick, A, B, Y).
```
NX2navi.exe launcher/native/nx2navi.c: swaps in a staged Python, starts app\<current>\launcher\start.pyw
python/ python.org's embeddable Python 3.14.9 with tkinter, numpy, lz4 and zstandard
python/ python.org's embeddable Python 3.14.9 with tkinter, numpy, lz4, zstandard and truststore
app/current.txt the version the stub starts; app/previous.txt the one before it
app/<version>/ launcher/, runtime/, recomp/, tools/, bin/nx2navi_xlate.exe, VERSION,
toolchain.lock.json (the pinned build tools), feed.json (where updates come from)
@@ -56,7 +59,8 @@ pydeps/ zbic, downloaded from PyPI on this PC
toolchain/msys64/ucrt64 MSYS2's clang and libraries, downloaded from the MSYS2 repository on this PC
toolchain/pkgs/ the downloaded package files
data/launcher.json settings; data/logs/launcher.log (and crash.log); data/cache/fonts
game/ the built game, unless another folder was chosen (its layout: README-BuildKit.md)
game/ the built game, unless another folder was chosen (its layout: README-BuildKit.md), with
saves/ for saves to import (below)
```
A path the Windows ANSI code page cannot hold breaks the MSYS2 tools, so the toolchain then goes to
@@ -67,12 +71,23 @@ A path the Windows ANSI code page cannot hold breaks the MSYS2 tools, so the too
`jobs.InstallJob`, one row per step:
1. **Gather Build Tools** (`launcher/toolchain.py`): the packages `toolchain.lock.json` pins (MSYS2 UCRT64 clang,
gcc, SDL3, Opus, libvpx, dav1d, zstd, Vulkan headers and loader, VMA and their dependencies: 31 packages, 170 MB),
gcc, lld, SDL3, Opus, libvpx, dav1d, zstd, Vulkan headers and loader, VMA and their dependencies: about 32
packages, 175 MB; and, when the release found the same clang version there, MSYS2 CLANG64's clang with the DLLs
it loads: 9 packages, about 100 MB),
each checked against its sha256 and unpacked into `toolchain/msys64.new`, which replaces `toolchain/msys64` once
complete. No MSYS2 shell or pacman runs. Then zbic: the wheel for the bundled Python from PyPI, checked against
PyPI's sha256, into `pydeps/`.
2. **Inspect Your Game** to **Compile**: `tools/make_game.py --nup <dump> --out <game> --msys <toolchain> --events`,
whose `@@` progress lines drive the rows (check, exefs, romfs, lift, compile).
whose `@@` progress lines drive the rows (check, exefs, romfs, lift, compile). How it keeps the build short:
- The RomFS image (26 GB, mostly disk work) is packed beside the lift and the compile (processor work), so a first
build takes about as long as the longer of the two. Only `romfs.c`, which includes the RomFS index's
`romfs_config.h`, waits for the image.
- Every logical processor compiles, the largest files first, so no big file starts last and runs on alone.
- The lifted code compiles with MSYS2 CLANG64's clang when the toolchain has it: the same LLVM release as
UCRT64's, built with profile-guided optimisation, about a fifth faster on this code and writing the same objects
(it reads UCRT64's headers; `make_game.py` `fast_clang` checks that both report the same version and target).
- The lifted code is compiled for the processor's own level, x86-64-v3 (AVX2, FMA) or v2 (`cpu_level`): the game
runs on the PC that builds it, and a generic build would do every fused multiply-add of the game in software.
3. **Prepare the Shaders**: make_game.py's last step translates every shader of the game into the game's shader
cache and the driver's own cache (`tools/prepare_shaders.py`), so play does not stop to compile them.
4. The game's Cetra Display is decoded for the window.
@@ -100,12 +115,19 @@ The stub swaps `python.next` in while no Python runs, then starts the new versio
`jobs.ContinueUpdateJob`:
4. **Update Build Tools**: only when the new release pins other packages.
5. **Translate the Code** and **Compile Changed Files**: `make_game.py --rebuild`. The exefs and romfs steps depend
only on the game files, so they are kept and the dump is not needed again; the lift runs again and the compile
step compiles only the files whose source, included headers, flags or compiler changed (`build-cache/obj/
objects.json`, from clang's dependency files).
6. **Swap In the New Build**: the link writes `ocarina.exe.new`, which replaces `ocarina.exe` only once it is
complete. A failed rebuild leaves the old build playable; Rebuild the Game tries again.
5. **Translate the Code** and **Compile Changed Files**: `make_game.py --rebuild`. Most updates compile almost
nothing:
- The exefs and romfs steps depend only on the game files: they are kept, and the dump is not needed again.
- The lift runs only when the lifter, its patch table or the bring-up fixes changed.
- The runtime comes prebuilt with the release (`app/<version>/lib/libnx2navirt.a`, compiled by
`make_game.py --make-runtime-lib` with the release's own compiler). Only the few runtime files that include a
header written per install (`layout.h`, `patches.h`, `romfs_config.h`) are compiled on the player's PC.
- Every other file compiles only when its source, a header it includes, the flags or the compiler changed
(`build-cache/obj/objects.json`, from clang's dependency files). The kit's folder is not part of that key, so the
objects of the version before are reused although each version lives in its own folder. The lifted code
includes only `cpu.h` and `simd.h`: a change there compiles it all again, which is rare.
6. **Swap In the New Build**: lld links `ocarina.exe.new`, which replaces `ocarina.exe` only once it is complete. A
failed rebuild leaves the old build playable; Rebuild the Game tries again.
7. **Prepare the Shaders** again for the new build; shaders the translator already prepared come from the cache.
8. Every version but the current and the previous one is deleted.
@@ -114,6 +136,32 @@ When the new version fails to start (exit code 86 from `start.pyw`, or any error
runs. Keep Build Cache off deletes the lifted C and the objects (about 5 GB) after each build but keeps
`build-cache/state.json`, so updates still rebuild without the dump (compiling everything).
## Importing a save
Import Saves (home, under Game Folder; `launcher/saves.py`) makes another save the game's own. The game folder has
a `saves` folder with a README for it: a save put there (its files, or the folder that holds them) is taken first,
and the row then reads "From the Saves Folder"; otherwise selecting the row asks for the folder. It recognises a
Switch save backup (the save's files), an emulator's save folder (nested folders named by user and title ids), and
another NX2navi game folder or its profile folder (the account folder inside). A folder with more than 4,000 files
or 1 GB is refused as picked by mistake.
The game keeps its save in `profile/save/<account>`, named after `profile/account.uid` (made as the game would when
it never ran). The import copies the save the game would load now (a finished commit still waiting in
`profile/save-tx`, else the account folder) to `saves/backup <date time>`, puts the new files in place and empties
`profile/save-tx`, so the game loads the imported save. A save taken from the saves folder moves to
`saves/imported <date time>`. It refuses while the game runs.
## Downloads
Every download (`launcher/net.py`) comes over HTTPS and is checked against a sha256. A server's certificate is checked
first by Python's OpenSSL against the certificates in Windows' stores and, when that refuses it, by Windows' own
certificate check (the truststore package of the bundled Python; not under Wine, whose crypt32 passes a chain that
stops at an issuer it does not know). OpenSSL sees only the roots the PC already holds, and it takes an expired
cross-signature still in a store before the server's valid one ("certificate has expired" on a valid chain); Windows
weighs every chain as its own programs do and downloads a missing root from Windows Update. A certificate both refuse
stops the step with what each check said, the PC's clock against the server's (the Date of a plain-HTTP request to
it), and what to check: the date, time and time zone, or security software that scans encrypted connections.
## Uninstalling
Settings > Uninstall NX2navi, or Settings > Apps > NX2navi (which runs `NX2navi.exe --uninstall`): the launcher
@@ -130,7 +178,9 @@ Built Game" is On. The dump is never touched.
## Releases
1. Bump `VERSION`; build the shader translator (`tools/build_xlate.py`).
2. With Python 3.14, numpy (the icon is drawn with it, `launcher/ui/art.py`) and uv:
2. With Python 3.14, numpy (the icon is drawn with it, `launcher/ui/art.py`) and uv, on Windows (the prebuilt runtime
needs MSYS2's own clang; on Linux add `--wine` to run it through Wine, else the release ships without it and
players compile `runtime/` themselves):
```powershell
uv run --no-project --python 3.14 --with numpy tools\make_launcher.py --xlate build\xlate --ffx amd_fidelityfx_vk.dll --feed owner/name --notes notes.md
@@ -140,7 +190,8 @@ Built Game" is On. The dump is never touched.
On Linux it cross-compiles the two programs with clang, lld and llvm-windres against the same MSYS2 packages and
unpacks Tcl/Tk with msiextract (msitools). It writes into `build/launcher-release`: `NX2navi-Setup-<version>.exe`
(the setup program with the launcher folder appended as a zip), `NX2navi-app-<version>.zip`,
(the setup program with the launcher folder appended as a zip), `NX2navi-app-<version>.zip` (with the prebuilt
runtime),
`NX2navi-runtime-<id>.zip` and `NX2navi-update.json`. `check_content` runs over the folder first.
3. Upload those four files to GitHub release `v<version>` of the feed repository, by hand or with `--publish`
(`gh release create`; add `--draft` to look before it goes out). Launchers see it at their next start.
@@ -151,10 +202,15 @@ repository keeps old package files, so a release keeps building with the compile
## Tests
`python tests/launcher_test.py` (Python 3.14): versions, settings, the toolchain resolver and installer against a
local fake repository, resumed and checked downloads, the GitHub and URL feeds, staging, switching, rollback and
local fake repository, resumed and checked downloads, the certificate checks against a local HTTPS server (its
certificates made with the `openssl` command), the GitHub and URL feeds, staging, switching, rollback and
pruning, unsafe archives, the font decoding on a system font wrapped like the game's, `make_game.py` on a synthetic
`.nup` (events, skipped steps, `--rebuild` without the dump, step keys), the compile step's object cache with the
system's C compiler, the install job with a fake `make_game.py`, and the window's selftest when a display is there.
`.nup` (events, skipped steps, `--rebuild` without the dump, step keys), the RomFS step beside the lift and compile
and its failure stopping the build, the compile step's object cache with the system's C compiler and its
largest-first order, its windres call under folders with spaces in their names, CLANG64's clang in the lock and the
install, the save import (the account folder, the save folders it recognises, backup, the commit journal, the saves
folder, refusals), the install job with a fake
`make_game.py`, and the window's selftest when a display is there.
## Requirements and limits