mirror of
https://github.com/tangerineturtle1/NX2navi.git
synced 2026-11-17 11:29:58 -04:00
Redraw the in-game menu with the game's own interface textures, and bring in the launcher's save import and faster builds.
In-game menu: the settings window, its tabs, rows, buttons and cursor, the performance overlay and the Preparing shaders box are drawn with the game's interface textures where the RomFS holds them (runtime/ui/menu_skin.cpp, perf_panel.cpp; plain shapes until they are decoded). Restart joins Continue and Quit, each behind a confirmation; VSync is an on/off switch; DLSS is listed only on NVIDIA GPUs; the menu sounds are quieter; Picture shape is gone and the setting descriptions are shorter; edge cases of input, layout and fonts are fixed. tools/menu_preview.py draws the menu without the game. Launcher: Import Saves takes a save from the game folder's saves/ folder or one the player picks; downloads fall back to Windows' own certificate check when Python's refuses a server; updates are faster; the title bar is drawn in the window and Navi is every icon. tools/make_game.py builds with CLANG64's clang where the launcher installed it, packs the RomFS beside the lift and the compile, takes a prebuilt runtime, and compiles the game's resources without cmd.exe, so a user folder with a space in its name builds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -49,9 +49,9 @@ With extracted folders instead, use `--exefs "C:\path\exefs" --romfs "C:\path\ro
|
||||
check that your dump is the right version, add `--check` (it builds nothing).
|
||||
|
||||
The build checks the version, unpacks the program files, packs the 26 GB RomFS image (every file is checked against
|
||||
the dump's own checksums), turns the game's code into C and compiles it (about 6 minutes on a 16-thread Ryzen 7; longer on fewer cores),
|
||||
then prepares every one of the game's shaders for your graphics card (about 5 more minutes), so the game does not stop
|
||||
to compile shaders while you play.
|
||||
the dump's own checksums) while it turns the game's code into C and compiles it (about 6 minutes on a 16-thread
|
||||
Ryzen 7; longer on fewer cores), then prepares every one of the game's shaders for your graphics card (about 5 more
|
||||
minutes), so the game does not stop to compile shaders while you play. The game it builds is for this PC's processor.
|
||||
You can use your PC meanwhile. If it stops,
|
||||
run the same command again: finished steps are skipped.
|
||||
|
||||
@@ -62,7 +62,9 @@ When it says `finished`, start `C:\Games\Ocarina\ocarina.exe`. You can delete `C
|
||||
|
||||
- **F1** opens the settings menu. Settings are saved in `nx2navi.ini` beside the game; saves go to `profile\`.
|
||||
- Every shader was prepared during the build. After an update of your graphics driver, the next start shows
|
||||
"Preparing shaders" once while the driver rebuilds them.
|
||||
"Preparing shaders" once while the driver rebuilds them. With a driver that has Vulkan's graphics pipeline library
|
||||
it lasts at most about 20 seconds once the shaders are translated, and the remaining pipelines are built in the
|
||||
background while you play.
|
||||
- **Screen** in the menu has Upscaling (DLSS on NVIDIA RTX cards, FSR 3.1 on any card), Frame generation (FSR 3.1; it
|
||||
needs DLSS or FSR 3.1 upscaling on), HDR (Native is the game's own HDR; turn HDR on in Windows first) and the
|
||||
resolution. Those marked "after restart" take effect when the game starts again.
|
||||
|
||||
@@ -85,9 +85,13 @@ keeps per-core data without locks and relies on that, so the host models it (`ru
|
||||
- Shaders: Ampere (SM86) code is lowered to Maxwell (`tools/xlate/AmpereLowering.cs`) and translated to SPIR-V by the
|
||||
pinned shader translator (`tools/vendor/ryujinx`, 1.3.2) with our patches (`tools/xlate/patches/`), built with .NET 10 NativeAOT as
|
||||
`nx2navi_xlate.exe` and run as separate processes. Translations are cached on disk; shaders and pipelines seen in
|
||||
play are prepared at the next boot. The game waits for all of it (`runtime/boot.c` `boot_wait`): about half a second
|
||||
play are prepared at the next boot. The game waits for it (`runtime/boot.c` `boot_wait`): about half a second
|
||||
when warm; after a translator, GPU or driver change a "Preparing shaders" screen shows the progress (`ui/menu.cpp`
|
||||
`menu_boot_frame`). `NX2NAVI_PREWARM_WAIT_MS` caps the wait, after which the rest finishes in the background.
|
||||
`menu_boot_frame`). With the graphics pipeline library the boot builds recorded pipelines in record order for at
|
||||
most 20 s after the shaders are ready and leaves the rest to the pipeline workers below normal priority; a draw
|
||||
that meets one before it is built gets a fast-linked pipeline (`vk_pipeline.c` `PREWARM_PIPE_MS`).
|
||||
`NX2NAVI_PREWARM_FULL=1`, and drivers without the library, wait for every pipeline. `NX2NAVI_PREWARM_WAIT_MS` caps
|
||||
the whole wait, after which the rest finishes in the background.
|
||||
- Mesh and task shaders run as `VK_EXT_mesh_shader`; indirect mesh draws read NVN's `{count, first}` record through a
|
||||
ring of Vulkan `{x, y, z}` records filled on the GPU.
|
||||
- Pipelines: `VK_EXT_graphics_pipeline_library` where the driver has it: stage libraries compiled ahead, a fast link
|
||||
|
||||
@@ -228,6 +228,13 @@ Before a commit:
|
||||
`git config core.hooksPath .githooks` runs it before each local commit.
|
||||
- `build\<ver>\nx2navi.exe --selftest`: layout, dispatch table and host-function registry, without game files.
|
||||
|
||||
A change to the in-game menu's look (`runtime/ui/menu_skin.cpp`, `option_rows.cpp`, `perf_panel.cpp`) can be looked at
|
||||
without game files: `python tools/menu_preview.py` builds the menu's own drawing code with the host's clang++ and
|
||||
writes the Screen tab and the full performance overlay to `build/menu-preview/` (a stand-in font, and the window's
|
||||
plain border in place of the game's ornamented frame). With `--romfs <extracted RomFS folder>` it draws the game's
|
||||
interface textures as the game does; `python tools/ui_textures.py <folder>` lists the textures there and dumps those
|
||||
matching `--match` to `build/ui-textures/`.
|
||||
|
||||
A change to the button prompts (`tools/make_romfs_patches.py --prompts`, `tools/button_icons.py`) passes
|
||||
`.venv-py314\Scripts\python tests\romfs_prompts_test.py`, which runs the tool on a RomFS it makes itself (no game
|
||||
files); a change to the settings passes `nx2navi.exe --probe-cfg`.
|
||||
|
||||
@@ -23,20 +23,23 @@ game. Saves and settings are never touched.
|
||||
|
||||
## Look
|
||||
|
||||
The window follows the game's options menu, with colours sampled from it: a wooden title band with the Triforce and
|
||||
the name, bronze label cells, dark value cells, orange On/Off choices, a white glowing cursor around the selected
|
||||
The window follows the game's options menu, with colours sampled from it: a wooden band across the whole top with
|
||||
Navi and the name, which is also the window's title bar (no system title bar: drag the band to move the window,
|
||||
minimise and close sit at its right end, Alt+F4 closes), bronze label cells, dark value cells, orange On/Off choices, a white glowing cursor around the selected
|
||||
row, the menu's filigree dividers and a description box for the selected row. All text is in the game's own Cetra
|
||||
Display: the launcher decodes it from the player's dump or built game into `data/cache/fonts` (as
|
||||
`runtime/menu_assets.c` does for the in-game menu) and uses the system's interface font until then. Every picture is
|
||||
drawn in code (`launcher/ui/art.py`); no texture, font or sound of the game is in the repository or the release.
|
||||
|
||||
The Triforce is also the icon of `NX2navi.exe`, `NX2navi-Setup.exe`, the window and the taskbar
|
||||
(`art.triforce_ico`, written into the release by `tools/make_launcher.py`).
|
||||
Navi (`art.NAVI`, 22 x 21 pixels, drawn in whole pixels at every size) is also the icon of `NX2navi.exe`,
|
||||
`NX2navi-Setup.exe`, the window and its taskbar button (`art.navi_ico`, written into the release by
|
||||
`tools/make_launcher.py`). The window sets that icon on itself, and the setup's shortcuts carry the window's taskbar
|
||||
identity (`NX2navi.Launcher`), so the taskbar groups and pins it under NX2navi rather than Python.
|
||||
|
||||
Screens (`launcher/ui/app.py`): first run (dump, install folder, Desktop Shortcut, Check for Updates at Start,
|
||||
Update Channel, Begin Installation); the steps of an install, update or rebuild with their progress, the build log's
|
||||
last lines and Cancel, and after a failure Try Again and Copy Report; home (Play, Updates, Game Folder, Rebuild the
|
||||
Game, Settings); settings (update check, Hide Launcher While Playing, Keep Build Cache, Update Channel, the folders,
|
||||
last lines and Cancel, and after a failure Try Again and Copy Report; home (Play, Updates, Game Folder, Import Saves,
|
||||
Rebuild the Game, Settings); settings (update check, Hide Launcher While Playing, Keep Build Cache, Update Channel, the folders,
|
||||
Open Logs Folder, Uninstall, Back); uninstall (keep or remove the built game).
|
||||
|
||||
Controls: the mouse (hover moves the cursor, a click selects); the keyboard (arrows, Enter or Space, Esc, F5 checks
|
||||
@@ -48,7 +51,7 @@ for updates); an XInput gamepad (d-pad or left stick, A, B, Y).
|
||||
|
||||
```
|
||||
NX2navi.exe launcher/native/nx2navi.c: swaps in a staged Python, starts app\<current>\launcher\start.pyw
|
||||
python/ python.org's embeddable Python 3.14.9 with tkinter, numpy, lz4 and zstandard
|
||||
python/ python.org's embeddable Python 3.14.9 with tkinter, numpy, lz4, zstandard and truststore
|
||||
app/current.txt the version the stub starts; app/previous.txt the one before it
|
||||
app/<version>/ launcher/, runtime/, recomp/, tools/, bin/nx2navi_xlate.exe, VERSION,
|
||||
toolchain.lock.json (the pinned build tools), feed.json (where updates come from)
|
||||
@@ -56,7 +59,8 @@ pydeps/ zbic, downloaded from PyPI on this PC
|
||||
toolchain/msys64/ucrt64 MSYS2's clang and libraries, downloaded from the MSYS2 repository on this PC
|
||||
toolchain/pkgs/ the downloaded package files
|
||||
data/launcher.json settings; data/logs/launcher.log (and crash.log); data/cache/fonts
|
||||
game/ the built game, unless another folder was chosen (its layout: README-BuildKit.md)
|
||||
game/ the built game, unless another folder was chosen (its layout: README-BuildKit.md), with
|
||||
saves/ for saves to import (below)
|
||||
```
|
||||
|
||||
A path the Windows ANSI code page cannot hold breaks the MSYS2 tools, so the toolchain then goes to
|
||||
@@ -67,12 +71,23 @@ A path the Windows ANSI code page cannot hold breaks the MSYS2 tools, so the too
|
||||
`jobs.InstallJob`, one row per step:
|
||||
|
||||
1. **Gather Build Tools** (`launcher/toolchain.py`): the packages `toolchain.lock.json` pins (MSYS2 UCRT64 clang,
|
||||
gcc, SDL3, Opus, libvpx, dav1d, zstd, Vulkan headers and loader, VMA and their dependencies: 31 packages, 170 MB),
|
||||
gcc, lld, SDL3, Opus, libvpx, dav1d, zstd, Vulkan headers and loader, VMA and their dependencies: about 32
|
||||
packages, 175 MB; and, when the release found the same clang version there, MSYS2 CLANG64's clang with the DLLs
|
||||
it loads: 9 packages, about 100 MB),
|
||||
each checked against its sha256 and unpacked into `toolchain/msys64.new`, which replaces `toolchain/msys64` once
|
||||
complete. No MSYS2 shell or pacman runs. Then zbic: the wheel for the bundled Python from PyPI, checked against
|
||||
PyPI's sha256, into `pydeps/`.
|
||||
2. **Inspect Your Game** to **Compile**: `tools/make_game.py --nup <dump> --out <game> --msys <toolchain> --events`,
|
||||
whose `@@` progress lines drive the rows (check, exefs, romfs, lift, compile).
|
||||
whose `@@` progress lines drive the rows (check, exefs, romfs, lift, compile). How it keeps the build short:
|
||||
- The RomFS image (26 GB, mostly disk work) is packed beside the lift and the compile (processor work), so a first
|
||||
build takes about as long as the longer of the two. Only `romfs.c`, which includes the RomFS index's
|
||||
`romfs_config.h`, waits for the image.
|
||||
- Every logical processor compiles, the largest files first, so no big file starts last and runs on alone.
|
||||
- The lifted code compiles with MSYS2 CLANG64's clang when the toolchain has it: the same LLVM release as
|
||||
UCRT64's, built with profile-guided optimisation, about a fifth faster on this code and writing the same objects
|
||||
(it reads UCRT64's headers; `make_game.py` `fast_clang` checks that both report the same version and target).
|
||||
- The lifted code is compiled for the processor's own level, x86-64-v3 (AVX2, FMA) or v2 (`cpu_level`): the game
|
||||
runs on the PC that builds it, and a generic build would do every fused multiply-add of the game in software.
|
||||
3. **Prepare the Shaders**: make_game.py's last step translates every shader of the game into the game's shader
|
||||
cache and the driver's own cache (`tools/prepare_shaders.py`), so play does not stop to compile them.
|
||||
4. The game's Cetra Display is decoded for the window.
|
||||
@@ -100,12 +115,19 @@ The stub swaps `python.next` in while no Python runs, then starts the new versio
|
||||
`jobs.ContinueUpdateJob`:
|
||||
|
||||
4. **Update Build Tools**: only when the new release pins other packages.
|
||||
5. **Translate the Code** and **Compile Changed Files**: `make_game.py --rebuild`. The exefs and romfs steps depend
|
||||
only on the game files, so they are kept and the dump is not needed again; the lift runs again and the compile
|
||||
step compiles only the files whose source, included headers, flags or compiler changed (`build-cache/obj/
|
||||
objects.json`, from clang's dependency files).
|
||||
6. **Swap In the New Build**: the link writes `ocarina.exe.new`, which replaces `ocarina.exe` only once it is
|
||||
complete. A failed rebuild leaves the old build playable; Rebuild the Game tries again.
|
||||
5. **Translate the Code** and **Compile Changed Files**: `make_game.py --rebuild`. Most updates compile almost
|
||||
nothing:
|
||||
- The exefs and romfs steps depend only on the game files: they are kept, and the dump is not needed again.
|
||||
- The lift runs only when the lifter, its patch table or the bring-up fixes changed.
|
||||
- The runtime comes prebuilt with the release (`app/<version>/lib/libnx2navirt.a`, compiled by
|
||||
`make_game.py --make-runtime-lib` with the release's own compiler). Only the few runtime files that include a
|
||||
header written per install (`layout.h`, `patches.h`, `romfs_config.h`) are compiled on the player's PC.
|
||||
- Every other file compiles only when its source, a header it includes, the flags or the compiler changed
|
||||
(`build-cache/obj/objects.json`, from clang's dependency files). The kit's folder is not part of that key, so the
|
||||
objects of the version before are reused although each version lives in its own folder. The lifted code
|
||||
includes only `cpu.h` and `simd.h`: a change there compiles it all again, which is rare.
|
||||
6. **Swap In the New Build**: lld links `ocarina.exe.new`, which replaces `ocarina.exe` only once it is complete. A
|
||||
failed rebuild leaves the old build playable; Rebuild the Game tries again.
|
||||
7. **Prepare the Shaders** again for the new build; shaders the translator already prepared come from the cache.
|
||||
8. Every version but the current and the previous one is deleted.
|
||||
|
||||
@@ -114,6 +136,32 @@ When the new version fails to start (exit code 86 from `start.pyw`, or any error
|
||||
runs. Keep Build Cache off deletes the lifted C and the objects (about 5 GB) after each build but keeps
|
||||
`build-cache/state.json`, so updates still rebuild without the dump (compiling everything).
|
||||
|
||||
## Importing a save
|
||||
|
||||
Import Saves (home, under Game Folder; `launcher/saves.py`) makes another save the game's own. The game folder has
|
||||
a `saves` folder with a README for it: a save put there (its files, or the folder that holds them) is taken first,
|
||||
and the row then reads "From the Saves Folder"; otherwise selecting the row asks for the folder. It recognises a
|
||||
Switch save backup (the save's files), an emulator's save folder (nested folders named by user and title ids), and
|
||||
another NX2navi game folder or its profile folder (the account folder inside). A folder with more than 4,000 files
|
||||
or 1 GB is refused as picked by mistake.
|
||||
|
||||
The game keeps its save in `profile/save/<account>`, named after `profile/account.uid` (made as the game would when
|
||||
it never ran). The import copies the save the game would load now (a finished commit still waiting in
|
||||
`profile/save-tx`, else the account folder) to `saves/backup <date time>`, puts the new files in place and empties
|
||||
`profile/save-tx`, so the game loads the imported save. A save taken from the saves folder moves to
|
||||
`saves/imported <date time>`. It refuses while the game runs.
|
||||
|
||||
## Downloads
|
||||
|
||||
Every download (`launcher/net.py`) comes over HTTPS and is checked against a sha256. A server's certificate is checked
|
||||
first by Python's OpenSSL against the certificates in Windows' stores and, when that refuses it, by Windows' own
|
||||
certificate check (the truststore package of the bundled Python; not under Wine, whose crypt32 passes a chain that
|
||||
stops at an issuer it does not know). OpenSSL sees only the roots the PC already holds, and it takes an expired
|
||||
cross-signature still in a store before the server's valid one ("certificate has expired" on a valid chain); Windows
|
||||
weighs every chain as its own programs do and downloads a missing root from Windows Update. A certificate both refuse
|
||||
stops the step with what each check said, the PC's clock against the server's (the Date of a plain-HTTP request to
|
||||
it), and what to check: the date, time and time zone, or security software that scans encrypted connections.
|
||||
|
||||
## Uninstalling
|
||||
|
||||
Settings > Uninstall NX2navi, or Settings > Apps > NX2navi (which runs `NX2navi.exe --uninstall`): the launcher
|
||||
@@ -130,7 +178,9 @@ Built Game" is On. The dump is never touched.
|
||||
## Releases
|
||||
|
||||
1. Bump `VERSION`; build the shader translator (`tools/build_xlate.py`).
|
||||
2. With Python 3.14, numpy (the icon is drawn with it, `launcher/ui/art.py`) and uv:
|
||||
2. With Python 3.14, numpy (the icon is drawn with it, `launcher/ui/art.py`) and uv, on Windows (the prebuilt runtime
|
||||
needs MSYS2's own clang; on Linux add `--wine` to run it through Wine, else the release ships without it and
|
||||
players compile `runtime/` themselves):
|
||||
|
||||
```powershell
|
||||
uv run --no-project --python 3.14 --with numpy tools\make_launcher.py --xlate build\xlate --ffx amd_fidelityfx_vk.dll --feed owner/name --notes notes.md
|
||||
@@ -140,7 +190,8 @@ Built Game" is On. The dump is never touched.
|
||||
|
||||
On Linux it cross-compiles the two programs with clang, lld and llvm-windres against the same MSYS2 packages and
|
||||
unpacks Tcl/Tk with msiextract (msitools). It writes into `build/launcher-release`: `NX2navi-Setup-<version>.exe`
|
||||
(the setup program with the launcher folder appended as a zip), `NX2navi-app-<version>.zip`,
|
||||
(the setup program with the launcher folder appended as a zip), `NX2navi-app-<version>.zip` (with the prebuilt
|
||||
runtime),
|
||||
`NX2navi-runtime-<id>.zip` and `NX2navi-update.json`. `check_content` runs over the folder first.
|
||||
3. Upload those four files to GitHub release `v<version>` of the feed repository, by hand or with `--publish`
|
||||
(`gh release create`; add `--draft` to look before it goes out). Launchers see it at their next start.
|
||||
@@ -151,10 +202,15 @@ repository keeps old package files, so a release keeps building with the compile
|
||||
## Tests
|
||||
|
||||
`python tests/launcher_test.py` (Python 3.14): versions, settings, the toolchain resolver and installer against a
|
||||
local fake repository, resumed and checked downloads, the GitHub and URL feeds, staging, switching, rollback and
|
||||
local fake repository, resumed and checked downloads, the certificate checks against a local HTTPS server (its
|
||||
certificates made with the `openssl` command), the GitHub and URL feeds, staging, switching, rollback and
|
||||
pruning, unsafe archives, the font decoding on a system font wrapped like the game's, `make_game.py` on a synthetic
|
||||
`.nup` (events, skipped steps, `--rebuild` without the dump, step keys), the compile step's object cache with the
|
||||
system's C compiler, the install job with a fake `make_game.py`, and the window's selftest when a display is there.
|
||||
`.nup` (events, skipped steps, `--rebuild` without the dump, step keys), the RomFS step beside the lift and compile
|
||||
and its failure stopping the build, the compile step's object cache with the system's C compiler and its
|
||||
largest-first order, its windres call under folders with spaces in their names, CLANG64's clang in the lock and the
|
||||
install, the save import (the account folder, the save folders it recognises, backup, the commit journal, the saves
|
||||
folder, refusals), the install job with a fake
|
||||
`make_game.py`, and the window's selftest when a display is there.
|
||||
|
||||
## Requirements and limits
|
||||
|
||||
|
||||
Reference in New Issue
Block a user