Merge the Linux pull requests into the 0.7.0 release line.

Eighteen commits, one per Linux pull request that was not in yet: the build fix for a source checkout, NVIDIA's shader
disk cache only with NVIDIA's driver, tools/run_game.py for a Linux game folder, a memory limit the low-memory handling
can read, the recording thread's core on Linux, shadow storage on Linux and Proton, the Steam Deck profile and prompts,
FSR 3.1 offered only where its library loads, DLSS frame generation through NGX on Linux, the AppImage with its window
and its workflow, and the processor level from /proc/cpuinfo. Two conflicts: the settings table keeps the area-load
settings under the new comment, and the example file keeps the lazy write-back as off by default and adds that it
stays off with shadow storage.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
chrg6
2026-11-10 07:31:55 -05:00
59 changed files with 3597 additions and 449 deletions

View File

@@ -25,7 +25,7 @@ downloads included, and keeps the game up to date. This page is the same build b
```
`zbic` unpacks the game's compressed program files. It is a separate GPL-2.0 package you install yourself; the kit
only calls it. `fonttools` and `pillow` draw the Xbox, PlayStation and keyboard and mouse button prompts, when the
only calls it. `fonttools` and `pillow` draw the Xbox, PlayStation, Steam Deck and keyboard and mouse button prompts, when the
kit has a `prompt-art` folder.
2. **MSYS2** from [msys2.org](https://www.msys2.org/), installed to the default `C:\msys64`. Open **MSYS2 UCRT64** from
@@ -53,7 +53,7 @@ The build checks the version, unpacks the program files, packs the 26 GB RomFS i
the dump's own checksums) while it turns the game's code into C and compiles it (about 6 minutes on a 16-thread
Ryzen 7; longer on fewer cores), then prepares every one of the game's shaders for your graphics card (about 5 more
minutes), so the game does not stop to compile shaders while you play. When the kit has a `prompt-art` folder, it
last draws the Xbox, PlayStation and keyboard and mouse button prompts from that art (about a minute) into the
last draws the Xbox, PlayStation, Steam Deck and keyboard and mouse button prompts from that art (about a minute) into the
`romfs-patches-*` folders beside the game. The game it builds is for this PC's processor. You can use your PC
meanwhile. If it stops, run the same command again: finished steps are skipped.
@@ -80,7 +80,9 @@ RADV). It takes clang, clang++ and lld from `PATH` and links against the system'
pinned 3.4.0 into `toolchain/vma/` and checks its hash.
2. The shader translator: a kit with `bin/nx2navi_xlate` has it; from a source checkout, build it once with
`python tools/build_xlate.py` (needs the .NET 10 SDK). The build copies it beside the game.
`python tools/build_xlate.py` (needs the .NET 10 SDK). The build copies it beside the game. A kit also carries the
runtime's own shaders compiled; from a source checkout the build compiles them first, with `glslc`, `spirv-as` and
`spirv-val` (on Arch the packages `shaderc` and `spirv-tools`).
3. Build, then play:
@@ -93,13 +95,42 @@ With 16 GB of RAM, add `--jobs 4`: the compiles run beside the 26 GB RomFS image
processor can fill memory and swap until the kernel stops processes of the desktop session (about 10 minutes on a
12-thread Ryzen with 4 jobs). Do not play or prepare shaders while it builds.
On NVIDIA, start the game with `NX2NAVI_SHADOW=0` if the 3D world stays black while the HUD draws (issue #72). The
profile, logs and settings are in the `--out` folder as on Windows.
The profile, logs and settings are in the `--out` folder as on Windows.
On NVIDIA the driver keeps the shaders it compiled in `nvidia-cache` beside `ocarina`, not in `~/.cache/nvidia`, where
one run adds at most 1 GiB while the preparation at the end of the build makes about 1.5 GB. The game sets the folder
and a limit of 8 GiB itself (`__GL_SHADER_DISK_CACHE_PATH` and `__GL_SHADER_DISK_CACHE_SIZE` in the environment take
precedence). Deleting the folder costs one slow start; `./ocarina --warm-shaders` fills it again.
one run adds at most 1 GiB while the preparation at the end of the build makes about 1.5 GB. The game makes and sets
the folder unless the environment sets `__GL_SHADER_DISK_CACHE_PATH`, and a limit of 8 GiB unless it sets
`__GL_SHADER_DISK_CACHE_SIZE`. When it cannot make or write the folder, it sets neither and the driver keeps its own
folder and limit; the log says which (`vk: NVIDIA's shader disk cache`). Deleting the folder costs one slow start;
`./ocarina --warm-shaders` fills it again.
The game reads the memory it may use: the limit of its cgroup v2 (`systemd-run --user --scope -p MemoryMax=16G
ocarina`, a container's) and the machine's `MemAvailable`, whichever leaves less. Linux does not fail an allocation when
memory runs short, it ends the process, so while less than the larger of 4 GB and a tenth of that limit is left the game
drops stale textures and evicts idle ones. The log names the limits at the first check (`platform: memory limits`). That
cannot give back what the game needs after its start: about 10 GB in Kokiri Forest at a 2160p game resolution.
AMD FSR 3.1 loads AMD's `amd_fidelityfx_vk.dll`, which only the Windows build can use: on Linux the menu offers neither
FSR 3.1 upscaling nor FSR 3.1 frame generation, and `taa = fsr3` or `framegen = fsr3` in `nx2navi.ini` leave the
game's own scaling and swapchain as they are (the log says so).
NVIDIA DLSS (RTX cards) needs the driver's own `libnvidia-ngx.so.1` on the library path (the proprietary driver's
utilities package, `nvidia-utils` on Arch) and NVIDIA's DLSS library `libnvidia-ngx-dlss.so.<version>` beside
`ocarina`. The AppImage's kit has the library in `bin/` with `libnvidia-ngx-dlss-NOTICE.txt`, which names its licence,
and the build copies both beside `ocarina` (`tools/make_appimage.py --no-dlss` leaves it out; such an AppImage
downloads the same pinned library from NVIDIA's repository after a build on a machine with NVIDIA's driver, checks its
SHA-256 and writes the notice, unless `NX2NAVI_NO_DLSS=1` is set or a library is already there). From a source checkout,
or to use another DLSS version, put the library there yourself: NVIDIA publishes it as
`lib/Linux_x86_64/rel/libnvidia-ngx-dlss.so.<version>` in [its DLSS repository](https://github.com/NVIDIA/DLSS), tagged
`v<version>`, and `tools/dlss_pin.py` names the version the kit carries. Later builds keep a library they did not
place. Without it DLSS does nothing on Linux.
DLSS Frame Generation (RTX 40 and 50 cards) runs through NGX itself on Linux, without Streamline. It needs NVIDIA's
`libnvidia-ngx-dlssg.so.<version>` beside `ocarina` (the same repository, `lib/Linux_x86_64/rel/`; no kit carries it)
and DLSS upscaling on when the game starts. *Frame generation* in the Screen menu then offers NVIDIA DLSS, which
switches at once, with a *DLSS multiplier* of 2x to 6x (above 2x on RTX 50 cards) or Dynamic, which generates only the
frames needed to reach the display's refresh rate (`framegen = dlss` and `fg_mult` in `nx2navi.ini`). The log says
`ngxfg: DLSS Frame Generation available` when NGX found the library.
`tools/make_appimage.py` packs this kit into one `NX2navi-Linux-<version>-x86_64.AppImage` with its own Python, for
players without a source checkout: it checks for the packages above, installs zbic with pip on its first run, picks
@@ -113,18 +144,20 @@ beside the game.
"Preparing shaders" once while the driver rebuilds them. With a driver that has Vulkan's graphics pipeline library
it lasts at most about 20 seconds once the shaders are translated, and the remaining pipelines are built in the
background while you play.
- **Screen** in the menu has Texture filtering (anisotropic, up to 16x, at once), Upscaling (DLSS on NVIDIA RTX cards, FSR 3.1 on any card), Frame generation (NVIDIA DLSS
on RTX 40 and 50 cards, with a DLSS multiplier of 2x, or 3x and 4x on RTX 50 cards; FSR 3.1 on any card; it needs
DLSS or FSR 3.1 upscaling on, and the frame limit then counts the frames on screen; with no limit chosen it is the
display's refresh rate), HDR (the game's picture tone mapped to your display, with its own grading rows, or Native, the game's own HDR; turn HDR on in Windows first) and the
- **Screen** in the menu has Texture filtering (anisotropic, up to 16x, at once), Upscaling (DLSS on NVIDIA RTX cards,
FSR 3.1 on any card where `amd_fidelityfx_vk.dll` loads), Frame generation (NVIDIA DLSS on RTX 40 and 50 cards, with
a DLSS multiplier of 2x, or 3x and 4x on RTX 50 cards; FSR 3.1 on any card where it loads; it needs DLSS or FSR 3.1
upscaling on, and the frame limit then counts the frames on screen; with no limit chosen it is the display's refresh
rate), HDR (the game's picture tone mapped to your display, with its own grading rows, or Native, the game's own HDR; turn HDR on in Windows first) and the
resolution. Those marked "after restart" take effect when the game starts again.
- **Controls** in the menu has Button prompts: the game's own, or Xbox, PlayStation or keyboard and mouse ones when
- **Controls** in the menu has Button prompts: the game's own, or Xbox, PlayStation, Steam Deck or keyboard and mouse ones when
the build drew them (a kit with `prompt-art`). It takes effect after a restart. The keyboard and mouse prompts show
the keys `nx2navi.ini` binds when the build runs: after rebinding keys, run
`py -I tools\make_game.py --rebuild --out "C:\Games\Ocarina"` to draw them again (it skips everything else).
- The build copies NVIDIA's DLSS library `nvngx_dlss.dll` (with `nvngx_dlss-NOTICE.txt`, which names its licence)
and FSR 3.1's `amd_fidelityfx_vk.dll` beside `ocarina.exe` from the kit's `bin\`. To use another DLSS version, put
your own `nvngx_dlss.dll` there: later builds keep it.
your own `nvngx_dlss.dll` there: later builds keep it. On Linux the library is `libnvidia-ngx-dlss.so.<version>`
(see "Linux").
- A kit with `bin\dlssg\` (made with `tools/make_kit.py --dlssg`) also brings DLSS Frame Generation on GeForce RTX 40
and 50 cards: the build copies `nx2navi_dlssg.dll`, NVIDIA Streamline's runtime, `nvngx_dlssg.dll` and their licence
texts beside `ocarina.exe`, and Frame generation in the Screen menu then offers DLSS too.

View File

@@ -235,17 +235,38 @@ NVIDIA's Linux driver compiles into a disk cache of its own. A process adds at m
folder, `~/.cache/nvidia`, is shared with every program. `--prepare-shaders` of game 1.1.1 builds 42,142 variants, about
1.5 GB: on the default limit about 30,700 of them were stored and the rest compiled again at their first draw in play.
`runtime/vk_device.c` therefore points `__GL_SHADER_DISK_CACHE_PATH` at `nvidia-cache` beside `ocarina` and the size at
8 GiB before the first Vulkan call, each unless the environment sets it, as it does `MESA_SHADER_CACHE_DIR` for RADV. The
8 GiB, each unless the environment sets it, as it does `MESA_SHADER_CACHE_DIR` for RADV, and only while NVIDIA's kernel
driver is loaded (`/proc/driver/nvidia/version`). The driver does not make that folder and caches nothing in a folder it
cannot write, so the game makes it, and when it cannot make or write it sets neither variable. A game run sets them in
`main` before any of its threads starts, since a thread can load the Vulkan driver before the instance is made. The
folder can be deleted; `ocarina --warm-shaders` fills it again.
Memory: on Windows the GPU imports guest memory and `GetWriteWatch` tracks the CPU's writes to it. On Linux,
re-protecting imported memory invalidates the driver's userptr buffer on every submit (about 1.3 ms per 16 MiB on
amdgpu), and importing pins the pages, which drops their write protection. Each CPU-visible pool therefore gets a
host-visible copy (shadow storage, `runtime/vk_memory.c`): the pages the CPU wrote go in before each submit, and the GPU's
writes come back at its fence. In play that costs about 0.2 ms a frame and doubles the memory of those pools (about
1.1 GB). `NX2NAVI_SHADOW=1` turns it on under Windows too. The Steam Deck's RADV has `VK_EXT_external_memory_host` as
writes come back at its fence. Pages both sides write keep a base copy, so each direction moves only its own bytes;
a VRAM mirror's write-back merge marks the pages it changed, and only those are compared at the fence; a lookup that
reads guest memory itself (the uniform snapshot, `vk_mem_lookup`) does not queue its pool for the copy-in. In Kokiri
Forest (1.1.1, RTX 4090) that costs about 1 ms a frame (copy-in 0.5 ms, write-back 0.6 ms), and the copies hold about
5 GB. `NX2NAVI_SHADOW=1` turns it on under Windows too. The Steam Deck's RADV has `VK_EXT_external_memory_host` as
well: the shadow copies are chosen for the measured cost above, not for a missing extension.
The device address table (`runtime/vk_gpa.c`) names a pool's shadow copy as it names an import, so shaders follow guest
pointers into either. A shader that reads through the table resolves nothing, so the pool it reaches is not queued for
the copy-in: a part whose pipelines declare the table is marked at its draws and dispatches, and its prologue brings
the shadow of every pool in the table up to date with the pages the CPU wrote and queues the same pages for those
pools' VRAM mirrors (`vk_memory.c` `vk_mem_gpa_sync`). That copies the written pages of every CPU-visible pool in
each such part, which the cost above does not include. What a shader stores through the table into a shadow copy does
not come back to guest memory. `--probe-vkmem` checks the read: a shader follows the table to a word of a pool that is
never bound, and must read `0x13579BDF` and, after a plain CPU write, `0x2468ACE0`. On Linux, run it for each backing:
```sh
for shadow in 0 1; do for memory in import mirror; do
NX2NAVI_GPA=1 NX2NAVI_SHADOW=$shadow NX2NAVI_MEMORY=$memory ./ocarina --probe-vkmem
done; done
```
The Deck's GPU (Van Gogh) has no native ASTC, so its textures go through the compute decoder (`runtime/vk_astc.c`,
`NX2NAVI_ASTC`) as on desktop AMD GPUs without it; frame times measured there include it.

View File

@@ -23,7 +23,7 @@ letters unless *Button prompts* says otherwise (below).
## Button prompts
`NX2NAVI_PROMPTS` (`prompts`, the *Button prompts* row on the Controls tab) picks the buttons the game's prompts show: `nintendo` (default, the game's own), `xbox`, `playstation` or `pc` (*Keyboard and mouse*). It changes three things the game draws:
`NX2NAVI_PROMPTS` (`prompts`, the *Button prompts* row on the Controls tab) picks the buttons the game's prompts show: `nintendo` (default, the game's own), `xbox`, `playstation`, `steamdeck` or `pc` (*Keyboard and mouse*). It changes three things the game draws:
- the button font every prompt in text uses (dialogue, the button guides, the controls screen): A B X Y, L R ZL ZR, Plus and Minus, the d-pad and the sticks with their directions become that pad's buttons (LB RB LT RT, Menu and View; L1 R1 L2 R2, Options and Create; its d-pad and sticks), or the keys. Power, Home and the console stay;
- the letter in the action prompt (top right);
@@ -35,13 +35,13 @@ A, B, X and Y show the button the *Button layout* presses: with `nintendo` (by p
The game reads the font and the interface textures once at start, so a change shows after a restart (the row says so; the menu's Restart button restarts). A button or key remapped on this tab keeps its old prompt until the folders are made again.
The prompts live in overlay folders next to the exe, `romfs-patches-xbox`, `romfs-patches-xbox-label`, `romfs-patches-playstation` and `romfs-patches-playstation-label` (by position and by label) and `romfs-patches-pc`, which `runtime/romfs.c` loads in place of the game's files. `tools/pad_prompts.py` draws them on the player's machine from the player's own RomFS and a folder of button art, which the repository does not hold. A launcher release or build kit made with `--prompt-art` carries that art (`prompt-art`, `tools/prompt_art.py`), and `tools/make_game.py` draws every folder in its last step, again whenever the art, the tools or the `bind_` lines change (so a rebuild after rebinding keys brings the keyboard prompts up to date). By hand:
The prompts live in overlay folders next to the exe, `romfs-patches-xbox`, `romfs-patches-xbox-label`, `romfs-patches-playstation`, `romfs-patches-playstation-label`, `romfs-patches-steamdeck` and `romfs-patches-steamdeck-label` (by position and by label) and `romfs-patches-pc`, which `runtime/romfs.c` loads in place of the game's files. `tools/pad_prompts.py` draws them on the player's machine from the player's own RomFS and a folder of button art, which the repository does not hold. A launcher release or build kit made with `--prompt-art` carries that art (`prompt-art`, `tools/prompt_art.py`), and `tools/make_game.py` draws every folder in its last step, again whenever the art, the tools or the `bind_` lines change (so a rebuild after rebinding keys brings the keyboard prompts up to date). By hand:
```
python tools\pad_prompts.py --art <art pack>\Source --game <install folder>
```
`--style xbox|playstation|pc` makes one style; the keys come from `<install folder>\nx2navi.ini` (`--binds` names another). The art folder holds `XGamepad/Default`, `XGamepad/Light`, `P5Gamepad/Default`, `P5Gamepad/Light`, `Keyboard_Mouse/Dark` and `Keyboard_Mouse/White` with the file names of `STYLES` and `KEY_ART` in the tool; a key it has no picture of is written on `Keyboard_Mouse/Blanks`. Without the folders the row says *prompt files missing* and the Switch prompts stay; so they do when a RomFS mod replaces one of the same files (*off: a RomFS mod replaces their files*). The log has a `RomFS overlay: ... button prompts` line for the folder in use. The tool needs fontTools, Pillow and zstandard (`tools/requirements-py314.txt`).
`--style xbox|playstation|steamdeck|pc` makes one style; `steamdeck` shows the Deck's own labels (A B X Y where an Xbox pad has them, L1 R1 L2 R2 and L3 R3, Menu and View) and is drawn from the Xbox and PlayStation pictures, as the art has no Deck style; the keys come from `<install folder>\nx2navi.ini` (`--binds` names another). The art folder holds `XGamepad/Default`, `XGamepad/Light`, `P5Gamepad/Default`, `P5Gamepad/Light`, `Keyboard_Mouse/Dark` and `Keyboard_Mouse/White` with the file names of `STYLES` and `KEY_ART` in the tool; a key it has no picture of is written on `Keyboard_Mouse/Blanks`. Without the folders the row says *prompt files missing* and the Switch prompts stay; so they do when a RomFS mod replaces one of the same files (*off: a RomFS mod replaces their files*). The log has a `RomFS overlay: ... button prompts` line for the folder in use. The tool needs fontTools, Pillow and zstandard (`tools/requirements-py314.txt`).
The actions are the game's own, as its *Basic Controls* screen (Plus menu, System) names them:
@@ -148,6 +148,8 @@ The PS button is not used. The Controls tab binds and remaps its buttons like an
`NX2NAVI_INPUT_RECORD=<file>` (`input_record` in `nx2navi.ini`) writes the merged input in the same format, one line per button press and per stick segment. The file replays as a script, which makes it the repro to attach to a bug report.
`tools/run_game.py <game folder> --save <save> --seconds <s> --wall <w> [--script <file>] [--shot <png>]` runs the game on a copy of its profile with a save swapped in, under the script (default `tools/input/load_save.txt`: title screen to the first save slot). `<save>` names one of the game's save backups (`profile/save-backup/<time>`; the game keeps the last three) or is a save folder or `.zip` from elsewhere, such as the zipped profile folder of a crash report; a single slot (`slot_NN`) replaces only that slot. The run ends once `<s>` seconds of game time passed after the save loaded, `--shot` saving that frame as a PNG and `--env NX2NAVI_SHOTS=w<from>-<to>:<step>` one every `<step>` seconds of game time along the way (480x270, full size with `NX2NAVI_SHOTS_FULL=1`), or after `<w>` seconds of wall time. `--env NX2NAVI_SHOT_DIR=<dir> --env NX2NAVI_SHOT_AFTER=<s> --env NX2NAVI_SHOT_EVERY=<s>` takes pictures by wall time instead, for a script that never reaches a place where game time counts. On Windows it also takes `build\<ver>` folders and the saves in `build\save-backup`.
`NX2NAVI_TEST_PAD=<lx>,<ly>,<rx>,<ry>` attaches an SDL virtual gamepad at start whose sticks rest at these raw SDL axis values (-32768 to 32767, down is +y), so a drifting or held stick can be tested without a controller: the values go through the gamepad mapping and `NX2NAVI_STICK_DEADZONE` like a real pad's, and `NX2NAVI_INPUT_RECORD` shows what reached the game. `NX2NAVI_TEST_PAD_AT=<ms>` attaches it that long after input starts instead and presses its south button a second later, which tests a controller switched on in play.
## RenderDoc capture (blocked)

View File

@@ -202,9 +202,10 @@ bin/ nx2navi_xlate, hactool, ninja, libSDL3.so.0
- The setup's temp folder is `.setup-temp` inside the install folder (`--temp` changes it): the AppImage is mounted
read-only, and a Steam Deck's `/home` is often too full for the generated C while the game goes to the SD card.
- The game keeps Mesa's shader cache in `mesa-cache` beside `nx2navi` unless `MESA_SHADER_CACHE_DIR` is set, for the
same reason: RADV fills its default folder in `~/.cache` up to 1 GB. NVIDIA's driver cache goes to `nvidia-cache`
beside it with a limit of 8 GiB unless `__GL_SHADER_DISK_CACHE_PATH` or `__GL_SHADER_DISK_CACHE_SIZE` is set: the
default limit of 1 GiB is smaller than what the shader preparation builds.
same reason: RADV fills its default folder in `~/.cache` up to 1 GB. With NVIDIA's driver, its cache goes to
`nvidia-cache` beside it unless `__GL_SHADER_DISK_CACHE_PATH` is set, with a limit of 8 GiB unless
`__GL_SHADER_DISK_CACHE_SIZE` is set: the default limit of 1 GiB is smaller than what the shader preparation builds.
When that folder cannot be made or written, the driver keeps its own folder and limit.
- The window follows `LANG` for English or Russian, cancels through SIGTERM to the CLI's process group (the CLI handles
it as Ctrl+Break on Windows) and opens the finished folder with `xdg-open`.
- Requirements beyond Windows': Linux 6.7 or later (the write watch, `docs/building.md`) and glibc 2.34.

View File

@@ -133,7 +133,7 @@ A path the Windows ANSI code page cannot hold breaks the MSYS2 tools, so the too
3 minutes without a count moving it says so, and that the step is still running.
4. **Draw the Button Prompts**, only when the version holds button art (`app/<version>/prompt-art`, a release made
with `--prompt-art`; `build.has_prompt_art`): make_game.py's last step runs `tools/pad_prompts.py` on that art
and writes the Xbox, PlayStation and keyboard and mouse prompts into `romfs-patches-*` beside the game (about
and writes the Xbox, PlayStation, Steam Deck and keyboard and mouse prompts into `romfs-patches-*` beside the game (about
30 s), which the game's Controls > Button prompts loads. The keyboard and mouse prompts show the keys of
`nx2navi.ini`'s `bind_` lines, so Rebuild the Game after rebinding keys draws them again; the step's key names
all it reads (the RomFS, the art, the tools, those lines), so it runs again only when one changed or a folder is