mirror of
https://github.com/tangerineturtle1/NX2navi.git
synced 2026-11-19 06:29:58 -04:00
Import the base tree (MIT, see LICENSE)
This commit is contained in:
148
docs/architecture.md
Normal file
148
docs/architecture.md
Normal file
@@ -0,0 +1,148 @@
|
||||
# Architecture
|
||||
|
||||
How the Switch build of *Ocarina of Time* becomes a native Windows x86-64 program. The upstream projects it builds on are listed under Credits in the [README](../README.md).
|
||||
|
||||
## Starting points
|
||||
|
||||
Two MIT projects cover different halves of the problem:
|
||||
|
||||
| | LAS Recomp | MacNativeGames |
|
||||
|---|---|---|
|
||||
| Game | *Link's Awakening* | *Ocarina of Time* 1.0.0–1.4.3 |
|
||||
| CPU | ARM64 lifted to C, runs on x86-64 | Original ARM64 code runs directly on Apple silicon |
|
||||
| OS boundary | IPC services and SVCs | ~163 `nn::*` SDK functions replaced by symbol, 38 SVCs |
|
||||
| Graphics | Own NVN layer on Vulkan, own Maxwell translator | NVN front end + Metal 4 back end, Ryujinx translator |
|
||||
| Useful to us | Lifter, NEON-to-C lowering, Windows memory layer, tests | Everything NX2navi-specific: host behaviour, NVN front end, version tables, patches, shader list |
|
||||
|
||||
MacNativeGames cannot run on x86-64 as is: its CPU path executes ARM64 code natively. LAS Recomp's host only covers what *Link's Awakening* calls. The port takes the CPU half from LAS Recomp and the game half from MacNativeGames, and writes the parts neither has: a lifter that scales to NX2navi, and a Vulkan back end for the NVN front end.
|
||||
|
||||
## Pipeline
|
||||
|
||||
### 1. Input
|
||||
|
||||
- Base 1.0.0 (`0100F2C0115B6000`) and one cumulative update (`0100F2C0115B6800`), NSP or XCI, plus the keys of the user's console.
|
||||
- ExeFS and RomFS extraction: MacNativeGames' Python extractor (Python 3.14 for `compression.zstd`, `.venv-py314`).
|
||||
- Update merging: hactool at `1d64a83` with MacNativeGames' BKTR patches, built by `tools/build_hactool.py`.
|
||||
- Verification: MacNativeGames' `expected-v*.json` hashes of ExeFS, RomFS and fonts.
|
||||
- Primary version: 1.4.3, the last update; performance work targets it. It replaced all three modules of the earlier updates (new SDK 20.5.22, RELR relocations, audio renderer REV15) and builds with `--version v143`, `recomp/patches-v143.txt`. 1.2.1 is a second target on the same build path; 1.0.0, 1.1.2 and 1.2.1 share `sdk`, `subsdk0` and `rtld` byte for byte, so only `main` differs among them. Saves load on either.
|
||||
- Windows driver: `tools/extract.py` runs the vendored extractors (`tools/vendor/macnativegames/`) and identifies the update by the SHA-256 of its `main`. hactool has no Darwin `funopen` here, so the base Program NCA is copied out of the NSP before the BKTR merge (about 17 GB of temporary space).
|
||||
|
||||
### 2. CPU: lift ARM64 to C
|
||||
|
||||
- Modules: `main`, `sdk`, `subsdk0`. `rtld` is dropped and linking is done by the host, as MacNativeGames does.
|
||||
- Base: LAS Recomp's hand-written decoder (`lifter.py`) and NEON/FP lowering (`simd_c.py`, `simd.h`), MIT. Output: one C function per guest function, direct calls for `BL`, an address-keyed table for indirect branches.
|
||||
- Rewritten for NX2navi scale:
|
||||
- generator runs in parallel processes, with bounded memory per worker;
|
||||
- jump tables resolved by our own symbolic pass instead of Unicorn, which keeps GPL code out of the shipped generator;
|
||||
- `LDXP`/`STXP` (128-bit exclusive pairs) supported;
|
||||
- no 64 MB limit on `main`; `subsdk*` loaded;
|
||||
- NZCV and the six most used general registers of each function kept in C locals instead of the `Cpu`
|
||||
(`recomp/flags.py`, `recomp/regs.py`): registers are stored back where they changed, before calls, SVCs and
|
||||
exits, and reloaded after them. A call reloads only X0-X18 and X30, as AAPCS64 has the callee preserve X19-X29 and
|
||||
SP; `gen.py --no-callee-saved-trust` reloads those too.
|
||||
- Coverage oracle: MacNativeGames' NSO rewriter lists every site where game code touches the OS (SVCs, TLS reads, counter reads, `x18`, exclusive windows, jump tables): 68,968 sites in 1.4.3 `main`. The lifter must handle every one.
|
||||
- Memory: guest address equals host address, all below 2^39 (IPC encodes 39-bit addresses). The heap, stacks and the alias region are reserved with `MEM_WRITE_WATCH`, so the graphics back end can ask which pages the CPU wrote since the last upload (`runtime/memory.c`).
|
||||
- Indirect branches go through `rt_dispatch` (`runtime/dispatch.c`), which looks the target up in the address table and tail-calls it.
|
||||
- Testing: LAS Recomp's differential harness, every lifted instruction class checked against Unicorn (test-only, never linked into the game).
|
||||
|
||||
### 3. Host
|
||||
|
||||
- Portable C from MacNativeGames: threads and synchronisation, IPC, memory services, fs/RomFS, time, account, applet, audio renderer and effects. The Apple pieces (mach VM, CommonCrypto, AudioToolbox, GameController) are replaced with Win32 and SDL3.
|
||||
- SDK boundary at the `nn::*` function level, matching MacNativeGames. Only `audren:u` and `sm:` are real IPC services.
|
||||
- Window, keyboard and mouse: Win32 (`gfx_window.c`, `input_win.c`). Gamepad and audio output: SDL3. Voice: libopus (BSD). Movies: the SDK's `movie::Decoder` is replaced by a host VP9 decoder over libvpx (BSD, static; `runtime/ov_movie.c`), because FFmpeg from MSYS2 is a GPL build.
|
||||
- zstd: most RomFS files of 1.4.3 are zstd frames. The patch set `zstd` sends the game's two decoder entry points to
|
||||
libzstd on the host (`ov_zstd.c`, `NX2NAVI_ZSTD_NATIVE`); the lifted decoder took almost three times as long.
|
||||
- Settings and menu: every key with its default is in one table (`cfg.c`), read from `nx2navi.ini` and `NX2NAVI_*`
|
||||
variables and written back when the in-game menu changes it. The menu and the performance overlay are Dear ImGui,
|
||||
drawn into the swapchain image (`runtime/ui/`); their text is in `ui_text.c` (English, Russian, Brazilian Portuguese,
|
||||
Simplified Chinese). The Chinese glyphs come from a system font (Microsoft YaHei, `plat_font_path`) merged into the
|
||||
font atlas.
|
||||
- OS calls the host makes go through `runtime/platform/` (`platform.h`, `win32.c`), so a Linux or macOS host adds one file instead of touching the rest. The split is not finished: `tools/check_platform.py` lists the Win32 uses left in the other files.
|
||||
|
||||
#### Threads and cores
|
||||
|
||||
Every guest thread is a host thread. NX2navi assumes Horizon's scheduler: three application cores, a thread runs only on a core in its affinity mask, and a higher-priority thread takes a core from a lower-priority one. The game keeps per-core data without locks and relies on that, so the host models it (`runtime/kernel.c`):
|
||||
|
||||
- Three virtual cores. A guest thread runs guest code only while it holds a core; the others wait in a priority-ordered list.
|
||||
- Preemption. When a waiting thread has a better priority than a holder on a core it may use, exactly one holder is flagged. The holder checks the flag at its next SVC and at every indirect call (`rt_dispatch`: two loads when nobody waits), checks again under the kernel lock that a better thread still waits, and only then hands its core over. The indirect-call check comes from SavageWindRecomp's BotW port; checking only at SVCs left a better thread waiting up to 26 ms.
|
||||
- Asynchronous preemption (from MacNativeGames 8bf0893, on unless `NX2NAVI_ASYNC_PREEMPT=0`). Game code can run for
|
||||
milliseconds with neither (`NX2NAVI_CORE_DIAG=1` measures it): a waiter that has spun 250 us suspends the holder and,
|
||||
if its host rip is in lifted code at host depth 0 (not inside a host call or SVC), moves it to `kern_ap_stub`, which
|
||||
saves the volatile registers and gives the core up there. A holder of a guest mutex counts at the priority of its
|
||||
best waiter, and one inside an SDK spin lock (TLR+0x100 `disable_count`) is not preempted: its interrupt flag is set
|
||||
and the switch happens at SVC 0x36. `NX2NAVI_ASYNC_PREEMPT=2` keeps those two without interrupting. The interruption is
|
||||
Win32 (SuspendThread, SetThreadContext); another host needs a signal handler there, as MacNativeGames has.
|
||||
- Host work gives the core back. Queue submits, presents, GPU waits, shader translation, `nn::fs` calls and NVN memory pool setup run without a core (`NVN_BLOCKING`, `kern_core_away`), as the Switch blocks those threads in IPC, so a thread blocked in the driver or on a file does not hold up guest threads.
|
||||
- Spin waits. Loops that only another thread can end (857 in 1.4.3, 864 in 1.2.1, found by `tools/spin_waits.py`, the
|
||||
nxnative search) send their back-edges through `rt_spin_gate`, which pauses the core once per pass and hands it to a
|
||||
waiter after 50 us of polling (`NX2NAVI_SPIN`, `NX2NAVI_SPIN_HANDOFF_US`).
|
||||
- `free_cores = 1` (`NX2NAVI_FREE_CORES=1`) turns the gate off, for experiments only: the game crashes or hangs, usually
|
||||
at start, because its job workers all report core 0 and share per-core memory (#212).
|
||||
|
||||
### 4. Graphics
|
||||
|
||||
- NVN front end from MacNativeGames (`nvn_api_*.c`, `nvn_obj.h`, texture layout and deswizzle). The game records command buffers through it; nothing reaches the GPU until the game submits.
|
||||
- Vulkan 1.3 back end in place of MacNativeGames' Metal one. At `QueueSubmitCommands` the recorded NVN commands are replayed into Vulkan command buffers (`vk_exec*.c`).
|
||||
- Memory pools. Apple's unified memory let MacNativeGames use the game's pools zero-copy. The back end imports the pools as host memory (`VK_EXT_external_memory_host`). On a discrete GPU, pools bound as buffers are mirrored in device-local memory and the pages the CPU wrote (write watch, above) are uploaded before a replay; idle pool regions are made read-only so the write-watch queries skip them (`vk_memory.c`). GPU-rendered textures are written back when the CPU reads them: on a pool invalidate, on a CPU copy, and for small targets once the CPU waits for a fence (`vk_texture.c`).
|
||||
- Replay thread. `NX2NAVI_REPLAY_ASYNC=1` moves the replay off the submitting thread onto a queue at most one frame behind the game; reads of GPU results wait for the newest writer instead of draining the queue (`vk_backend.c`). It is off by default. Where the game waits for the GPU it overlaps the replay with the game threads (a GTX 1650 in the Depths, issue #88), a gain measured only there; while it was the default, a GTX 1660 Ti at the 60 fps cap with the GPU at 26 % reported uneven motion (#659). It adds up to a frame of latency; `replay_async = 1` turns it on.
|
||||
- Recorder thread. The Vulkan calls the replay makes into its command buffer are made by a thread of their own (`vk_rec.c`, after Savage Wind Recomp and LAS Recomp): each `vkCmd*` copies its arguments into a packet of a ring and the recorder makes the calls in order; calls into other command buffers are made at once; the recorder also ends the command buffer, and the submit waits until it has. In the town on an RX 9070 the replay takes 17% less time on the submitting thread. It is off by default on AMD GPUs, where the driver loses the device or hangs the GPU at startup with it (`NX2NAVI_VK_RECORDER=1` forces it on, `=0` makes every call on the submitting thread).
|
||||
- Uniform snapshot. The game rewrites per-object uniform blocks on other threads soon after it submits the frame
|
||||
that reads them, while the replay still records it. At the start of a submission's replay every uniform range it
|
||||
binds from CPU-written pools is copied, and the draws bind the copy (`vk_usnap.c`, after MacNativeGames'
|
||||
`NX2NAVI_UBO_SNAPSHOT`); without it, late draws read another object's matrix.
|
||||
- Driver data. The NVN driver keeps a per-stage constant bank (c0) that shaders read for things like point size, dispatch size and base instance. The back end fills it the way the driver does.
|
||||
- Shaders: the Ryujinx translator 1.3.2 with MacNativeGames' patches and ours (`tools/xlate/patches/`), built with .NET 10 NativeAOT as `nx2navi_xlate.exe` and run as a separate process. Its SPIR-V goes to Vulkan directly. `tools/prepare_shaders.py` and `nx2navi.exe --prepare-shaders` translate and compile the variants of MacNativeGames' shared list ahead of play (29,587 for 1.4.3); the rest are translated on first use and cached on disk.
|
||||
- Pipelines: `VK_EXT_graphics_pipeline_library` where the driver has it. Shader stages are compiled ahead into libraries; a draw that meets a new state links them at once (fast link), and an optimised pipeline replaces the linked one when a background thread finishes it. Pipeline keys seen in play are saved and built at boot. A draw whose pipeline is not ready is skipped when one of its pass's targets was rendered in the previous frame, and waits otherwise, so one-shot passes are never lost; a full-screen draw is skipped as well rather than waiting (`NX2NAVI_PIPE_SKIP`); the stage libraries a waiting draw needs are built on idle workers in parallel (`NX2NAVI_PIPE_WAIT`). Without the extension, a missing pipeline is built on a worker and the draw waits or is skipped by the same rule.
|
||||
- Resolution. Patch sets make the game read its output, swapchain and scene sizes from the patch page
|
||||
(MacNativeGames' sites); `resolution.c` sets them from `output_res` and `render_scale` and grows the texture heap
|
||||
and the application pool to match. When the window is larger than the output, `vk_fsr.c` runs AMD FSR 1 (EASU,
|
||||
RCAS) on the present. An `output_res` that is not 16:9 (more than 1% off, `res_is_16x9`) also switches on the
|
||||
`aspect` patch set, which hands the output's ratio to the camera, and `romfs.c` `overlay_init` loads the
|
||||
`romfs-patches-<w>x<h>` overlay for it when it exists (`tools/make_romfs_patches.py --aspect`, laid out by
|
||||
`tools/ultrawide_layout.py`; the installer builds fifteen sizes: 21:9 2560x1080, 3440x1440, 3840x1600, 5120x2160,
|
||||
32:9 3840x1080, 5120x1440, 16:10 1280x800, 1680x1050, 1920x1200, 2560x1600 and 3:2 1920x1280, 2160x1440, 2256x1504,
|
||||
2880x1920, 3000x2000). A wider layout moves the HUD to the
|
||||
screen's corners, a narrower one to its top and bottom edges. Other sizes get the camera of their shape and the 16:9
|
||||
HUD stretched over it. The `aspect` setting (`fit`, the default; `stretch`) decides what the present does
|
||||
when the window's shape differs from the output's: `fit` letterboxes, where a 16:9 game in a wider window used to be
|
||||
stretched.
|
||||
- HDR (`hdr = 1`, `vk_hdr.c`). The game tonemaps its scene into an r11g11b10f target, upscales it and draws the HUD
|
||||
into another float target, and a last draw copies that into the 8-bit sRGB window texture it presents. With HDR on,
|
||||
the tonemap's fragment shader (recognised by its constants and resource names, not a hash) runs as
|
||||
`shaders/hdr_tonemap.frag`, which keeps highlights up to the display's peak; the present reads the float target the
|
||||
window texture was copied from (any other last draw into it, such as a menu or a movie, leaves the window texture,
|
||||
shown at the HUD's white); a compute pass turns the picture into PQ code values in Rec. 2020 for an HDR10
|
||||
swapchain, which FSR 1 or the blit then scale; the in-game menu is drawn into an RGBA8 image and blended on at the
|
||||
HUD's white. Without an HDR10 surface format (Windows with "Use HDR" off, a compositor without colour management)
|
||||
the swapchain stays SDR. `NX2NAVI_SHOTS` reads the window texture, so its pictures clip highlights in HDR.
|
||||
- DLAA (`vk_dlaa.c`, `dlaa`, off by default). NVIDIA DLSS at the scene's own size, through the driver's NGX core loaded
|
||||
at run time (no SDK code compiled in). It runs on the R11G11B10F light target right before the bloom chain's first
|
||||
pass, so post effects and the HUD read the anti-aliased picture. The scene's draws get a sub-pixel Halton viewport
|
||||
jitter; camera motion vectors are reprojected from the scene depth through this and last frame's view-projection of
|
||||
the scene camera block (constant bank c4), in double precision on the CPU. Per-object motion (`dlaa_objmv`,
|
||||
`vk_mvprev.c`, the per-object section of `vk_exec_draw.c`, translator patch 0015): solid scene draws whose position
|
||||
reads an object's own uniform blocks (bones c5, place c7, c6) are drawn a second time with a vertex shader variant
|
||||
that computes last frame's position from last frame's blocks (the game writes them into two copies that take turns;
|
||||
the uniform snapshot keeps both) and writes the difference into the motion image. `NX2NAVI_MV_CHECK` measures the
|
||||
vectors against the picture, `NX2NAVI_DLAA_TIMES` times the passes on the GPU. `fxaa` switches the game's own FXAA pass.
|
||||
- Occlusion queries never block the CPU; a result that is not ready counts as visible.
|
||||
- ASTC textures are uploaded as they are where the device reports `textureCompressionASTC_LDR`, and decoded by a compute shader everywhere else (`vk_astc.c`, `NX2NAVI_ASTC`). The CPU decoder (`tex_astc*.c`) is the fallback and the test oracle.
|
||||
- Draw state. A draw whose program, bound state and textures repeat reuses the previous draw's shaders and skips rebuilding its descriptor sets; descriptor sets are written through update templates (`vk_exec_draw.c`, `vk_pipeline.c`).
|
||||
- Allocation: Vulkan Memory Allocator. Debugging: Vulkan validation layers, per-pass frame dumps (`NX2NAVI_DUMP`, `vk_dump.c`) and a flight recorder (`NX2NAVI_FLIGHT`). RenderDoc cannot capture, because it hides `VK_EXT_external_memory_host` from the application.
|
||||
|
||||
Maxwell behaviour the back end has to match, and the NX2navi bugs other ports fixed: [maxwell.md](maxwell.md).
|
||||
|
||||
Why Vulkan rather than D3D12 or wgpu:
|
||||
|
||||
- The translator emits SPIR-V, which Vulkan consumes as is. D3D12 would add a SPIR-V to DXIL step; wgpu would add SPIR-V to WGSL through Naga and drop features the game uses (bindless textures, some atomics).
|
||||
- LAS Recomp's Vulkan back end is a working reference on Windows.
|
||||
- The same back end runs on Linux.
|
||||
|
||||
### 5. Toolchain
|
||||
|
||||
- Generated C: MSYS2 UCRT64 Clang + LLD. The generated code uses GNU extensions (`__int128`, `musttail`, vector types) that `cl.exe` cannot compile.
|
||||
- Host code: the same Clang, C11.
|
||||
- Builds: `tools/build_game.py` writes a Ninja file and runs MSYS2 Ninja. Every object is hashed before linking, because a faulty RAM module once flipped bits in stored objects.
|
||||
- Analysis: Ghidra with the Switch loader, Capstone, `llvm-objdump`.
|
||||
- Players: the installer ([installer.md](installer.md)) builds the game code on the player's machine with a bundled llvm-mingw, so no recompiled code is ever distributed.
|
||||
|
||||
263
docs/building.md
Normal file
263
docs/building.md
Normal file
@@ -0,0 +1,263 @@
|
||||
# Building from source
|
||||
|
||||
Windows 10 or later, an x86-64 CPU with SSE4.2, and a GPU with Vulkan 1.3 and `VK_EXT_external_memory_host`. The
|
||||
default build targets x86-64-v3 (AVX2/FMA); a CPU without them builds with `--cpu v2` instead and runs slower (see
|
||||
"Build"). Allow about 60 GB for the work folder. Extraction wants about 55 GB free while it runs and keeps about 35 GB
|
||||
afterwards.
|
||||
|
||||
Players who only want to play can use the [installer](installer.md) instead: it builds the game on their machine
|
||||
without this toolchain.
|
||||
|
||||
## Toolchain
|
||||
|
||||
Versions are the ones the build was verified with; newer patch releases are fine.
|
||||
|
||||
| Tool | Version | Install | Used for |
|
||||
|---|---|---|---|
|
||||
| MSYS2 UCRT64 | clang/lld 23.1.3 | msys2.org installer to `C:\msys64`, packages below | compiling the generated C and the runtime |
|
||||
| Visual Studio Build Tools | 2026, C++ workload, Windows SDK 10.0.26100 | winget `Microsoft.VisualStudio.BuildTools` with `--add Microsoft.VisualStudio.Workload.VCTools --includeRecommended` | the linker .NET NativeAOT calls |
|
||||
| .NET SDK | 10.0 | winget `Microsoft.DotNet.SDK.10` | the shader translator (`nx2navi_xlate.exe`) |
|
||||
| Vulkan SDK | 1.4.363 | winget `KhronosGroup.VulkanSDK` | validation layers, `vulkaninfo` |
|
||||
| uv | 0.13 | standalone installer from astral.sh | the two Python environments |
|
||||
| hactool | 1.4.0 @ `1d64a83` | `.venv\Scripts\python tools\build_hactool.py`, see below | NCA and update extraction |
|
||||
|
||||
MSYS2 packages, after `pacman -Syuu` (run it twice; the first pass updates the core and exits):
|
||||
|
||||
```sh
|
||||
pacman -S --needed --noconfirm mingw-w64-ucrt-x86_64-{clang,lld,ninja,gcc,make,pkgconf,sdl3,opus,libvpx,zstd,vulkan-devel,\
|
||||
vulkan-memory-allocator,vulkan-validation-layers,shaderc,spirv-tools} base-devel git unzip
|
||||
```
|
||||
|
||||
`ffmpeg` from MSYS2 is a GPLv3 build and is never linked; the movies go through libvpx.
|
||||
|
||||
PATH:
|
||||
|
||||
- Add `C:\Program Files (x86)\Microsoft Visual Studio\Installer` to the user PATH: NativeAOT needs `vswhere.exe`, or
|
||||
the publish fails at the link step (MSB3073).
|
||||
- Another `cmake` or `ninja` earlier on PATH (a 32-bit winlibs MinGW, for example) is picked up by accident. The build
|
||||
scripts call Ninja from `C:\msys64\ucrt64\bin` themselves; from a shell, put `C:\msys64\ucrt64\bin` first.
|
||||
|
||||
### hactool
|
||||
|
||||
`tools/build_hactool.py` fetches SciresM/hactool at `1d64a83` into `toolchain\src\hactool`, applies the three patches in
|
||||
`tools/hactool/` and builds a static `toolchain\hactool\hactool.exe` with MSYS2 UCRT64 gcc (a few seconds):
|
||||
|
||||
- `portable.patch`: the parts of MacNativeGames' `build_hactool()` that run on Windows, the `MNG_TITLE_KEYS` variable and
|
||||
the two BKTR fixes in `nca.c` for the subsection table padding and the subsection layout of newer cumulative updates;
|
||||
- `unicode.patch`: a manifest with `<activeCodePage>UTF-8</activeCodePage>`, so paths under a non-ASCII user profile
|
||||
reach it intact;
|
||||
- `mbedtls-asm-clobbers.patch`: the `memory` and `cc` clobbers the bundled mbedtls 2.6.1's x86-64 multiply asm lacks.
|
||||
Without them a clang build (the Linux release) fails every RSA signature check and the update merge stops.
|
||||
|
||||
hactool reads `-k <prod.keys>`; MSYS2 shells set `HOME`, which hides its default key location, so pass `-k` there.
|
||||
|
||||
## Build
|
||||
|
||||
From the repository root, in PowerShell. Every step that depends on the game version takes it as `<ver>`: `v143` for
|
||||
1.4.3, `v121` for 1.2.1 or `v110` for 1.1.0. The tools default to `v121` when `--version` is left out. Saves load on any
|
||||
of them.
|
||||
|
||||
1. Python environments (once). `.venv` runs the lifter and the build; `.venv-py314` runs the extraction scripts, which
|
||||
need the stdlib `compression.zstd` of Python 3.14:
|
||||
|
||||
```powershell
|
||||
uv venv --python 3.12 .venv
|
||||
uv pip install --python .venv/Scripts/python.exe -r tools/requirements.txt
|
||||
uv venv --python 3.14 .venv-py314
|
||||
uv pip install --python .venv-py314/Scripts/python.exe -r tools/requirements-py314.txt
|
||||
```
|
||||
|
||||
2. Copy `config.example.toml` to `config.local.toml` and set the paths to your base game, the update and your console's
|
||||
keys file. `config.local.toml` is ignored by git. The tools read the keys file themselves.
|
||||
|
||||
3. Extract and verify. This writes `build\extracted\v0` (1.0.0) and `build\extracted\<ver>` for each update in
|
||||
`updates`, identified by its hash; a hash mismatch stops the run. With another `[work] dir` in `config.local.toml`,
|
||||
`extracted` goes there instead, and `build_game.py` reads the ExeFS and RomFS from there:
|
||||
|
||||
```powershell
|
||||
.venv-py314\Scripts\python tools\extract.py
|
||||
```
|
||||
|
||||
4. Shader translator (no game files; again after a change to its sources, and `tools\make_release.py` stops on an older
|
||||
one). This publishes `build\xlate\nx2navi_xlate.exe`:
|
||||
|
||||
```powershell
|
||||
.venv\Scripts\python tools\build_xlate.py
|
||||
```
|
||||
|
||||
5. Lift the modules to C, write the RomFS overlay (the frame-rate fix), then compile and link
|
||||
`build\<ver>\nx2navi.exe`:
|
||||
|
||||
```powershell
|
||||
.venv\Scripts\python recomp\gen.py --exefs build\extracted\<ver>\exefs --out generated\<ver> --patches recomp\patches-<ver>.txt
|
||||
.venv-py314\Scripts\python tools\make_romfs_patches.py --version <ver>
|
||||
.venv\Scripts\python tools\build_game.py --version <ver>
|
||||
```
|
||||
|
||||
`--cpu v2` targets x86-64-v2 (SSE4.2, no AVX2/FMA) for older CPUs; the default `v3` needs AVX2. The v2 build links
|
||||
`runtime/fma.c`, a correctly rounded software fma the FP lowering needs, and runs slower. Inline assembly,
|
||||
target attributes and intrinsics are not bound by `-march`, so a v2 build runs `tools/check_isa.py` over the runtime
|
||||
objects before it links: any AVX, BMI or other instruction above x86-64-v2 outside a function listed there as
|
||||
dispatched by a runtime CPU check stops the build.
|
||||
|
||||
`gen.py` sizes its emit pool from the CPU count and the free memory; `--jobs N` sets it, and so does the environment variable `NX2NAVI_LIFT_JOBS=N`, which the installer's lift step honours too (it has no `--jobs`). When a worker runs out of memory, the pool restarts with half the workers (`emit: out of memory with N workers, retrying with M`), down to one; only an out-of-memory error with one worker fails the step.
|
||||
|
||||
Lifting takes about 15 s, a clean compile about 200 s on 24 threads; later builds recompile only what changed.
|
||||
A compiler cache in front of clang shortens it further: with `SCCACHE` set to an sccache executable, `build_game.py`
|
||||
writes it into the ninja file ahead of `cc`, and every lift rewrites the generated chunks with the same contents, so
|
||||
a second build of the same version hits all of their objects:
|
||||
|
||||
```powershell
|
||||
$env:SCCACHE = "path\to\sccache.exe"
|
||||
.venv\Scripts\python tools\build_game.py --version <ver>
|
||||
```
|
||||
`--patches` builds in the frame-rate, resolution and other patch sets, switchable at run time. The link copies
|
||||
`SDL3.dll`, `libiconv-2.dll` and `libopus-0.dll` next to the exe.
|
||||
For Xbox button prompts, run `make_romfs_patches.py` twice more, with `--prompts xbox` and `--prompts xbox-label`:
|
||||
they write `build\<ver>\romfs-patches-xbox` and `romfs-patches-xbox-label` ([controls.md](controls.md), "Xbox
|
||||
button prompts"), which `build_game.py` copies next to `nx2navi.exe` like `romfs-patches`. `--prompts pc` writes
|
||||
`romfs-patches-pc`, the keyboard and mouse prompts ([controls.md](controls.md), "PC button prompts").
|
||||
|
||||
6. Prepare the shaders (once per build folder and translator). It needs `refs\MacNativeGames` (`tools\fetch_refs.sh`)
|
||||
for the shared shader list:
|
||||
|
||||
```powershell
|
||||
.venv-py314\Scripts\python tools\prepare_shaders.py build\<ver> --version <ver>
|
||||
```
|
||||
|
||||
Without this step the first launch compiles shaders and pipelines during play and stutters until the caches fill.
|
||||
The AMD driver keeps its pipeline cache per exe path and holds a limited amount: preparing another build folder can
|
||||
evict this one's. `nx2navi.exe --warm-shaders` rebuilds it from the shader cache.
|
||||
When the driver's shader compiler crashes on a variant (NVIDIA 615.78.08 on an RTX 5080 does, issue #30),
|
||||
`prepare_shaders.py` runs the driver step again with `--warm-shaders`, up to three times, without the variants the
|
||||
driver was building at the crash. `shader-prepare-skip.txt` next to the exe lists them with the GPU and driver
|
||||
version; they build at their first draw instead, another driver version builds them again, and so does deleting the
|
||||
file.
|
||||
|
||||
7. Run. The game keeps running when its window loses the focus (`assume_focus = 1` in `nx2navi.ini`, the default; the
|
||||
Game tab switches it). With 0 it pauses, as it does on the Switch when Home is pressed.
|
||||
|
||||
```powershell
|
||||
build\<ver>\nx2navi.exe
|
||||
```
|
||||
|
||||
Settings live in `nx2navi.ini` next to `nx2navi.exe` ([nx2navi.ini.example](../nx2navi.ini.example) lists them). Saves go to
|
||||
`build\profile` for `v121` and `build\profile-<ver>` for other versions (`build_game.py --profile` changes it).
|
||||
|
||||
### Portable folder
|
||||
|
||||
`.venv\Scripts\python tools\package_build.py --version <ver>` writes `build\<ver>-portable`, which runs on another of
|
||||
your machines without MSYS2, Python or .NET. `romfs.bin` stays out unless you pass `--with-romfs`; otherwise put it in
|
||||
`romfs\` next to the exe or point `NX2NAVI_ROMFS` at its folder. Run `nx2navi.exe --prepare-shaders` once in the copied
|
||||
folder. The folder holds code and data derived from your game: for your own machines only.
|
||||
|
||||
### Profile-guided build
|
||||
|
||||
`build_game.py --pgo <file>.profdata` compiles the generated code at -O2 with an execution profile, and the runtime
|
||||
with it too: hot paths laid out straight, hot calls inlined, code that never ran compiled for size. On 1.4.3 the fight
|
||||
bench ran 12-15% faster per frame and a fight outside the training about 6%. The installer builds with
|
||||
`data/pgo-<ver>.profdata` of the release when the update has one, else at -O1.
|
||||
|
||||
A profile only fits the generated C it was trained on: functions are matched by name and control-flow hash, so a
|
||||
function whose C changes loses its counts without a warning. Retrain after a lifter or generator change, from a clean
|
||||
`recomp/`. The instrumented build needs llvm-mingw, as MSYS2's clang has no profile runtime:
|
||||
|
||||
```powershell
|
||||
$llvm = "toolchain\llvm-mingw-<date>\llvm-mingw-<date>-ucrt-x86_64"
|
||||
.venv\Scripts\python tools\build_game.py --version v143 --out build\v143-pgogen --toolchain $llvm --pgo gen
|
||||
```
|
||||
|
||||
Play `build\v143-pgogen\nx2navi.exe` through varied scenes (fights, towns, riding, building, the Depths, menus) for 20-30
|
||||
minutes; it is slower than a normal build. It writes `build\v143-pgogen\pgo\*.profraw` every minute and at exit.
|
||||
Scripted runs from saves (`tools/run_game.py`) add to the same file. Then:
|
||||
|
||||
```powershell
|
||||
.venv\Scripts\python tools\pgo.py merge build\v143-pgogen --version v143
|
||||
```
|
||||
|
||||
writes `build\pgo\pgo-v143.profdata` and its record (`.json`: the `recomp/` tree it fits). `make_release.py` ships every
|
||||
profile of `build\pgo` in `data\` and warns when a record names another `recomp/` tree than the one released.
|
||||
|
||||
## Linux (in progress)
|
||||
|
||||
The runtime builds and runs on x86-64 Linux with a window, input and menu on SDL3 (`*_sdl.c`) in place of the Win32
|
||||
files (`*_win.c`); `runtime/platform/linux.c` implements `platform.h`. Status: the selftest and the probes pass; 1.2.1
|
||||
plays at 60 fps on RADV (RX 7800 XT) and NVIDIA (RTX 3060); 1.4.3 builds, probes and plays on RADV (RX 6600, and the
|
||||
Vega 7 of a Ryzen 5 4600G at about 34 fps). Long play sessions and the Steam Deck itself are not verified yet.
|
||||
|
||||
Requirements:
|
||||
|
||||
- Linux 6.7 or later: the write watch uses userfaultfd write protection with `PAGEMAP_SCAN`. SteamOS 3.8 qualifies.
|
||||
- A Vulkan 1.3 driver with `VK_EXT_external_memory_host` (RADV, NVIDIA).
|
||||
- Packages on Arch:
|
||||
|
||||
```sh
|
||||
pacman -S --needed clang lld ninja make sdl3 opus libvpx zstd vulkan-headers vulkan-icd-loader \
|
||||
vulkan-validation-layers shaderc spirv-tools dotnet-sdk uv git
|
||||
```
|
||||
|
||||
Vulkan Memory Allocator is a single header (AUR `vulkan-memory-allocator`). Without it, `build_game.py` fetches the
|
||||
version the Windows release pins (3.4.0) into `toolchain/vma/` on first use and checks its hash.
|
||||
|
||||
The steps are the Windows ones with POSIX paths: `.venv/bin/python` in place of `.venv\Scripts\python`, and `nx2navi` in
|
||||
place of `nx2navi.exe`. `build_game.py` takes clang, lld and Ninja from PATH (or the ones in `<dir>/bin` with
|
||||
`--toolchain <dir>`) and links against the system libraries, with `libstdc++` static and an `$ORIGIN` rpath;
|
||||
`build_hactool.py` builds `hactool` with make. `tools/prepare_shaders.py` reads the shader list from
|
||||
`refs/MacNativeGames` at `origin/HEAD`, which must be the pinned commit with its history (the list format changed later).
|
||||
|
||||
The probes open a window, so from an SSH session they need the desktop session's display, or they stop at `gpu 0
|
||||
skipped: no graphics+compute+present queue family`: on Wayland `XDG_RUNTIME_DIR=/run/user/$(id -u)
|
||||
WAYLAND_DISPLAY=wayland-1 SDL_VIDEODRIVER=wayland` (the socket name is in `$XDG_RUNTIME_DIR`), on X11 `DISPLAY=:0`.
|
||||
|
||||
Memory: on Windows the GPU imports guest memory and `GetWriteWatch` tracks the CPU's writes to it. On Linux,
|
||||
re-protecting imported memory invalidates the driver's userptr buffer on every submit (about 1.3 ms per 16 MiB on
|
||||
amdgpu), and importing pins the pages, which drops their write protection. Each CPU-visible pool therefore gets a
|
||||
host-visible copy (shadow storage, `runtime/vk_memory.c`): the pages the CPU wrote go in before each submit, and the GPU's
|
||||
writes come back at its fence. In play that costs about 0.2 ms a frame and doubles the memory of those pools (about
|
||||
1.1 GB). `NX2NAVI_SHADOW=1` turns it on under Windows too. The Steam Deck's RADV has `VK_EXT_external_memory_host` as
|
||||
well: the shadow copies are chosen for the measured cost above, not for a missing extension.
|
||||
|
||||
The Deck's GPU (Van Gogh) has no native ASTC, so its textures go through the compute decoder (`runtime/vk_astc.c`,
|
||||
`NX2NAVI_ASTC`) as on desktop AMD GPUs without it; frame times measured there include it.
|
||||
|
||||
## Checks
|
||||
|
||||
Before a commit:
|
||||
|
||||
- `python tools/check_content.py --staged`: nothing game-derived is staged. CI runs the same check;
|
||||
`git config core.hooksPath .githooks` runs it before each local commit.
|
||||
- `build\<ver>\nx2navi.exe --selftest`: layout, dispatch table and host-function registry, without game files.
|
||||
|
||||
A change to the button prompts (`tools/make_romfs_patches.py --prompts`, `tools/button_icons.py`) passes
|
||||
`.venv-py314\Scripts\python tests\romfs_prompts_test.py`, which runs the tool on a RomFS it makes itself (no game
|
||||
files); a change to the settings passes `nx2navi.exe --probe-cfg`.
|
||||
|
||||
The lifter (`recomp/`) and the CPU headers (`runtime/cpu.h`, `runtime/simd.h`) are checked against Unicorn. Every lifter
|
||||
change passes these, all run with `.venv`:
|
||||
|
||||
| Test | Checks | Game files |
|
||||
|---|---|---|
|
||||
| `tests\diff\lift.py` | instructions sampled from the extracted modules; `--coverage` lists every word the lifter rejects or traps | yes |
|
||||
| `tests\diff\simd.py` | the whole FP/SIMD encoding space | no |
|
||||
| `tests\diff\atomic.py` | exclusive load/store sequences and ARMv8.1 atomics | no |
|
||||
| `tests\diff\func.py` | whole functions as the generator emits them, loops, jump tables and calls included (calls go to a stub that checks the registers at the call) | yes |
|
||||
| `tests\flags_check.py` | the function-local flag helpers of `runtime\cpu.h` against its packed ones | no |
|
||||
| `tests\excl_mt.py` | the exclusive monitor helpers under contention and ABA, on several threads | no |
|
||||
| `tests\jumptables_check.py` | `recomp/jumptables.py` against a Unicorn oracle; must report 0 disagreements, on 1.4.3 too | yes |
|
||||
|
||||
A `--cpu v2` build passes the same differential tests with `--cflags=-march=x86-64-v2`; they compile `runtime/fma.c`
|
||||
too, so they check the software fma the v2 build links rather than the C runtime's.
|
||||
|
||||
Runtime probes, each a mode of `nx2navi.exe` (the list is at the top of `runtime/main.c`): `--probe-kernel`,
|
||||
`--probe-vkexec`, `--probe-nvn-tex` and `--probe-shader` cover most runtime changes. `--probe-shader` and the draw
|
||||
part of `--probe-vkexec` translate shaders of the census below: they look for `build/<ver>/shaders/jobs.json` of the
|
||||
exe's own version first, then of the other one.
|
||||
|
||||
Shaders: `.venv-py314\Scripts\python tools\shader_census.py --version <ver> --translate` extracts every Maxwell shader
|
||||
to `build\<ver>\shaders\` and runs it through the translator and `spirv-val`; failure groups go to `failures.txt`.
|
||||
`tools\xlate_smoke.py` checks the translator alone, without game files.
|
||||
|
||||
## Reverse engineering (optional)
|
||||
|
||||
- Ghidra 12.1.2 with Adubbz's Ghidra-Switch-Loader in `toolchain\ghidra\` (JDK 21) opens the game modules (NSO).
|
||||
- `tools\fetch_refs.sh` clones the upstream projects this port builds on into `refs\`, read-only.
|
||||
74
docs/cheats.md
Normal file
74
docs/cheats.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# Cheats
|
||||
|
||||
The port reads Atmosphère cheat files, the format Switch homebrew and the emulators use. It ships no cheat files: a cheat file goes into a mod in the `mods` folder next to `nx2navi.exe`, like any other mod ([mods.md](mods.md)). Open the menu (F1) and switch the cheats on the Cheats tab. The switches are saved under `[cheats]` in `nx2navi.ini`.
|
||||
|
||||
## Adding a cheat file
|
||||
|
||||
1. Find the cheat file for your update. Cheat sites and collections name it after the first 8 bytes of the game's build id:
|
||||
|
||||
| Update | File |
|
||||
|---|---|
|
||||
| 1.2.1 | `9B4E43650501A4D4.txt` |
|
||||
| 1.4.3 | `277178B7DBA1B6D4.txt` |
|
||||
|
||||
The Cheats tab names the file the installed game reads when no mod holds one, and `logs\nx2navi.log` says `cheats: no enabled mod holds cheats/<id>.txt`.
|
||||
2. Put it in the `cheats` folder of a mod, a folder of its own under `mods`:
|
||||
|
||||
```
|
||||
mods\
|
||||
My cheats\
|
||||
cheats\
|
||||
277178B7DBA1B6D4.txt
|
||||
```
|
||||
|
||||
A download laid out for the Switch's SD card or an emulator works as it is: unpack it into a folder of its own under `mods`, and the port finds `atmosphere\contents\0100F2C0115B6000\cheats`, `contents\0100F2C0115B6000\cheats` or `0100F2C0115B6000\cheats` inside it. The file name's case does not matter.
|
||||
3. Start the game. Cheats that write the game's memory work now.
|
||||
4. Cheats that change the game's code need the game rebuilt with them. When the game starts and finds such cheats it did not build, it opens the setup's rebuild window, which builds them in (a few minutes; the shaders stay prepared) and starts the game again. Until then they are greyed out with "after the rebuild". When the rebuild fails or you cancel it, the game starts without them and does not try again for the same cheats; "Rebuild the game now" on the Cheats tab tries again. `mods_rebuild = off` under `[mods]` in `nx2navi.ini` turns the automatic rebuild off; running `NX2navi-Setup.exe` on the same folder does the same job by hand. Saves and settings are kept.
|
||||
5. Switch the cheats on the Cheats tab of the menu.
|
||||
|
||||
The Cheats tab lists the cheats grouped by mod, in the order of the Mods tab. A mod switched off on the Mods tab, or one the port cannot use, contributes no cheats; the change applies at the next start. The Mods tab says how many cheats each mod brought.
|
||||
|
||||
## The file format
|
||||
|
||||
Standard Atmosphère: a `[name]` line starts a cheat, `{name}` a master cheat that is always on, and the cheat's opcodes follow as 8-digit hex words. A name with no opcodes is shown as a heading. The file's title line (the one with `TID=`) is not shown.
|
||||
|
||||
```
|
||||
[The Legend of Zelda: Ocarina of Time v1.4.3 TID=0100F2C0115B6000 BID=277178B7DBA1B6D4]
|
||||
|
||||
[Player]
|
||||
[A code cheat]
|
||||
040A0000 XXXXXXXX YYYYYYYY
|
||||
|
||||
[A data cheat (Hold ZL+X)]
|
||||
80000104
|
||||
580F0000 XXXXXXXX
|
||||
640F0000 00000000 YYYYYYYY
|
||||
20000000
|
||||
```
|
||||
|
||||
The words above are placeholders: `XXXXXXXX` an offset, `YYYYYYYY` a value.
|
||||
|
||||
Each cheat's switch is `cheat_<hash> = 1`, where the hash is FNV-1a 32 of `<mod folder name>/<cheat name>`. A name repeated in one file gets `#2`, `#3` and so on. The build names each code cheat the same way, so renaming a mod's folder resets its cheats' switches to off and makes the next start rebuild its code cheats. Rename the folder before you switch its cheats on.
|
||||
|
||||
## How cheats run
|
||||
|
||||
The game's code is compiled ahead of time, so a cheat cannot write new instructions into it while the game runs. The port sorts each cheat by what it writes (`tools/cheats.py` at setup, `runtime/cheats.c` in game):
|
||||
|
||||
| Kind | What the cheat does | How the port runs it |
|
||||
|---|---|---|
|
||||
| code | Opcode 0 writes into main's code | Built into the game at setup as a patch set (`recomp/patches.py`); switches at once. |
|
||||
| key | Code writes inside a button condition (`8` mask, writes, `21` else, the original words, `20`) | A patch set that is on while the cheat is on and the buttons are held. |
|
||||
| data | Writes data only: heap, main's data, pointer chains, button conditions | Runs in a cheat VM after dmnt's, opcodes 0 to A, every presented frame while on. |
|
||||
| master | `{name}` with code writes | A patch set that is always on. |
|
||||
| restore | Writes back only the game's original instructions | Not needed: switching a cheat off restores them. Greyed out. |
|
||||
|
||||
A cheat whose code writes call or jump into a code cave the cheat writes itself (into main's zero padding) gets the cave inlined at the call site. The setup builds the code cheats of every mod in the `mods` folder, switched on or off, so switching a mod on later needs only a restart. A code cheat added or edited after the last build waits for the rebuild; a data cheat runs at the next start.
|
||||
|
||||
Not supported, and shown greyed out with the reason:
|
||||
|
||||
- the 0xC and 0xF opcode extensions;
|
||||
- opcode 5 with a register offset;
|
||||
- code writes mixed with other opcodes, except the button pattern above;
|
||||
- code writes that change the CPU flags differently from the game's instruction, or code caves with branches or pc-relative instructions.
|
||||
|
||||
`build-cache\cheats.status` next to `nx2navi.exe`, written by the setup, records which cheats were built and how; the log line `cheats: ...` sums them up at start. When the code cheats cannot be built, the setup warns and installs the game without them.
|
||||
183
docs/controls.md
Normal file
183
docs/controls.md
Normal file
@@ -0,0 +1,183 @@
|
||||
# Controls
|
||||
|
||||
Input comes from the first SDL3 gamepad, the keyboard and the mouse, merged into one Switch controller (`runtime/input_win.c`). Buttons are ORed; each stick takes the source with the larger deflection. Keys and mouse buttons count only while the game window is in the foreground. `NX2NAVI_INPUT=none` disables both (automated runs). `NX2NAVI_*` settings can also be set in `nx2navi.ini` next to `nx2navi.exe` (`input = none`, `motion = 0`, `rumble = 0`); the environment overrides the file. `NX2NAVI_RUMBLE_STRENGTH` is the rumble strength in percent (0 to 300, default 130; 0 is the same as `rumble = 0`). `NX2NAVI_RUMBLE_CURVE` is the exponent applied to the amplitude before the strength (0.2 to 1.5, default 0.6; lower makes weak pulses stronger, 1 is linear). `NX2NAVI_RUMBLE_TRIGGERS=1` also drives the impulse triggers of an Xbox pad.
|
||||
|
||||
`NX2NAVI_PAD_LAYOUT` (`pad_layout`, the *Button layout* row on the Controls tab, live) picks which face button is Switch A:
|
||||
|
||||
- `nintendo`: by position, as on a Switch (the table below).
|
||||
- `xbox`: south A, east B, west X, north Y, as an Xbox pad labels them; on a PlayStation pad cross is A, circle B, square X, triangle Y.
|
||||
- `auto` (default): `nintendo` for a pad whose east button SDL reports labelled A (Switch Pro Controller, Nintendo-layout pads), `xbox` for the others.
|
||||
|
||||
The log names the layout in effect (`input: gamepad layout ...`); the menu shows it under *Automatic*. Which controller's buttons the game's prompts show is `NX2NAVI_PROMPTS` (see Xbox button prompts below). 0.1.9's `pad_layout = xbox_prompts` is read as `pad_layout = nintendo` with `prompts = xbox`, and the menu writes it that way.
|
||||
|
||||
| Switch | Gamepad (by position, Nintendo layout) | Keyboard and mouse | Action in the world |
|
||||
|---|---|---|---|
|
||||
| Left stick | left stick | W A S D | move |
|
||||
| Right stick | right stick | mouse; arrow keys as a second source | camera |
|
||||
| Y | west | left mouse button | attack |
|
||||
| ZL | left trigger (over 50%) | right mouse button, Z | shield, lock on |
|
||||
| ZR | right trigger (over 50%) | F, X | bow |
|
||||
| X | north | Space | jump |
|
||||
| B | south | Left Shift | run, cancel |
|
||||
| A | east | E | interact, pick up |
|
||||
| L | left shoulder | Q | abilities |
|
||||
| R | right shoulder | R | throw weapon |
|
||||
| Left stick click | left stick click | Left Ctrl | crouch |
|
||||
| Right stick click | right stick click | C | first-person view |
|
||||
| D-pad left, up, right, down | D-pad | 1 2 3 4 | quick-select |
|
||||
| Plus | Start | Tab | menu, inventory |
|
||||
| Minus | Back | M | map |
|
||||
| Right stick shaken left and right | shake the right stick | G | detach a fused part from what object holds |
|
||||
|
||||
Esc is unbound in the game; it opens the settings menu and releases the mouse cursor (see Mouse camera). Z and X duplicate ZL and ZR so menu pages turn with two neighbouring keys; the menu tabs (L, R) are Q and R. The mouse wheel is unbound. The bindings are the tables of `input_map` in `runtime/input_map.c`: the keys (platform-neutral key ids) and one gamepad table per layout (gamepad inputs by position); `runtime/input_win.c` translates Win32 virtual keys and SDL buttons.
|
||||
|
||||
## Xbox button prompts
|
||||
|
||||
`NX2NAVI_PROMPTS` (`prompts`, the *Button prompts* row on the Controls tab, read at start) picks the controller whose buttons the game's prompts show: `nintendo` (default) keeps the game's own, `xbox` shows an Xbox controller's, `pc` the keyboard and mouse (PC button prompts, below).
|
||||
|
||||
- L, R, ZL, ZR, Plus and Minus become LB, RB, LT, RT, Menu and View, and the d-pad prompts a cross with an arrow for each direction meant. The letters on the keys are cut out in the game's own text font.
|
||||
- A, B, X and Y name the label of the button the pad layout presses, so each prompt names the button to press. `pad_layout = nintendo` plays an Xbox pad by position: the right button, labelled B, does Switch A (interact, confirm), and the bottom one, labelled A, does Switch B (run, cancel), so the prompts swap A with B and X with Y. `xbox` and `auto` put Switch A on the button labelled A, and the letters stay.
|
||||
|
||||
| Switch | `nintendo` layout | `xbox` or `auto` layout |
|
||||
|---|---|---|
|
||||
| A, B, X, Y | B, A, Y, X | A, B, X, Y |
|
||||
| L, R | LB, RB | LB, RB |
|
||||
| ZL, ZR | LT, RT | LT, RT |
|
||||
| Plus, Minus | Menu, View | Menu, View |
|
||||
| D-pad | cross | cross |
|
||||
|
||||
The action prompt (`DoCommand_00`) shows its button in a diamond whose shape gives the face buttons' positions as on a Switch, with the other three letters around it. With `nintendo` its letters are swapped with the rest and every one names the button in that place. With `xbox` and `auto` the letters are right but the diamond keeps the Switch positions: A shows on its right, while the button labelled A is at the bottom. The sticks keep the Switch icons, and so do prompts drawn as fixed images (some tutorial animations).
|
||||
|
||||
The prompts are part of the game's fonts, textures and messages, so they follow the layout and not the bindings. A button remapped with a `bind_` line or on the Controls tab keeps the prompt the game draws for it, which then names the wrong button. Use the Xbox prompts with the default bindings.
|
||||
|
||||
The game reads its UI archive and fonts once at boot, so the prompts change at the next start, also after `pad_layout` changes between `nintendo` and the others (the menu says *prompts change after restart*); the buttons change at once. Under that note the menu offers *Restart now*: after a confirmation it writes `nx2navi.ini`, closes the game as *Quit game* does and starts the game again with the same command line, which waits until the old process is gone before it opens the log and the saves (`runtime/gfx_present.c`, `gfx_window_closed`; `plat_restart` and `plat_restart_wait` in `runtime/platform/win32.c` and `linux.c`). Progress since the last save is lost, as with *Quit game*; while the game writes a save, the confirmation waits. With `prompts = xbox` the game loads `romfs-patches-xbox` (`pad_layout = nintendo`) or `romfs-patches-xbox-label` (`xbox`, `auto`) next to `romfs-index.bin` in place of `romfs-patches` (`runtime/romfs.c`, `overlay_init`; `runtime/input_map.c`, `input_prompts_folder`): the same 60 fps patches, with the prompts drawn from the game's own files by `tools/make_romfs_patches.py --prompts xbox` or `--prompts xbox-label` (`tools/button_icons.py`, after MacNativeGames' `make_button_icons.py`), which the setup runs next to the plain one:
|
||||
|
||||
- every prompt but the button guide in play is text in the button font, `scft/External.bfotf` in each `Font/Font*.Nin_NX_NVN.bfarc.zs`: glyphs U+E040 to U+E043 (A B X Y), U+E044 to U+E047 (L R ZL ZR), U+E04B and U+E04C (Plus, Minus), U+E049 and U+E060 to U+E065 (the d-pad). fontTools writes the font again, so the archives grow and the folder also carries the resource size table (`System/Resource/ResourceSizeTable.Product.<n>...rsizetable.zs`) with their entries raised by as much; without that the game stops at start.
|
||||
- the button guide in play draws L and ZL from the images `Nt_KeyTexL_00` and `Nt_KeyTexZL_00` of the Common archive's `timg/__Combined.bntx`, painted with the same icons. The action prompt draws its button from `Nt_KeyTexA/B/X/Y_00`, whose image data is swapped for `nintendo`, and the diamond's letters are text the game takes from `LayoutMsg/DoCommand_00.msbt` of each language's `Mals` archive (and the layout's own text panes behind it), swapped the same way.
|
||||
|
||||
The Switch prompts stay, with a note under *Button prompts* and a `RomFS overlay: ...` line in the log, when the folder is missing (an install made by an older setup, or one where the setup could not prepare it and showed a warning; running the setup again on the install folder writes it), when it fails its check, with a game resolution of another shape than 16:9 (its own UI layout wins; 16:10 and 21:9), and when the `romfs_mods` folder replaces one of its files: `romfs-patches` then loads in its place, where two overlays replacing one file would stop the game. In a source build, `.venv-py314\Scripts\python tools\make_romfs_patches.py --version <ver> --prompts xbox` (and `--prompts xbox-label`) writes `build\<ver>\romfs-patches-xbox` (`-label`); it needs fontTools from `tools/requirements-py314.txt`. `--xbox-prompts` is the older name of `--prompts xbox`.
|
||||
|
||||
## PC button prompts
|
||||
|
||||
`prompts = pc` (*PC* on the *Button prompts* row) shows the key or mouse button the default keyboard bindings press each Switch button with, whatever the pad layout. The game loads `romfs-patches-pc` in place of `romfs-patches`, written by `tools/make_romfs_patches.py --prompts pc` (`button_icons.py`, `pc_icons`), which the setup runs next to the Xbox ones:
|
||||
|
||||
| Switch | Prompt |
|
||||
|---|---|
|
||||
| A, B, X, Y | E, Shift, Space, left click |
|
||||
| L, R, ZL, ZR | Q, R, right click, F |
|
||||
| Plus, Minus | Tab, M |
|
||||
| D-pad | 1 (left), 2 (up), 3 (right), 4 (down), as a cross or the keys of the directions meant |
|
||||
| Left stick, its press | W A S D (or the keys of the directions meant), Ctrl |
|
||||
| Right stick, its press | the mouse, with an arrow for each direction meant, C |
|
||||
|
||||
- The glyphs keep the width of the game's own, so the right-aligned guides do not run into their text. A key is a rounded square with its label cut out in the game's own text fonts; a mouse button is a mouse with that button filled and the others hollow.
|
||||
- The button guide in play draws every button from images of the Common archive (`Nt_KeyTexA/B/X/Y/L/ZL_00`, `Nt_KeyTexLSt_00` and `_01`, the stick tilted and pressed): all of them are painted with the same icons. The letters of the action prompt's diamond are one character of the Normal font each: E, an arrow for Shift, a bar for Space, L and R for the mouse buttons.
|
||||
- Like the Xbox prompts they follow the defaults and not the bindings. `--binds <nx2navi.ini>` draws the first key of each `bind_` line of that file instead, for a folder made by hand.
|
||||
- At a game resolution of another shape than 16:9 the game loads `romfs-patches-pc-<w>x<h>` in place of `romfs-patches-<w>x<h>` when it is there: the same prompts with that shape's UI layout, `make_romfs_patches.py --prompts pc --aspect <w>x<h> --out <install>\romfs-patches-pc-<w>x<h>`. The setup does not write those, so without one the Switch prompts stay (*off at a game resolution not 16:9*).
|
||||
|
||||
## Window keys
|
||||
|
||||
| Key | Gamepad | Action |
|
||||
|---|---|---|
|
||||
| F1 or Esc | Minus and Plus together | open or close the settings menu; the game gets no input while it is open |
|
||||
| F10 | L3 + R3 (both stick clicks) | performance overlay: off, frame rate, full panel |
|
||||
| F11 | | borderless fullscreen on or off, saved to `nx2navi.ini` |
|
||||
|
||||
The Minus+Plus combo never reaches the game. In the menu the arrow keys, the D-pad or the left stick move, Enter or A selects, Q and E or L and R switch tabs, Esc or B goes back (Switch button names, as in the table above).
|
||||
|
||||
## Remapping
|
||||
|
||||
The *Controls* tab of the settings menu lists, under the button layout, every Switch button, the eight stick directions the keys drive (movement and camera) and the shake key, each with two key cells and a controller cell. Select a cell, press A or Enter, then press the new key, mouse button or controller button; Esc or five seconds without a press cancel, and a controller button cancels a key cell. Delete, Backspace or X clears the selected cell. A key another action has moves to the new one (the description names the action it left); a controller button swaps with the one it replaces. A remapped controller button is physical: from then on the face buttons (A B X Y) stay where they are in both layouts, until the buttons are put back as the layout in use has them or *Reset all buttons* is chosen. F1, F8, F10, F11 and Esc cannot be bound (the menu, the flight recorder and the window keep them). Punctuation keys are named by their place on a US keyboard; the menu shows the character the keyboard layout in use puts there.
|
||||
|
||||
The tab writes `nx2navi.ini` lines, one per Switch button or stick direction: `bind_<button> = <entry>, <entry>, ...`. The button is `A B X Y L R ZL ZR PLUS MINUS LEFT UP RIGHT DOWN LCLICK RCLICK`, or a stick direction `LSTICK_LEFT LSTICK_UP LSTICK_RIGHT LSTICK_DOWN` (movement, W A S D by default) and `RSTICK_...` (camera, the arrow keys), or `SHAKE` (keys only, G by default). An entry is `pad:<name>` or `key:<name>`:
|
||||
|
||||
- pad names: `south east west north l r zl zr start back lstick rstick dpad_left dpad_up dpad_right dpad_down lpaddle1 rpaddle1 lpaddle2 rpaddle2` (`zl`, `zr` are the triggers; the paddles are the back buttons of an Xbox Elite, DualSense Edge or Steam Controller, upper pair 1, lower pair 2, and no Switch button has them by default)
|
||||
- key names: `a` to `z`, `0` to `9`, `f2` to `f7`, `f9`, `f12`, `num0` to `num9`, `space tab enter backspace lshift rshift lctrl rctrl lalt capslock insert delete home end pageup pagedown up down left right`, `num_add num_sub num_mul num_div num_decimal`, `minus equals lbracket rbracket semicolon apostrophe comma period slash backslash grave`, `mouse_l mouse_r mouse_m mouse_x1 mouse_x2`
|
||||
- `key:default` and `pad:default` keep that part's defaults (`pad:default` is each layout's own button).
|
||||
- `none` leaves the button unbound.
|
||||
|
||||
A line replaces every default of that button, pad and keyboard, so list the keys you still want or use `key:default`. Pad names are physical positions, so a line means the same in both layouts and replaces the layout's own button for that Switch button. Stick directions take keys only: the controller's sticks and the mouse camera always work. Buttons without a line keep the table above. Names are case-insensitive. An invalid entry logs one `input: bind ...` line and is skipped; a line without a valid entry changes nothing. The `NX2NAVI_BIND_<button>` environment variables work like any other setting. A changed line applies at once (`runtime/input_win.c` builds the tables again at its next poll).
|
||||
|
||||
A pad that reports its face buttons swapped (A/B and X/Y, as some GuliKit models do) is fixed with:
|
||||
|
||||
```
|
||||
bind_A = pad:south, key:E
|
||||
bind_B = pad:east, key:lshift
|
||||
bind_X = pad:west, key:space
|
||||
bind_Y = pad:north, key:mouse_l
|
||||
```
|
||||
|
||||
## Mouse camera
|
||||
|
||||
`mouse_mode = direct` (the default) turns the camera by the raw mouse motion on game 1.4.3, through hooks in the game's camera code (`runtime/mouse_direct.c`, the `mouse_direct` sites of `recomp/patches-v143.txt`). None of them changes a step of the gamepad alone. The free-orbit camera receives raw-count
|
||||
angular increments at its final spherical-angle output: 0.08 degrees per count on both axes at sensitivity 1, with
|
||||
no time multiplier, curve or carry into later frames. The controller's persistent pitch/yaw move with the output.
|
||||
Mouse input is removed from its stick path first, so the camera's later vertical signed-square curve and velocity
|
||||
filters cannot process it a second time. Other camera modes receive input after the radial dead zone, unit-circle
|
||||
clamp and stick response exponent; object has a separate hook after its manipulation input's power curves.
|
||||
The game's camera inversion and gamepad response remain in effect. `mouse_sens` scales raw counts; its scale
|
||||
differs from the stick-emulation mode. Start at 1.
|
||||
The Controls tab switches between *Direct* and *Stick emulation* (`mouse_mode = stick`).
|
||||
The orbit camera keeps its desired pitch at the final angular output, including stationary frames, so a later game
|
||||
filter cannot recenter it. The persistent filter state accounts for the camera's current vertical offset instead of
|
||||
accumulating corrections. A gamepad vertical input or the game's lock-on/override releases that mouse target.
|
||||
While L is held, mouse counts move the ability wheel's virtual cursor, which stays where it was when the mouse stops.
|
||||
The horizontal quick lists (D-pad up/left/right: materials, shields and weapons) use separate 32 ms right-stick steps,
|
||||
with at least 32 ms of neutral between them. At sensitivity 1, 32 horizontal counts request a step; a large flick
|
||||
cannot queue continued scrolling after the mouse stops. Vertical movement is ignored in these lists. Switching or
|
||||
releasing a selector resets its cursor/step state without sending the last movement to the camera. Selector input is
|
||||
not consumed by the direct camera or object hooks.
|
||||
The mouse never stands in for a stick shake: to detach a fused part, press G (`bind_SHAKE`). A press swings the right
|
||||
stick left and right for 0.8 s, holding the key keeps it swinging. The arrow keys or the gamepad's right stick shaken by
|
||||
hand work too. A mouse gesture for it would also fire while a held object is turned back and forth.
|
||||
A step of the gamepad alone, with no mouse motion and no pitch the mouse holds, reaches the camera exactly as without
|
||||
the hooks.
|
||||
The mapping below is still used when the camera-input hook is unavailable or the game disables camera input.
|
||||
Games 1.1.0 and 1.2.1 have no hooks and keep stick emulation whatever `mouse_mode` says.
|
||||
|
||||
In stick-emulation mode, mouse motion (raw input) becomes right-stick deflection. Each presented frame takes the counts moved since the last frame, divides by the frame time and sets the deflection to `counts per second * NX2NAVI_MOUSE_SENS / 3000` of full, held for that frame, so the turn rate does not depend on the frame rate. This is not mouselook: the game's stick curve and dead zone still apply, so very slow movements do nothing and a fast flick saturates. The mouse deflection competes with the gamepad and arrow keys by the larger-deflection rule.
|
||||
|
||||
`NX2NAVI_MOUSE_SENS` (0 to 10, default 1; higher turns faster, 0 turns the mouse camera and the cursor capture off). While the window is the foreground window the cursor is hidden and confined to the client area. It is released on focus loss, minimise, while F11 changes the window, during a size or move loop and while a text entry is open. Esc releases it by hand: the cursor shows, the mouse camera and mouse buttons stop, and the window can be moved, minimised or closed. A mouse click inside the client area captures it again; that click does not reach the game. Activating the window by a click on its title bar or border does not capture, so it can be dragged or closed right away; Alt+Tab or a click in the client area captures as before. `NX2NAVI_INPUT=none` disables the capture too.
|
||||
|
||||
`NX2NAVI_STICK_DEADZONE` (0 to 50, default 8) is the gamepad stick dead zone in percent of full deflection, the same for both sticks. It is round: a stick inside the circle reads centred, and the rest of its travel is scaled to start at 0 at the edge, so full deflection still reaches full. Raise it for a stick that drifts. The game's own dead zone applies after it. `NX2NAVI_LSTICK_DEADZONE` and `NX2NAVI_RSTICK_DEADZONE` (the *Left/Right stick dead zone* rows under *Analog sticks* on the Controls tab) set one stick each and fall back to `NX2NAVI_STICK_DEADZONE` while unset, so a worn stick can get a larger zone than the other. With a gamepad connected each row shows its stick's deflection now, before the dead zone (*stick now*): let go of the stick and set the row a few percent above the value it rests at.
|
||||
|
||||
The stick direction bits (`StickLLeft` to `StickRDown`) are derived from the sticks at 50% deflection. A gamepad plugged in later is picked up within one second; with none connected the log says `input: no gamepad`.
|
||||
|
||||
## Mouse in the game's menus
|
||||
|
||||
On game 1.4.3, while one of the game's menus has its focus box (the frame the d-pad moves: the title, the save list, the inventory and its System tab, Options, shops, dialogue choices), the cursor is released from the mouse camera and shown in the middle of the window. The item under the pointer takes the focus as if the d-pad had moved there, with the game's highlight, sound and description; a left click is A on it, a right click is B. The keys and the pad keep working, and the focus moves only when the pointer moves. This happens only while the keyboard and mouse are the device used last: after a gamepad button press or a stick pushed past half way the menus are as on the console, with no pointer, until the mouse moves or a key is pressed (`input_kbm_last` in `runtime/input.c`). The ability wheel, the d-pad strips, Autobuild, the Purah Pad's camera and the map stay gameplay: the mouse keeps the camera there. When the menu closes the cursor is captured again.
|
||||
|
||||
`runtime/menu_cursor.c` reads eui's screens at each pad read and moves the focus with the writes `eui::BoxCursorMgr::moveBoxCursor` makes, so no game code is called. `menu_cursor = 0` (`NX2NAVI_MENU_CURSOR=0`, live; the *Mouse in the game's menus* row of the Controls tab) turns it off; `NX2NAVI_MENU_CURSOR_LOG=1` logs every screen it reads (by default the first six), with the layout size, the node count and the hit panes it uses.
|
||||
|
||||
## Nintendo Switch 2 controllers
|
||||
|
||||
Windows does not present the Switch 2 Pro Controller as a gamepad SDL can open (the log says `input: no gamepad`), so it needs a bridge program that wakes it and creates a virtual gamepad. The DS4Windows fork by hbashton ([github.com/hbashton/DS4Windows](https://github.com/hbashton/DS4Windows), tested with release VIIPERRC4.5.1, a pre-release) with the output set to an emulated DualSense gives buttons, rumble, gyro and accelerometer, and reads the sticks through the controller's calibration (they rest at 0 to 2%). The game reads its virtual DualSense directly; the log then shows `input: gamepad "DualSense Wireless Controller" (rumble, gyro and accelerometer, ...)` and `input: motion sensors on`. Its release notes also list Joy-Con 2 support, which is untested here.
|
||||
|
||||
Bridges that output a ViGEm DualShock 4 (ProCon2Windows, connectyourpro as tested) give buttons and rumble but no motion: their virtual DualShock 4 reports a gyro and an accelerometer that never change, so motion aiming and object rotation do nothing. Run one bridge at a time; Steam Input can also claim the controller, so close Steam if the bridge does not find it.
|
||||
|
||||
## Scripted input
|
||||
|
||||
`NX2NAVI_INPUT_SCRIPT=<file>` replays presses. It is ORed with real input and still applies under `NX2NAVI_INPUT=none`. One event per line, `#` starts a comment:
|
||||
|
||||
```
|
||||
<frame> <button> [hold_frames] # A B X Y L R ZL ZR PLUS MINUS LEFT UP RIGHT DOWN LCLICK RCLICK; hold defaults to 4
|
||||
<frame> LSTICK|RSTICK <x> <y> [hold_frames] # -32767..32767, up is +y
|
||||
<frame> MOUSE <dx> <dy> # direct-mode raw counts, once; down is +dy
|
||||
@<ms> <button> [hold_ms] # timed on the world clock
|
||||
@<ms> LSTICK|RSTICK <x> <y> [hold_ms]
|
||||
@<ms> MOUSE <dx> <dy>
|
||||
```
|
||||
|
||||
`<frame>` counts 60 Hz ticks of game time since the first frame (presented frames at 60 fps), so the title menus get the presses at the same game time at any frame limit. `@<ms>` counts milliseconds of the world clock: game time stepped since the end of the first save load (the player leaves the title screen's origin), so a timed event lands at the same game time at any frame rate and load length. A later load, such as fast travel, still shifts what follows. In a script with timed lines the untimed ones stop when the world clock starts, as the load ends at another tick in each run. Each event logs once (`input: script frame N tick T: A`). `tools/input/load_save.txt` loads the first save from the title screen.
|
||||
|
||||
`NX2NAVI_INPUT_RECORD=<file>` (`input_record` in `nx2navi.ini`, or Input recording in the menu) writes the merged input in the same format, one line per button press and per stick segment: events before the world clock starts by tick, the rest timed; a button or stick held when the world clock starts is cut there and goes on as a timed event. The file replays as a script, which makes it the repro to attach to a bug report.
|
||||
|
||||
In direct mouse mode the recording also writes one `MOUSE` event per consumed raw displacement. These are one-shot
|
||||
displacements rather than held stick segments. They replay without cursor capture, including with `input = none`;
|
||||
the game must be accepting camera input when the event reaches its camera update.
|
||||
|
||||
`NX2NAVI_TEST_PAD=<lx>,<ly>,<rx>,<ry>` attaches an SDL virtual gamepad at start whose sticks rest at these raw SDL axis values (-32768 to 32767, down is +y), so a drifting or held stick can be tested without a controller: the values go through the gamepad mapping and `NX2NAVI_STICK_DEADZONE` like a real pad's, and `NX2NAVI_INPUT_RECORD` shows what reached the game.
|
||||
|
||||
## RenderDoc capture (blocked)
|
||||
|
||||
`NX2NAVI_RDOC=<frame>[,...]` (`runtime/rdoc_win.c`; `NX2NAVI_RDOC_DIR`, `NX2NAVI_RDOC_DLL`) loads RenderDoc before `vkCreateInstance`. RenderDoc hides `VK_EXT_external_memory_host`, which the back end requires for guest pool imports (`runtime/vk_memory.c`), so device creation fails under it. A capture needs a copy-based pool fallback first.
|
||||
238
docs/installer.md
Normal file
238
docs/installer.md
Normal file
@@ -0,0 +1,238 @@
|
||||
# Installer
|
||||
|
||||
A public release that turns the player's own game files into a playable NX2navi folder. The release carries no game
|
||||
data and no code derived from the game: the installer builds the game on the player's machine. The installed folder
|
||||
holds only the game; the installer and its tools stay outside it.
|
||||
|
||||
## What the player does
|
||||
|
||||
1. Downloads `NX2navi-Setup-<version>-windows.zip` from GitHub Releases (none is published yet), unpacks it anywhere and
|
||||
starts `NX2navi-Setup.exe`.
|
||||
2. Picks:
|
||||
- the base game (XCI or NSP) and the 1.1.0, 1.2.1 or 1.4.3 update (NSP); an NSZ or XCZ (compressed by the nsz
|
||||
tool) works as it is, the setup unpacks it itself;
|
||||
- their `prod.keys`;
|
||||
- optionally the firmware folder (`system/Contents/registered` of an unpacked firmware or an emulator's install),
|
||||
needed for Chinese, Korean and Traditional Chinese text;
|
||||
- the game language and the install folder.
|
||||
3. Waits for the setup, one time. The window shows each step and its progress.
|
||||
4. Plays `nx2navi.exe` from the install folder. The installer and the source NSP or XCI (or NSZ, XCZ) can be deleted.
|
||||
|
||||
The install folder (`main.c` `portable_paths()` reads this layout; it stays portable):
|
||||
|
||||
```
|
||||
<install folder>/
|
||||
nx2navi.exe, SDL3.dll, libopus-0.dll, nx2navi_xlate.exe
|
||||
nvngx_dlss.dll NVIDIA DLSS, downloaded by the setup's DLSS step when the player keeps it ticked
|
||||
nx2navi.ini language; the other keys as commented defaults (nx2navi.ini.example)
|
||||
exefs/, romfs/ extracted from the player's files
|
||||
shared-fonts/ the Switch system fonts, with Chinese, Korean or Traditional Chinese text
|
||||
profile/ saves
|
||||
mods/ the player's mods, one folder per mod (created by the menu's "Open mods folder")
|
||||
logs/ nx2navi.log of the last run, nx2navi.1.log to nx2navi.4.log of the runs before it
|
||||
shader and pipeline caches prepared by the setup; shader-cache/shader-cache.pack holds the translations
|
||||
nx2navi-install.json what built this folder: release version, input hashes, finished steps
|
||||
```
|
||||
|
||||
## Updating the port
|
||||
|
||||
The player runs a newer installer on the same folder. It reads `nx2navi-install.json`, keeps `romfs/`, `exefs/`,
|
||||
`shared-fonts/`, `profile/` and `nx2navi.ini` after checking their hashes, rebuilds `nx2navi.exe` and re-prepares the
|
||||
shaders. A newer release that changes nothing game-derived skips the rebuild.
|
||||
|
||||
Given an update file other than the recorded one (1.1.0, then 1.2.1, then 1.4.3), the installer extracts and builds again from it
|
||||
and keeps `profile/`. Saves carry over both ways between 1.2.1 and 1.4.3: 1.4.3 converts a 1.2.1 save on load, and 1.2.1
|
||||
loads the saves 1.4.3 wrote.
|
||||
|
||||
## Setup steps
|
||||
|
||||
1. **Check** before writing anything: an NSZ or XCZ input is unpacked first (below), the files open with the keys, the update is 1.1.0, 1.2.1 or 1.4.3 (by hash, as
|
||||
`tools/extract.py`), the keys are new enough for the update, enough space is free on the target disk (about 36 GiB at the peak of a
|
||||
first install, 20 GiB after it), the CPU is at least x86-64-v2 (SSE4.2; with AVX2/FMA the faster `v3` build is used,
|
||||
otherwise the `v2` one), a Vulkan 1.3 GPU exists. Each failure is one clear message.
|
||||
2. **Fonts** with Chinese, Korean or Traditional Chinese text and a firmware folder: extract the Switch system fonts
|
||||
from the firmware into `shared-fonts/` (MacNativeGames' `extract_shared_fonts.py`, which verifies every NCA layer
|
||||
and BFTTF header). Without a firmware folder the setup warns and goes on; the game then lacks those glyphs.
|
||||
3. **Extract** ExeFS and RomFS into the install folder and verify the hashes.
|
||||
4. **Lift** the ExeFS to C (`recomp/gen.py`, `recomp/patches-v121.txt` or `patches-v143.txt`) into a temp folder next
|
||||
to the installer. The code cheats of the mods' cheat files and the mods' ExeFS patch files become patch sets first
|
||||
(`tools/cheats.py`, `tools/exefs_patches.py`, status in `build-cache/cheats.status` and `exefs.status`); a file the
|
||||
tools refuse costs those cheats or patches, not the install.
|
||||
5. **Compile** the generated C, `modules.c`, `romfs.c`, `patches.c` and `ov_zstd.c` (they bake the per-install layout, RomFS
|
||||
hashes and the game version's patch page) and link
|
||||
them with the prebuilt runtime library for the host's level (`libnx2navirt.a` or `libnx2navirt-v2.a`) into `nx2navi.exe`
|
||||
(`build_game.py --runtime-lib`; about 3 min on 24 cores).
|
||||
6. **Prepare** (only when the runtime, the translator or the game version changed, not for a new lift) the RomFS
|
||||
patches and the shaders (`tools/make_romfs_patches.py`, `tools/prepare_shaders.py`,
|
||||
`nx2navi.exe --prepare-shaders`) in the install folder, then fold the shader cache's loose files (four per shader, about
|
||||
117,000) into `shader-cache/shader-cache.pack` (`tools/pack_shader_cache.py`). nx2navi.exe reads the pack first and
|
||||
writes the translations it makes in play as loose files; running the tool again with the game closed folds those in.
|
||||
The patches are written four times: `romfs-patches`, `romfs-patches-xbox` and `romfs-patches-xbox-label`
|
||||
(`--prompts`: Xbox button prompts for `prompts = xbox`, the first for `pad_layout = nintendo`, the second for `xbox`
|
||||
and `auto`), and `romfs-patches-pc` (keyboard and mouse prompts for `prompts = pc`). The three prompt folders are
|
||||
optional: when one cannot be made, the setup warns and finishes.
|
||||
When the graphics driver crashes while building the shaders, `prepare_shaders.py` runs that part again without the
|
||||
variants it was building ([building.md](building.md), step 6), and the setup finishes with a warning that names how
|
||||
many were left out.
|
||||
7. **DLSS** (the window's tick box, on by default, or `--dlss`; Windows with an NVIDIA RTX GPU): download
|
||||
`nvngx_dlss.dll` from NVIDIA's own DLSS SDK repository (github.com/NVIDIA/DLSS, a pinned tag, NVIDIA RTX SDK
|
||||
license) into the install folder and check its size and SHA-256 (`steps.py`, `Dlss`). The release does not ship
|
||||
the library; NGX loads it from the folder of nx2navi.exe for `dlaa` and the DLSS modes. A DLSS library already in the
|
||||
folder (a newer one the player put there) is kept. Without an RTX GPU the step only logs and does nothing; a failed
|
||||
download warns, the game runs without DLSS, and the next run of the setup tries again.
|
||||
8. **Clean up** the temp folder, except the generated C and the objects (about 1.3 GB), and write
|
||||
`build-cache/rebuild.txt` in the install folder.
|
||||
|
||||
A cancelled or failed setup resumes from the last finished step (`nx2navi-install.json`).
|
||||
|
||||
## Rebuilding for the mods
|
||||
|
||||
The generated C and the objects stay in the temp folder, so the next lift rewrites only the chunks whose C changed and
|
||||
Ninja recompiles only those: a new or edited code cheat costs the lift (under a minute on 24 cores, a few minutes on 4)
|
||||
and a few dozen chunks, not a full compile, and the shaders stay prepared.
|
||||
|
||||
`build-cache/rebuild.txt` holds the command that opens the setup window in its rebuild mode on the install folder
|
||||
(`installer/gui.py --rebuild <install folder>`, started through `python -c` so the release folder is on `sys.path`
|
||||
whatever the working folder), one argument per line. When an enabled mod has code cheats or ExeFS patch files the build
|
||||
lacks, `nx2navi.exe` runs it at start and exits (`runtime/cheats.c` `rebuild_if_needed`); the
|
||||
window runs the setup on the folder at once and starts the game again when it succeeds. A failed or cancelled rebuild
|
||||
offers to start the game with the code it has; `build-cache/rebuild.skip` then keeps the same files from starting it
|
||||
again, until "Rebuild the game now" on the Cheats or Mods tab. `[mods] mods_rebuild = off` turns it off. The rebuild
|
||||
needs the release folder the setup ran from: a moved or deleted release folder means running a setup again.
|
||||
|
||||
## Command line
|
||||
|
||||
`installer/` (stdlib only; `installer/layout.py` maps the release folder and, in a repository, the dev toolchain):
|
||||
|
||||
```
|
||||
python -m installer --base FILE --update FILE --keys FILE --lang CODE --dest DIR [--firmware DIR] [--root DIR] [--temp DIR] [--verify]
|
||||
```
|
||||
|
||||
- On a folder with `nx2navi-install.json` only `--dest` is needed; the game files only when the extraction must run again.
|
||||
- `--firmware DIR` is the folder holding the firmware NCAs: an unpacked firmware, or the emulators' installed
|
||||
`Contents/registered` (the parent `Contents` folder works too). It is read only with Chinese, Korean or Traditional
|
||||
Chinese text, and only when `shared-fonts/` is not already in the install folder.
|
||||
- stdout: one JSON event per line, types `start`, `plan`, `step` (started, done, skipped), `progress` (step, phase,
|
||||
done/total/unit), `space` (bytes needed and free on the install disk and the temp disk), `warning`, `error`, `end` (`installer/events.py`). Messages are ids with parameters;
|
||||
`installer/text/en.json` holds their English text. The human-readable log goes to `<temp>/logs/`.
|
||||
- Exit codes: 0 done, 1 failed, 2 bad command line, 3 cancelled. The window cancels with Ctrl+Break to the
|
||||
installer's process group.
|
||||
- Each step records a key over its inputs (runner.py). A step with the recorded key and intact output is skipped, so
|
||||
resume and update are one rule; output kept from another release is re-hashed first.
|
||||
- Peak space on the install disk is about 36 GiB, not 22: the extraction copies both Program NCAs (16.3 GiB) next to
|
||||
the merged RomFS (15.9 GiB) before it deletes them. The check computes the exact figure.
|
||||
- Window (`python -m installer.gui`, `installer/gui.py` draws, `installer/guilogic.py` decides): form, step list with
|
||||
progress, space line, collapsible log, Cancel (Ctrl+Break to the CLI's process group, then it waits for the exit),
|
||||
finish screen with Open folder. English, Russian, Brazilian Portuguese or Simplified Chinese, chosen from the Windows
|
||||
display language (zh-CN and zh-SG give Chinese; the Traditional Chinese regions give English), switchable in the
|
||||
window; every sentence is an id in `installer/text/<en|ru|pt-BR|zh-Hans>.json`. A Chinese window still starts with
|
||||
English as the game language, because Chinese game text needs the Switch shared fonts, which the setup extracts only
|
||||
from a firmware folder. A folder with
|
||||
`nx2navi-install.json` needs only the folder. `--selftest [--shots DIR]` builds every screen in every language and
|
||||
closes itself.
|
||||
- `tests/installer_gui_test.py`: event model, form, catalogs, cancel, the self-closing window and one end-to-end run
|
||||
of the CLI in `--verify` mode (needs `build/install-test` and `build/extracted/v121`; see its docstring).
|
||||
- `tests/installer_test.py`: bookkeeping with fake steps. `tests/installer_e2e.py`: a dev-mode run with the files of
|
||||
`config.local.toml`.
|
||||
|
||||
## Release layout
|
||||
|
||||
The unpacked release; `tools/make_release.py` assembles it and `--zip` packs it:
|
||||
|
||||
```
|
||||
NX2navi-Setup.exe starts python/pythonw.exe -m installer.gui (installer/launcher/launcher.c, no console)
|
||||
installer/ the installer: steps, CLI, window, en/ru/zh-Hans text catalog
|
||||
python/ Python 3.14.9 embeddable + numpy, lz4, cryptography, cffi, pycparser (+ tkinter for the window)
|
||||
recomp/, tools/ the lifter and the build, extraction and shader scripts the steps run (copied from the repo)
|
||||
llvm/ llvm-mingw, x86_64-only subset (Apache-2.0 with LLVM exception; mingw-w64 parts permissive)
|
||||
runtime/ libnx2navirt.a, libnx2navirt-v2.a and the headers the generated code and modules.c/romfs.c/patches.c
|
||||
include
|
||||
bin/ nx2navi_xlate.exe, hactool.exe, SDL3.dll, libopus-0.dll, ninja.exe
|
||||
data/ shader_list.tsv (MacNativeGames, MIT); pgo-<ver>.profdata + .json, the execution profile an
|
||||
update's code and the runtime libraries compile with (building.md, "Profile-guided build")
|
||||
THIRD_PARTY_NOTICES.md
|
||||
nx2navi.ini.example, VERSION the default ini; the release string (the repository's VERSION plus the commit)
|
||||
```
|
||||
|
||||
- SDL3 comes from the upstream release, not MSYS2: MSYS2's DLL imports the LGPL libiconv.
|
||||
- Every bundled third-party piece gets a row in `THIRD_PARTY_NOTICES.md`.
|
||||
- The release contains no code, data, text or assets of the game; `make_release.py` runs `tools/check_content.py` over
|
||||
the folder before it finishes. The execution profiles name lifted functions by address (`f_<address>`) and hold
|
||||
branch counts and control-flow hashes of the generated C.
|
||||
|
||||
## Switch system fonts
|
||||
|
||||
Simplified Chinese, Traditional Chinese and Korean text needs the Switch system fonts (`0-standard.bfttf` and the other
|
||||
nn::pl shared fonts). The game does not ship them. Without them it exits before its first frame in those languages
|
||||
(`rt_fatal` in `runtime/ov_system.c`). Supply them from your own console, in either way:
|
||||
|
||||
- Give the setup the firmware folder (`--firmware DIR`, or the window's firmware field); the Fonts step extracts them to
|
||||
`shared-fonts/` in the install folder.
|
||||
- Run `python tools/extract_fonts.py --firmware <dir>` with `.venv-py314` and the keys from `config.local.toml`, or
|
||||
point `NX2NAVI_SHARED_FONTS` at a folder that holds the extracted fonts.
|
||||
|
||||
## Linux release
|
||||
|
||||
`tools/make_release_linux.py` builds the same setup for Linux and packs it as `NX2navi-Setup-x86_64.AppImage` (the static
|
||||
type-2 runtime: no libfuse2 needed on the host). The player makes it executable and starts it; on a Steam Deck, from
|
||||
Desktop Mode. The command-line setup needs the desktop session's display (`DISPLAY` or `WAYLAND_DISPLAY`) because preparing the
|
||||
shaders opens a window; the check step fails at once without it. Building the release also needs `make` and `pkg-config`. The bundled Tcl/Tk 9.0.4 is rebuilt with
|
||||
Xft against the sniper sysroot so Russian text uses system TrueType fonts. The setup builds the game into the folder they pick, as on Windows, and the game is `nx2navi` there with
|
||||
`libSDL3.so.0` beside it. The folder inside the AppImage:
|
||||
|
||||
```
|
||||
AppRun starts python/bin/python3 -m installer.gui (no bytecode written into the read-only mount)
|
||||
nx2navi-setup.desktop, nx2navi-setup.png
|
||||
installer/, recomp/, tools/, data/, licenses/, THIRD_PARTY_NOTICES.md, nx2navi.ini.example, VERSION as on Windows
|
||||
python/ python-build-standalone 3.14 (with tkinter) + the installer's packages; a .pth puts the folder,
|
||||
tools/ and recomp/ on sys.path
|
||||
llvm/ LLVM 23 (clang, lld, llvm-ar) and llvm/sysroot: the files of the Steam Runtime 3 "sniper" SDK the
|
||||
game's compile and link read (about 130 files)
|
||||
runtime/ libnx2navirt.a, libnx2navirt-v2.a, headers, the install-side sources, nx2navi.png; deps/ with the Vulkan
|
||||
headers, VMA and a static zstd
|
||||
bin/ nx2navi_xlate, hactool, ninja, libSDL3.so.0
|
||||
```
|
||||
|
||||
- What we compile is built against the sniper sysroot (glibc 2.31): the game, hostcheck, hactool and the translator
|
||||
need no newer glibc. The game links Opus, libvpx, zstd and libstdc++ statically and needs only glibc, the system's
|
||||
Vulkan loader and the SDL3 beside it. The setup itself needs glibc 2.34 (Ubuntu 22.04, Debian 12, Fedora 35,
|
||||
SteamOS 3): the official LLVM binaries and the `cryptography` wheel are built against it. `make_release_linux.py`
|
||||
checks every ELF file in the folder against that floor (`check_glibc`), and the setup's check step stops with one
|
||||
message on an older system.
|
||||
- The setup's temp folder is `.setup-temp` inside the install folder (`--temp` changes it): the AppImage is mounted
|
||||
read-only, and a Steam Deck's `/home` is often too full for the generated C while the game goes to the SD card.
|
||||
- The game keeps Mesa's shader cache in `mesa-cache` beside `nx2navi` unless `MESA_SHADER_CACHE_DIR` is set, for the
|
||||
same reason: RADV fills its default folder in `~/.cache` up to 1 GB.
|
||||
- The window follows `LANG` for English or Russian, cancels through SIGTERM to the CLI's process group (the CLI handles
|
||||
it as Ctrl+Break on Windows) and opens the finished folder with `xdg-open`.
|
||||
- Requirements beyond Windows': Linux 6.7 or later (the write watch, `docs/building.md`) and glibc 2.34.
|
||||
- Building the release needs one game version extracted and lifted (`tools/extract.py`, `recomp/gen.py`, either
|
||||
version): the runtime library includes the generated `layout.h`, whose shared part (`MOD_COUNT`, `MOD_MAIN_BASE`)
|
||||
is the same for both. `tools/make_release.py` takes `generated/v121`, else `generated/v143`.
|
||||
|
||||
## Constraints
|
||||
|
||||
- The installer steps contain no Win32 calls: Linux runs the same steps with its own compiler ("Linux release").
|
||||
- The keys file path is passed to the tools; key values never appear in the UI, logs or files.
|
||||
- The window is a front end over the CLI (`python -m installer --base ... --update ... --keys ... --lang ... --dest
|
||||
...`), which prints one JSON progress event per line. Tests and CI use the CLI.
|
||||
- Paths may hold any Unicode characters (a Cyrillic user name puts the temp and install folders under it). Every exe
|
||||
the setup runs or builds reads its arguments and paths as UTF-8 through an `activeCodePage` manifest: hactool
|
||||
(our patch, [building.md](building.md)), nx2navi.exe (`runtime/nx2navi.rc`), ninja (upstream). The Python steps write
|
||||
their files and logs as UTF-8, and `build_game.py` quotes paths with spaces on its Ninja command lines.
|
||||
- A game file whose data fails its hash checks or ends early stops the setup with `check.input_damaged`, which names
|
||||
the file and asks to copy it again (`extract.DamagedFile`). A failed merge checks the base Program NCA alone to tell
|
||||
which file is damaged.
|
||||
- Unsigned exe: SmartScreen warns on first start.
|
||||
- NSZ and XCZ input (game files compressed by the nsz tool: NCAs stored as NCZ, their sections decrypted and
|
||||
compressed with zstd): the check step tells them by their content, not their name (an NSP or XCI holding NCZ files),
|
||||
so the file dialog has one filter for all four. It unpacks each into `<temp>/unpacked/<base|update>.nsp`
|
||||
(`tools/ncz.py`), an NSP of the NSZ's files or of the XCZ's secure partition, and the extraction reads that.
|
||||
Unpacking encrypts the sections again, so every NCA comes back byte for byte, and checks each against the hash its
|
||||
name carries; a mismatch is `check.input_damaged`. It needs the unpacked size free in the temp folder (checked
|
||||
first), runs at about 500 MB/s, and an interrupted setup reuses a finished NSP of the same file. The extract step
|
||||
deletes the folder when it is done. An XCZ update is refused like an XCI one.
|
||||
|
||||
## Later
|
||||
|
||||
- Re-warm the driver cache on start after a move, GPU or driver change.
|
||||
235
docs/maxwell.md
Normal file
235
docs/maxwell.md
Normal file
@@ -0,0 +1,235 @@
|
||||
# Maxwell / NVN behaviour a port must reproduce
|
||||
|
||||
Reference for diagnosing graphics bugs. Each item: what Maxwell (GM20B, 3D class `B197`, compute `B1C0`) and NVN do,
|
||||
what Vulkan / AMD (RDNA4, our GPU0) do instead, how emulators and ports handle it, and the source. Items are ordered by
|
||||
how likely they are to cause visible artifacts. "Ours:" marks what our runtime does (grep-level look, see section 7;
|
||||
our code is cited by file and function). Licensing: the hardware/suyu are GPL, cited for behaviour only.
|
||||
|
||||
Source shorthands:
|
||||
|
||||
- **OGD**: NVIDIA open-gpu-doc `classes/3d/clb197.h`, `classes/compute/clb1c0.h` (MIT-style),
|
||||
<https://github.com/NVIDIA/open-gpu-doc/tree/master/classes>. Offsets below are method offsets in `clb197.h`.
|
||||
- **G80TEX**: Mesa nouveau `src/gallium/drivers/nouveau/nv50/g80_texture.xml.h` (MIT),
|
||||
<https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/gallium/drivers/nouveau/nv50/g80_texture.xml.h>.
|
||||
- **RYU-SU**: Ryujinx `Ryujinx.Graphics.Gpu/Engine/Threed/StateUpdater.cs` (MIT), mirror at 1.1.1137:
|
||||
<https://git.dawidpotocki.com/mirror/ryujinx/plain/src/Ryujinx.Graphics.Gpu/Engine/Threed/StateUpdater.cs?h=1.1.1137>.
|
||||
- **RYU-SH**: Ryujinx shader translator vendored at `refs/MacNativeGames/vendor/ryujinx/src/Ryujinx.Graphics.Shader/`
|
||||
(Ryujinx 1.3.2, MIT). Paths below are relative to it.
|
||||
- **NXN**: nxnative's graphics notes, `refs/nxnative/docs/agents/` (MIT). nxnative has rewritten them since; each
|
||||
row states the behaviour and fix it took from them.
|
||||
- **MNG**: `refs/MacNativeGames` @ `b75be8cfa704df3701e86130c8ced268e79d6b11` (MIT); line numbers below are at that
|
||||
commit.
|
||||
- **DK3D**: `refs/deko3d` @ `373bfd43d8b2d805fa2eb7d406dbb823032df958` (zlib, fincs), the homebrew Maxwell driver for Switch. Written
|
||||
against the hardware, not the NVN API: what it programs is what the GPU needs. Paths relative to `source/`. `.def` files give
|
||||
methods as word indices (`engine_3d.def`); byte offset = index x 4, so `0x5A1` = OGD `0x1684`. No conditional rendering, no
|
||||
provoking-vertex conversion, no 8-bit indices in it.
|
||||
- **YZ-2305 / YZ-2306**: the hardware progress reports May / June 2023,
|
||||
<https://the hardware-mirror.github.io/entry/the hardware-progress-report-may-2023/>,
|
||||
<https://the hardware-mirror.github.io/entry/the hardware-progress-report-jun-2023/> (behaviour only).
|
||||
- **VKSPEC**: Vulkan 1.4 specification, <https://registry.khronos.org/vulkan/specs/latest/html/vkspec.html>;
|
||||
**GLSL450**: SPIR-V GLSL.std.450 extended instructions, <https://registry.khronos.org/SPIR-V/specs/unified1/GLSL.std.450.html>.
|
||||
|
||||
## 1. Render targets
|
||||
|
||||
| # | Maxwell / NVN | Vulkan / AMD | Emulators / ports | Source |
|
||||
|---|---|---|---|---|
|
||||
| 1.1 | **Colour target the fragment shader does not write keeps its old texel**, even with blending enabled. | Undefined value written/blended. | NXN: zero write mask and no blend for unwritten outputs (`reflection colors`). Already fixed here. | NXN; MNG `games/<game>/tests/metal_semantics/probe.m:5` |
|
||||
| 1.2 | **Unwritten vertex outputs read as (0, 0, 0, 1)** in the next stage, including the unwritten components of a partly written output (VS writes xyz, FS divides by w). | FS input with no VS output: interface mismatch, value undefined; an unwritten component of a written output is undefined (AMD: whatever the register holds, often 0). | Ryujinx initialises exactly the components the next stage reads, but only when stages are linked (`Translation/Translator.cs:212-227`, `NextInputAttributesComponents`); position starts (0,0,0,1) (`:259`). NXN Link's Awakening: sun shadow lookups divided by w = 0, scene dim, no shadows. Ours: missing outputs padded with (0, 0, 0, 1), unlinked outputs initialised by translator patch 0006 (section 7, check 2). | NXN |
|
||||
| 1.3 | **Clears honour the scissor and the clear's own channel / stencil write mask**, and may come between draws inside a pass. `SET_CLEAR_SURFACE_CONTROL` (0x10f8) has `RESPECT_STENCIL_MASK`, `USE_CLEAR_RECT`, `USE_SCISSOR0`, `USE_VIEWPORT_CLIP0`. | `vkCmdClearAttachments` takes a rect but **ignores the colour write mask**; load-op clears are whole-attachment. | NXN/MNG draw a triangle with the clear's mask (`clear_color_tri`, `clear_ds_tri`); NX2navi uses mid-pass, partial-mask and partial-scissor clears (NXN). the hardware: "red dot" was clears hitting the wrong RT index (YZ-2305, PR 10243). Hardware recipe (DK3D): `CLEAR_BUFFERS` (`0x674` = 0x19d0) carries the colour channel mask (bits 2-5), depth (0), stencil (1), target id (6-9) and layer (10-20) in one word; `ClearBufferFlags` (`0x43E` = 0x10f8) is set once to `StencilMask | Scissor` (`maxwell/gpu_3d_base.cpp:69`), so every clear is cut by scissor 0 and its stencil bits by the stencil front write mask register, and DK3D's colour clear does not program `ColorWriteMask` (`0x680`). A stencil clear writes `StencilFrontMask` (`0x4E7`) and `ClearStencil` (`0x368`) just before the clear and restores the shadowed state after (`gpu_3d_base.cpp:497-510`). Multisample raster is switched off around a colour clear (`:459, :465`). Arrays: an MME macro repeats the clear for every layer (`maxwell/clear_buffers.mme:5-22`). No pass concept: clears sit between draws. Ours: masked colour and stencil clears are **skipped** (check 3). | OGD 0x10f8; NXN; DK3D `maxwell/gpu_3d_base.cpp:444-512`, `engine_3d.def:305, 644` |
|
||||
| 1.4 | **Conditional rendering** (`SET_RENDER_ENABLE_A..C` 0x1550-0x1558, NVN `SetRenderEnableConditional`) compares two 64-bit counters at `addr` and `addr+16` on the GPU, in command order. NX2navi wraps nearly every clear in it. A `TIMESTAMP` report writes 0 to the counter field. | `VK_EXT_conditional_rendering` tests one 32-bit word against zero. | NXN: honouring it fixed the black 3D scene (light-volume depth never cleared). deko3d has no conditional rendering at all (no `SET_RENDER_ENABLE` in `engine_3d.def`), so it cannot settle the compare rule. Ours: evaluated on the CPU at record time; a report from an unfinished submission counts as 0 (check 5). | OGD; NXN |
|
||||
| 1.5 | **Depth mode** is runtime state: `nvnDeviceSetDepthMode` near = -w (GL, NVN default) or 0. Register `SET_VIEWPORT_Z_CLIP` (0x0d7c) `NEGATIVE_W_TO_POSITIVE_W` / `ZERO_TO_POSITIVE_W`. | Vulkan clips z < 0 unless `VK_EXT_depth_clip_control` `negativeOneToOne`. | Ryujinx infers the mode from the viewport transform, falling back to the register (`GetDepthMode`, RYU-SU). NXN: wrong mode either clips everything (black window) or puts all depth in 0.5..1 (DOF blurs the screen). Ours: `negativeOneToOne` from NVN state. DK3D programs both halves: `SetDepthMode` (`0x35F` = 0xd7c) selects the pre-transform clip range, and the viewport transform maps z to [near, far] itself: scale z = `0.5 * (far - near)` and offset z = `0.5 * (near + far)` for -1..1, scale `far - near` and offset `near` for 0..1 (`maxwell/gpu_3d_base.cpp:143-147, 336-343`). Same maths as Vulkan `negativeOneToOne`. | OGD 0x0d7c; NXN-64, NXN; DK3D `maxwell/gpu_3d_base.cpp:136-147, 321-362` |
|
||||
| 1.6 | **Depth clamp / clip**: `SET_VIEWPORT_CLIP_CONTROL` (0x193c) has separate `PIXEL_MIN_Z`/`PIXEL_MAX_Z` CLIP or CLAMP and geometry-clip modes. | `depthClampEnable` also disables z clipping (unless `VK_EXT_depth_clip_enable`). RADV had a dynamic depth clamp driver bug causing NX2navi vertex explosions. | Ryujinx: clamp = `!(clip & DepthClampDisabled)` (RYU-SU `UpdateDepthClampState`). the hardware disabled dynamic depth clamp on affected RADV (YZ-2305, PR 10262). DK3D has one switch, `depthClampEnable`, and writes `VIEW_VOLUME_CLIP_CONTROL` (`0x64F` = 0x193c) with `0x101A` when on and `0x181D` when off (default at setup is `0x181D`); it labels the bits "todo: figure out what this actually does". Bits 3-4 (`DepthClampNear/Far`) are set in both values, bits 0-2 and 11 differ. There is no separate depth-clip toggle. The viewport near/far registers (`Viewport` array words 2-3) carry the clamp range and are swapped when far < near (`:349-350`). Vulkan clamp to [min, max] of the viewport depth matches. | OGD 0x193c; DK3D `maxwell/gpu_3d_state.cpp:42`, `gpu_3d_base.cpp:135-139, 349-350`, `engine_3d.def:621` |
|
||||
| 1.7 | **Depth formats**: D24S8 is native; D24 quantises to 2^-24 everywhere in 0..1. Polygon offset units are in D24 steps for D24 targets; `SET_DEPTH_BIAS_CONTROL` (0x1110) `DEPTH_FORMAT_DEPENDENT`. | RX 9070 has no D24S8 attachment; D32F has finer steps near 0 and a per-primitive exponent-based r. EQUAL / LEQUAL tests between a pre-pass and a later pass, and decal biases, behave differently. | the hardware rescales depth-bias units when D24 is emulated as D32F; Ryujinx passes `units / 2` (register is twice the API value) (RYU-SU `UpdateDepthBiasState`). MNG: D16 load-clear 0.5 → 32767 vs fragment 0.5 → 32768 made object/Recall classification tiles visible. Ours: D24S8 → D32_SFLOAT_S8, bias not rescaled (check 4). DK3D writes `PolygonOffsetUnits` (`0x56F` = 0x15bc) as `constantFactor * 2.0f`, clamp to `0x61F`, factor to `0x55B` (`maxwell/gpu_3d_state.cpp:148-157`), confirming the x2 register. | OGD 0x1110, 0x15bc, 0x156c; MNG `docs/how-it-works.md:57-61`; DK3D `maxwell/gpu_3d_state.cpp:154` |
|
||||
| 1.8 | **Position invariance**: identical position math gives identical depth across programs (pre-pass then EQUAL / LEQUAL). | Only for outputs declared `Invariant`. | Ryujinx decorates position `Invariant` (`CodeGen/Spirv/Declarations.cs:571`) and marks float ops `NoContraction` unless `QueryHostReducedPrecision` (`CodeGen/Spirv/Instructions.cs:2047`). NXN: Odyssey ground z-fought without it; LA's foliage drawn with depth EQUAL vanished when the pre-pass failed. Ours: translator default (all NoContraction). | NXN |
|
||||
| 1.9 | **Sampling a texture that is also a bound render target**: Maxwell's texture cache is not coherent with ROP writes; NVN requires `nvnCommandBufferBarrier(INVALIDATE_TEXTURE)` between writing and sampling (methods `INVALIDATE_TEXTURE_DATA_CACHE` 0x1338, `..._NO_WFI` 0x1288). Reading a different level/layer, or read-only depth while testing, is fine. | Same image as attachment and sampled in one pass is a feedback loop: undefined unless `VK_EXT_attachment_feedback_loop_layout`; on AMD DCC/HTILE-compressed data is not readable by the sampler without decompression. | the hardware: pixelated object glow fixed with barriers on attachment feedback loops (YZ-2305, PR 10402). BotW black textures on GCN worked around with `AMD_DEBUG=nohyperz` (HTILE off) (<https://git.openpunk.com/OpenPunk/the hardware/wiki/commit/bd401539420182cbbf8bc638b042536a7e690c3f>). Ours: an NVN barrier ends the pass and inserts a full memory barrier; no detection of a sampled texture that is a current attachment (check 10). DK3D barrier recipe (`maxwell/gpu_base.cpp:49-115`): the ordering part and the cache part are separate. Ordering: Tiles = `TiledCacheBarrier` (`0x3DF`), Fragments = `FragmentBarrier` (`0x378`), Primitives = `WaitForIdle` (`0x44`), Full = gpfifo `SetReference` + new gpfifo entry with `NoPrefetch`, so later command and indirect-parameter fetch happens after everything finished. Tiles and Fragments need an implicit WFI before a cache invalidate. Caches (each opt-in): texture data `InvalidateTextureDataCache` (`0x3DD`, WFI) or `..NoWfi` (`0x4A2`); shader `InvalidateShaderCaches` (`0x369`: instruction, global data, constant); descriptors `InvalidateTextureHeaderCacheNoWfi` (`0x50A`) + `InvalidateSamplerCacheNoWfi` (`0x509`); zcull `0x4B2`; L2 gpfifo `MemOpB` `L2FlushDirty` + `L2SysmemInvalidate`. The texture-cache invalidate is its own flag; ordering alone does not refresh it. Every queue flush ends with image + shader + descriptor + L2 invalidation "to ensure the visibility of CPU updates between flushes" (`:28-47`): CPU writes into textures and descriptors reach the GPU at the next submit boundary, not mid-submit (2.2). | OGD; DK3D `maxwell/gpu_base.cpp:28-115` |
|
||||
| 1.10 | **Duplicate attachments**: the same texture bound at two colour slots with write mask 0 on one does not interfere (ROP writes only enabled channels). | Tile-based or compressed attachments may store the unwritten copy over the written one. | NXN BotW G-buffer (c1 = c2, c3 = c4): leaves the duplicate slot unattached (`rt_dup_slot`). | NXN |
|
||||
| 1.11 | **Per-target write masks**: `RtColorMask[i]`, or entry 0 for all when `RtColorMaskShared`. `SET_SINGLE_CT_WRITE_CONTROL` (0x0f90). A colour target with format 0 is disabled. | Per-attachment `colorWriteMask`. | RYU-SU `UpdateRtColorMask`, `IsRtEnabled` ("Colors are disabled by writing 0 to the format"). | OGD 0x0f90 |
|
||||
| 1.12 | **sRGB render target views**: NVN encodes on write when the view format is sRGB over a linear texture. Maxwell also has a global `SET_SRGB_WRITE` (0x15b8). | Format of the attachment view decides; needs `MUTABLE_FORMAT`. | NXN Mario: attaching the base texture stored linear values, picture darker. Ours: RT views take the view format (`view_format`). | OGD 0x15b8; NXN |
|
||||
| 1.13 | **Compressed and 3D targets**: agl's GPU BC4 encoder renders blocks through an `RGBA16UI` view, one plane of a 32×32×32 BC4 3D texture per pass. NVN renders a 3D texture slice by slice. | Needs `BLOCK_TEXEL_VIEW_COMPATIBLE` + `EXTENDED_USAGE`, `2D_ARRAY_COMPATIBLE`. | NXN shadow textures (`bcrt_attach`). the hardware: wrong block depth on 3D slices gave Gloom textures on terrain (YZ-2305, PR 10433). Already fixed here. | NXN |
|
||||
| 1.14 | **Draw with no render target** (fragment shader only writes an SSBO, 1×1 viewport) runs normally. | Needs an attachment-less pass with explicit render area. | NXN `bare_dims`: cave ambient light came back. Ours: area of viewport 0 (`vk_exec.c` `resolve_rt`). | NXN |
|
||||
| 1.15 | **Window origin**: NVN default lower-left (GL); NX2navi switches to upper-left. Y flip and scissor flip follow. | Vulkan upper-left; negative viewport height to flip. | Ryujinx flips scissor against screen height when Y-negate is on (RYU-SU `UpdateScissorState`). | NXN |
|
||||
| 1.16 | **Multiple viewports / scissors** (16), selected by `ViewportIndex` output. | Needs `multiViewport` and `shaderOutputViewportIndex`. | Ours: one viewport, one scissor; differing scissors logged (`nvn_api_cmd.c`). | OGD 0x192c |
|
||||
| 1.17 | **Fragment output saturation** per target: `SET_PS_SATURATE` (0x13a8) clamps output n to 0..1 or -1..1 in fixed function. NVN does not expose it (GLSLC emits `.SAT` in code). | None. | Not emulated by Ryujinx; listed so a clamp difference is not chased in the wrong place. | OGD 0x13a8 |
|
||||
| 1.18 | **New memory reads as zero** (fresh pool pages); feedback effects never clear their state texture. | New image content undefined. | NXN `tex_zero_fill`. | NXN |
|
||||
| 1.19 | **MSAA**: `nvnCommandBufferDownsample` resolves; depth-stencil downsamples are copies the game does itself. | `vkCmdResolveImage` colour only; integer formats pick one sample. | NXN LA: stencil lost in the half-size depth copy, no AO. | NXN |
|
||||
|
||||
## 2. Textures and samplers
|
||||
|
||||
| # | Maxwell / NVN | Vulkan / AMD | Emulators / ports | Source |
|
||||
|---|---|---|---|---|
|
||||
| 2.1 | **Automatic block height**: NVN truncates `height_in_blocks * 1.5 / 8` before picking the GOB power of two (deko3d rounds up). Wrong H reads padding: 160×160 BC3 warp pad half black. Allocation size is the same, so size checks do not catch it. Mip levels shrink the block height while the level fits in half of it. | Opaque optimal tiling. | MNG fixed (40 SDK-measured cases, `games/<game>/tests/texture_auto_layout/probe.c`). Ours: truncating form already (`runtime/nvn_tex.c`, the block-height pick). DK3D reads: height in block rows (rounded up), x 1.5, then `(h + 7) / 8` rounded up, then the largest power of two <= that, capped at 16 GOBs (`dk_image.cpp:444-447`, `pickTileSize` `:13-20`); 3D uses `depth + depth / 2` directly as the GOB count (`:451-454`); video images are fixed at 2 GOBs (`:438`). Per level, the shift drops while `unit << (shift - 1) >= level dimension` with unit 8 rows / 1 slice (`adjustTileSize` `:23-37`, used at `:89-91`), width tile is always 0. Array layers are padded to the block shrunk to the full height / depth (`alignLayerSize` `:57-72`). Every one of these matches `nvn_tex.c` except the rounding. | MNG `docs/how-it-works.md:47-53`; DK3D `dk_image.cpp:13-37, 57-72, 80-103, 436-455` |
|
||||
| 2.2 | **Textures filled by the CPU with no NVN flush** (NX2navi terrain: two 121-layer ASTC 8×8 arrays, 42 MB each; BC4 material textures read back with `ReadTexels` and copied into a 5120×5120 roughness atlas). The CPU and GPU share memory, so no flush is needed on the Switch. | Host images are copies; must detect CPU writes. | NXN write-protects texture pages (`tex_dirty_upload`); zero ASTC decodes magenta, zero BC black. the hardware: small CPU writes into GPU-used pages not invalidated gave slow rain/snow particles and teleport particle corruption (YZ-2306, PR 10942). Ours: `GetWriteWatch` generations (`vk_memory.c`, `mem_written_pages`). | NXN-175 |
|
||||
| 2.3 | **Same storage, new texture object every frame** (transient targets, deferred handles). Same storage with a different descriptor reinterprets the same block-linear bytes. | Two `VkImage`s over one allocation with optimal tiling do not share a defined layout (AMD tiling and DCC metadata are per image); a new image starts `UNDEFINED`. | NXN shares host textures by `(pool, offset, description)`; MNG keeps a "live texture at address" redirect. the hardware: whole areas black for a frame on camera changes from texture-cache alias/overlap ordering (YZ-2305, PR 10418). Ours: GPU-only pool textures are `VkImage`s bound onto pool memory (check 7). | NXN; MNG `docs/how-it-works.md:67-71` |
|
||||
| 2.4 | **Deferred texture handles** (`BindTexturesDeferred`): the GPU reads the handle from memory when the command runs; BotW records command blocks once and writes handles later. | n/a | NXN BotW: handles copied at record time were zero, 3D scene black. | NXN |
|
||||
| 2.5 | **Border colours**: any RGBA (TSC words 4-7, plus sRGB border fields). NX2navi/BotW grass interaction map clamps to (0.5, 0.5, 0.5, 0.5), its "untouched" value. | Three fixed colours unless `VK_EXT_custom_border_color` (+ `customBorderColorWithoutFormat`); integer formats need `INT_CUSTOM`. | MNG emulates in the shader; with transparent black, grass away from the player lay flat and showed no cut (MNG issue #49). Ours: custom border when supported, else transparent black (check 11). | G80TEX; MNG `games/<game>/third_party/d2r_gfx/gfx/nvn/nvn_mtl.m:5041-5045` |
|
||||
| 2.6 | **sRGB decode** is a TIC bit (`G80_TIC_2_SRGB_CONVERSION`) separate from the format; TSC has its own `SRGB_CONVERSION` (word 0 bit 13). NVN exposes sRGB as formats. | Format decides. | NVN-level translation by format is enough. | G80TEX:197, :377 |
|
||||
| 2.7 | **Swizzles** per texture and per view (`TextureBuilderSetSwizzle`, `TextureViewSetSwizzle`); BotW layout captures rebuild a texture on the same storage with (1, 1, 1, R) or alpha 1. Default per-format mapping R8 → (R, 0, 0, 1). | `VkComponentMapping` on the view. | NXN: font atlas white boxes, HUD dark boxes without it. Ours: mapped (`vk_exec_draw.c` `get_view`). | NXN |
|
||||
| 2.8 | **Sample dimension comes from the instruction**: a non-array sample of an array texture reads layer 0; 1,512 BotW shaders sample a cube as a cube array. | View type must match the declared type. | NXN: 2D array bound as 2D gave one pale colour (washed-out scene); cubes created as 6-layer arrays with cube / cube-array views. Ours: views per resource type (`get_view`). | NXN |
|
||||
| 2.9 | **LOD**: TSC min/max LOD clamp, LOD bias, `TWO_D_NO_MIPMAP` texture type, a min filter without mips stays on the base level; TIC carries its own max anisotropy and aniso quality fields. | `minLod`/`maxLod`, `mipLodBias` limited by `maxSamplerLodBias`. | the hardware: anisotropy heuristics for mods forcing trilinear (YZ-2306, PR 10744). Ours: non-mip filter → `maxLod = minLod + 0.25`, bias clamped (`vk_texture.c` `make_sampler`). | G80TEX |
|
||||
| 2.10 | **Texture gather rounding**: Maxwell sub-texel precision differs from AMD/Intel; edge texels picked by TLD4 differ by one. | Implementation-defined `subTexelPrecisionBits`. | the hardware adds a sub-pixel offset to gathers on non-NVIDIA hosts: fixed BotW shadow artifacts at object edges ("shader_recompiler: Add subpixel offset for correct rounding at ImageGather", <https://git.openpunk.com/OpenPunk/the hardware/src/commit/780240e6979b198e7bd10feaad5399b8b4b63762>, title only). Ours: the translator biases gather coordinates by the host's `subTexelPrecisionBits` on every non-NVIDIA GPU (`vk_caps.c` → job `gather_bias` → Ryujinx `TexturePass.InsertCoordGatherBias`); NX2navi on RADV drew dashed dark lines along wall and floor edges in the prologue caves without it. `NX2NAVI_GATHER_BIAS=0` turns it off. | — |
|
||||
| 2.11 | **Gather offsets**: `TLD4.PTP` takes four per-texel offsets, possibly from registers. | `ConstOffsets` must be constant. | Ours: `QueryHostSupportsNonConstantTextureOffset() => false` makes Ryujinx emulate register offsets (`tools/xlate/Program.cs`). | RYU-SH `Translation/Optimizations` (TexturePass) |
|
||||
| 2.13 | **TXD gradient register order**: NX2navi's terrain shaders fill the gradient registers of a `TXD` per coordinate (du/dx, du/dy, dv/dx, dv/dy). | Vulkan's `Grad` takes dPdx (du/dx, dv/dx) and dPdy (du/dy, dv/dy). | Ryujinx's `EmitTxd` takes the first two registers as dPdx and the last two as dPdy, so the mip level came from the wrong derivatives and was too sharp on terrain faces turned across the screen: patches that followed the faces crawled when the camera moved (1.4.3, the Depths; found by the user from their F9 frame, a gradient sample's coordinate and gradients traced back to `DPdx` and `DPdy` of the interpolants). Ours: `gfx_shader.c` `patch_txd_transpose` swaps the two inner components of every two-component gradient pair in the loaded SPIR-V (the shader cache is untouched). `NX2NAVI_TXD_TRANSPOSE=0` turns it off. Only 2D gradients are handled; 3D and cube ones are left as translated. | — |
|
||||
| 2.12 | **Depth compare** (`TSC` depth compare + `TIC_4_DEPTH_TEXTURE`), min/max reduction filtering. | `compareEnable`; reduction mode needs `samplerFilterMinmax` and is not allowed with compare. | Ours: reduction dropped when compare is on (`vk_texture.c` `make_sampler`). | G80TEX |
|
||||
| 2.13 | **ASTC / BC**: native ASTC; zero ASTC block = error colour on hosts. | AMD and NVIDIA desktop GPUs lack ASTC: decode to RGBA8. the hardware offered ASTC recompression to BC1/BC3 for low VRAM (YZ-2305, PR 10398). | Ours: native upload where the device samples ASTC, else a compute-shader decode (`vk_astc.c`, `NX2NAVI_ASTC`), CPU decode (`tex_astc.c`) as the fallback. | `runtime/vk_astc.c` |
|
||||
| 2.14 | **Linear CPU-written textures** (`FLAGS_LINEAR`, small ramps, glyph caches): CPU writes visible to the GPU directly. | Must re-upload or use linear images over host memory. | NXN Mario: sky black without its 64×1 RGBA16F ramps. | NXN |
|
||||
|
||||
## 3. Shader math
|
||||
|
||||
| # | Maxwell | Vulkan / AMD | Emulators / ports | Source |
|
||||
|---|---|---|---|---|
|
||||
| 3.1 | **FMNMX and `.SAT` with NaN return the other operand / 0** (IEEE minNum). Games rely on this to stop NaN spreading (one NaN pixel blown up by bloom or temporal passes flickers or fills the screen). | `GLSL.std.450 FMin/FMax/FClamp`: result undefined when an operand is NaN (NMin/NMax are the NaN-dropping forms). AMD hardware min/max usually drops NaN; the compiler may still assume no NaN. | Ryujinx emits FMin/FMax/FClamp (`CodeGen/Spirv/Instructions.cs:336, 956, 972`; saturate = FClamp 0..1, `Instructions/InstEmitAluHelper.cs:419-423`). NXN translated shaders use `isnan` selects; MNG measured Metal min/max/saturate drop NaN (`probe.m:1-4`). | GLSL450; NXN |
|
||||
| 3.2 | **FMZ multiply mode** (`FMUL/FFMA/...` field `Fmz`, `.FMZ`): 0 × anything = 0, including Inf/NaN (DX9 rule). | IEEE: 0 × Inf = NaN. | Ryujinx decodes `Fmz` (`Decoders/InstDecoders.cs:306-309, 1901`) but no emitter reads it. Ours: same. Census needed (check 13). | RYU-SH |
|
||||
| 3.3 | **Denormals**: per-instruction `.FTZ`. | AMD flushes fp32 denormals by default; `VK_KHR_shader_float_controls` sets it per shader. | Ryujinx ignores `.FTZ`. Low risk. | RYU-SH (no `Ftz` use in `Instructions/`) |
|
||||
| 3.4 | **Warp size 32**: `SHFL`, `VOTE`, `S2R LANEID`, ballots are 32-bit. | AMD RDNA runs wave32 or wave64 per shader (the driver chooses; RADV fragment default wave64); `VK_EXT_subgroup_size_control` `requiredSubgroupSize = 32` fixes it. | Ryujinx adapts lane masks to `QueryHostSubgroupSize()` (default 32, `IGpuAccessor.cs:218`; `Instructions/InstEmitWarp.cs:72, 106`, `InstEmitMove.cs:175-185`). Ours: not overridden, no required size (check 6). | <https://docs.mesa3d.org/envvars.html> (`RADV_PERFTEST=pswave32`) |
|
||||
| 3.5 | **Discard (`KIL`)** keeps the quad's derivatives valid (demote semantics). | `OpKill` terminates; later derivatives in the quad undefined. | Ryujinx emits `OpKill` (`CodeGen/Spirv/Instructions.cs:508`). | — |
|
||||
| 3.6 | **Implicit-LOD sampling in divergent control flow** works on Maxwell (TEX unit uses quad neighbours). | Undefined in non-uniform control flow; AMD whole-quad mode usually works. | — | VKSPEC (derivatives) |
|
||||
| 3.7 | **Integer division** does not exist (compiled to `XMAD`/`IMAD` sequences); MUFU.RCP(0) = +Inf, LG2(0) = -Inf. | Same IEEE edge cases through `1/x`, `Log2`. | Low risk. | — |
|
||||
| 3.8 | **FP64** in some NX2navi cutscene shaders. | No FP64 on some hosts. | the hardware FP64 → FP32 (YZ-2306, PR 10693); ours: `softfloat64` job flag (`tools/xlate/Program.cs` `QueryHostSupportsShaderFloat64`). | — |
|
||||
| 3.9 | **Integer vertex inputs read as raw bits** (bone indices `R8UI`). | Float-declared inputs convert integers. | NXN Mario: stretched characters; re-translated with `uint` inputs. Ours: attribute types passed to `QueryGraphicsState` (`Program.cs`). | NXN |
|
||||
| 3.10 | **BFE / BFI fields past bit 31** (as PTX `bfe` / `bfi`): a field that runs past bit 31 is cut there; one that starts past it extracts 0 (signed: bit 31 repeated) and inserts nothing. NX2navi 1.4.3's Depths terrain vertex shaders (42 variants) take the position from a register, `((2y + x) * 8) & 0xff` with 5-bit x and y, so up to 248. | `OpBitField*Extract` / `OpBitFieldInsert` undefined when offset or offset + count exceeds 32; AMD's bitfield instructions take offset and width modulo 32. | Ryujinx passes the fields through. UltraTears saw the Depths walls and ground missing on the Steam Deck and a Linux RX 570 (RADV). Ours: clamped where the field is not a constant inside the word (translator patch 0012). RX 9070 on Windows drew the Depths the same before and after. | PTX ISA `bfe`, `bfi` |
|
||||
| 3.11 | **SHF by 0** gives the unshifted word (C for `SHF.L`, A for `SHF.R`). | Ryujinx's funnel shifts the other word by 32: undefined in SPIR-V; AMD shifts take the low 5 bits and give A \| C. | the hardware fixed Depths terrain gaps in its funnel shift (PR 10459). Ours: a select on a zero amount (translator patch 0013). In 1.4.3 every register SHF pairs its word with 0, so the result was right already. | — |
|
||||
| 3.12 | **Registers read before any write** on a path (paths the structurizer adds included) have no defined value in the game's code; UltraTears reports 0 from the NVIDIA driver. | An `OpVariable` without an initializer loads an undefined value; Mesa folds undefined values freely. | Ours: on Mesa drivers (RADV, ANV, NVK; `vk_caps_driver`) every function-local variable starts at 0 (`OpConstantNull`, job flag `zero_locals`, translator patch 0016; `NX2NAVI_ZERO_LOCALS` overrides). Elsewhere the load stays undefined: a zero start on every driver (the old patch 0014) made the whole picture shake by about 0.9 output pixels a frame with DLSS or DLAA on an RTX 4080 SUPER (#612). It also moved the SPIR-V hash of the game's FXAA shader out of `FXAA_FS` (`vk_dlaa.c` `vk_dlaa_skip_draw`), the likely cause of the shake; that path is not traced. | — |
|
||||
|
||||
## 4. Memory model
|
||||
|
||||
| # | Maxwell / Tegra | Vulkan / AMD discrete | Emulators / ports | Source |
|
||||
|---|---|---|---|---|
|
||||
| 4.1 | **One memory**: the CPU reads GPU-written buffers and textures after a fence; the GPU sees CPU writes after `FlushMappedRange` or, in practice, immediately (NX2navi skips flushes). Block-linear layout is defined, so a buffer copy into texture storage is a texture upload. | Images live in VRAM with private tiling; buffer bytes copied into image memory are not texels. | the hardware PR 10942 (small CPU writes); MNG alias visibility barriers for copies through two buffers over one allocation. Ours: GPU-written ranges of CPU-visible pools marked at fence completion; GPU-only pools nothing (check 8). | MNG `docs/how-it-works.md:67-71` |
|
||||
| 4.2 | **Counter reports** (`ReportCounter`, `SET_REPORT_SEMAPHORE_A..D` 0x1b00): `{counter, timestamp}`; TIMESTAMP puts 0 in the counter; SAMPLES_PASSED counts visible samples; ZCULL stats. | Occlusion queries are separate objects; results copied to a buffer. | NXN NX2navi: timestamp counter 0 matters for conditional clears. Ours: every report writes `{0, time}`; occlusion counts are always 0 (check 5). DK3D writes `SET_REPORT_SEMAPHORE_OFFSET` (`0x6C0` = 0x1b00) with payload `0` and `OPERATION=Counter`, `STRUCTURE_SIZE=FourWords` (`dk_variable.cpp:193-240`): TIMESTAMP = unit `Crop`, counter `Payload`, written when the ROP has retired every earlier draw; SAMPLES_PASSED = unit `Crop`, counter `0x15`; ZCULL stats = four one-word reports at +0, +4, +8, +12. TIMESTAMP_TOP is not a 3D report: a gpfifo `Semaphore` release of 16 bytes with `ReleaseWfiDisable`, written when the command processor reaches it, without waiting for earlier work (`:246-253`). So `TIMESTAMP - TIMESTAMP_TOP` around a span is its GPU duration. Layout: two 64-bit words, counter then timestamp (`Primer.md:578`). Each report is followed by a tiled-cache flush (`:256`). `ResetCounter` (`0x54C` = 0x1530) zeroes SAMPLES_PASSED; it is the reset for the timestamp types too (`:282-300`). Unit: `dkTimestampToNs(ts) = ts * 625 / 384`, i.e. **614.4 MHz** GPU ticks (1.63 ns), not the 19.2 MHz system tick (`include/deko3d.h:1367-1373`) (check 21). | OGD 0x1b00; NXN; DK3D `dk_variable.cpp:183-300`, `include/deko3d.h:1367` |
|
||||
| 4.3 | **Semaphores / events** written by the GPU in order; CPU polls them. | Writes on a completion thread after the fence. | Ours: `vk_completion_add`, submit order. | — |
|
||||
| 4.4 | **Inline uniform updates** (`UpdateUniformBuffer`) happen in command order between draws; reading the buffer at record time gives the last value. | Must snapshot. | NXN: snapshotted at encode time; Mario's wave sim diverged when two writes reordered. | NXN |
|
||||
| 4.5 | **GPU-written index buffers** (8-bit indices written earlier in the same submission). | No `uint8` index type without `VK_EXT_index_type_uint8`; CPU widening reads stale bytes. | NXN widens on the GPU (`wide_run`). Ours: CPU widening at record time; device-local byte indices skipped (check 9). | NXN |
|
||||
| 4.6 | **Compute shared memory**: 48 KiB per block on GM20B. | AMD 64 KiB. | Ours: `QueryComputeSharedMemorySize` from job (`Program.cs`). | OGD `clb1c0.h` |
|
||||
| 4.7 | **Virtual memory pools** map physical pool pages; the same bytes reachable at two GPU addresses. | Two `VkBuffer`s over one allocation need alias-aware barriers. | MNG `MTL4VisibilityOptionResourceAlias`. | MNG `docs/how-it-works.md:67-71` |
|
||||
| 4.8 | **Command list calls**: a `CallCommands` / `CallList` is not a GPU subroutine. DK3D `dkCmdBufCallList` stores a `Call` control entry (`dk_cmdbuf.cpp:327-338`); `dkQueueSubmitCommands` walks the entries on the CPU and recurses into the called list at submit (`dk_queue.cpp:287-296`), emitting gpfifo entries that point at the command words, which the GPU fetches later (prefetch can be switched off per entry, `:176`). So list structure is fixed at submit, command words are read when fetched. | n/a | Ours: `OP_CALL` is walked at replay, `CallCommands` inlines the recording at record time (`vk_exec.c` `walk`, `nvn_api_cmd.c` `inline_recording`). Equivalent for control flow; deferred bind handles are the data read late (2.4). | DK3D `dk_cmdbuf.cpp:327`, `dk_queue.cpp:280-298` |
|
||||
|
||||
## 5. Fixed function
|
||||
|
||||
| # | Maxwell / NVN | Vulkan / AMD | Emulators / ports | Source |
|
||||
|---|---|---|---|---|
|
||||
| 5.1 | **Provoking vertex: last** (GL convention, NVN default; `SET_PROVOKING_VERTEX` 0x1684, NVN 55 `SetProvokingVertexMode`). Flat-interpolated outputs (instance/particle ids, material indices, flat normals) come from the last vertex. | **First vertex** unless `VK_EXT_provoking_vertex` with `provokingVertexMode = LAST_VERTEX`. | Ours: last-vertex mode where `provokingVertexLast` is available (`vk_pipeline.c` `create_graphics`; check 1). DK3D: `ProvokingVertexLast` (`0x5A1`) is set to 1 at queue setup and per rasterizer state from `DkProvokingVertex` (`maxwell/gpu_3d_base.cpp:76`, `gpu_3d_state.cpp:67`); `DkRasterizerState` defaults to Last (`include/deko3d.h:823`). Confirms last as the hardware-side default. | OGD 0x1684; `runtime/nvn_ext.c`; DK3D `include/deko3d.h:794-823` |
|
||||
| 5.2 | **Primitive restart**: any index value, compared against the raw index for any index size; can apply to lists (`SET_DA_PRIMITIVE_RESTART_TOPOLOGY_CHANGE`) and to non-indexed draws (`..._VERTEX_ARRAY` 0x0de8). | Restart only at all-ones of the index type; lists need `primitiveTopologyListRestart`. | Ryujinx enables it for draw-arrays only on GL (RYU-SU comment). NXN: NX2navi terrain strips use 0xffff. Ours: non-all-ones restart not replayed, logged. DK3D: `PrimitiveRestartEnable` (`0x591` = 0x1644) + `PrimitiveRestartIndex` (`0x592` = 0x1648), index written only when enabled; any value; `PrimitiveRestartWithDrawArrays` (`0x37A` = 0x0de8) is forced 0 at setup (`maxwell/gpu_3d_ibo.cpp:21-30`, `gpu_3d_base.cpp:73`). Nothing on lists vs strips beyond that. | OGD 0x1644, 0x1648; DK3D `maxwell/gpu_3d_ibo.cpp:21-30` |
|
||||
| 5.3 | **Point size**: `SET_POINT_SIZE` (0x1518) when the shader does not write it; GLSLC clamps a written size to a driver constant. | Point size must be written by the shader. | Ryujinx writes the `PointSize` from graphics state at VS start (`Translation/EmitterContext.cs:84-88`); ours reads the `SetPointSize` state from the support buffer (translator patch 0008). | NXN |
|
||||
| 5.4 | **Line width**, smooth lines (`SET_LINE_WIDTH_FLOAT` 0x13b0, `SET_ANTI_ALIASED_LINE`). | `wideLines`, `VK_EXT_line_rasterization`. | Ours: 1 px, logged. | OGD |
|
||||
| 5.5 | **Polygon offset units** are twice the API value in the register; NVN API takes GL units. | `depthBiasConstantFactor` in format-dependent r. | Ryujinx `units / 2f` from the register (RYU-SU). Ours takes NVN API units (right) but D32F instead of D24 (1.7). DK3D `constantFactor * 2.0f` (1.7). | OGD 0x15bc, 0x187c; DK3D `maxwell/gpu_3d_state.cpp:154` |
|
||||
| 5.6 | **Dual-source blending**: SRC1 factors read the shader's second output. | Needs fragment output `Location 0, Index 1`. | Ryujinx declares `Index` only when `DualSourceBlendEnable` (`Translation/Translator.cs:196`, `CodeGen/Spirv/Declarations.cs:608-617`). Ours: SRC1 factors mapped (`vk_pipeline.c` `blend_factor`) but the translator is always given `false` (`Program.cs` `QueryGraphicsState`) (check 12). | — |
|
||||
| 5.7 | **Alpha-to-coverage** with dither (`SET_ALPHA_TO_COVERAGE_DITHER_CONTROL` 0x12e0), alpha test (fixed function on Maxwell). | A2C without dither; no alpha test. | Ryujinx emulates alpha test and A2C dither in the shader from graphics state; ours passes `AlphaTestOp.Always`, A2C false to the translator (`Program.cs` `QueryGraphicsState`). | OGD 0x12e0, 0x16b4 |
|
||||
| 5.8 | **Logic ops** on colour targets. | `logicOp` feature; ignored on float targets. | Ours: in the pipeline key. | OGD 0x19c4 |
|
||||
| 5.9 | **Vertex formats**: `*SCALED` 8/16-bit, RGB10A2 SNORM/SCALED, 3-component formats, 32-bit scaled. | No 32-bit SCALED; 3-component formats often unsupported. | Ours: 32-bit scaled read as integers (logged), 3-component widened (`vk_pipeline.c` vertex format table). | — |
|
||||
| 5.10 | **Transform feedback**, rasterizer discard. | `VK_EXT_transform_feedback`. | Ours: rasterizer discard without XFB = no draw (`nvn_api_cmd.c`). | OGD 0x0744 |
|
||||
| 5.11 | **Depth bounds, sample mask, conservative raster**. | Optional features. | Ours: ignored, logged (`nvn_ext.c`). | OGD 0x0fa4 |
|
||||
| 5.12 | **Blend factors on targets without alpha** (RGBX emulated as RGBA). | Destination alpha reads the stored value. | Ryujinx replaces DST_ALPHA factors on no-alpha targets (`FilterBlendFactor`, RYU-SU). | RYU-SU |
|
||||
|
||||
## 6. Known BotW / NX2navi issues in other projects
|
||||
|
||||
| Game | Symptom | Cause | Fix / link |
|
||||
|---|---|---|---|
|
||||
| NX2navi | 3D scene black | conditional clears ignored | NXN |
|
||||
| NX2navi | cave magenta / black | CPU-filled ASTC/BC with no flush never uploaded | NXN |
|
||||
| NX2navi | stone blocks unlit by torches | roughness atlas built from `ReadTexels` of never-uploaded BC4 | NXN |
|
||||
| NX2navi | BC4 3D texture draws lost | compressed render target | NXN |
|
||||
| NX2navi | missing ambient light, bloom halo | draw with no render target skipped | NXN |
|
||||
| NX2navi | warp pad half black | block height rounding | MNG `docs/how-it-works.md:49-53` |
|
||||
| NX2navi | coloured squares with object / Recall | D16 clear vs fragment rounding with depth EQUAL | MNG `docs/how-it-works.md:57-61` |
|
||||
| NX2navi/BotW | grass flat away from Link, no cut | custom border colour (0.5) | MNG `nvn_mtl.m:5041-5045` |
|
||||
| NX2navi | cloud flicker, missing geometry in the Depths | shader recompiler bugs | YZ-2305 PR 10234 |
|
||||
| NX2navi | red dot on screen | clear targeted the wrong RT index | YZ-2305 PR 10243 |
|
||||
| NX2navi | scrambled Link/terrain at 2× | buffer-cache clears during sync | YZ-2305 PR 10249 |
|
||||
| NX2navi | vertex explosions (RADV) | driver dynamic depth clamp bug | YZ-2305 PR 10262 |
|
||||
| NX2navi | pixelated object glow | attachment feedback loop without barriers | YZ-2305 PR 10402 |
|
||||
| NX2navi | areas black for a frame on camera changes | texture-cache alias/overlap order | YZ-2305 PR 10418 |
|
||||
| NX2navi | Gloom textures on terrain | block-depth adjustment on 3D slices | YZ-2305 PR 10433 |
|
||||
| NX2navi | terrain gaps at distance (Depths) | one-character shader recompiler error | YZ-2305 PR 10459 |
|
||||
| NX2navi | mangled textures on RDNA3 | not found (AMD test case) | YZ-2305 |
|
||||
| BotW/NX2navi | slow rain/snow, particle corruption on teleport | small CPU writes into GPU-used pages not invalidated | YZ-2306 PR 10942 |
|
||||
| NX2navi | AO broken above 1008p mods | rescale thresholds | YZ-2306 PR 10675 |
|
||||
| NX2navi | "detective mode" Link on AMD | AMD EDS3 colour-blend driver regression | YZ-2306 PR 10946 |
|
||||
| BotW | shadow artifacts at object edges (AMD) | gather rounding | the hardware commit 780240e (2.10) |
|
||||
| NX2navi | dashed dark lines along wall and floor edges, changing with the camera angle (AMD, RADV) | gather rounding | gather bias (2.10) |
|
||||
| NX2navi | blocky foliage shadows that jump instead of sliding | not a GPU difference: the engine sizes its sun shadow map to the area it covers, down to about 400 near the ground, and each change of size moves the shadow texels | fixed size, `shadow_res` (the shadow set of `recomp/patches-v121.txt`; the community mods force 1024 the same way) |
|
||||
| BotW | black textures on GCN | HTILE (depth compression) | `AMD_DEBUG=nohyperz` wiki (1.9) |
|
||||
| BotW | 3D scene black | deferred texture handles | NXN |
|
||||
| BotW | albedo scattered squares | duplicate colour attachments | NXN |
|
||||
| BotW | HUD dark boxes, black stamina wheel | capture texture swizzle rebuild | NXN |
|
||||
|
||||
## 7. Checks for our runtime
|
||||
|
||||
Grep-level look on 2026-10-22, statuses updated 2026-10-26; "saw" is what the grep and short reads showed, not a test result.
|
||||
|
||||
1. **Provoking vertex.** Vulkan draws flat outputs from the first vertex, NVN from the last. `vk_pipeline.c` enables
|
||||
`provokingVertexLast` and chains `VkPipelineRasterizationProvokingVertexStateCreateInfoEXT{LAST_VERTEX}` in
|
||||
`create_graphics`; without the feature it logs that flat outputs come from the first vertex. **Fixed.**
|
||||
2. **Unwritten vertex outputs.** `spirv_add_outputs` (`vk_pipeline.c`) adds outputs initialised to (0, 0, 0, 1) for
|
||||
FS inputs the VS lacks. Our translator never links stages, so Ryujinx's `InitializeVertexOutputs` would initialise
|
||||
no user output; translator patch 0006 initialises the unlinked outputs. **Fixed.**
|
||||
3. **Masked clears.** `vk_exec.c` skips colour clears with a partial channel mask (`W_CLEAR_MASK`); `clear_ds` skips stencil
|
||||
clears with a partial mask (`W_STENCIL_MASK`). NX2navi issues such clears (NXN). Check the log for
|
||||
`colour clear with channel mask` and `stencil clear with mask`. DK3D (1.3) confirms what the replay must do: colour clear = scissor 0 + channel mask, stencil clear = `value & mask` merged under the front write mask; so a draw with `colorWriteMask` / `stencilWriteMask` set to the clear mask is the exact equivalent. **Missing.**
|
||||
4. **D24 → D32F.** `vk_formats.c` maps D24S8 to `D32_SFLOAT_S8_UINT`; `vk_exec_draw.c` `record_fixed` passes NVN units
|
||||
unscaled. Depth EQUAL passes and decal offsets can z-fight. Look at: per-pass depth-test op histogram, EQUAL passes
|
||||
against D24 targets. **N/A for NX2navi**: a census of the field and fight saves (2026-10-22) saw depth targets
|
||||
0x34 (DEPTH32F) and 0x32 (DEPTH16) only; biased draws (units -5 and 1, factor -0.5 and -0.2) are on DEPTH32F.
|
||||
5. **Counters and conditional rendering.** `vk_exec_xfer.c` (as of 2026-10-22): every report writes `{0, time}`;
|
||||
SAMPLES_PASSED is always 0; a condition is read on the CPU at record time and an unfinished report reads 0. Grep the
|
||||
NVN log for counter types other than TIMESTAMP; any occlusion-driven visibility (flares, culling) is off. **Fixed 2026-10-22**:
|
||||
SAMPLES_PASSED counts come from Vulkan occlusion queries between RESET and REPORT (`vk_exec_xfer.c`, ~4 queries
|
||||
per frame in the field, ~21 in a fight); the game's CPU reads them (guest f_70010c3298) to cull objects.
|
||||
6. **Subgroup size.** The translator assumes 32; RDNA4 may run fragment shaders in wave64. `vk_caps.c` and
|
||||
`vk_pipeline.c` `pin_subgroup32` pin stages to 32 lanes with `VkPipelineShaderStageRequiredSubgroupSizeCreateInfo`
|
||||
where `subgroupSizeControl` allows it, and otherwise tell the translator the GPU's own size
|
||||
(`NX2NAVI_VK_SUBGROUP_PIN=0` forces that). **Fixed.**
|
||||
7. **Images aliased onto GPU-only pool memory.** `vk_texture.c` `bind_pool_memory` binds each `VkImage` into its pool
|
||||
allocation; textures are keyed by address plus descriptor, and each new image starts `UNDEFINED` (`init_image`). Two descriptors over the
|
||||
same storage, or bytes written through a buffer, do not see each other's data on AMD. Check: count distinct keys per
|
||||
address per frame. **Risk** (the hardware PR 10418 class).
|
||||
8. **GPU writes into CPU-visible pools.** `note_gpu_write` (`vk_exec_xfer.c`) stamps pages when the fence
|
||||
completes, so a texture sampled later in the same submission keeps its old upload; GPU-only pools are skipped.
|
||||
MNG saw at most one buffer copy per frame in NX2navi, so low priority. **Risk.**
|
||||
9. **Index buffers.** `vk_exec_draw.c` `op_draw_idx`: byte indices widened on the CPU at record time, skipped in
|
||||
device-local memory; `restart_for`: a non-all-ones restart index logged and dropped. Census (2026-10-22): 130-170 byte-index draws/frame widened on the
|
||||
CPU, 0 device-local, 0 non-all-ones restarts. **OK**; `VK_KHR_index_type_uint8` (supported) would drop the CPU cost.
|
||||
10. **Feedback loops.** Census (2026-10-22): 425/frame (field) to 667 (fight), G-buffer material/decal shaders
|
||||
sampling the 1600x900 targets of the open pass. The terrain G-buffer pass reads its own albedo, normal and R32F
|
||||
targets at the fragment's pixel; on the RX 9070 the reads returned stale data as ~16x14 px dark rectangles on the
|
||||
ground, and the sage spirit next to Link was not drawn. `vk_exec_draw.c` (feedback loops) now detects such a draw:
|
||||
with `VK_EXT_attachment_feedback_loop_layout` and `dynamicRenderingLocalRead` the target moves to the feedback
|
||||
layout and a by-region barrier precedes the draw; without them the pass restarts behind a full barrier
|
||||
(`NX2NAVI_FEEDBACK`). **Fixed.**
|
||||
11. **Border colours.** `vk_texture.c` `make_sampler`: custom colour only with `customBorderColorWithoutFormat`, else transparent
|
||||
black; integer formats use the float path. Confirm the RX 9070 reports the feature (grass map needs 0.5).
|
||||
**OK if supported.**
|
||||
12. **Dual-source blend.** `vk_pipeline.c` `blend_factor` maps SRC1 factors; `Program.cs` `QueryGraphicsState` always passes
|
||||
`dualSourceBlendEnable = false`, so no `Index 1` output exists. Log pipelines with SRC1 factors. **Mismatch if used.**
|
||||
13. **FMZ / NaN.** Count `FMUL/FFMA/FMUL32I/FFMA32I` words with the `Fmz` field set (`Decoders/InstDecoders.cs:1901`
|
||||
bit positions) in `build/v121/shaders/`; FMin/FMax/FClamp NaN results are undefined (3.1). Census (2026-10-22):
|
||||
FMZ in 1 instruction of 12397 shaders, FTZ in 12241 shaders, FMNMX in 7457. The translator emits GLSL.std.450
|
||||
FMin/FMax (Ryujinx `GenerateMinimum`), not NMin/NMax. **FMZ: no fix needed. FMNMX: OK** (RX 9070 test: FMin/FMax(NaN, 1) = 1 both orders, FClamp(NaN, 0, 1) = 0).
|
||||
14. **Point size / line width.** Census: `SetLineWidth 4.5` ~1/frame (2 line draws into a 144x144 R8 target),
|
||||
`SetPointSize 5` ~14/frame; 4 point VS rely on it. Point size: translator patch 0008 reads it from the support
|
||||
buffer, **fixed**. Line width: 1 px, non-default values logged (`nvn_ext.c`), **mismatch, minor**.
|
||||
15. **Block height.** `nvn_tex.c` truncates like NVN (2.1). DK3D rounds up (`dk_image.cpp:446`); every other step matches. **OK.**
|
||||
16. **Depth mode and clamp.** `vk_pipeline.c` `create_graphics` (`negativeOneToOne` from `nvn_depth_minus_w`, `depthClampEnable`
|
||||
from NVN state). DK3D agrees on the z maths (1.5) and the clamp switch (1.6). **OK.**
|
||||
17. **Depth bias argument order.** `vk_exec_draw.c` `record_fixed` (units, clamp, factor); DK3D takes the same three and doubles units for the register (1.7). **OK.**
|
||||
18. **Sampler LOD.** `vk_texture.c` `make_sampler`. **OK.**
|
||||
19. **Viewport / scissor arrays.** Single viewport (`vk_exec_draw.c` `record_fixed`), extra scissors logged. Census: 0 differing. **OK.**
|
||||
|
||||
20. **Depth-compare sampling in compute** (VUID-vkCmdDispatch-None-06479): one compute shader samples a 4x4 R32F
|
||||
CPU-written placeholder through an R32_SFLOAT view with compare on, ~1/frame. Needs a D32 copy. **Minor.**
|
||||
21. **Report timestamps: unit and instant.** `write_counter` (`vk_exec_xfer.c`) writes host `ns * 12 / 625` (19.2 MHz
|
||||
ticks) at fence completion for every report; `nvnDeviceGetTimestampInNanoseconds` converts back with the same factor
|
||||
(`nvn_ext.c`). DK3D: GPU ticks convert at 625/384 ns (614.4 MHz), TIMESTAMP is written when the ROP has retired
|
||||
the earlier work, TIMESTAMP_TOP when the command processor reaches it (4.2). Two consequences for the engine's GPU time (dynamic resolution and speed follow it):
|
||||
(a) if the 1.2.1 consumer scales raw report ticks with its own constant rather than the NVN conversion call, our
|
||||
19.2 MHz unit is 32x off from the hardware's; find that consumer before fixing the unit; (b) the delta needs
|
||||
`vkCmdWriteTimestamp2` with `TOP_OF_PIPE` for TIMESTAMP_TOP and `BOTTOM_OF_PIPE` for TIMESTAMP, per submission
|
||||
segment, converted to whichever unit (a) shows. **Mismatch (instant); unit unverified.**
|
||||
|
||||
## 8. Sources not read yet
|
||||
|
||||
- Mesa NVK (`src/nouveau/vulkan`, MIT): how it maps Vulkan provoking vertex, depth clip, restart, feedback loops onto B197.
|
||||
- envytools rnndb `graph/gf100_3d.xml`, `tsc.xml`, `tic.xml` (MIT): field-level TIC/TSC meanings (LOD snap, trilinear opt).
|
||||
- Ryujinx `Ryujinx.Graphics.Gpu/Image/TextureCache.cs`, `Texture.cs` (MIT): alias/overlap rules, depth texture swizzle.
|
||||
- Ryujinx NX2navi/BotW PRs and issues (GitHub repo gone; mirrors blocked or 404 on 2026-10-22).
|
||||
- the hardware progress reports 2023-08 to 2024-02 and suyu/sudachi trackers (behaviour only).
|
||||
- switchbrew wiki NVN and GPU pages.
|
||||
107
docs/mods.md
Normal file
107
docs/mods.md
Normal file
@@ -0,0 +1,107 @@
|
||||
# Mods
|
||||
|
||||
NX2navi Recomp loads the mods players make for the Switch emulators (Ryujinx, the hardware and its forks): texture and model
|
||||
packs, gameplay and text changes from GameBanana and similar sites, a mod manager's merged output, code patches (`.ips`,
|
||||
`.pchtxt`) and Atmosphère cheat files. No mod or cheat file ships with the port.
|
||||
|
||||
## Install a mod
|
||||
|
||||
1. Open the settings menu (F1, Esc, or Minus and Plus together on a controller), go to the Mods tab and choose
|
||||
**Open mods folder**. That is the `mods` folder next to `nx2navi.exe`; the menu creates it when it is missing.
|
||||
2. Make one folder per mod in it and unpack the mod's archive into that folder. The mod's own folders (`romfs`,
|
||||
`exefs`, `cheats`) must sit directly inside the mod's folder:
|
||||
|
||||
```
|
||||
mods\
|
||||
Better Grass\
|
||||
romfs\
|
||||
...
|
||||
Fast Travel Anywhere\
|
||||
exefs\
|
||||
patch.pchtxt
|
||||
My cheats\
|
||||
cheats\
|
||||
277178B7DBA1B6D4.txt
|
||||
```
|
||||
|
||||
An archive laid out for a Switch SD card or an emulator works as it is. Inside the mod's folder the port also finds
|
||||
`atmosphere\contents\0100F2C0115B6000\romfs`, `contents\0100F2C0115B6000\romfs` and `0100F2C0115B6000\romfs`, the
|
||||
same places for `exefs` and `cheats`, and `atmosphere\exefs_patches\<name>\` for code patches.
|
||||
3. Start the game. A mod that patches the game's code makes the game rebuild itself first (see Code patches below).
|
||||
4. Open the Mods tab to check the mod. Each row says how many game files the mod replaces and adds, how many cheats it
|
||||
brings, and why a part of it is not used.
|
||||
|
||||
## Order and switches
|
||||
|
||||
The Mods tab lists every folder in `mods`. Select a mod and press A or Enter to switch it on or off. The top face button
|
||||
or Ctrl+Up moves it up, the left face button or Ctrl+Down moves it down. When two mods change the same file, the
|
||||
higher one wins; the lower mod's row says how many of its files the higher one covers. Changes apply after a restart.
|
||||
|
||||
The menu saves the order and the switches in `nx2navi.ini`, under `[mods]`, by folder name: `mods_order` and
|
||||
`mods_disabled`. A folder the list does not name yet, such as a new or renamed one, goes below the named ones.
|
||||
|
||||
### Mods that change the same files
|
||||
|
||||
Two mods that edit the same game file do not combine: the higher one wins the whole file. Merge such mods with
|
||||
a mod manager and set its export location to an empty folder inside
|
||||
`mods`, for example `mods\merged`. The exported folder then counts as one mod here. Mods packaged for a mod manager only
|
||||
(`.tkcl` files) go the same way.
|
||||
|
||||
The port merges the resource size table on its own: at every start with mods on, it combines the tables the mods ship
|
||||
and raises the entries for files a mod made larger or added. A separate RESTBL mod is not needed.
|
||||
|
||||
## Code patches (ExeFS)
|
||||
|
||||
A mod with `.ips` or `.pchtxt` files patches the game's code. A patch is for one game update: an `.ips` file is named
|
||||
after the build id of the update's code, a `.pchtxt` file names it on its `@nsobid` line. The game's code is compiled
|
||||
ahead of time, so these patches are built into `nx2navi.exe`:
|
||||
|
||||
1. The first start after such a mod is added closes the game and opens the setup's rebuild window. The rebuild takes
|
||||
a few minutes (the shaders stay prepared), then the game starts again. Saves and settings are kept.
|
||||
2. Until the rebuild succeeds, the whole mod stays off, its other files included.
|
||||
3. After the rebuild the mod switches on and off from the Mods tab like any other, without another rebuild.
|
||||
|
||||
The rebuild needs the setup folder the game was installed from. When that folder was moved or deleted, run
|
||||
`NX2navi-Setup.exe` again on the same install folder: it does the same job. `mods_rebuild = off` under `[mods]` in
|
||||
`nx2navi.ini` turns the automatic rebuild off.
|
||||
|
||||
A code patch over one of the port's own patches (the 60 FPS patch, the resolution patch and the other entries under
|
||||
**Built into the port**) turns that built-in patch off while the mod is on. The mod's row names it.
|
||||
|
||||
## Cheats
|
||||
|
||||
A cheat file is part of a mod: `mods\<mod>\cheats\<build id>.txt`, switched cheat by cheat on the Cheats tab. Cheats
|
||||
that change the game's code need the same rebuild as code patches. [cheats.md](cheats.md) explains the file names for
|
||||
each game update, the rebuild and which cheats cannot run.
|
||||
|
||||
## Renaming a mod folder
|
||||
|
||||
The port keys a mod by its folder name. A renamed folder counts as a new mod:
|
||||
|
||||
- it moves below the mods the list names, and comes back switched on;
|
||||
- its cheats come back switched off;
|
||||
- its code patches and code cheats are built again, so the next start runs the rebuild.
|
||||
|
||||
Rename mod folders before you set them up in the menu, not after.
|
||||
|
||||
## What does not work
|
||||
|
||||
| On the Mods tab | Why | What to do |
|
||||
|---|---|---|
|
||||
| This folder has no romfs, exefs or cheats files. | The archive was unpacked one level too deep (`mods\Mod\Mod\romfs`), or it holds no game files. | Move the folders up so `romfs` sits directly in the mod's folder. |
|
||||
| ... files are made for another game version and are left out. | The mod's files carry another update's version in their names (`.Product.<number>.`). | Download the mod's release for your game update. |
|
||||
| Its code patches (exefs) are made for another game version. | The patch files name another update's build id. | Same: get the release for your update. |
|
||||
| This mod brings its own game code (a subsdk plugin or a replaced main). | Mods such as UltraCam ship compiled Switch code. The port cannot load it. | No workaround; the whole mod stays off. |
|
||||
| This mod patches the game code (exefs) in a way the port cannot apply. | The patch writes the game's data, not its code, or changes code in a form the port cannot build. | No workaround; the reason follows on the row. |
|
||||
| The folder name contains \|. | The mod list uses `\|` between names. | Rename the folder. |
|
||||
|
||||
A mod never stops the game from starting. Whatever the port cannot use is left out and shown on the Mods tab.
|
||||
|
||||
## When something looks wrong
|
||||
|
||||
1. Switch the newest mod off on the Mods tab and restart. If the problem goes away, check that the mod is made for your
|
||||
game update and does not fight another mod over the same files.
|
||||
2. `logs\nx2navi.log` next to `nx2navi.exe` names every mod at the start (lines starting with `mods:`), the cheats
|
||||
(`cheats:`) and the resource size table (`RomFS: resource size table:`). Open log folder on the Advanced tab of
|
||||
the menu shows the logs.
|
||||
3. When reporting a problem, attach `nx2navi.log` and list your mods with the links you downloaded them from.
|
||||
109
docs/tester-guide.md
Normal file
109
docs/tester-guide.md
Normal file
@@ -0,0 +1,109 @@
|
||||
# Tester guide: running NX2navi Recomp on your GPU
|
||||
|
||||
This guide is for testing the build on a GPU other than the developer's (AMD RX 9070). You build it from source with
|
||||
your own copy of the game and your own console keys. Nothing from the game is in this repository, and nothing you build
|
||||
from it may be shared.
|
||||
|
||||
## What you need
|
||||
|
||||
- Windows 10 or later, an x86-64 CPU with SSE4.2 (AVX2/FMA recommended: they build the default, faster version), about
|
||||
60 GB free.
|
||||
- A GPU with Vulkan 1.3 and `VK_EXT_external_memory_host`, with a current driver.
|
||||
- *Ocarina of Time* base game (1.0.0) and the 1.1.0, 1.2.1 or 1.4.3 update, dumped from your own Switch as NSP or XCI.
|
||||
- `prod.keys` dumped from your own Switch. Never send it to anyone, and never paste it into a log or an issue.
|
||||
|
||||
## Build
|
||||
|
||||
1. Follow [building.md](building.md): the toolchain, then build steps 1-6 (Python environments, `config.local.toml`
|
||||
with your own paths, extraction, the shader translator, lifting and linking, shader preparation). The result is
|
||||
`build\<ver>\nx2navi.exe`, with `<ver>` the game version you built (`v143` or `v121`).
|
||||
2. Check the build without the game: `build\<ver>\nx2navi.exe --selftest` must print a pass.
|
||||
3. To run the build on another of your machines, package a portable folder ([building.md](building.md), "Portable
|
||||
folder"). The folder holds game code: for your own machines only.
|
||||
4. The installer ([installer.md](installer.md)) is the other route to a machine without the toolchain: it builds the
|
||||
game there from your own files. No installer release is published yet.
|
||||
|
||||
## Run and collect the log
|
||||
|
||||
Every run writes its log to `logs\nx2navi.log` next to `nx2navi.exe`; the four runs before it are kept as `nx2navi.1.log` to
|
||||
`nx2navi.4.log`, and Open log folder in the settings menu's Advanced tab (F1) shows them. Copy the log before you start the
|
||||
game again. A file stops at 64 MiB. Started with `2> file`, the game writes its log to that file instead.
|
||||
|
||||
From `cmd.exe` in the repository root, with the validation layer on:
|
||||
|
||||
```bat
|
||||
set NX2NAVI_VK_VALIDATION=1
|
||||
build\<ver>\nx2navi.exe
|
||||
```
|
||||
|
||||
`NX2NAVI_VK_VALIDATION=1` turns on the Vulkan validation layer (it comes with the Vulkan SDK). It makes the game slower;
|
||||
do one run with it and one without (`set NX2NAVI_VK_VALIDATION=`).
|
||||
|
||||
If the game stops with a lost device (the log says `device lost` or `VK_ERROR_DEVICE_LOST`), the lines starting with
|
||||
`vk: pass marks` name the render pass or dispatch the GPU was running. AMD GPUs write them by default. On an NVIDIA GPU,
|
||||
`set NX2NAVI_PASS_CRUMBS=1` (or GPU pass marks: On in the Advanced tab) before the run that reproduces the crash.
|
||||
|
||||
The game keeps running behind other windows (`assume_focus = 0` in `nx2navi.ini` pauses it instead). Shaders the preparation did not cover compile on first use, so
|
||||
the first minutes of play can stutter.
|
||||
Play into the open field (load a save, walk around for a minute, fight something if you can), then close the window.
|
||||
|
||||
## Choosing the GPU
|
||||
|
||||
A laptop with an iGPU and a discrete GPU, or two cards, lists every GPU at the top of the log (`vk: gpu 0: ...`).
|
||||
|
||||
- `set NX2NAVI_GPU_NAME=Radeon` picks the first usable GPU whose name contains that text (case does not matter).
|
||||
- `set NX2NAVI_GPU_INDEX=1` picks GPU 1 from that list.
|
||||
|
||||
The line `vk: using gpu N: <name>` says which one ran.
|
||||
|
||||
## Mods
|
||||
|
||||
Mods and cheats made for the emulators go into `mods/` next to `nx2navi.exe`, one folder per mod. [mods.md](mods.md) covers
|
||||
installing them, their order, code patches, what does not work and what to check when something looks wrong;
|
||||
[cheats.md](cheats.md) covers cheat files. The log names every mod at the start (`mods: <name>: ...`); when you
|
||||
report a problem with mods, list them with their download links.
|
||||
|
||||
## What to send back
|
||||
|
||||
1. `logs\nx2navi.log` from each run. It contains no keys and no game files. The lines that matter most start with
|
||||
`vk caps:` (GPU, driver, features and the fallbacks taken), `vk: gpu`, `validation` and `gfx_shader:`.
|
||||
2. GPU model, driver version, CPU, RAM, and whether it is a laptop. Copy system info in the settings menu's Advanced
|
||||
tab (F1) puts these and your changed settings on the clipboard. Say which game update you ran.
|
||||
3. What you saw: how far it got (menu, loading, field), roughly how smooth it ran, and screenshots of anything wrong
|
||||
(black or missing areas, flickering, wrong colours, missing grass). Win+Shift+S is fine.
|
||||
4. If it crashed or hung: what you were doing. The log of that run ends with the crash report, so send it whole.
|
||||
5. For a bug you can trigger again: a recording of your input (`set NX2NAVI_INPUT_RECORD=input.txt`, see
|
||||
[controls.md](controls.md), "Scripted input") and the save you started from.
|
||||
|
||||
## Steam Deck
|
||||
|
||||
Build or install on Linux ([installer.md](installer.md), "Linux release"), in Desktop Mode. The game then runs in either
|
||||
mode; for Game Mode, add `nx2navi` from the install folder to Steam as a non-Steam game.
|
||||
|
||||
The game recognises the Deck (LCD and OLED) and uses its device profile for every setting `nx2navi.ini` does not set:
|
||||
fullscreen, the screen's own 1280x800 (16:10: the camera and interface take that shape), the game's own 3D scale,
|
||||
vsync and a 40 fps limit. These are starting points, not measurements. What helps most:
|
||||
|
||||
1. The performance overlay (F10, or both stick clicks L3 + R3) in full mode: the frame rate, mean and longest frame
|
||||
time over a minute in Lookout Landing, in the Depths and in the Sky, with the settings named.
|
||||
2. The same with `fps_limit` 30, 45 and 60 and `render_scale` 50, 67 and game, so the defaults can be picked from
|
||||
measurements. Steam's own performance overlay (the ... button) adds power draw and temperatures.
|
||||
3. LCD or OLED, SteamOS version (`uname -r` in Desktop Mode's terminal), and the game update.
|
||||
4. Anything that looks stretched, squeezed or cut off, with a screenshot. At 16:10 the menus keep their 16:9
|
||||
proportions in the middle of the taller screen, the HUD sits at the top and bottom edges, and full-screen
|
||||
backgrounds reach them; pre-rendered cutscenes play stretched.
|
||||
|
||||
`device_profile = desktop` in `nx2navi.ini` turns the profile off; `deck` turns it on elsewhere, to try the Deck's
|
||||
settings on a PC.
|
||||
|
||||
## Switches for the developer's questions
|
||||
|
||||
Only set these when asked; each forces a fallback the build picks by itself on GPUs that need it.
|
||||
|
||||
| Variable | Effect |
|
||||
|---|---|
|
||||
| `NX2NAVI_VK_NO_FLOAT64=1` | Translate shaders without 64-bit floats |
|
||||
| `NX2NAVI_VK_SSBO_ALIGN=64` | Treat storage buffers as needing 64-byte alignment |
|
||||
| `NX2NAVI_VK_SUBGROUP_PIN=0` | Do not pin shaders to 32 lanes; tell the translator the GPU's own subgroup size |
|
||||
|
||||
Each switch makes the first launch translate every shader again.
|
||||
Reference in New Issue
Block a user