diff --git a/AGENTS.md b/AGENTS.md index 7124b47d..99981cc9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,6 @@ # NX2navi -Static recompilation of *Ocarina of Time* (Switch, ARM64) into a native Windows x86-64 exe with a Vulkan renderer. Design: [docs/architecture.md](docs/architecture.md). Toolchain, build commands and tests: [docs/building.md](docs/building.md). +Static recompilation of *Ocarina of Time* (Switch 2, ARM64, game version 1.1.1) into a native Windows x86-64 exe with a Vulkan renderer. Design: [docs/architecture.md](docs/architecture.md). Toolchain, build commands and tests: [docs/building.md](docs/building.md). ## Game content boundary diff --git a/README.md b/README.md index e72c2725..911df4df 100644 --- a/README.md +++ b/README.md @@ -1,92 +1,56 @@ -# NX2navi +# Ocarina Recomp -**NX2navi** is a static recompilation of *The Legend of Zelda: Ocarina of Time* (Nintendo Switch, 2023) into a native x86-64 application for Windows and Linux ([docs/architecture.md](docs/architecture.md)). +**Ocarina Recomp** is a static recompilation of *The Legend of Zelda: Ocarina of Time* for Nintendo Switch 2 (game +version 1.1.1) into a native x86-64 application for Windows, with a Vulkan renderer +([docs/architecture.md](docs/architecture.md)). > [!IMPORTANT] -> **No game files are included.** You need your own legally obtained copy of *Ocarina of Time*, dumped from your own console: the base game and the 1.1.0, 1.2.1 or 1.4.3 update. Instructions for obtaining those files will not be provided. +> **No game files are included.** You need your own legally obtained copy of the game, dumped from your own console. +> Instructions for obtaining those files will not be provided. > > The game is built from your files on your machine, so the result contains the game. It is for your own use. Do not share it. ## Status -Early development. The game is playable on Windows and Linux; visual bugs and crashes are still expected. +Early development. Kokiri Forest and the areas around it play at 60 fps; visual bugs and crashes are still expected. Report problems as issues, following [CONTRIBUTING.md](CONTRIBUTING.md). ## Features - Native code, not emulation -- Game updates 1.1.0, 1.2.1 and 1.4.3; 1.2.1 and 1.4.3 saves carry over between them -- Takes NSP and XCI game files, and compressed NSZ and XCZ as they are: the setup unpacks them itself -- 60 fps by default; up to 144 fps or the monitor's refresh rate (experimental) -- Custom resolutions (1080p, 2k...), supersampling and AMD FSR 1 upscaling -- NVIDIA DLSS / DLAA on RTX GPUs: the setup downloads `nvngx_dlss.dll` from NVIDIA's DLSS SDK repository - (github.com/NVIDIA/DLSS, NVIDIA RTX SDK license) when it finds an RTX GPU; untick it in the setup to skip it -- HDR10 output with peak, game and HUD brightness settings -- 16:10 (Steam Deck, handhelds), 3:2 (Surface, Framework), 21:9 and 32:9 (ultrawide) game resolutions, with the camera and the interface in that shape -- Anisotropic texture filtering up to 16x, for sharp ground into the distance -- In-game settings menu (F1) and performance overlay (F10) -- Discord Rich Presence ("Playing NX2navi" with the play time), off by default, own application id supported -- Setup and in-game menu in English, Russian, Brazilian Portuguese and Simplified Chinese -- Keyboard, mouse and gamepad with remapping, rumble and gyro +- 60 fps, at the game's own resolution +- AMD FSR 1 or bilinear scaling to the window, with a sharpness setting; black bars or stretch +- Colour profiles and brightness, contrast, saturation and gamma settings +- In-game settings menu (F1, or Minus and Plus together on a gamepad) and performance overlay (F10) +- Keyboard, mouse and gamepads with remapping, Nintendo or Xbox button layout, rumble and gyro; the game follows + whichever connected gamepad you press - Stereo or 5.1 surround sound output -- Xbox button prompts: LB, RB, LT, RT, Menu and View, and A, B, X and Y as labelled on the button to press, with either button layout -- Shaders prepared at install instead of during play -- Mods made for emulators: unpack a mod into the `mods` folder next to `nx2navi.exe`; switch and order them in the menu. RomFS - mods, a mod manager's merged output and ExeFS code patches (`.ips`, `.pchtxt`) work; the game rebuilds itself for code patches - ([docs/mods.md](docs/mods.md)) -- Atmosphère cheats: put a cheat file into a mod and switch its cheats in the menu ([docs/cheats.md](docs/cheats.md)) - -## Planned - -- VRR +- Docked or handheld console mode +- Graphics cards with less than 20 GiB of video memory keep the game's textures compressed (BC3), about half the + video memory at the same speed ## Installing -### Windows +The setup program for this game is not ready yet. [docs/building.md](docs/building.md) describes building from +source. Settings are in the in-game menu and in `nx2navi.ini` ([nx2navi.ini.example](nx2navi.ini.example) lists every +key). Controls: [docs/controls.md](docs/controls.md). -Requirements: Windows 10 or later, an x86-64 CPU with SSE4.2 (AVX2/FMA machines get a faster build), a GPU with Vulkan 1.3 and `VK_EXT_external_memory_host`, about 36 GiB free during setup (20 GiB after). - -1. Download `NX2navi-Setup--windows.zip` from [Releases](../../releases/latest) and unpack it. -2. Start `NX2navi-Setup.exe`. If SmartScreen warns about an unsigned app, choose *More info*, then *Run anyway*. -3. Pick your game files, the language and an install folder, and press *Start*. The setup builds the game once. With - an NVIDIA RTX GPU it also downloads NVIDIA DLSS, unless you untick *Download NVIDIA DLSS*; turn DLSS on in the - menu (F1). -4. Play `nx2navi.exe` from the install folder. - -### Linux - -Requirements: as on Windows, plus Linux 6.7 or later and glibc 2.34 or later (Ubuntu 22.04, Debian 12, Fedora 35, SteamOS 3). - -1. Download `NX2navi-Setup-x86_64.AppImage` from [Releases](../../releases/latest), or build it ([docs/installer.md](docs/installer.md), "Linux release"). -2. Make it executable (`chmod +x NX2navi-Setup-x86_64.AppImage`) and start it. On a Steam Deck, do this in Desktop Mode. -3. Pick your game files, the language and an install folder, and press *Start*. -4. Play `nx2navi` from the install folder. - -To update, run a newer setup on the same folder; saves and settings are kept. Settings are in the in-game menu and in `nx2navi.ini` ([nx2navi.ini.example](nx2navi.ini.example) lists every key). Controls: [docs/controls.md](docs/controls.md). - -## Credits - -NX2navi builds on the projects below. What was taken from each, with its license, is listed in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). - -- [LAS Recomp](https://github.com/peiikai/LASRecomp) -- [MacNativeGames](https://github.com/ly05010419/MacNativeGames) -- [d2r-mobile](https://github.com/jhaizhou-ops/d2r-mobile) and [nxnative](https://github.com/seedds/nxnative) -- [SavageWindRecomp](https://github.com/Revivedx/SavageWindRecomp) -- [Ryujinx](https://github.com/Ryujinx/Ryujinx) -- [deko3d](https://github.com/devkitPro/deko3d) -- [hactool](https://github.com/SciresM/hactool) -- [SDL 3](https://libsdl.org), [Dear ImGui](https://github.com/ocornut/imgui), [AMD FSR 1](https://github.com/GPUOpen-Effects/FidelityFX-FSR), [Vulkan Memory Allocator](https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator), [libopus](https://opus-codec.org), [libvpx](https://chromium.googlesource.com/webm/libvpx), [Zstandard](https://github.com/facebook/zstd) -- [Switchbrew](https://switchbrew.org) and the Switch homebrew community +Requirements: Windows 10 or later, an x86-64 CPU with SSE4.2 (AVX2/FMA machines get a faster build), a GPU with Vulkan +1.3, `VK_EXT_external_memory_host` and `VK_EXT_mesh_shader`. ## License -The project's own code is under the [MIT License](LICENSE); third-party code keeps its own license ([THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)). The license grants nothing for Nintendo's game. +The project's own code is under the [MIT License](LICENSE); third-party code keeps its own license +([THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)). The license grants nothing for Nintendo's game. -NX2navi is an independent, non-commercial fan project, not affiliated with or endorsed by Nintendo. *The Legend of Zelda*, *Ocarina of Time* and *Nintendo Switch* are trademarks of Nintendo. +Ocarina Recomp is an independent, non-commercial fan project, not affiliated with or endorsed by Nintendo. *The Legend +of Zelda*, *Ocarina of Time* and *Nintendo Switch* are trademarks of Nintendo. ## Building from source -[docs/building.md](docs/building.md) lists the toolchain and the steps. Everything the build writes stays under `build\` and `generated\`, which git ignores; run `tools/check_content.py` before a commit. Testers: [docs/tester-guide.md](docs/tester-guide.md). +[docs/building.md](docs/building.md) lists the toolchain and the steps. Everything the build writes stays under +`build\` and `generated\`, which git ignores; run `tools/check_content.py` before a commit. -Most of the code and documentation is written by AI coding agents (Claude Code) under the maintainer's direction; the maintainer plays every build and decides what is merged. [AGENTS.md](AGENTS.md) holds the rules they work under. +Most of the code and documentation is written by AI coding agents (Claude Code) under the maintainer's direction; the +maintainer plays every build and decides what is merged. [AGENTS.md](AGENTS.md) holds the rules they work under. diff --git a/docs/architecture.md b/docs/architecture.md index a64bfb68..d2b952fd 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,148 +1,113 @@ # Architecture -How the Switch build of *Ocarina of Time* becomes a native Windows x86-64 program. The upstream projects it builds on are listed under Credits in the [README](../README.md). - -## Starting points - -Two MIT projects cover different halves of the problem: - -| | LAS Recomp | MacNativeGames | -|---|---|---| -| Game | *Link's Awakening* | *Ocarina of Time* 1.0.0–1.4.3 | -| CPU | ARM64 lifted to C, runs on x86-64 | Original ARM64 code runs directly on Apple silicon | -| OS boundary | IPC services and SVCs | ~163 `nn::*` SDK functions replaced by symbol, 38 SVCs | -| Graphics | Own NVN layer on Vulkan, own Maxwell translator | NVN front end + Metal 4 back end, Ryujinx translator | -| Useful to us | Lifter, NEON-to-C lowering, Windows memory layer, tests | Everything NX2navi-specific: host behaviour, NVN front end, version tables, patches, shader list | - -MacNativeGames cannot run on x86-64 as is: its CPU path executes ARM64 code natively. LAS Recomp's host only covers what *Link's Awakening* calls. The port takes the CPU half from LAS Recomp and the game half from MacNativeGames, and writes the parts neither has: a lifter that scales to NX2navi, and a Vulkan back end for the NVN front end. +How the Switch 2 build of *Ocarina of Time* (game version 1.1.1) becomes a native Windows x86-64 program. Code taken +from other projects, with its license, is listed in [THIRD_PARTY_NOTICES.md](../THIRD_PARTY_NOTICES.md). ## Pipeline ### 1. Input -- Base 1.0.0 (`0100F2C0115B6000`) and one cumulative update (`0100F2C0115B6800`), NSP or XCI, plus the keys of the user's console. -- ExeFS and RomFS extraction: MacNativeGames' Python extractor (Python 3.14 for `compression.zstd`, `.venv-py314`). -- Update merging: hactool at `1d64a83` with MacNativeGames' BKTR patches, built by `tools/build_hactool.py`. -- Verification: MacNativeGames' `expected-v*.json` hashes of ExeFS, RomFS and fonts. -- Primary version: 1.4.3, the last update; performance work targets it. It replaced all three modules of the earlier updates (new SDK 20.5.22, RELR relocations, audio renderer REV15) and builds with `--version v143`, `recomp/patches-v143.txt`. 1.2.1 is a second target on the same build path; 1.0.0, 1.1.2 and 1.2.1 share `sdk`, `subsdk0` and `rtld` byte for byte, so only `main` differs among them. Saves load on either. -- Windows driver: `tools/extract.py` runs the vendored extractors (`tools/vendor/macnativegames/`) and identifies the update by the SHA-256 of its `main`. hactool has no Darwin `funopen` here, so the base Program NCA is copied out of the NSP before the BKTR merge (about 17 GB of temporary space). +- The ExeFS and RomFS of the user's own dump. Everything extracted or generated stays under `build/`, `generated/` or + the user's own folders, never in the repository (`tools/check_content.py`). ### 2. CPU: lift ARM64 to C -- Modules: `main`, `sdk`, `subsdk0`. `rtld` is dropped and linking is done by the host, as MacNativeGames does. -- Base: LAS Recomp's hand-written decoder (`lifter.py`) and NEON/FP lowering (`simd_c.py`, `simd.h`), MIT. Output: one C function per guest function, direct calls for `BL`, an address-keyed table for indirect branches. -- Rewritten for NX2navi scale: - - generator runs in parallel processes, with bounded memory per worker; - - jump tables resolved by our own symbolic pass instead of Unicorn, which keeps GPL code out of the shipped generator; - - `LDXP`/`STXP` (128-bit exclusive pairs) supported; - - no 64 MB limit on `main`; `subsdk*` loaded; - - NZCV and the six most used general registers of each function kept in C locals instead of the `Cpu` +- Modules: `main`, `sdk` and the `subsdk*` modules. `rtld` is dropped; the host links the modules. +- A hand-written decoder (`recomp/lifter.py`) and NEON/FP lowering (`recomp/simd_c.py`, `runtime/simd.h`) emit one C + function per guest function, direct calls for `BL` and an address-keyed table for indirect branches. + - The generator runs in parallel processes, with bounded memory per worker. + - Jump tables are resolved by a symbolic pass of our own (`recomp/jumptables.py`). + - `LDXP`/`STXP` (128-bit exclusive pairs) are supported. + - NZCV and the six most used general registers of each function live in C locals instead of the `Cpu` (`recomp/flags.py`, `recomp/regs.py`): registers are stored back where they changed, before calls, SVCs and - exits, and reloaded after them. A call reloads only X0-X18 and X30, as AAPCS64 has the callee preserve X19-X29 and - SP; `gen.py --no-callee-saved-trust` reloads those too. -- Coverage oracle: MacNativeGames' NSO rewriter lists every site where game code touches the OS (SVCs, TLS reads, counter reads, `x18`, exclusive windows, jump tables): 68,968 sites in 1.4.3 `main`. The lifter must handle every one. -- Memory: guest address equals host address, all below 2^39 (IPC encodes 39-bit addresses). The heap, stacks and the alias region are reserved with `MEM_WRITE_WATCH`, so the graphics back end can ask which pages the CPU wrote since the last upload (`runtime/memory.c`). -- Indirect branches go through `rt_dispatch` (`runtime/dispatch.c`), which looks the target up in the address table and tail-calls it. -- Testing: LAS Recomp's differential harness, every lifted instruction class checked against Unicorn (test-only, never linked into the game). + exits, and reloaded after them. A call reloads only X0-X18 and X30, as AAPCS64 has the callee preserve X19-X29 + and SP; `gen.py --no-callee-saved-trust` reloads those too. +- Memory: guest address equals host address, all below 2^39 (IPC encodes 39-bit addresses). The heap, stacks and the + alias region are reserved with `MEM_WRITE_WATCH`, so the graphics back end can ask which pages the CPU wrote since + the last upload (`runtime/memory.c`). +- Indirect branches go through `rt_dispatch` (`runtime/dispatch.c`), which looks the target up in the address table + and tail-calls it. +- Testing: a differential harness checks every lifted instruction class against Unicorn (test-only, never linked into + the game). ### 3. Host -- Portable C from MacNativeGames: threads and synchronisation, IPC, memory services, fs/RomFS, time, account, applet, audio renderer and effects. The Apple pieces (mach VM, CommonCrypto, AudioToolbox, GameController) are replaced with Win32 and SDL3. -- SDK boundary at the `nn::*` function level, matching MacNativeGames. Only `audren:u` and `sm:` are real IPC services. -- Window, keyboard and mouse: Win32 (`gfx_window.c`, `input_win.c`). Gamepad and audio output: SDL3. Voice: libopus (BSD). Movies: the SDK's `movie::Decoder` is replaced by a host VP9 decoder over libvpx (BSD, static; `runtime/ov_movie.c`), because FFmpeg from MSYS2 is a GPL build. -- zstd: most RomFS files of 1.4.3 are zstd frames. The patch set `zstd` sends the game's two decoder entry points to - libzstd on the host (`ov_zstd.c`, `NX2NAVI_ZSTD_NATIVE`); the lifted decoder took almost three times as long. +- The SDK boundary is at the `nn::*` function level: threads and synchronisation, IPC, memory, fs/RomFS, time, + account, applet, hid, audio renderer and effects run as host code. Only `audren:u` and `sm:` are real IPC services. +- Window, keyboard and mouse: Win32 (`gfx_window_win.c`, `input_win.c`). Gamepads and audio output: SDL3 (every + connected gamepad is open; the game reads the one pressed last, `input.c`). Voice: libopus. Movies: a host VP9 + decoder over libvpx replaces the SDK's `movie::Decoder` (`runtime/ov_movie.c`). - Settings and menu: every key with its default is in one table (`cfg.c`), read from `nx2navi.ini` and `NX2NAVI_*` variables and written back when the in-game menu changes it. The menu and the performance overlay are Dear ImGui, - drawn into the swapchain image (`runtime/ui/`); their text is in `ui_text.c` (English, Russian, Brazilian Portuguese, - Simplified Chinese). The Chinese glyphs come from a system font (Microsoft YaHei, `plat_font_path`) merged into the - font atlas. -- OS calls the host makes go through `runtime/platform/` (`platform.h`, `win32.c`), so a Linux or macOS host adds one file instead of touching the rest. The split is not finished: `tools/check_platform.py` lists the Win32 uses left in the other files. + drawn into the swapchain image (`runtime/ui/`); their text is in `ui_text.c`. +- OS calls the host makes go through `runtime/platform/` (`platform.h`, `win32.c`). #### Threads and cores -Every guest thread is a host thread. NX2navi assumes Horizon's scheduler: three application cores, a thread runs only on a core in its affinity mask, and a higher-priority thread takes a core from a lower-priority one. The game keeps per-core data without locks and relies on that, so the host models it (`runtime/kernel.c`): +Every guest thread is a host thread. The game assumes Horizon's scheduler: three application cores, a thread runs +only on a core in its affinity mask, and a higher-priority thread takes a core from a lower-priority one. The game +keeps per-core data without locks and relies on that, so the host models it (`runtime/kernel.c`): -- Three virtual cores. A guest thread runs guest code only while it holds a core; the others wait in a priority-ordered list. -- Preemption. When a waiting thread has a better priority than a holder on a core it may use, exactly one holder is flagged. The holder checks the flag at its next SVC and at every indirect call (`rt_dispatch`: two loads when nobody waits), checks again under the kernel lock that a better thread still waits, and only then hands its core over. The indirect-call check comes from SavageWindRecomp's BotW port; checking only at SVCs left a better thread waiting up to 26 ms. -- Asynchronous preemption (from MacNativeGames 8bf0893, on unless `NX2NAVI_ASYNC_PREEMPT=0`). Game code can run for - milliseconds with neither (`NX2NAVI_CORE_DIAG=1` measures it): a waiter that has spun 250 us suspends the holder and, - if its host rip is in lifted code at host depth 0 (not inside a host call or SVC), moves it to `kern_ap_stub`, which - saves the volatile registers and gives the core up there. A holder of a guest mutex counts at the priority of its - best waiter, and one inside an SDK spin lock (TLR+0x100 `disable_count`) is not preempted: its interrupt flag is set - and the switch happens at SVC 0x36. `NX2NAVI_ASYNC_PREEMPT=2` keeps those two without interrupting. The interruption is - Win32 (SuspendThread, SetThreadContext); another host needs a signal handler there, as MacNativeGames has. -- Host work gives the core back. Queue submits, presents, GPU waits, shader translation, `nn::fs` calls and NVN memory pool setup run without a core (`NVN_BLOCKING`, `kern_core_away`), as the Switch blocks those threads in IPC, so a thread blocked in the driver or on a file does not hold up guest threads. -- Spin waits. Loops that only another thread can end (857 in 1.4.3, 864 in 1.2.1, found by `tools/spin_waits.py`, the - nxnative search) send their back-edges through `rt_spin_gate`, which pauses the core once per pass and hands it to a - waiter after 50 us of polling (`NX2NAVI_SPIN`, `NX2NAVI_SPIN_HANDOFF_US`). -- `free_cores = 1` (`NX2NAVI_FREE_CORES=1`) turns the gate off, for experiments only: the game crashes or hangs, usually - at start, because its job workers all report core 0 and share per-core memory (#212). +- Three virtual cores. A guest thread runs guest code only while it holds a core; the others wait in a + priority-ordered list. +- Preemption. When a waiting thread has a better priority than a holder on a core it may use, exactly one holder is + flagged. The holder checks the flag at its next SVC and at every indirect call (`rt_dispatch`), checks again under + the kernel lock that a better thread still waits, and only then hands its core over. +- Asynchronous preemption (on unless `NX2NAVI_ASYNC_PREEMPT=0`): a waiter that has spun 250 us suspends the holder + and, if its host rip is in lifted code at host depth 0, moves it to `kern_ap_stub`, which saves the volatile + registers and gives the core up there. A holder of a guest mutex counts at the priority of its best waiter, and one + inside an SDK spin lock is not preempted. +- Host work gives the core back. Queue submits, presents, GPU waits, shader translation, `nn::fs` calls and NVN memory + pool setup run without a core (`NVN_BLOCKING`, `kern_core_away`). +- Spin waits. Loops that only another thread can end send their back-edges through `rt_spin_gate`, which pauses the + core once per pass and hands it to a waiter after 50 us of polling (`NX2NAVI_SPIN`, `NX2NAVI_SPIN_HANDOFF_US`). ### 4. Graphics -- NVN front end from MacNativeGames (`nvn_api_*.c`, `nvn_obj.h`, texture layout and deswizzle). The game records command buffers through it; nothing reaches the GPU until the game submits. -- Vulkan 1.3 back end in place of MacNativeGames' Metal one. At `QueueSubmitCommands` the recorded NVN commands are replayed into Vulkan command buffers (`vk_exec*.c`). -- Memory pools. Apple's unified memory let MacNativeGames use the game's pools zero-copy. The back end imports the pools as host memory (`VK_EXT_external_memory_host`). On a discrete GPU, pools bound as buffers are mirrored in device-local memory and the pages the CPU wrote (write watch, above) are uploaded before a replay; idle pool regions are made read-only so the write-watch queries skip them (`vk_memory.c`). GPU-rendered textures are written back when the CPU reads them: on a pool invalidate, on a CPU copy, and for small targets once the CPU waits for a fence (`vk_texture.c`). -- Replay thread. `NX2NAVI_REPLAY_ASYNC=1` moves the replay off the submitting thread onto a queue at most one frame behind the game; reads of GPU results wait for the newest writer instead of draining the queue (`vk_backend.c`). It is off by default. Where the game waits for the GPU it overlaps the replay with the game threads (a GTX 1650 in the Depths, issue #88), a gain measured only there; while it was the default, a GTX 1660 Ti at the 60 fps cap with the GPU at 26 % reported uneven motion (#659). It adds up to a frame of latency; `replay_async = 1` turns it on. -- Recorder thread. The Vulkan calls the replay makes into its command buffer are made by a thread of their own (`vk_rec.c`, after Savage Wind Recomp and LAS Recomp): each `vkCmd*` copies its arguments into a packet of a ring and the recorder makes the calls in order; calls into other command buffers are made at once; the recorder also ends the command buffer, and the submit waits until it has. In the town on an RX 9070 the replay takes 17% less time on the submitting thread. It is off by default on AMD GPUs, where the driver loses the device or hangs the GPU at startup with it (`NX2NAVI_VK_RECORDER=1` forces it on, `=0` makes every call on the submitting thread). -- Uniform snapshot. The game rewrites per-object uniform blocks on other threads soon after it submits the frame - that reads them, while the replay still records it. At the start of a submission's replay every uniform range it - binds from CPU-written pools is copied, and the draws bind the copy (`vk_usnap.c`, after MacNativeGames' - `NX2NAVI_UBO_SNAPSHOT`); without it, late draws read another object's matrix. -- Driver data. The NVN driver keeps a per-stage constant bank (c0) that shaders read for things like point size, dispatch size and base instance. The back end fills it the way the driver does. -- Shaders: the Ryujinx translator 1.3.2 with MacNativeGames' patches and ours (`tools/xlate/patches/`), built with .NET 10 NativeAOT as `nx2navi_xlate.exe` and run as a separate process. Its SPIR-V goes to Vulkan directly. `tools/prepare_shaders.py` and `nx2navi.exe --prepare-shaders` translate and compile the variants of MacNativeGames' shared list ahead of play (29,587 for 1.4.3); the rest are translated on first use and cached on disk. -- Pipelines: `VK_EXT_graphics_pipeline_library` where the driver has it. Shader stages are compiled ahead into libraries; a draw that meets a new state links them at once (fast link), and an optimised pipeline replaces the linked one when a background thread finishes it. Pipeline keys seen in play are saved and built at boot. A draw whose pipeline is not ready is skipped when one of its pass's targets was rendered in the previous frame, and waits otherwise, so one-shot passes are never lost; a full-screen draw is skipped as well rather than waiting (`NX2NAVI_PIPE_SKIP`); the stage libraries a waiting draw needs are built on idle workers in parallel (`NX2NAVI_PIPE_WAIT`). Without the extension, a missing pipeline is built on a worker and the draw waits or is skipped by the same rule. -- Resolution. Patch sets make the game read its output, swapchain and scene sizes from the patch page - (MacNativeGames' sites); `resolution.c` sets them from `output_res` and `render_scale` and grows the texture heap - and the application pool to match. When the window is larger than the output, `vk_fsr.c` runs AMD FSR 1 (EASU, - RCAS) on the present. An `output_res` that is not 16:9 (more than 1% off, `res_is_16x9`) also switches on the - `aspect` patch set, which hands the output's ratio to the camera, and `romfs.c` `overlay_init` loads the - `romfs-patches-x` overlay for it when it exists (`tools/make_romfs_patches.py --aspect`, laid out by - `tools/ultrawide_layout.py`; the installer builds fifteen sizes: 21:9 2560x1080, 3440x1440, 3840x1600, 5120x2160, - 32:9 3840x1080, 5120x1440, 16:10 1280x800, 1680x1050, 1920x1200, 2560x1600 and 3:2 1920x1280, 2160x1440, 2256x1504, - 2880x1920, 3000x2000). A wider layout moves the HUD to the - screen's corners, a narrower one to its top and bottom edges. Other sizes get the camera of their shape and the 16:9 - HUD stretched over it. The `aspect` setting (`fit`, the default; `stretch`) decides what the present does - when the window's shape differs from the output's: `fit` letterboxes, where a 16:9 game in a wider window used to be - stretched. -- HDR (`hdr = 1`, `vk_hdr.c`). The game tonemaps its scene into an r11g11b10f target, upscales it and draws the HUD - into another float target, and a last draw copies that into the 8-bit sRGB window texture it presents. With HDR on, - the tonemap's fragment shader (recognised by its constants and resource names, not a hash) runs as - `shaders/hdr_tonemap.frag`, which keeps highlights up to the display's peak; the present reads the float target the - window texture was copied from (any other last draw into it, such as a menu or a movie, leaves the window texture, - shown at the HUD's white); a compute pass turns the picture into PQ code values in Rec. 2020 for an HDR10 - swapchain, which FSR 1 or the blit then scale; the in-game menu is drawn into an RGBA8 image and blended on at the - HUD's white. Without an HDR10 surface format (Windows with "Use HDR" off, a compositor without colour management) - the swapchain stays SDR. `NX2NAVI_SHOTS` reads the window texture, so its pictures clip highlights in HDR. -- DLAA (`vk_dlaa.c`, `dlaa`, off by default). NVIDIA DLSS at the scene's own size, through the driver's NGX core loaded - at run time (no SDK code compiled in). It runs on the R11G11B10F light target right before the bloom chain's first - pass, so post effects and the HUD read the anti-aliased picture. The scene's draws get a sub-pixel Halton viewport - jitter; camera motion vectors are reprojected from the scene depth through this and last frame's view-projection of - the scene camera block (constant bank c4), in double precision on the CPU. Per-object motion (`dlaa_objmv`, - `vk_mvprev.c`, the per-object section of `vk_exec_draw.c`, translator patch 0015): solid scene draws whose position - reads an object's own uniform blocks (bones c5, place c7, c6) are drawn a second time with a vertex shader variant - that computes last frame's position from last frame's blocks (the game writes them into two copies that take turns; - the uniform snapshot keeps both) and writes the difference into the motion image. `NX2NAVI_MV_CHECK` measures the - vectors against the picture, `NX2NAVI_DLAA_TIMES` times the passes on the GPU. `fxaa` switches the game's own FXAA pass. +- NVN 2 front end (`nvn_api_*.c`, `nvn_obj.h`, texture layout and deswizzle, the Switch 2 GOB layout). The game + records command buffers through it; nothing reaches the GPU until the game submits. +- Vulkan 1.3 back end. At `QueueSubmitCommands` the recorded NVN commands are replayed into Vulkan command buffers on + the submitting thread (`vk_exec*.c`); a recorder thread makes the `vkCmd*` calls from a ring of packets + (`vk_rec.c`). +- Memory pools are imported as host memory (`VK_EXT_external_memory_host`). On a discrete GPU, pools bound as buffers + are mirrored in device-local memory and the pages the CPU wrote (write watch) plus the ranges the game flushes are + uploaded before a replay (`vk_memory.c`); without the mirrors the GPU reads them over PCIe and Kokiri Forest runs at + 7 fps. GPU-rendered textures are written back when the CPU reads them. +- Uniform snapshot. The game rewrites per-object uniform blocks soon after it submits the frame that reads them; at + the start of a submission's replay every uniform range it binds from CPU-written pools is copied, and the draws bind + the copy (`vk_usnap.c`). +- Driver data. The NVN driver keeps a per-stage constant bank (c0). The back end fills it as the driver does: storage + and uniform tables, push constants and, for NVN 2 compute, the dispatch's group counts in bytes 0-0xb. +- Storage buffers the shaders reach through pointer tables in memory are bound at the address the table holds + (`GfxShaderRes.ptr`, `vk_exec_draw.c ptr_addr`). +- Bindless textures: the texture and sampler pools are descriptor arrays kept in step with the game's pool writes + (`vk_exec_draw.c pool_sync`); slots that resolve alike share one lookup. +- Shaders: Ampere (SM86) code is lowered to Maxwell (`tools/xlate/AmpereLowering.cs`) and translated to SPIR-V by the + Ryujinx shader translator 1.3.2 with our patches (`tools/xlate/patches/`), built with .NET 10 NativeAOT as + `nx2navi_xlate.exe` and run as separate processes. Translations are cached on disk; shaders and pipelines seen in + play are prepared at the next boot, which waits for them at most `NX2NAVI_PREWARM_WAIT_MS` and then lets them finish + in the background. +- Mesh and task shaders run as `VK_EXT_mesh_shader`; indirect mesh draws read NVN's `{count, first}` record through a + ring of Vulkan `{x, y, z}` records filled on the GPU. +- Pipelines: `VK_EXT_graphics_pipeline_library` where the driver has it: stage libraries compiled ahead, a fast link + at a new state, an optimised pipeline replacing it from a background thread. A draw whose pipeline is not ready is + skipped when one of its pass's targets was rendered in the previous frame, and waits otherwise. Compute pipelines + are compiled outside the pipeline lock, and one compile serves every thread that needs it. +- The game's tensor-core upscaler is replaced by a decode-and-scale pass (`vk_tonemap.c`); `vk_fsr.c` runs AMD FSR 1 + (EASU, RCAS) on the present when the window is larger than the picture. +- ASTC textures are uploaded as they are where the device samples ASTC, and decoded by a compute shader everywhere + else (`vk_astc.c`); under 20 GiB of video memory the decoder encodes them to BC3. - Occlusion queries never block the CPU; a result that is not ready counts as visible. -- ASTC textures are uploaded as they are where the device reports `textureCompressionASTC_LDR`, and decoded by a compute shader everywhere else (`vk_astc.c`, `NX2NAVI_ASTC`). The CPU decoder (`tex_astc*.c`) is the fallback and the test oracle. -- Draw state. A draw whose program, bound state and textures repeat reuses the previous draw's shaders and skips rebuilding its descriptor sets; descriptor sets are written through update templates (`vk_exec_draw.c`, `vk_pipeline.c`). -- Allocation: Vulkan Memory Allocator. Debugging: Vulkan validation layers, per-pass frame dumps (`NX2NAVI_DUMP`, `vk_dump.c`) and a flight recorder (`NX2NAVI_FLIGHT`). RenderDoc cannot capture, because it hides `VK_EXT_external_memory_host` from the application. +- Allocation: Vulkan Memory Allocator. Debugging: Vulkan validation layers, per-pass frame dumps (`NX2NAVI_DUMP`, + `vk_dump.c`), a sampling profiler (`NX2NAVI_PROFILE`, `tools/prof_report.py`) and a flight recorder + (`NX2NAVI_FLIGHT`). RenderDoc cannot capture, because it hides `VK_EXT_external_memory_host` from the application. -Maxwell behaviour the back end has to match, and the NX2navi bugs other ports fixed: [maxwell.md](maxwell.md). - -Why Vulkan rather than D3D12 or wgpu: - -- The translator emits SPIR-V, which Vulkan consumes as is. D3D12 would add a SPIR-V to DXIL step; wgpu would add SPIR-V to WGSL through Naga and drop features the game uses (bindless textures, some atomics). -- LAS Recomp's Vulkan back end is a working reference on Windows. -- The same back end runs on Linux. +Maxwell behaviour the back end has to match: [maxwell.md](maxwell.md). ### 5. Toolchain -- Generated C: MSYS2 UCRT64 Clang + LLD. The generated code uses GNU extensions (`__int128`, `musttail`, vector types) that `cl.exe` cannot compile. +- Generated C: MSYS2 UCRT64 Clang + LLD. The generated code uses GNU extensions (`__int128`, `musttail`, vector + types) that `cl.exe` cannot compile. - Host code: the same Clang, C11. -- Builds: `tools/build_game.py` writes a Ninja file and runs MSYS2 Ninja. Every object is hashed before linking, because a faulty RAM module once flipped bits in stored objects. -- Analysis: Ghidra with the Switch loader, Capstone, `llvm-objdump`. -- Players: the installer ([installer.md](installer.md)) builds the game code on the player's machine with a bundled llvm-mingw, so no recompiled code is ever distributed. - +- Builds write a Ninja file and run MSYS2 Ninja. Every object is hashed before linking, because a faulty RAM module + once flipped bits in stored objects. diff --git a/docs/controls.md b/docs/controls.md index b4218192..759c3980 100644 --- a/docs/controls.md +++ b/docs/controls.md @@ -1,6 +1,14 @@ # Controls -Input comes from the first SDL3 gamepad, the keyboard and the mouse, merged into one Switch controller (`runtime/input_win.c`). Buttons are ORed; each stick takes the source with the larger deflection. Keys and mouse buttons count only while the game window is in the foreground. `NX2NAVI_INPUT=none` disables both (automated runs). `NX2NAVI_*` settings can also be set in `nx2navi.ini` next to `nx2navi.exe` (`input = none`, `motion = 0`, `rumble = 0`); the environment overrides the file. `NX2NAVI_RUMBLE_STRENGTH` is the rumble strength in percent (0 to 300, default 130; 0 is the same as `rumble = 0`). `NX2NAVI_RUMBLE_CURVE` is the exponent applied to the amplitude before the strength (0.2 to 1.5, default 0.6; lower makes weak pulses stronger, 1 is linear). `NX2NAVI_RUMBLE_TRIGGERS=1` also drives the impulse triggers of an Xbox pad. +Input comes from SDL3 gamepads, the keyboard and the mouse, merged into one Switch controller (`runtime/input.c`, +`runtime/input_win.c`). Every connected gamepad is kept open and the game reads the one pressed last; a gamepad +switched on in play is found within a second. Buttons are ORed; each stick takes the source with the larger +deflection. Keys and mouse buttons count only while the game window is in the foreground. `NX2NAVI_INPUT=none` +disables both (automated runs). `NX2NAVI_*` settings can also be set in `nx2navi.ini` next to the exe (`input = none`, +`motion = 0`, `rumble = 0`); the environment overrides the file. `NX2NAVI_RUMBLE_STRENGTH` is the rumble strength in +percent (0 to 300, default 130; 0 is the same as `rumble = 0`). `NX2NAVI_RUMBLE_CURVE` is the exponent applied to the +amplitude before the strength (0.2 to 1.5, default 0.6; lower makes weak pulses stronger, 1 is linear). +`NX2NAVI_RUMBLE_TRIGGERS=1` also drives the impulse triggers of an Xbox pad. `NX2NAVI_PAD_LAYOUT` (`pad_layout`, the *Button layout* row on the Controls tab, live) picks which face button is Switch A: @@ -8,9 +16,11 @@ Input comes from the first SDL3 gamepad, the keyboard and the mouse, merged into - `xbox`: south A, east B, west X, north Y, as an Xbox pad labels them; on a PlayStation pad cross is A, circle B, square X, triangle Y. - `auto` (default): `nintendo` for a pad whose east button SDL reports labelled A (Switch Pro Controller, Nintendo-layout pads), `xbox` for the others. -The log names the layout in effect (`input: gamepad layout ...`); the menu shows it under *Automatic*. Ocarina's prompts always show the Switch letters: the Xbox and PC prompt folders below are made from NX2navi's files and do not exist for Ocarina, so its menu has no *Button prompts* row. 0.1.9's `pad_layout = xbox_prompts` is read as `pad_layout = nintendo` with `prompts = xbox`, and the menu writes it that way. +The log names the layout in effect (`input: gamepad layout ...`) and each gamepad's first press +(`input: gamepad "...": first press`); the menu shows the layout under *Automatic*. The game's prompts show the Switch +letters. -The actions are Ocarina's, as its own *Basic Controls* screen (Plus menu, System) names them: +The actions are the game's own, as its *Basic Controls* screen (Plus menu, System) names them: | Switch | Gamepad (by position, Nintendo layout) | Keyboard and mouse | Action in the world | |---|---|---|---| @@ -33,55 +43,7 @@ The actions are Ocarina's, as its own *Basic Controls* screen (Plus menu, System | Plus | Start | Tab | menu: items, equipment, collection, system | | Minus | Back | M | map | -The game's *Swap Jump Controls* option trades B and X (jump on B, put away on X); the bindings stay on the Switch buttons, so Space then puts away and Left Ctrl jumps. Left Shift is A, so holding it dashes; E presses A once for talking and opening. Esc is unbound in the game; it opens the settings menu and releases the mouse cursor (see Mouse camera). Z and X duplicate ZL and ZR on neighbouring keys. The mouse wheel is unbound. The right-stick shake (`bind_SHAKE`, NX2navi's object) has no default key and no row in Ocarina's menu. The bindings are the tables of `input_map` in `runtime/input_map.c`: the keys (platform-neutral key ids) and one gamepad table per layout (gamepad inputs by position); `runtime/input_win.c` translates Win32 virtual keys and SDL buttons. - -## Xbox button prompts - -NX2navi only: the folders are made from NX2navi's UI archive and fonts (`Font/Font*.Nin_NX_NVN.bfarc.zs`); Ocarina's are `Font2/*.Nin_Ounce_NVN2` and have no such folders. - -`NX2NAVI_PROMPTS` (`prompts`, the *Button prompts* row on the Controls tab, read at start) picks the controller whose buttons the game's prompts show: `nintendo` (default) keeps the game's own, `xbox` shows an Xbox controller's, `pc` the keyboard and mouse (PC button prompts, below). - -- L, R, ZL, ZR, Plus and Minus become LB, RB, LT, RT, Menu and View, and the d-pad prompts a cross with an arrow for each direction meant. The letters on the keys are cut out in the game's own text font. -- A, B, X and Y name the label of the button the pad layout presses, so each prompt names the button to press. `pad_layout = nintendo` plays an Xbox pad by position: the right button, labelled B, does Switch A (interact, confirm), and the bottom one, labelled A, does Switch B (run, cancel), so the prompts swap A with B and X with Y. `xbox` and `auto` put Switch A on the button labelled A, and the letters stay. - -| Switch | `nintendo` layout | `xbox` or `auto` layout | -|---|---|---| -| A, B, X, Y | B, A, Y, X | A, B, X, Y | -| L, R | LB, RB | LB, RB | -| ZL, ZR | LT, RT | LT, RT | -| Plus, Minus | Menu, View | Menu, View | -| D-pad | cross | cross | - -The action prompt (`DoCommand_00`) shows its button in a diamond whose shape gives the face buttons' positions as on a Switch, with the other three letters around it. With `nintendo` its letters are swapped with the rest and every one names the button in that place. With `xbox` and `auto` the letters are right but the diamond keeps the Switch positions: A shows on its right, while the button labelled A is at the bottom. The sticks keep the Switch icons, and so do prompts drawn as fixed images (some tutorial animations). - -The prompts are part of the game's fonts, textures and messages, so they follow the layout and not the bindings. A button remapped with a `bind_` line or on the Controls tab keeps the prompt the game draws for it, which then names the wrong button. Use the Xbox prompts with the default bindings. - -The game reads its UI archive and fonts once at boot, so the prompts change at the next start, also after `pad_layout` changes between `nintendo` and the others (the menu says *prompts change after restart*); the buttons change at once. Under that note the menu offers *Restart now*: after a confirmation it writes `nx2navi.ini`, closes the game as *Quit game* does and starts the game again with the same command line, which waits until the old process is gone before it opens the log and the saves (`runtime/gfx_present.c`, `gfx_window_closed`; `plat_restart` and `plat_restart_wait` in `runtime/platform/win32.c` and `linux.c`). Progress since the last save is lost, as with *Quit game*; while the game writes a save, the confirmation waits. With `prompts = xbox` the game loads `romfs-patches-xbox` (`pad_layout = nintendo`) or `romfs-patches-xbox-label` (`xbox`, `auto`) next to `romfs-index.bin` in place of `romfs-patches` (`runtime/romfs.c`, `overlay_init`; `runtime/input_map.c`, `input_prompts_folder`): the same 60 fps patches, with the prompts drawn from the game's own files by `tools/make_romfs_patches.py --prompts xbox` or `--prompts xbox-label` (`tools/button_icons.py`, after MacNativeGames' `make_button_icons.py`), which the setup runs next to the plain one: - -- every prompt but the button guide in play is text in the button font, `scft/External.bfotf` in each `Font/Font*.Nin_NX_NVN.bfarc.zs`: glyphs U+E040 to U+E043 (A B X Y), U+E044 to U+E047 (L R ZL ZR), U+E04B and U+E04C (Plus, Minus), U+E049 and U+E060 to U+E065 (the d-pad). fontTools writes the font again, so the archives grow and the folder also carries the resource size table (`System/Resource/ResourceSizeTable.Product....rsizetable.zs`) with their entries raised by as much; without that the game stops at start. -- the button guide in play draws L and ZL from the images `Nt_KeyTexL_00` and `Nt_KeyTexZL_00` of the Common archive's `timg/__Combined.bntx`, painted with the same icons. The action prompt draws its button from `Nt_KeyTexA/B/X/Y_00`, whose image data is swapped for `nintendo`, and the diamond's letters are text the game takes from `LayoutMsg/DoCommand_00.msbt` of each language's `Mals` archive (and the layout's own text panes behind it), swapped the same way. - -The Switch prompts stay, with a note under *Button prompts* and a `RomFS overlay: ...` line in the log, when the folder is missing (an install made by an older setup, or one where the setup could not prepare it and showed a warning; running the setup again on the install folder writes it), when it fails its check, with a game resolution of another shape than 16:9 (its own UI layout wins; 16:10 and 21:9), and when the `romfs_mods` folder replaces one of its files: `romfs-patches` then loads in its place, where two overlays replacing one file would stop the game. In a source build, `.venv-py314\Scripts\python tools\make_romfs_patches.py --version --prompts xbox` (and `--prompts xbox-label`) writes `build\\romfs-patches-xbox` (`-label`); it needs fontTools from `tools/requirements-py314.txt`. `--xbox-prompts` is the older name of `--prompts xbox`. - -## PC button prompts - -NX2navi only, as the Xbox prompts above. - -`prompts = pc` (*PC* on the *Button prompts* row) shows the key or mouse button the default keyboard bindings press each Switch button with, whatever the pad layout. The game loads `romfs-patches-pc` in place of `romfs-patches`, written by `tools/make_romfs_patches.py --prompts pc` (`button_icons.py`, `pc_icons`), which the setup runs next to the Xbox ones: - -| Switch | Prompt | -|---|---| -| A, B, X, Y | E, Shift, Space, left click | -| L, R, ZL, ZR | Q, R, right click, F | -| Plus, Minus | Tab, M | -| D-pad | 1 (left), 2 (up), 3 (right), 4 (down), as a cross or the keys of the directions meant | -| Left stick, its press | W A S D (or the keys of the directions meant), Ctrl | -| Right stick, its press | the mouse, with an arrow for each direction meant, C | - -- The glyphs keep the width of the game's own, so the right-aligned guides do not run into their text. A key is a rounded square with its label cut out in the game's own text fonts; a mouse button is a mouse with that button filled and the others hollow. -- The button guide in play draws every button from images of the Common archive (`Nt_KeyTexA/B/X/Y/L/ZL_00`, `Nt_KeyTexLSt_00` and `_01`, the stick tilted and pressed): all of them are painted with the same icons. The letters of the action prompt's diamond are one character of the Normal font each: E, an arrow for Shift, a bar for Space, L and R for the mouse buttons. -- Like the Xbox prompts they follow the defaults and not the bindings. `--binds ` draws the first key of each `bind_` line of that file instead, for a folder made by hand. -- At a game resolution of another shape than 16:9 the game loads `romfs-patches-pc-x` in place of `romfs-patches-x` when it is there: the same prompts with that shape's UI layout, `make_romfs_patches.py --prompts pc --aspect x --out \romfs-patches-pc-x`. The setup does not write those, so without one the Switch prompts stay (*off at a game resolution not 16:9*). +The game's *Swap Jump Controls* option trades B and X (jump on B, put away on X); the bindings stay on the Switch buttons, so Space then puts away and Left Ctrl jumps. Left Shift is A, so holding it dashes; E presses A once for talking and opening. Esc is unbound in the game; it opens the settings menu and releases the mouse cursor (see Mouse camera). Z and X duplicate ZL and ZR on neighbouring keys. The mouse wheel is unbound. The bindings are the tables of `input_map` in `runtime/input_map.c`: the keys (platform-neutral key ids) and one gamepad table per layout (gamepad inputs by position); `runtime/input_win.c` translates Win32 virtual keys and SDL buttons. ## Window keys @@ -95,16 +57,16 @@ The Minus+Plus combo never reaches the game. In the menu the arrow keys, the D-p ## Remapping -The *Controls* tab of the settings menu lists, under the button layout, every Switch button and the eight stick directions the keys drive (movement and camera), each with two key cells and a controller cell. Each row's description says what the button does in Ocarina. Select a cell, press A or Enter, then press the new key, mouse button or controller button; Esc or five seconds without a press cancel, and a controller button cancels a key cell. Delete, Backspace or X clears the selected cell. A key another action has moves to the new one (the description names the action it left); a controller button swaps with the one it replaces. A remapped controller button is physical: from then on the face buttons (A B X Y) stay where they are in both layouts, until the buttons are put back as the layout in use has them or *Reset all buttons* is chosen. F1, F8, F10, F11 and Esc cannot be bound (the menu, the flight recorder and the window keep them). Punctuation keys are named by their place on a US keyboard; the menu shows the character the keyboard layout in use puts there. +The *Controls* tab of the settings menu lists, under the button layout, every Switch button and the eight stick directions the keys drive (movement and camera), each with two key cells and a controller cell. Each row's description says what the button does in the game. Select a cell, press A or Enter, then press the new key, mouse button or controller button; Esc or five seconds without a press cancel, and a controller button cancels a key cell. Delete, Backspace or X clears the selected cell. A key another action has moves to the new one (the description names the action it left); a controller button swaps with the one it replaces. A remapped controller button is physical: from then on the face buttons (A B X Y) stay where they are in both layouts, until the buttons are put back as the layout in use has them or *Reset all buttons* is chosen. F1, F8, F10, F11 and Esc cannot be bound (the menu, the flight recorder and the window keep them). Punctuation keys are named by their place on a US keyboard; the menu shows the character the keyboard layout in use puts there. -The tab writes `nx2navi.ini` lines, one per Switch button or stick direction: `bind_