6 Commits

Author SHA1 Message Date
ruckus7382sfgsgdf
afbb0d140a Triforce v0.3.2
The above-60 fixes (the pick-up counter, water ripples, sound, moving platforms, the time of day, cutscene cues) and the
RDNA 1 subgroup fix, ported from NotRexed/Hookshot: Jordan Ella, hckrman101, klimovich008, Bruno Baccari.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-12 01:29:02 -06:00
ruckus7382sfgsgdf
362d576c7d Triforce v0.3.1
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 22:49:28 -06:00
ruckus7382sfgsgdf
a939e9d0d1 Triforce v0.3.0
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-11 20:19:23 -06:00
ruckus7382sfgsgdf
035977a061 Triforce v0.2.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 22:37:38 -06:00
ruckus7382sfgsgdf
b855d86841 CONTRIBUTING: Triforce's own page (links, file names, logs)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 22:32:16 -06:00
ruckus7382sfgsgdf
a6bb7ffa09 CONTRIBUTING: Triforce's own, with its links, file names and logs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 22:31:36 -06:00
25 changed files with 3392 additions and 366 deletions

View File

@@ -1,12 +1,13 @@
# Contributing to Hookshot
# Contributing to Triforce
Hookshot is a static recompilation of the Switch 2 *The Legend of Zelda: Ocarina of Time* remake: the game's AArch64
code is lifted to C, the Horizon OS and NVN2 are implemented natively, and the GPU work is replayed on Vulkan. No
emulator is involved, and the game is built on your PC from your own copy.
Triforce is a native PC port of the Switch 2 *The Legend of Zelda: Ocarina of Time* remake. The game's code is
translated to C and built on your own PC from your own copy of the game; the system and the graphics are rebuilt
natively, with the graphics replayed on Vulkan. No emulator is involved. Triforce builds on the open-source Hookshot
project (GPL-3.0), and its code keeps that license.
This page covers what to send, how to build and test it, and the rules that keep the project out of trouble. The
[development guide](docs/development.md) has the full pipeline and its commands; the [design notes](docs/design.md)
have the architecture and the lessons the code follows.
have the architecture.
## The rule with no exceptions
@@ -14,273 +15,135 @@ have the architecture and the lessons the code follows.
never will. That includes:
- the `.nup`, `romfs`, `exefs`, `.nso`, `.nca`, `.nsp`, `.xci` or `.nsz`/`.xcz` files, and any title keys;
- `work/` and everything in it: the extracted dump, `image.bin`, the lifted C, test builds, runs and screenshots;
- `hookshot.exe`, `hookshot_vk.dll`, `hookshot_programs.bin` (and its `hookshot_programs.idx`), `hookshot_pipelines.bin`,
`mkw_vk_pipelines.bin` and other build output;
- saves (`user/`), `hookshot_options.ini`, `hookshot_env.txt` and `language.txt`.
- `work/` and everything in it: the extracted dump, `image.bin`, the translated C, test builds, runs and screenshots;
- `Triforce.exe`, `triforce_vk.dll`, `triforce_shaders.bin`, `triforce_programs.bin` (and `.idx`),
`triforce_pipelines.bin`, `mkw_vk_pipelines.bin` and every other file a build or a run makes;
- saves (`user/`), `triforce_options.ini`, `triforce_env.txt` and `language.txt`.
Do not open an issue or a PR asking where to get the game, how to decrypt it, or for someone else's files. A
screenshot, a video and `hookshot_log.txt` are fine and often necessary; the log holds no game data.
Do not open an issue or a pull request asking where to get the game, how to decrypt it, or for someone else's files.
A screenshot, a video and `triforce_log.txt` are fine and often necessary; the log holds no game data.
`.gitignore` covers most of this, but check `git status` before you commit. If you are not sure whether a file is
game data, leave it out.
## Ways to help
## Reporting a problem
What to work on first, and the steps from claiming a problem to its release, are in the [workflow](docs/workflow.md):
priorities by severity and by how many players run the affected hardware, and the hardware tiers a change must not
break. Claim in Discord #dev-work before you start.
Open an [issue](https://github.com/ruckus7382sfgsgdf/CBeta0.2/issues) and include:
- the Triforce version (the release name, or the first line of `triforce_log.txt`);
- your graphics card, its driver version, and your Windows version;
- where you were in the game and what you did;
- the logs:
- **a game problem:** `triforce_log.txt` from your game folder (the folder with `Triforce.exe`). If you started the
game again after the problem, send `triforce_log.1.txt` too: each start keeps the four runs before it as
`triforce_log.1.txt` (the last) to `triforce_log.4.txt`.
- **a Setup or build problem:** `setup.log` from `%LOCALAPPDATA%\Programs\Triforce`, and for a failed build the text
from **Show details** on the build page.
Before you report:
- Try the [newest release](https://github.com/ruckus7382sfgsgdf/CBeta0.2/releases) first.
- Search the open issues; if yours is there, add your details to it.
- For graphics or performance problems, test without ReShade, overlays, shader-cache tools and frame-generation
injectors: they change what Triforce does.
- If graphics look wrong after an update, clearing the shader caches often helps: close the game and delete
`triforce_programs.bin`, `triforce_programs.idx`, `triforce_pipelines.bin`, `triforce_pipeline_states.bin`,
`triforce_computes.bin`, `mkw_vk_pipelines.bin` and `triforce_prepared.bin` from the game folder (never
`triforce_shaders.bin`). The next start prepares them again.
One topic per issue or pull request, in English, please.
## Ways to help
| | What |
|---|---|
| **Test on other hardware** | NVIDIA cards are the most tested; the README's [status table](readme.md#project-status) says what works where, and [tested configurations](docs/tested-configurations.md) lists the runs behind it (build, hardware, driver, where, fps). Reports from AMD and Intel cards, other NVIDIA drivers, other distributions and SteamOS are useful even when everything works: add a row there. |
| **Play further** | The tested area is the start of the game. Reaching new areas, cutscenes, menus and items finds problems the developer has not seen. |
| **Report what is broken** | Use the [issue forms](https://github.com/NotRexed/Hookshot/issues/new/choose): game problem, build problem or idea. Include the version, your hardware, where you were and `hookshot_log.txt`. |
| **Send a fix** | Fork, fix, test, open a pull request. Small, focused changes are easiest to review and merge. |
| **Improve the docs** | The README, the development guide and the design notes are part of the project. Corrections and clearer steps are welcome. |
## Before you open an issue or a PR
- Try the [newest release](https://github.com/NotRexed/Hookshot/releases/latest) first. It includes every fix before it.
- Search the open issues. If yours is already there, add your details instead of opening another.
- English, please.
- One topic per issue or pull request. Do not mix a fix with a refactor or a batch of unrelated changes.
- Report the version you actually ran: the release name, or the first line of `hookshot_log.txt`.
- Test without ReShade, shader-cache tools, overlays and frame-generation injectors when you report a graphics or
performance problem. They change what Hookshot does.
| **Test on other hardware** | NVIDIA cards are the most tested. Reports from AMD and Intel cards and other drivers are useful even when everything works. |
| **Play further** | Reaching new areas, cutscenes, menus and items finds problems nobody has seen yet. |
| **Report what is broken** | As above, with the version, your hardware, where you were and the log. |
| **Send a fix** | Fork, fix, test, open a pull request. Small, focused changes are easiest to review. |
| **Improve the docs** | Corrections and clearer steps are welcome. |
## Development setup
You need your own supported dump: program `0400486024DEC000`, update v131072, build `e0f3ccfa`. Other versions do
not work, and the build reads the dump where it lies without changing it. Builds need about 60 GB free and a run uses
about 10 GB of RAM. Build on an SSD if you can: a hard drive works, but takes much longer, and the builder warns you
before it starts.
You need your own supported copy of the game: program `0400486024DEC000`, update v131072, build `e0f3ccfa`. Other
versions do not work, and the build reads your files where they are without changing them. A build needs about 60 GB
free and takes around 40 minutes; a run uses about 10 GB of RAM. Build on an SSD if you can.
### Windows, from the command line
The easy way to get every tool a build needs is **Triforce Setup** from the
[releases](https://github.com/ruckus7382sfgsgdf/CBeta0.2/releases): it installs the tools (Python with its packages,
llvm-mingw, the Vulkan headers, FFmpeg, SDL2) into `%LOCALAPPDATA%\Programs\Triforce\tools`.
1. Install Python 3 from [python.org](https://www.python.org/downloads/) (not the Microsoft Store build: `python -I`
does not see packages installed there) and the pipeline's packages:
```powershell
python -m pip install -r requirements.txt
```
(the second line: the button prompts' packages. Without them the game shows the Switch's icons)
To build from a clone instead:
1. Install Python 3 from [python.org](https://www.python.org/downloads/) (not the Microsoft Store build) and the
pipeline's packages: `python -m pip install -r requirements.txt`.
2. Get [llvm-mingw](https://github.com/mstorsjo/llvm-mingw) (the `ucrt-x86_64` build) and set `HOOKSHOT_MINGW` to its
folder. Without it the build falls back to emsdk's clang at `C:\emsdk` with Visual Studio's C++ tools and the
Windows SDK (`HOOKSHOT_MSVC=1` forces that route).
3. Copy the pieces a clone does not carry from the `tools` folder Hookshot Builder installs. Run `HookshotBuilder.exe`
from the latest release once (releases no longer carry a `HookshotBuilder.zip`): the folder is
`%LOCALAPPDATA%\HookshotBuilder\versions\<version>\HookshotBuilder\tools`.
| From `tools` | To | Variable |
|---|---|---|
| `tools\cpm` | anywhere | `RX_CPM` points at it (Vulkan headers and FFmpeg; without it, system headers or `HOOKSHOT_*_HEADERS`, the standalone build stops) |
| `tools\wgpu-native` | anywhere | `WGPU_NATIVE` points at it (the WebGPU fallback renderer; without it the game is built without one) |
| `tools\third_party` | `third_party\` | (its `.dll`, `.exe`, `.lib` and `.sha1` files, to the same places: `a64emit`, `lld-link`, `zbic` and SDL2) |
| `tools\work` | `work\` | (hot-function list, link order, profile and icon; without it the game builds but runs slower) |
folder.
3. From the installed tools folder, point `RX_CPM` at `tools\cpm` and `WGPU_NATIVE` at `tools\wgpu-native`, and copy
`tools\third_party` into `third_party\` and `tools\work` into `work\`.
4. Run everything with `python -I` from the repository root.
### Linux
(Some environment variables still carry the older `HOOKSHOT_` name; they work as documented.).
Either use the `Hookshot-<version>-x86_64.AppImage` from the Releases page, or build the environment and the AppImage
yourself:
### Triforce Setup
```sh
scripts/linux/deps.sh # LLVM, Python packages, FFmpeg with dav1d, SDL2, Vulkan headers
scripts/linux/appimage.sh # the AppImage, in build-linux/
```
The AppImage needs glibc 2.35 or newer and a Vulkan 1.3 driver with mesh shaders. `HOOKSHOT_JOBS` sets how many files
compile at once. The README's [Linux section](readme.md#on-linux) has the details and the runtime paths.
### NVIDIA DLSS (optional)
DLSS (`gpu/vk/vk_dlss.cpp`) is built in only with NVIDIA's DLSS SDK, which is NVIDIA's, under NVIDIA's own license, and
is never committed, bundled or uploaded: not in a PR, not in a release, not in a CI artifact. Fetch it for your own
builds with `scripts/ngx_sdk.py` (it prints NVIDIA's license notice and needs `--accept-nvidia-license` or a typed yes),
then build with `HOOKSHOT_NGX_SDK=<sdk folder>`:
```powershell
python -I scripts/ngx_sdk.py --platform windows C:\path\ngx-sdk # headers, nvsdk_ngx_s.lib, nvngx_dlss.dll
$env:HOOKSHOT_NGX_SDK = "C:\path\ngx-sdk"
python -I nx2/make_game.py --nup "C:\path\your-game.nup" --out "C:\Games\Hookshot-dlss"
```
On Windows the llvm-mingw build links NVIDIA's static `nvsdk_ngx_s.lib` (built for Microsoft's static runtime) into
`hookshot_vk.dll`, with `gpu/vk/vk_ngx_msvc_rt.cpp` and `host/mingw_msvc_shims.c` for what it needs of that runtime.
`make_game.py` copies `nvngx_dlss.dll` into the game folder's `ngx` folder; never put NGX's core, `_nvngx.dll`, beside
the game (it is the driver's). On Linux it is `--platform linux` (or `scripts/linux/ngx-sdk.sh`), with `libnvsdk_ngx.a`
and `libnvidia-ngx-dlss.so`. A Windows game with DLSS can be built on Linux too, with `RX_TARGET=windows` as below.
CI's `vulkan-dlss` workflow (`.github/workflows/vulkan-dlss.yml`) checks only that `hookshot_vk.dll` still links with the
SDK. It runs by hand, and on a push or pull request only when a file DLSS's build depends on changes; it accepts NVIDIA's
license for that run alone and uploads nothing.
Never share a game folder built with the SDK. A change to the pinned release updates the file table and SHA-256s in
`scripts/ngx_sdk.py` (and `tests/test_ngx_sdk.py`), and the release named in the builder's license text
(`builder/HookshotBuilder/MainWindow.xaml.cs`).
### Hookshot Builder
The builder is a .NET 8 WPF app. Open `builder/HookshotBuilder.sln` in Visual Studio 2022 with the .NET desktop
workload, or build it from a developer prompt:
```powershell
dotnet build builder/HookshotBuilder.sln
```
You only need Visual Studio for the builder itself. The game build uses llvm-mingw.
The Windows app's **Build workers** choice applies to builds started from that window. **Automatic** keeps the existing
`HOOKSHOT_JOBS` override, or the build tools' free-memory policy when no override is set. A number overrides it for that
build's child process only; more workers need more memory and are not always faster. This choice is not saved: a new
window, including one started to update a game, uses Automatic again.
The launcher lives in `setup/`: a .NET Framework 4.8 WPF app, built with `powershell -File setup\build.ps1` (needs the
C# compiler from Visual Studio Build Tools). `setup\pack_tools.ps1` packs the tools zip a release carries. Never run a
development build's install, update or uninstall without `TRIFORCE_TEST_ROOT` set to a scratch folder: it would act
on your real install.
## Building
| Command | What it does |
|---|---|
| `python -I nx2/make_game.py --nup "C:\path\your-game.nup" --out "C:\Games\Hookshot" --language en-US` | Build a complete game folder from a `.nup`. A build takes about 40 minutes. |
| `python -I nx2/make_game.py --romfs "C:\path\romfs" --exefs "C:\path\exefs" --out "C:\Games\Hookshot"` | The same, from extracted folders. |
| `python -I nx2/make_game.py --nup "C:\path\your-game.nup" --out "C:\Games\Triforce" --language en-US` | Build a complete game folder from a `.nup`. |
| `python -I nx2/make_game.py --romfs "C:\path\romfs" --exefs "C:\path\exefs" --out "C:\Games\Triforce"` | The same, from extracted folders. |
| `python -I nx2/make_game.py --check --nup "C:\path\your-game.nup"` | Check the game and its version without building. |
| `python -I nx2/dump.py --dump <folder with exefs/ and romfs/> --set` | Record the dump in `work/dump.txt`. |
| `python -I nx2/lift.py --standalone --write` | Lift the game to C in `work/sa/gen`. |
| `python -I nx2/build.py --standalone --jobs 4` | Compile `work/sa/build/hookshot.exe` and `hookshot_vk.dll`. |
| `powershell -File scripts/run-vk.ps1 -Window -Seconds 120` | Run the standalone build with the Vulkan back end and take screenshots in `work/runs/<name>/shots`. |
| `python -I scripts/prof.py <stacks file>` | Read a profile sampled with `HOS_PROFILE`, `HOS_SAMPLE` and `HOS_SAMPLE_FILE`. |
Use a separate `--work` folder or `OOT_WORK` if you keep more than one dump around. The lifted C and every build are
local artifacts; they are never committed.
The translated C and every build are local files; they are never committed.
## Testing
### Game-free checks
These need no dump and are what CI runs, on every pull request and every push to `main`. One command runs them all:
The game-free checks need no game files:
```powershell
python -I -B tests/run_all.py
```
It runs every `tests/test_*.py` and `nx2/test_*.py` in its own process, the C++ unit tests under `gpu/vk/tests` and a
byte-compile of every tracked Python file, goes on after a failure, and ends with a table (slowest first) and every
failure's output. Name a check to run only the ones whose path contains it (`python -I -B tests/run_all.py ub_`). A new
`test_*.py` file is picked up by itself: no workflow to edit.
It runs every `tests/test_*.py` and `nx2/test_*.py`, the C++ unit tests under `gpu/vk/tests`, and a byte-compile of
every Python file, and ends with a table of results. Name a check to run only the ones whose path contains it. A new
`test_*.py` file is picked up by itself.
Some checks compile a small C or C++ fixture and need a compiler (`g++`, `gcc` or `clang++` on PATH), the SPIR-V
validator (`spirv-val`), the SPIR-V or Vulkan headers (`HOOKSHOT_SPIRV_HEADERS` / `HOOKSHOT_VULKAN_HEADERS`, or the CPM
cache in `RX_CPM` or `build-linux/deps/cpm`), or the `zstandard` package. A check whose tool is missing is skipped and
says so; CI passes `--strict`, which fails it instead. A check that cannot run in CI (a GPU, the game, a Visual Studio
prompt), or runs on one OS only, says so in its own file with `RUN_ALL_SKIP`; `--no-skip` runs it anyway.
When you add a fix, add a game-free test where the behavior can be tested without the game. For a renderer or runtime
fix, say what you saw before and after, on which GPU and driver, and where in the game.
### Checks that need your dump or a run
## Code conventions
These are slow and local, and they need `work/dump.txt` from `nx2/dump.py --set`:
- Match the file you are editing: its indentation, comment style and naming. Do not reformat unrelated code.
- Do not modify `third_party/`. Its parts are unmodified upstream copies, each with its notice in its own
`PROVENANCE.md`.
- New switches use the `RX_` prefix in the renderer and `HOS_` in the runtime, are off unless asked for, and cost
nothing while off.
- Rendering fixes must reproduce what the game itself does, not tuned factors that only look right on one screen.
- Python runs with `-I`; tests stay on the standard library. Keep line endings LF.
| Command | What it checks |
|---|---|
| `python -I nx2/test_insn.py` | Every instruction form the game uses, Unicorn against the lifted C. |
| `python -I nx2/test_leaf.py --calls` | Whole functions and call trees on the relocated image, native against Unicorn. |
| `python -I nx2/census.py --nx2` | The instruction census per module. |
| `python -I nx2/lower_test.py work/lowfail` | Shader programs that failed to lower. |
| `powershell -File scripts/run-vk.ps1 -Window` | An in-game run: screenshots, the flash detector, frame dumps and the profiler. |
## Pull requests
For a graphics fix, look at the screenshots in `work/runs/<name>/shots` and delete them afterwards; they reach
gigabytes and are derived from the game. The switches for scripted input, screenshots, frame dumps and profiles are in
the development guide's [reference](docs/development.md#switches-for-testing-and-debugging).
### When you add a fix or a check
- Add a game-free test where the behavior can be tested without the dump. The existing tests are small Python files
under `tests/` or `nx2/` that read source files and run them, or compile a tiny C++ fixture.
- Name it `tests/test_*.py` or `nx2/test_*.py`: `tests/run_all.py` picks it up by itself and CI runs it on every pull
request, so there is no workflow to edit. What it needs from the runner it declares in its own file, after its
docstring and imports, as plain literals `tests/run_all.py` reads without importing the test (its docstring has
the details); there is no central list to add a line to:
```python
RUN_ALL_ARGS = ["--cxx", "{cxx}", "--headers", "{vulkan_headers}"] # a compiler, headers, spirv-val ...
RUN_ALL_MODULES = ["zstandard"] # packages it imports
RUN_ALL_SKIP = {"windows": "POSIX signals: the Linux guard"} # cannot run in CI there (a GPU, the game,
# one OS only), and why
```
`{name}` is a tool the runner finds (`cxx`, `vulkan_headers`, `spirv_headers`, `spirv_val`, `glslang`, `dotnet`);
a check whose tool is missing is skipped, or fails with `--strict`. A test that needs none of this declares nothing.
- Keep the test fast: every check shares one CI job with a 20-minute limit. Make it pass on both Ubuntu and Windows
where it can.
- If the fix is in the renderer or the runtime, say what you saw before and after, on which GPU and driver, and in
which part of the game.
## Code and data conventions
- Match the file you are editing: there is no formatter configuration, so use the indentation, comment style and
naming around you. Do not reformat unrelated code in the same change.
- Keep OoT-specific code in new files where possible. `nx2/`, `host/` and `gpu/` are kept close to mkw-recomp so
upstream fixes can be applied as a patch; keeping the paths and the shared files recognizable is what makes that
work. Each sync names its new mkw-recomp base commit in the commit message.
- Do not modify `third_party/`. Its parts are unmodified upstream copies with their notices in `PROVENANCE.md`; add a
new folder and a provenance note rather than editing one.
- Treat the local mkw-recomp and Eglantine checkouts as read-only references: read from them, run them, never edit
them.
- New switches use the `RX_` prefix in the renderer and `HOS_` in the runtime. They are off, or neutral, unless
asked for, and they do not cost anything while off. Document them where the surrounding switches are documented.
- The GPU replay rules in [design notes, section 4](docs/design.md#4-the-lessons-from-mkw-as-rules-here) are not
suggestions: nothing the replay reads may come from live guest memory, waits are honoured per queue, and texture
and sampler registrations stay in submission order. A change that reads guest memory at replay time is a
regression, even if it looks right on one frame.
- Python runs with `-I`. Tests stay on the standard library so CI needs no install step; the pipeline's existing
dependencies (lz4, numpy, capstone, unicorn, zstandard) are the exception, not a reason to add more. So are the
button prompts' (fontTools, Pillow, astc-encoder-py, texture2ddecoder: `nx2/prompt_requirements.txt`), accepted for
`nx2/prompt_assets.py` and `nx2/prompt_draw.py` alone, at build time. Maintainers' agents reviewing a change: check
that these four are still used only there, and flag any other use. Keep line endings LF (`.gitattributes`).
- New code has to be GPL-compatible. See below.
## Commits and pull requests
1. Fork the repository and branch from `main`. Name the branch for the change: `fix/...`, `linux/...`, `builder/...`,
`docs/...`.
2. Keep the pull request to one topic. Bug fixes do not carry refactors, and two unrelated fixes are two PRs.
3. Write the title as a short imperative sentence, as the history does: "Stop scanning storage-buffer tables after
one pass", "Linux: a crash report gives the whole host stack". Start the body with why the PR is needed, in one or
two plain sentences a player could follow: the bug, the missing feature or the cost it removes ("The menu's text
is cut off at 4K", "First starts spend 20 s waiting on a few shaders"). Then say what changed, how it was tested
(GPU, driver, game area, fps before and after), and link the issue it fixes.
4. Include screenshots when the change shows on screen: graphics, the menu, the start-up screen, an overlay, a
message. Before and after for a fix, the new screen for a feature, and the sizes or settings it was checked at. A
screenshot of the game is fine; the rule above is about game files, not pictures of the game.
5. Run the relevant checks. CI must pass; a change that only works on your machine is not ready.
6. Rebase or merge `main` when it has moved, and resolve conflicts in your branch rather than in the review.
7. Do not include build output, dumps, generated files or a `.gitignore` bypass. Check `git status`.
Commit subjects decide the next release: write them as Conventional Commits (`feat:`, `fix:`, and `!` or
`BREAKING CHANGE:` for a break). The version rules, branches and changelog format are in
[docs/versioning.md](docs/versioning.md).
Maintainer commits record AI assistance with `Assisted-by:` and `Co-Authored-By:` trailers. If you used a tool, add
the trailers that describe it; if you wrote the change yourself, there is nothing to add.
1. Fork the repository and branch from `main`.
2. One topic per pull request: a fix does not carry a refactor.
3. Title: a short imperative sentence. Body: why it is needed, what changed, how it was tested (GPU, driver, game
area, fps before and after), and the issue it fixes.
4. Screenshots for anything that shows on screen, before and after.
5. No build output, dumps or generated files. Check `git status`.
## Licensing
Hookshot is free software under the [GNU General Public License, version 3 or later](LICENSE). By submitting a change
you agree to license it under those terms. Only send code you can license that way. Do not paste code from a project
with an incompatible license, and do not paste decompiled game code or anything generated from the dump into a PR.
Triforce is free software under the [GNU General Public License, version 3 or later](LICENSE), like the Hookshot code
it builds on. By submitting a change you agree to license it under those terms. Do not paste code from a project with
an incompatible license, and never paste decompiled game code or anything generated from the game files.
The third-party parts (Eden, suyu, Sirit, the Vulkan Memory Allocator, SDL, Opus, Dear ImGui, zstd, the Libertinus
fonts, zbic and the a64-kit tools) keep their own notices. Preserve them, and do not relicense their code.
fonts, zbic and the a64-kit tools) keep their own notices. Preserve them.
NVIDIA's DLSS SDK is not one of them: it is never in the repository, and nothing of it may be added (see
[NVIDIA DLSS](#nvidia-dlss-optional)).
## Where to talk
- **Issues:** [use the forms](https://github.com/NotRexed/Hookshot/issues/new/choose).
- **Discord:** https://discord.gg/EajnD2AJD9. Use the `hookshot-discussion` channel; the server is mainly for
web ports, and that channel is where Hookshot talk belongs.
- **Newest release:** https://github.com/NotRexed/Hookshot/releases/latest.
Thanks for testing, reporting and sending fixes. Every report from a different GPU and every area played further than
before moves the port forward.
Thanks for testing, reporting and sending fixes. Every report from a different GPU and every area played further
moves the port forward.

View File

@@ -96,8 +96,8 @@ bool UseShuffleFallback(EmitContext& ctx) {
(ctx.profile.has_broken_spirv_fragment_shuffle && ctx.stage == Stage::Fragment);
}
Id ShuffleAbsolute(EmitContext& ctx, Id value, Id src_thread_id) {
if (!UseShuffleFallback(ctx)) {
Id ShuffleAbsolute(EmitContext& ctx, Id value, Id src_thread_id, bool broadcast) {
if (!broadcast) {
return ctx.OpGroupNonUniformShuffle(ctx.U32[1], SubgroupScope(ctx), value, src_thread_id);
}
Id result{ctx.u32_zero_value};
@@ -111,8 +111,9 @@ Id ShuffleAbsolute(EmitContext& ctx, Id value, Id src_thread_id) {
}
Id ShuffleRelative(EmitContext& ctx, Id value, Id delta, Id src_thread_id, spv::Op op) {
if (!ctx.profile.support_shuffle_relative || UseShuffleFallback(ctx)) {
return ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id));
const bool broadcast{UseShuffleFallback(ctx)};
if (!ctx.profile.support_shuffle_relative || broadcast) {
return ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id), broadcast);
}
const Id result{ctx.OpGroupNonUniformShuffleXor(ctx.U32[1], SubgroupScope(ctx), value, delta)};
ctx.opcode_rewrites.emplace_back(result.value, op);
@@ -260,7 +261,8 @@ Id EmitShuffleIndex(EmitContext& ctx, IR::Inst* inst, Id value, Id index, Id cla
return ctx.OpSelect(ctx.U32[1], in_range, BroadcastLane(ctx, value, lane.U32() & 31),
value);
}
const Id shuffled{ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id))};
const Id shuffled{
ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id), UseShuffleFallback(ctx))};
return ctx.OpSelect(ctx.U32[1], in_range, shuffled, value);
}
@@ -316,7 +318,7 @@ Id EmitShuffleButterfly(EmitContext& ctx, IR::Inst* inst, Id value, Id index, Id
// (hookshot) through the broadcasts where shuffles are broken (UseShuffleFallback: AMD's fragment stages, Lichii #273),
// as the absolute and relative shuffles are; #273 left SHFL.BFLY native, and some of Kokiri Forest's plants use it
const Id shuffled{UseShuffleFallback(ctx)
? ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id))
? ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id), true)
: ctx.OpGroupNonUniformShuffleXor(ctx.U32[1], SubgroupScope(ctx), value, mask)};
return ctx.OpSelect(ctx.U32[1], in_range, shuffled, value);
}
@@ -328,7 +330,8 @@ Id EmitQuadBroadcast(EmitContext& ctx, Id value, Id lane) {
const Id base{ctx.OpBitwiseAnd(ctx.U32[1], GetThreadId(ctx), ctx.Const(~3u))};
const Id local_lane{ctx.OpBitwiseAnd(ctx.U32[1], lane, ctx.Const(3u))};
const Id src_thread_id{ctx.OpBitwiseOr(ctx.U32[1], base, local_lane)};
return ShuffleAbsolute(ctx, value, src_thread_id);
return ShuffleAbsolute(ctx, value, src_thread_id,
ctx.profile.has_broken_spirv_subgroup_shuffle);
}
Id EmitQuadSwap(EmitContext& ctx, Id value, Id direction) {

View File

@@ -1190,11 +1190,46 @@ void state_record(const Program& p, const NvnState& s, const Targets& at, VkPrim
if (!state_file) state_open_locked();
if (state_file && state_seen.insert(h).second) fwrite(&r, sizeof r, 1, state_file), fflush(state_file);
}
/// (perf push 11-11) Which shipped list this folder's notes were last made from, in triforce_states_applied.bin: the list's
/// contents and the translator, hashed. The lists were gone through only inside a prepare (a first start, a new translator, a
/// new seed): a folder prepared already that got a longer shipped list (an update with the same translator, a list copied in)
/// never noted its pipelines, and built each in play when first drawn (a lab folder with a 12067-pipeline list: 2687 built in
/// play, the list not read once). 0: there is no shipped list
uint64_t state_shipped_id() {
FILE* f = fopen(state_list_path("triforce_pipeline_states_shipped.bin").c_str(), "rb");
if (!f) return 0;
uint64_t h = 0xCBF29CE484222325ull ^ program_cache_stamp(), total = 0;
std::vector<uint8_t> buf(1 << 16);
for (size_t n; (n = fread(buf.data(), 1, buf.size(), f)) > 0; total += n)
for (size_t i = 0; i < n; i++) h = (h ^ buf[i]) * 1099511628211ull;
fclose(f);
return total > 12 ? (h | 1) : 0;
}
/// Whether the shipped list is one the notes were not made from yet
bool state_shipped_new() {
if (getenv("RX_VK_NO_STATE_LIST")) return false;
const uint64_t id = state_shipped_id();
if (!id) return false;
uint64_t had = 0;
if (FILE* f = fopen(state_list_path("triforce_states_applied.bin").c_str(), "rb")) {
if (fread(&had, 8, 1, f) != 1) had = 0;
fclose(f);
}
return had != id;
}
/// (prepare_run, the seed open) every pipeline of the state lists (the player's own and the shipped one): its program translated
/// when the program cache lacks it, then noted in triforce_pipelines.bin for precompile_run to build. RX_VK_NO_STATE_LIST=1: off
void state_list_prepare(void (*progress)(size_t done, size_t total)) {
if (getenv("RX_VK_NO_STATE_LIST")) return;
const auto t0 = std::chrono::steady_clock::now();
const uint64_t shipped_id = state_shipped_id();
struct Applied { // (written when the lists were gone through, however this returns)
uint64_t id;
~Applied() {
if (!id) return;
if (FILE* f = fopen(state_list_path("triforce_states_applied.bin").c_str(), "wb")) fwrite(&id, 8, 1, f), fclose(f);
}
} applied{shipped_id};
std::vector<StateRec> recs = state_list_read(state_list_path("triforce_pipeline_states_shipped.bin"));
const size_t shipped = recs.size();
for (const StateRec& r : state_list_read(state_list_path("triforce_pipeline_states.bin"))) recs.push_back(r);
@@ -1220,6 +1255,9 @@ void state_list_prepare(void (*progress)(size_t done, size_t total)) {
rec_program[i] = it->second;
}
std::atomic<size_t> next{0}, made{0}, missing{0};
// (perf push 11-11: why a program of the list has no key. 0 a header-only stage, 1 mesh on a GPU without, 2 a stage the
// seed lacks or has as another kind. Measured on OoT: all 2178 of 6150 are the last, vertex and fragment both absent)
std::atomic<size_t> why_missing[3]{};
const auto worker = [&] {
ub_thread_off(); // (a background thread: see ub_thread_off; the pipeline state list's builds raced the replay's updates)
SetThreadPriority(GetCurrentThread(), THREAD_PRIORITY_BELOW_NORMAL);
@@ -1230,7 +1268,15 @@ void state_list_prepare(void (*progress)(size_t done, size_t total)) {
std::unique_ptr<Program> p;
// (a mesh program on a GPU without mesh shaders is not made; one with a header-only stage is made in play)
if (!r.code[3] && (!r.mesh || dev.mesh_shader)) p = seed_program_of(r.code, r.mesh != 0, r.vattr, r.color_types, &want[i].key);
if (!want[i].key) missing++;
if (!want[i].key) {
missing++;
why_missing[r.code[3] ? 0 : r.mesh && !dev.mesh_shader ? 1 : 2]++;
static std::atomic<int> told{0};
if (getenv("RX_VK_SEED_CHECK") && told++ < 40)
logf("vk: state list: no key for %016llx %016llx %016llx %016llx mesh %u",
(unsigned long long)r.code[0], (unsigned long long)r.code[1], (unsigned long long)r.code[2],
(unsigned long long)r.code[3], r.mesh);
}
if (p) {
if (!p->failed) made++;
for (Stage& st : p->stages)
@@ -1311,6 +1357,10 @@ void state_list_prepare(void (*progress)(size_t done, size_t total)) {
"by their textures (%zu made), %zu noted, %zu whose program variant was not made, in %.1f s", recs.size(), shipped,
want.size(), made.load(), missing.load(), variant_of.size(), variants_made.load(), noted_n, no_variant,
std::chrono::duration<double>(std::chrono::steady_clock::now() - t0).count());
if (missing.load())
logf("vk: state list: the %zu programs without a key: %zu with a header-only stage, %zu mesh (none on this GPU), %zu with a "
"stage the game's shader list lacks (made in play)", missing.load(), why_missing[0].load(), why_missing[1].load(),
why_missing[2].load());
}
// ---- (hookshot) Stage libraries, as TotK Recomp's runtime/vk_pipeline.c: the part of a pipeline that costs is its shaders, and with
@@ -2060,8 +2110,10 @@ struct HandleHit {
uint64_t destroy_gen;
bool sampling; // (its use reads the texture's contents)
uint64_t tex_epoch; // (the texture epoch it was last touched in)
bool known = true; // (Triforce: the slot is registered: with no view, a texture whose image is not there yet)
};
TLS_OBJECT(handle_hits, std::unordered_map<uint64_t, HandleHit>)
thread_local bool view_slot_known = true; // (Triforce: the last texture_view_of's slot was a registered one)
// (hookshot) RX_VK_TRACE_TEX=<w>x<h>,...: each draw sampling a texture of that size is logged with its state, every 30th frame, 600 lines: a
// model that does not show is found by its textures, and whether the game draws it, and how, read from the line
thread_local const Texture* trace_tex_hit = nullptr;
@@ -2125,6 +2177,7 @@ VkImageView texture_view_of(uint32_t handle, VkImageViewType want, ViewUse use,
}
if (tex_out) *tex_out = it->second.tex;
if (level_out) *level_out = it->second.level;
view_slot_known = it->second.known;
return it->second.view;
}
ParLock lk;
@@ -2141,6 +2194,31 @@ VkImageView texture_view_of(uint32_t handle, VkImageViewType want, ViewUse use,
if (tex) level = image_level(tex, has_view ? view_level(&v) : 0);
}
if (!view) { // (what a shader reads as the dummy instead: once per texture slot)
{ // (Triforce, diagnostics) RX_VK_LOG_DUMMY=1: every 10 s, the slots that were resolved to the dummy in the last second
static const bool log_dummy = getenv("RX_VK_LOG_DUMMY") != nullptr;
if (log_dummy) {
struct Seen { int64_t n = 0; std::chrono::steady_clock::time_point last; char desc[200]; };
static std::unordered_map<uint32_t, Seen> seen;
static auto next = std::chrono::steady_clock::now() + std::chrono::seconds(10);
const auto now = std::chrono::steady_clock::now();
Seen& d = seen[handle & 0xFFFFF];
d.n++, d.last = now;
if (!known) snprintf(d.desc, sizeof d.desc, "not registered");
else snprintf(d.desc, sizeof d.desc, "%s; target %d format %#x %ux%ux%u levels %u gpu %#llx view %s (format %#x target %d) want %d use %d",
tex ? "no view of it" : "no image for it", t.d.target, t.d.format, t.d.width, t.d.height, t.d.depth, t.d.levels,
(unsigned long long)t.gpu, has_view ? "yes" : "no", has_view && v.set_format ? v.format : 0,
has_view && v.set_target ? v.target : -1, (int)want, (int)use);
if (now >= next) {
next = now + std::chrono::seconds(10);
int k = 0;
for (auto& [slot, e] : seen)
if (now - e.last < std::chrono::seconds(1) && k++ < 48)
logf("vk: dummy: slot %#x resolved %lld times in 10 s: %s", slot, (long long)e.n, e.desc);
logf("vk: dummy: %d slots resolved to the dummy in the last second", k);
for (auto& [slot, e] : seen) e.n = 0;
}
}
}
static std::unordered_map<uint32_t, int> told;
if (told.size() < 64 && !told[handle & 0xFFFFF]++) {
if (!known)
@@ -2153,7 +2231,8 @@ VkImageView texture_view_of(uint32_t handle, VkImageViewType want, ViewUse use,
has_view && v.set_format ? v.format : 0, has_view && v.set_target ? v.target : -1, (int)want, (int)use);
}
}
handle_hits()[key] = {view, tex, level, version, dgen, use == ViewUse::Sample || use == ViewUse::Storage, texture_epoch()};
handle_hits()[key] = {view, tex, level, version, dgen, use == ViewUse::Sample || use == ViewUse::Storage, texture_epoch(), known};
view_slot_known = known;
if (tex_out) *tex_out = tex;
if (level_out) *level_out = level;
return view;
@@ -2328,6 +2407,7 @@ bool dump_sampled_wanted(uint64_t key) {
} // (RX_VK_DUMP: the storage images the descriptors let the program write)
TLS_OBJECT(dump_inputs, std::string) // (and what it samples)
int64_t st_dummy_textures; // (textures sampled as the dummy: no image for the handle)
int64_t st_dummy_known; // (Triforce: of those, registered textures: their image not made or not uploaded yet)
void dump_input(char kind, uint32_t index, uint32_t handle, const Texture* tex) {
char b[160];
@@ -2877,7 +2957,7 @@ void build_descriptors(const NvnState& s, const Program& p) {
tex ? tex->width : 0, tex ? tex->height : 0, tex ? tex->layers : 0, tex ? (void*)tex->image : nullptr,
(void*)view, view ? "" : " (dummy)");
if (view && tex && tex->gpu_written) draw_sampled().push_back(tex);
if (!view) view = dummies.view[(int)vt], st_dummy_textures += handle != 0;
if (!view) view = dummies.view[(int)vt], st_dummy_textures += handle != 0, st_dummy_known += handle != 0 && view_slot_known;
dw().imgs.push_back({sampler_for((int)(handle >> 20), d.is_depth), view, VK_IMAGE_LAYOUT_GENERAL});
}
w.pImageInfo = &dw().imgs[first];
@@ -2942,6 +3022,10 @@ TLS_OBJECT(pushed, std::array<Pushed, 2>) // [graphics, compute]
/// to it)
VkPipelineCache shared_pipeline_cache() { return pcache; }
void draw_skip_report(double frames) { skip_report_impl(frames), fine_report(frames); }
/// (Triforce) how many texture bindings have sampled the dummy so far (a texture with no image yet: left for a later frame, not
/// made, evicted): RX_VK_FRAME_TRACE's last column, the measure of materials showing flat while their textures come in
/// Registered textures only: OoT binds one handle of no registered slot (0x3126f) in ~18 places a frame, always the dummy
int64_t draw_dummy_textures() { return st_dummy_known; }
void descriptors_disturbed() {
pushed()[0].cmd = pushed()[1].cmd = VK_NULL_HANDLE;
}
@@ -7102,6 +7186,15 @@ bool prepare_run(void (*progress)(size_t done, size_t total), bool background) {
logf("vk: the shader list was hashed (%.0f MB in %.0f ms): written since the last check, or too close to it to go by its time",
prepared_for.seed_size / 1048576.0, hash_ms);
if (why == PrepareWhy::None && !getenv("RX_VK_PREPARE_AGAIN")) {
// (perf push 11-11: a shipped state list the notes were not made from, in a folder prepared already: its pipelines are
// noted now, for the precompile that follows, as a prepare would have. See state_shipped_id)
if (!background && state_shipped_new()) {
logf("vk: the shipped pipeline list is not the one this folder's notes were made from: its pipelines are noted now");
if (seed_open()) {
state_list_prepare(nullptr);
seed_close();
}
}
if (getenv("RX_VK_SEED_CHECK")) seed_open();
if (prepared_for.warmed == warmed || background) return false;
// (translated, and the driver's part is another driver's, or of before the libraries were wanted: that part alone, no
@@ -7279,6 +7372,47 @@ void prepare_background() {
}).detach();
}
// ---- (perf push 11-11) The precompile on a PC with little RAM. Every pipeline built here is kept for the session, with its
// program: measured at a fresh Market (RTX 4070 SUPER), the process holds 12.9 GB in RAM without the precompile, 13.4 GB with
// 4704 graphics and 275 compute pipelines, 15.7 GB with 8550 and 1650; after a tour of ten areas 18.6 against 20.6 GB. What
// the precompile is for is the compile, which the caches keep (ours, mkw_vk_pipelines.bin, and the driver's): with them warm,
// the same runs without it built 1462 and 4046 pipelines in play in 357 and 1639 ms in all (0.2-0.4 ms each), with as many
// frames over 40, 100 and 250 ms as with it. So where RAM is short (under 20 GB in the PC: the 16 GB ones, whose logs show the
// system taking pages back all through play) a pipeline is built before the game once, on the first start that has its note,
// and that start records it (triforce_pipelines.bin.built, for this translator and driver); later starts leave it to its first
// draw, and neither it nor its program is in RAM until then.
// The price, same tour: of ~4050 pipelines then built in play, ten or twelve submissions spend 10-65 ms on them (some in frames
// that were slow anyway, an area's arrival; three or four make a 40 ms frame of their own), and one in two tours had a
// pipeline the caches missed (508 ms). So it is not the default: RX_VK_PRECOMPILE_KEEP=auto: this rule where the PC has under
// 20 GB of RAM; =0: on any PC; unset or 1: every start builds and keeps all, as before ----
constexpr uint64_t kBuiltComputes = 0xC04D50555445D001ull; // (in the record: the compute programs' pipelines were built)
bool precompile_keeps_all() {
const char* e = getenv("RX_VK_PRECOMPILE_KEEP");
if (!e) return true;
if (strcmp(e, "auto")) return atoi(e) != 0;
#ifdef _WIN32
MEMORYSTATUSEX ms{};
ms.dwLength = sizeof ms;
if (GlobalMemoryStatusEx(&ms)) return ms.ullTotalPhys >= (20ull << 30);
#endif
return true;
}
std::unordered_set<uint64_t> built_read(uint64_t stamp) {
std::unordered_set<uint64_t> keys;
if (FILE* f = fopen((pipe_notes_path() + ".built").c_str(), "rb")) {
uint64_t v = 0;
if (fread(&v, 8, 1, f) == 1 && v == stamp)
while (fread(&v, 8, 1, f) == 1) keys.insert(v);
fclose(f);
}
return keys;
}
void built_write(uint64_t stamp, const std::unordered_set<uint64_t>& keys) {
std::vector<uint64_t> all{stamp};
all.insert(all.end(), keys.begin(), keys.end());
if (FILE* f = fopen((pipe_notes_path() + ".built").c_str(), "wb")) fwrite(all.data(), 8, all.size(), f), fclose(f);
}
void precompile_run(void (*progress)(size_t done, size_t total)) {
const auto t0 = std::chrono::steady_clock::now();
// (a start that dies in here would die in here every time, the notes being what it builds from: a player saw the console reach
@@ -7312,12 +7446,29 @@ void precompile_run(void (*progress)(size_t done, size_t total)) {
std::lock_guard lk(state_mu);
if (!state_file) state_open_locked();
}
// (little RAM, see precompile_keeps_all: the pipelines an earlier start built for these caches are left to their first draw)
const bool light = !precompile_keeps_all();
const uint64_t built_stamp = warmed_stamp(gpl_wanted());
std::unordered_set<uint64_t> built;
const size_t noted_n = notes.size();
bool computes_built = false;
if (light) {
built = built_read(built_stamp);
computes_built = built.count(kBuiltComputes) != 0;
std::erase_if(notes, [&](const std::unique_ptr<PipeNote>& n) { return built.count(n->key) != 0; });
logf("vk: precompile: RAM spared (RX_VK_PRECOMPILE_KEEP=1: as before): %zu of %zu noted pipelines were built by "
"an earlier start and are left to their first draw%s", noted_n - notes.size(), noted_n,
computes_built ? ", the compute programs' too" : "");
}
std::vector<uint64_t> building; // (their keys, for the record once they are built)
if (light)
for (const auto& n : notes) building.push_back(n->key);
std::unordered_set<uint64_t> wanted;
for (const auto& n : notes) wanted.insert(n->prog);
// (the compute programs: those play dispatched (triforce_computes.bin), or every kept one when there is no list yet)
const char* which = getenv("RX_VK_PRECOMPILE_COMPUTES");
const std::unordered_set<uint64_t> used = computes_read();
const bool all_computes = which ? !strcmp(which, "all") : used.empty();
const std::unordered_set<uint64_t> used = computes_built ? std::unordered_set<uint64_t>{} : computes_read();
const bool all_computes = !computes_built && (which ? !strcmp(which, "all") : used.empty());
if (!all_computes && !(which && !strcmp(which, "none"))) wanted.insert(used.begin(), used.end());
std::vector<Program*> computes, fallbacks;
const size_t programs_n = program_cache_load_some(wanted, [&](Program& p) {
@@ -7431,6 +7582,13 @@ void precompile_run(void (*progress)(size_t done, size_t total)) {
} else {
for (std::thread& t : threads) t.join();
delete pre;
// (little RAM: what this start built is on disk before the record says so, and the next start leaves it out)
if (light && !prepare_stopped() && (!building.empty() || !computes_built)) {
built.insert(building.begin(), building.end());
built.insert(kBuiltComputes);
pipeline_cache_save(true);
built_write(built_stamp, built);
}
}
logf("vk: precompile: %zu programs, %zu graphics and %zu compute pipelines, %zu mesh fallback programs in %.0f ms (%.0f ms "
"of it the caches opened, the notes and programs read)", programs_n, graphics_n, computes_n, fallbacks_n,

View File

@@ -65,6 +65,7 @@ void dry_marks_begin(); // (at the start of each dry pass: the last targets it
void draw_stats(int64_t* draws, int64_t* skipped, int64_t* dispatches, int64_t* pipelines);
void draw_stats2(int64_t* passes, int64_t* mesh, double* pipeline_ms);
void draw_stats3(int64_t* pending_skips, int64_t* waits, double* wait_ms);
int64_t draw_dummy_textures(); // (Triforce: texture bindings that sampled the dummy so far: RX_VK_FRAME_TRACE)
/// (hookshot) pipelines linked from stage libraries in the draw, and the libraries asked of the driver for them: had, and cold;
/// of the cold, those a worker has built since, and the workers' time in them. False: no stage libraries (RX_VK_GPL)
bool draw_stats4(int64_t* links, double* link_ms, int64_t* lib_hits, int64_t* lib_cold, int64_t* lib_built, double* lib_build_ms);

View File

@@ -210,6 +210,9 @@ void memory_drop_shadow(uint64_t guest, uint64_t size); // (a write the captur
/// when any write-back ever did
uint64_t memory_writeback_value(uint64_t guest, uint64_t size, bool* ever);
void memory_note_readback(uint64_t guest, uint64_t size); // (PR 430) a CPU read of GPU results: see memory_rb_refresh
/// (Triforce) around the write of an NVN event's word by a command buffer's signal: its write-back is not one the captures of
/// its page wait for (vk_memory.cpp). This thread's bindings while on
void memory_bind_event_write(bool on);
bool memory_rb_refresh(VkCommandBuffer up);
void memory_rb_recording_begun(); // (hookshot) at a recording's start, on its thread: the submission it joins (memory_rb_refresh)
void memory_rb_segments_begin(uint64_t base, uint64_t n); // (and a parallel replay's segments: their order, base on)
@@ -219,11 +222,12 @@ void memory_note_gpu_write(uint64_t gpu, uint64_t size);
uint64_t vk_timeline_completed();
void vk_timeline_wait(uint64_t v);
uint32_t memory_generation();
uintptr_t memory_stamp_who(uint64_t guest, uint32_t* gen, uintptr_t* anchor); // (RX_VK_STAMP_SRC: who wrote that page last)
/// 1: a page of [guest, guest + size) was written after generation gen; 0: none was
bool memory_written_since(uint64_t guest, uint64_t size, uint32_t gen);
void memory_stamp(uint64_t guest, uint64_t size); // a write the watch does not see (GPU writes, host writes)
/// The page runs of [guest, guest + size) written after gen, as byte ranges from guest (appended to out)
void memory_written_runs(uint64_t guest, uint64_t size, uint32_t gen, std::vector<std::pair<uint64_t, uint64_t>>& out);
void memory_written_runs(uint64_t guest, uint64_t size, uint32_t gen, std::vector<std::pair<uint64_t, uint64_t>>& out, bool tex = false);
uint64_t memory_pages_collected(); // (statistics)
/// An NVN format as Vulkan stores it (vk_formats.cpp): ASTC is decoded to RGBA8, block_w x block_h
@@ -250,6 +254,7 @@ struct ImageMove {
};
/// The free space inside the textures' pool's blocks, and what the blocks hold in all; false without the pool
bool textures_pool_usage(uint64_t* block_bytes, uint64_t* free_bytes);
bool target_floor_kept(); // (the render targets' pool keeps a floor of blocks for the session: RX_VK_TARGET_FLOOR_MB)
bool image_compact_begin(uint64_t max_bytes_a_pass, uint32_t max_moves_a_pass);
bool image_compact_active();
/// A pass begun: the moves VMA asks for. False: nothing left to move, the compaction is over (then *freed_bytes and
@@ -271,6 +276,19 @@ bool create_buffer(uint64_t size, VkBufferUsageFlags usage, bool host_visible, V
void destroy_buffer(VkBuffer buffer, VmaAllocation allocation);
/// A buffer in VRAM only (within the driver's budget), or false
bool create_device_buffer(uint64_t size, VkBufferUsageFlags usage, VkBuffer& buffer, VmaAllocation& allocation, bool dedicated = false);
/// (Triforce) device memory for a piece of a sparse buffer, and its release (vk_vma.cpp)
bool vma_piece_alloc(uint64_t size, uint64_t alignment, uint32_t type_bits, VmaAllocation& allocation, VkDeviceMemory& memory,
uint64_t& offset);
void vma_piece_free(VmaAllocation allocation);
/// (Triforce) the mirrors' pieces made resident in this recording, bound before it is submitted (vk_memory.cpp)
void memory_mirror_binds_flush();
bool memory_mirror_bind_wait(VkSemaphoreSubmitInfo* wait); // (and the wait a queue submission makes for those binds)
/// (Triforce, vk_memory.cpp) The memory handling played on NVIDIA cards only (mirrors in pieces, compacted textures): on there,
/// and on trial on a card of another make until triforce_memtrial.txt says it passed or failed. tick: at the 600th presents;
/// device_lost: from device_lost()
bool memory_trial_on();
void memory_trial_tick();
void memory_trial_device_lost();
/// Device-local memory this process uses and what the driver allows it now (VK_EXT_memory_budget), summed over the
/// device-local heaps; false when unknown
bool vram_usage(uint64_t* used, uint64_t* budget, uint64_t* block_free = nullptr);

View File

@@ -885,6 +885,7 @@ void prepare_with_screen() {
std::thread worker([] {
const bool update = prepare_is_update(); // (an update, RX_VK_PREPARE_BACKGROUND=1: what the notes name now, the rest in play)
const bool prepared = !update && prepare_run(prep_progress, false);
if (update) state_list_run(); // (as the start without a screen does, below: the state lists' pipelines before the precompile)
if (prepared && !prepare_stopped()) { // (a first start only: the pipelines the notes predict, into the driver's cache)
prep_done.store(0), prep_total.store(0);
prep_stage.store(1);

File diff suppressed because it is too large Load Diff

View File

@@ -180,6 +180,8 @@ void queue_flush_locked() { // (dev.present_mu held)
si.pCommandBufferInfos = pend.cbs.data();
si.signalSemaphoreInfoCount = n;
si.pSignalSemaphoreInfos = sig;
VkSemaphoreSubmitInfo bind_wait; // (Triforce: after the memory of the mirrors' new pieces is bound, vk_memory.cpp)
if (memory_mirror_bind_wait(&bind_wait)) si.waitSemaphoreInfoCount = 1, si.pWaitSemaphoreInfos = &bind_wait;
PROF(PROF_QSUBMIT);
const VkResult res = vkQueueSubmit2(dev.queue, 1, &si, VK_NULL_HANDLE);
static int failures;
@@ -355,6 +357,8 @@ void submit_recording(Recording& r, bool signal_value = true, uint64_t signal_at
return;
}
PROF(PROF_QSUBMIT);
VkSemaphoreSubmitInfo bind_wait; // (Triforce: after the memory of the mirrors' new pieces is bound, vk_memory.cpp)
if (memory_mirror_bind_wait(&bind_wait)) si.waitSemaphoreInfoCount = 1, si.pWaitSemaphoreInfos = &bind_wait;
const VkResult res = vkQueueSubmit2(dev.queue, 1, &si, r.fence);
r.submitted = res == VK_SUCCESS;
static int failures;
@@ -732,7 +736,9 @@ void replay(VkCommandBuffer& cmd, NvnState& s, const uint8_t* p, const uint8_t*
end_pass(cmd);
GprofSpan gp(cmd, "event signal (barrier, write, barrier)");
full_barrier(cmd);
memory_bind_event_write(true); // (Triforce: not a write-back the captures of its page wait for)
gpu_write(cmd, e.gpu, &v, 4);
memory_bind_event_write(false);
break;
}
case NOP_WAIT_EVENT: { // (a wait for an event another queue's work signals: submit_job orders them)
@@ -1282,10 +1288,76 @@ void maybe_split(VkCommandBuffer& cmd) {
// (hookshot) One cause of those is gone: the dry pass did not mark the targets of a clear (NOP_CLEAR_COLOR in replay()). With
// that, Castle Town's market and eight other places showed no black frame on 3 to 8 threads; the Kakariko walk was not run
// again, so it stays off. On a 9950X3D the replay's wall time in the market goes from 10.9 ms a frame to 7.75 on 3 threads.
int par_threads() {
static const int n = std::clamp(getenv("RX_VK_PARALLEL") ? atoi(getenv("RX_VK_PARALLEL")) : 0, 0, 8);
// (Triforce) RX_VK_PARALLEL_AUTO=<n> (2 to 8): the parallel replay on <n> threads where it is the faster of the two, measured
// as the game runs. On three threads it is faster where the replay is the frame's limit and slower where the GPU is: at the
// Castle Town save 67.1 fps against 58.8 (RTX 4070 SUPER, 2560 x 1440, windowed), on a tour of Hyrule Field, Kakariko and
// Lake Hylia 59.5 and 61.9 against 64.8 and 66.7 (its recordings done sooner, the game's thread only waits longer for the
// GPU, and the dry pass and the hand-over are not free); the flicker counts of the frame log were the same either way.
// par_tune_present tries the other way for a moment every so often and keeps the one whose frames are shorter.
// RX_VK_PARALLEL=<n> still means always, as before.
// EXPERIMENTAL, off unless asked for: its first runs (2026-11-11) gained nothing, 54.7 fps against 58.9 standing in Castle Town
// and 60.4 against 61.7 on the tour. The frame time of a scene with a crowd moves by 2 ms within seconds (one trial read 18.59
// and then 16.44 ms for the same way), more than the difference being measured, so the trials decide on noise
int par_threads_max() {
static const int n = [] {
if (const char* e = getenv("RX_VK_PARALLEL")) return std::clamp(atoi(e), 0, 8);
if (const char* e = getenv("RX_VK_PARALLEL_AUTO")) return std::clamp(atoi(e), 0, 8);
return 0;
}();
return n;
}
bool par_auto() {
static const bool v = !getenv("RX_VK_PARALLEL") && par_threads_max() >= 2;
return v;
}
bool par_auto_on = false; // (the replay thread's: the parallel replay is the one in use now)
int par_threads() {
const int n = par_threads_max();
return par_auto() && !par_auto_on ? 0 : n;
}
/// At each present (the replay thread): the tuner. Every kEvery presents it times kTake frames as they are, kTake the other way
/// (after kSettle to settle) and kTake as they were again, and changes over when the other way's mean frame was at least 4%
/// shorter than both: a scene that changed under the trial does not pass for a gain. Frames over 60 ms (a load) are left out
void par_tune_present() {
if (!par_auto()) return;
constexpr int kEvery = 900, kTake = 80, kSettle = 24;
static auto last = std::chrono::steady_clock::now();
const auto now = std::chrono::steady_clock::now();
const double ms = std::chrono::duration<double, std::milli>(now - last).count();
last = now;
static int phase = 0, n = 0, cnt[3];
static double sum[3];
static int64_t frames = 0, next_trial = 600, st_trials = 0, st_changes = 0;
frames++;
if (phase == 0) {
if (frames < next_trial) return;
phase = 1, n = 0;
sum[0] = sum[1] = sum[2] = 0, cnt[0] = cnt[1] = cnt[2] = 0;
return;
}
const int k = phase - 1;
n++;
if (n > (phase == 1 ? 0 : kSettle) && ms <= 60.0) sum[k] += ms, cnt[k]++;
if (n < (phase == 1 ? kTake : kSettle + kTake)) return;
n = 0;
if (phase == 1) {
par_auto_on = !par_auto_on, phase = 2;
} else if (phase == 2) {
par_auto_on = !par_auto_on, phase = 3;
} else {
phase = 0, st_trials++;
const bool ok = cnt[0] >= kTake * 3 / 4 && cnt[1] >= kTake * 3 / 4 && cnt[2] >= kTake * 3 / 4;
const double a = cnt[0] ? sum[0] / cnt[0] : 0, b = cnt[1] ? sum[1] / cnt[1] : 0, c = cnt[2] ? sum[2] / cnt[2] : 0;
const bool better = ok && b < std::min(a, c) * 0.96;
if (better) par_auto_on = !par_auto_on, st_changes++;
next_trial = frames + (better ? kEvery : kEvery * 2);
if (better || st_trials <= 8 || st_trials % 20 == 0)
logf("vk: parallel replay, trial %lld: %s %.2f ms a frame, the other way %.2f, %s again %.2f: %s (%lld changes so far)",
(long long)st_trials, par_auto_on != better ? "parallel" : "serial", a, b, par_auto_on != better ? "parallel" : "serial", c,
better ? (par_auto_on ? "parallel from now on" : "serial from now on") : (ok ? "as it was" : "not measured (a load)"),
(long long)st_changes);
}
}
struct ParPool {
std::mutex mu;
std::condition_variable cv, done;
@@ -1531,6 +1603,8 @@ const uint8_t* replay_handle(VkCommandBuffer& cmd, const uint8_t* h, bool may_st
si.pCommandBufferInfos = cb;
std::lock_guard lk(dev.present_mu);
queue_flush_locked();
VkSemaphoreSubmitInfo bind_wait; // (Triforce: after the memory of the mirrors' new pieces is bound, vk_memory.cpp)
if (memory_mirror_bind_wait(&bind_wait)) si.waitSemaphoreInfoCount = 1, si.pWaitSemaphoreInfos = &bind_wait;
const VkResult res = vkQueueSubmit2(dev.queue, 1, &si, j->r->fence);
j->r->submitted = res == VK_SUCCESS;
if (res != VK_SUCCESS) logf("vk: segment submit %llu failed (%d)", (unsigned long long)j->r->value, (int)res);
@@ -2559,6 +2633,7 @@ uint32_t crc32(const uint8_t* p, size_t n, uint32_t c = 0xFFFFFFFFu) {
/// closed (issue 1275: captures kept 25 -> 6527, staging 256 -> 1408 MB). Now the report once (a thread that sees the loss
/// too waits for it, instead of ending the process before it is written), then the process ends as replay_fatal does
[[noreturn]] void device_lost() {
memory_trial_device_lost(); // (Triforce: with the memory handling on trial on this card: off from the next start)
static std::once_flag once;
std::call_once(once, checkpoints_report);
replay_fatal("the GPU device was lost (VK_ERROR_DEVICE_LOST): the driver stopped or reset the GPU, and nothing sent "
@@ -2837,7 +2912,8 @@ void status_tick() {
std::string dir(exe, n);
FILE* f = fopen((dir.substr(0, dir.find_last_of("\\/") + 1) + "triforce_frames.csv").c_str(), "w");
if (f) fprintf(f, "frame,ms,pipe_waits,pipe_wait_ms,left_out,translate_ms,uploads,upload_mb,textures,memsync_ms,pipelines_ms,"
"uploads_ms,programs_ms,slot_wait_ms,gpu_behind_ms,draw_ms,desc_ms,cmds_ms,tex_lookup_ms,textures_gone\n");
"uploads_ms,programs_ms,slot_wait_ms,gpu_behind_ms,draw_ms,desc_ms,cmds_ms,tex_lookup_ms,textures_gone,"
"dummies,draws\n");
return f;
}();
if (trace && last_ns) {
@@ -2851,12 +2927,17 @@ void status_tick() {
const auto dp = [](int i) { return (prof_ns[i] - l_prof[i]) / 1e6; };
static uint64_t l_gone;
const uint64_t gone = texture_destroy_generation();
fprintf(trace, "%llu,%.2f,%lld,%.2f,%lld,%.1f,%lld,%.2f,%zu,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%llu\n",
static int64_t l_dummy, l_draws; // (Triforce: the bindings that sampled the dummy, and the draws, this frame)
const int64_t dummy = draw_dummy_textures();
int64_t draws_now = 0, sk_now = 0, di_now = 0, pl_now = 0;
draw_stats(&draws_now, &sk_now, &di_now, &pl_now);
fprintf(trace, "%llu,%.2f,%lld,%.2f,%lld,%.1f,%lld,%.2f,%zu,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%llu,%lld,%lld\n",
(unsigned long long)frames, (now_ns - last_ns) / 1e6, (long long)(pw - l_pw), pwms - l_pwms, (long long)(ps - l_ps),
tr - l_tr, (long long)(up - l_up), (upb - l_upb) / 1048576.0, texture_count(), dp(PROF_MEMSYNC), dp(PROF_PIPE),
dp(PROF_UPLOAD), dp(PROF_PROGRAM), dp(PROF_SLOTWAIT), (st_wait_ns - l_wait) / 1e6, dp(PROF_DRAW), dp(PROF_DESC),
dp(PROF_CMDS), dp(PROF_TEXGET), (unsigned long long)(gone - l_gone));
l_gone = gone;
dp(PROF_CMDS), dp(PROF_TEXGET), (unsigned long long)(gone - l_gone), (long long)(dummy - l_dummy),
(long long)(draws_now - l_draws));
l_gone = gone, l_dummy = dummy, l_draws = draws_now;
if (frames % 60 == 0) fflush(trace);
l_ps = ps, l_pw = pw, l_pwms = pwms, l_up = up, l_upb = upb, l_tr = tr, l_wait = st_wait_ns;
std::memcpy(l_prof, prof_ns, sizeof l_prof);
@@ -2964,6 +3045,7 @@ void do_present(const nvn_tex* t, const int crop[4]) {
full_barrier(r.cmd);
presents++;
presents_now = presents;
par_tune_present(); // (Triforce: RX_VK_PARALLEL_AUTO)
wide_sides_tick();
if (presents == 1) hang_watch_start();
{ // (EvilBoyce, diagnostics) RX_VK_WATCH_CPU=<present>:<gpu address>[,...] (hex addresses): from that present on, the
@@ -4178,7 +4260,7 @@ bool replay_start() {
VkSemaphoreCreateInfo si{VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO};
vkCreateSemaphore(dev.device, &si, nullptr, &f.acquired);
}
if (par_threads() >= 2 && !init_pools_make(pi)) return false; // (last: nothing after it fails and leaves them made)
if (par_threads_max() >= 2 && !init_pools_make(pi)) return false; // (last: nothing after it fails and leaves them made)
logf("vk: replay ready (%s; push descriptors %s (%u), extended dynamic state 3 %s, custom border colours %s)",
dev.offscreen ? "offscreen" : "window", dev.push_descriptor ? "yes" : "no", dev.max_push_descriptors,
dev.eds3_blend ? "yes" : "no", dev.custom_border_color ? "yes" : "no");

View File

@@ -566,11 +566,9 @@ Shader::Profile make_profile() {
p.support_scaled_attributes = false; // (*_UI2F / *_I2F read as integers, converted in the shader: R32 has no scaled format)
p.support_multi_viewport = true;
p.support_sampled_image_array_nonuniform_indexing = true;
// (hookshot) a graphics stage the GPU cannot pin to 32 lanes runs at its default: 64 on AMD, where only compute is pinned
// (vk_main.cpp). The translator then keeps each 32-lane guest warp to its half of the host's, as Eden does on AMD; the
// long path is right at 32 lanes too. The ballots and shuffles of the game's fragment shaders ran across both halves
p.warp_size_potentially_larger_than_guest =
dev.subgroup_size > 32 && (dev.sg32_stages & VK_SHADER_STAGE_ALL_GRAPHICS) != VK_SHADER_STAGE_ALL_GRAPHICS;
// Unpinned SPIR-V 1.6 stages can use any supported subgroup size, even when the default is 32.
// The stage-specific profiles below keep the short path only where pin_subgroup actually pins 32 lanes.
p.warp_size_potentially_larger_than_guest = dev.subgroup_max > 32;
// (RX_VK_WARP64=0 or 1, a test: the graphics stages' long warp path off or on, whatever the GPU; in the cache's stamp)
if (const char* e = getenv("RX_VK_WARP64")) p.warp_size_potentially_larger_than_guest = e[0] == '1';
p.lower_left_origin_mode = false;
@@ -1225,9 +1223,11 @@ uint64_t disk_stamp() {
const uint32_t mesh = (dev.mesh_shader ? 1u : 0u) | (dev.mesh_fallback ? 2u : 0u) | (dev.task_mesh_fallback ? 4u : 0u);
add(&mesh, sizeof mesh);
// (and what of the GPU the translator is told (make_profile): its warps' lanes and the driver's shuffle workaround, and
// which stages it pins to 32 lanes, which keep the short path: profile_graphics)
// which stages it pins to 32 lanes, which keep the short path: profile_graphics. An explicit warp override bypasses
// the pinned graphics profile even if the base profile already has the requested width)
const uint32_t lanes[] = {profile().warp_size_potentially_larger_than_guest ? 1u : 0u, profile().max_subgroup_size,
profile().has_broken_spirv_fragment_shuffle ? 1u : 0u, (uint32_t)dev.sg32_stages};
profile().has_broken_spirv_fragment_shuffle ? 1u : 0u, (uint32_t)dev.sg32_stages,
getenv("RX_VK_WARP64") ? 1u : 0u};
add(lanes, sizeof lanes);
// (the records keep Shader::Info as it lies in memory: a cache written by another C++ library (the MSVC build's, then the
// llvm-mingw one's libc++) has other layouts, and reading it crashed at start. The library and the sizes, so it starts again)

View File

@@ -241,7 +241,21 @@ std::unordered_map<uint64_t, std::vector<Pooled>> target_pool; // by pool_key_
uint64_t pool_bytes, pool_n, st_pool_reused, st_pool_made, st_pool_destroyed;
bool pool_pressed; // (VRAM near what the driver allows: texture_frame's budget)
bool compact_on() { // (hookshot, RX_VK_COMPACT=1: the textures in a pool of their own, compacted: compact_step)
static const bool v = getenv("RX_VK_COMPACT") && atoi(getenv("RX_VK_COMPACT")) != 0;
// (Triforce) on by default on NVIDIA cards (RX_VK_COMPACT=0: off; =1: on, any card). Measured there: an RTX 4070 SUPER
// played with it since 2026-11-10 (a 584,000-present session, 132 compactions, no fault; 85 -> 80.6 fps after a six-area
// tour against 81 -> 64 without), and with the allowance held to 7300 MB (RX_VK_TEST_ALLOWED_MB, an 8 GB card's) a tour of
// Hyrule Field, Kakariko, Lake Hylia and Kokiri peaked at 7165 MB of VRAM against 8565 without (over the allowance in 2 of
// 12 reports against 11) at 69 fps against ~64. On an 8 GB card what is over the allowance is paged to system memory, and
// the textures and render targets evicted to get back under it are made and uploaded again, 58 to 190 ms a frame (an
// RX 5700 XT's and an RTX 3060's logs, 2026-11-11). Not measured on AMD or Intel: as it was there until someone does
// (Triforce, later that day) and on trial on the other makes' cards: memory_trial_on (vk_memory.cpp), which turns it off by
// itself on a PC where a session with it loses the GPU or two end in a crash
static const bool v = [] {
if (const char* e = getenv("RX_VK_COMPACT")) return atoi(e) != 0;
const bool on = memory_trial_on();
if (on) logf("vk: the textures' VRAM is compacted (RX_VK_COMPACT=0: off)");
return on;
}();
return v;
}
uint64_t pool_cap() {
@@ -284,8 +298,11 @@ void pool_put(Dead& d) {
/// presents: destroyed, at most 16 a present (64 while VRAM is short). Returns how many
size_t pool_trim() {
std::lock_guard lk(pool_mu);
const uint64_t cap = pool_pressed ? 0 : pool_cap();
const size_t most = pool_pressed ? 64 : 16;
// (Triforce, 11-11: with a floor of blocks kept (target_floor_kept) the pool is not emptied while VRAM is short: what it gave
// back went into blocks the floor keeps, the driver's count did not move, and every target of the next frames was made new)
const bool give_back = pool_pressed && !target_floor_kept();
const uint64_t cap = give_back ? 0 : pool_cap();
const size_t most = give_back ? 64 : 16;
size_t n = 0;
for (auto it = target_pool.begin(); it != target_pool.end() && n < most;) {
auto& v = it->second;
@@ -1297,6 +1314,7 @@ struct UpSlow { int64_t ns; uint32_t w, h, layers, nvn; int route; uint64_t byte
std::vector<UpSlow> up_slowest;
std::mutex up_mu; // (uploads may come from a parallel replay's threads)
std::atomic<int64_t> frame_upload_ns{0}; // (this frame's time in upload(), every thread's)
std::atomic<int> frame_astc_admitted{0}; // (Triforce: this frame's new ASTC textures asked about so far: upload_defer)
std::atomic<int64_t> st_deferred{0}, st_deferred_frame{0};
void upload_frame_report() {
std::lock_guard lk(up_mu);
@@ -1589,11 +1607,13 @@ bool upload_inner(Texture& tex, const Ranges* dirty) {
/// What the CPU wrote of a texture's storage since gen, as byte ranges from its start. A texture in a virtual
/// (sparse) pool is looked at page by page through its mapping, and is all dirty when its mapping changed.
bool texture_dirty(Texture& tex, uint32_t gen, Ranges& out) {
/// same_ok (Triforce): a capture's stamp of a page with the bytes of its last copy is no write (vk_memory.cpp page_gen_tex): only
/// for an uploaded texture's "written since my upload?"
bool texture_dirty(Texture& tex, uint32_t gen, Ranges& out, bool same_ok = false) {
out.clear();
constexpr uint64_t kVirtual = 1ull << 40, kVPage = 1ull << 16;
if (tex.gpu < kVirtual) {
memory_written_runs(tex.guest, tex.guest_size, gen, out);
memory_written_runs(tex.guest, tex.guest_size, gen, out, same_ok);
return !out.empty();
}
// (the page walk only when the GPU mappings changed: the terrain's sparse arrays span over 1 GB, 64 KB pages,
@@ -1661,7 +1681,7 @@ bool texture_dirty(Texture& tex, uint32_t gen, Ranges& out) {
const size_t remaps = out.size();
for (const Texture::Run& r : tex.runs) {
in_run.clear();
memory_written_runs(r.guest, r.n, gen, in_run);
memory_written_runs(r.guest, r.n, gen, in_run, same_ok);
for (const auto& [lo, hi] : in_run) {
if (out.size() > remaps && out.back().second == r.off + lo) out.back().second = r.off + hi;
else out.push_back({r.off + lo, r.off + hi});
@@ -1693,6 +1713,14 @@ void destroy_later(Texture& tex) {
}
d.memory = tex.sparse_chunks;
dead.push_back(std::move(d));
if (tex.twin3d) { // (Triforce: its one-slice 3D copy goes with it)
Dead t{tex.last_value, tex.twin3d, tex.twin3d_alloc, {}, {}};
for (auto& kv : tex.twin3d_views)
if (kv.second) t.views.push_back(kv.second);
dead.push_back(std::move(t));
tex.twin3d = VK_NULL_HANDLE, tex.twin3d_alloc = nullptr;
tex.twin3d_views.clear();
}
total_bytes -= std::min(total_bytes, tex.bytes + tex.pad);
}
@@ -2175,6 +2203,7 @@ VkCommandBuffer upload_command_buffer() {
}
bool texture_uploads_recorded() {
if (memory_rb_refresh(up_cmd)) up_used = true; // (PR 430: readback pages from guest memory, last of the uploads)
memory_mirror_binds_flush(); // (Triforce: the memory of the mirrors' new pieces, bound before this recording is submitted)
memory_table_flush(); // (vk_memory.cpp: the page table's changes, after this recording's mirror copies)
if (up_used) full_barrier(up_cmd); // (before the recording that reads them)
return up_used;
@@ -2315,6 +2344,13 @@ namespace {
bool upload_defer(const Texture& tex) {
static const int64_t budget_ns = (int64_t)(getenv("RX_VK_UPLOAD_BUDGET_MS") ? atof(getenv("RX_VK_UPLOAD_BUDGET_MS")) : 40.0) * 1000000;
if (!budget_ns || !upload_defer_ok || tex.uploaded || !tex.astc || tex.gpu_written) return false;
// (Triforce) and the first RX_VK_UPLOAD_MIN_ASTC of a frame (24) are never left for later. The budget counts every upload of
// the frame, and a frame whose other uploads took it all left every new ASTC texture on the placeholder: on a card at its
// VRAM allowance the render targets evicted and made again are uploaded from memory on the CPU, 58 to 190 ms a frame for
// frames on end (an RX 5700 XT's log, 2026-11-11: 8,586 textures left for later in one session), and the materials stayed
// flat white or blue for as long as that went on. An ASTC upload is decoded on the GPU: its part of such a frame is small
static const int min_astc = getenv("RX_VK_UPLOAD_MIN_ASTC") ? std::max(0, atoi(getenv("RX_VK_UPLOAD_MIN_ASTC"))) : 24;
if (frame_astc_admitted.fetch_add(1, std::memory_order_relaxed) < min_astc) return false;
if (frame_upload_ns.load(std::memory_order_relaxed) < budget_ns) return false;
st_deferred.fetch_add(1, std::memory_order_relaxed), st_deferred_frame.fetch_add(1, std::memory_order_relaxed);
return true;
@@ -2402,7 +2438,7 @@ Texture* texture_touch(Texture* tex, bool for_sampling) {
"%llu)", tex->id, (unsigned long long)tex->gpu, tex->width, tex->height, tex->layers, tex->nvn_format,
(unsigned long long)frame_no, (unsigned long long)tex->last_written);
}
if (tex->uploaded && !gpu_dirty && !texture_dirty(*tex, since, dirty)) return tex;
if (tex->uploaded && !gpu_dirty && !texture_dirty(*tex, since, dirty, !tex->gpu_written)) return tex;
const uint32_t gen = memory_generation();
// (storage render targets took last, which the CPU has not written since: nothing to upload, the memory holds
// nothing the GPU's images did not replace. The game's transient heaps move with the dynamic resolution, and a
@@ -2488,6 +2524,35 @@ Texture* texture_touch(Texture* tex, bool for_sampling) {
// (hookshot) a BC3 image whose last upload failed waits before the next try: its upload stages the whole ASTC source (some
// 200 MB for a terrain array), not to be repeated every submission while the memory it needs is not there
if (tex->bc && !tex->uploaded && tex->bc_retry_frame > frame_no) return tex;
// (Triforce, RX_VK_STAMP_SRC=1, a diagnostic: a texture of 1 MB or more uploaded again: who wrote its first dirty page (the
// write watch 1, a capture 2, else a caller of memory_stamp, placed by memory_stamp's own address), and whether its memory
// holds other bytes than at its last upload at all. 400 lines)
static const bool stamp_src = getenv("RX_VK_STAMP_SRC") != nullptr;
if (stamp_src && tex->uploaded && tex->guest_size >= (1u << 20)) {
static int told;
static std::unordered_map<uint32_t, int> times; // (RX_VK_STAMP_SRC=<n>: only a texture's n-th such upload and later)
static const int from_nth = std::max(1, atoi(getenv("RX_VK_STAMP_SRC")));
const uint64_t h = guest_hash(*tex);
if (++times[tex->id] >= from_nth && told++ < 400) {
uint64_t part = 0;
for (const auto& r : dirty) part += r.second - r.first;
const uint64_t off = dirty.empty() ? 0 : dirty[0].first;
const uint64_t guest = tex->gpu < (1ull << 40) ? tex->guest + off : host->gpu_guest(tex->gpu + off);
uint32_t pg = 0;
uintptr_t anchor = 0;
const uintptr_t who = guest ? memory_stamp_who(guest, &pg, &anchor) : 0;
logf("vk: stamp src: #%u %#llx %ux%u x%u NVN %#x uploaded again (%d) at frame %llu: %llu of %llu KB dirty in %zu ranges%s; bytes %s; "
"page %#llx gen %u (uploaded at %u) by %s%+lld", tex->id, (unsigned long long)tex->gpu, tex->width, tex->height,
tex->layers, tex->nvn_format, times[tex->id], (unsigned long long)frame_no, (unsigned long long)(part >> 10),
(unsigned long long)(tex->guest_size >> 10), dirty.size(), gpu_dirty ? " (a GPU buffer wrote it)" : "",
!tex->verify_hash ? "not hashed before" : h == tex->verify_hash ? "THE SAME as at its last upload" : "changed",
(unsigned long long)guest, pg, since, who == 1 ? "the write watch" : who == 2 ? "a capture" : who ? "memory_stamp" : "unknown",
who > 2 ? (long long)(who - anchor) : 0ll);
}
tex->verify_hash = h;
} else if (stamp_src && tex->guest_size >= (1u << 20)) {
tex->verify_hash = guest_hash(*tex);
}
const auto up0 = std::chrono::steady_clock::now();
const bool up_ok = upload(*tex, partial || sparse_written ? &dirty : nullptr);
{ // (an upload over 15 ms: what it was, 60 of them)
@@ -2565,6 +2630,84 @@ Texture* texture_touch(Texture* tex, bool for_sampling) {
return tex;
}
/// (Triforce, 2026-11-11) A flat texture sampled by a program whose instruction is a 3D sample: a one-slice 3D image with its
/// texels, and a 3D view of that. Vulkan has no 3D view of a 2D image, so such a binding had no view and read zeros (a null
/// descriptor). The console's GPU samples by the texture's own header whatever the instruction says: a flat texture's texel at
/// (u, v), the third coordinate unused, which a one-slice 3D image gives for any w. OoT's water in Zora's River binds one of
/// the game's 4 x 4 textures so (slot 0x102, 44 lookups in 10 s; its ripples ran in slow motion). Small plain textures only:
/// one layer copied, up to 1024 x 1024, not depth, not kept as BC3. Copied again at a lookup after the texture's contents
/// changed (an upload, or the GPU writing it): the lookups are cached by slot, so one the GPU draws every frame is a frame or
/// more behind here. RX_VK_NO_TWIN3D=1: no such view, as before
VkImageView twin3d_view(Texture* tex, VkFormat f, uint32_t level0, uint32_t nlevels, uint32_t layer, const VkComponentMapping& cm) {
static const bool off = getenv("RX_VK_NO_TWIN3D") != nullptr;
if (off || tex->type != VK_IMAGE_TYPE_2D || tex->sparse || tex->bc || tex->samples != VK_SAMPLE_COUNT_1_BIT ||
tex->aspect != VK_IMAGE_ASPECT_COLOR_BIT || is_bc(tex->format) || is_astc(tex->format) ||
(uint64_t)tex->width * tex->height > 1024ull * 1024 || !tex->image)
return VK_NULL_HANDLE;
VkCommandBuffer cmd = VK_NULL_HANDLE;
if (!tex->twin3d) {
VkImageCreateInfo ci{VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO};
ci.imageType = VK_IMAGE_TYPE_3D;
ci.format = tex->format;
ci.extent = {tex->width, tex->height, 1};
ci.mipLevels = tex->levels;
ci.arrayLayers = 1;
ci.samples = VK_SAMPLE_COUNT_1_BIT;
ci.tiling = VK_IMAGE_TILING_OPTIMAL;
ci.usage = VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT;
ci.flags = VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT; // (the view's format may be another of its class: sRGB or not)
if (!create_image(ci, tex->twin3d, tex->twin3d_alloc, false, false, false)) {
tex->twin3d = VK_NULL_HANDLE, tex->twin3d_alloc = nullptr;
return VK_NULL_HANDLE;
}
cmd = upload_cmd();
VkImageMemoryBarrier2 b{VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER_2};
b.srcStageMask = b.dstStageMask = VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT;
b.dstAccessMask = VK_ACCESS_2_MEMORY_READ_BIT | VK_ACCESS_2_MEMORY_WRITE_BIT;
b.oldLayout = VK_IMAGE_LAYOUT_UNDEFINED, b.newLayout = VK_IMAGE_LAYOUT_GENERAL;
b.srcQueueFamilyIndex = b.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
b.image = tex->twin3d;
b.subresourceRange = {VK_IMAGE_ASPECT_COLOR_BIT, 0, VK_REMAINING_MIP_LEVELS, 0, 1};
VkDependencyInfo di{VK_STRUCTURE_TYPE_DEPENDENCY_INFO};
di.imageMemoryBarrierCount = 1, di.pImageMemoryBarriers = &b;
vkCmdPipelineBarrier2(cmd, &di);
tex->twin3d_value = ~0ull, tex->twin3d_written = ~0ull;
static int told;
if (told++ < 12)
logf("vk: texture #%u %#llx %ux%u levels %u NVN %#x is sampled as a 3D texture: a one-slice 3D copy of it made", tex->id,
(unsigned long long)tex->gpu, tex->width, tex->height, tex->levels, tex->nvn_format);
}
if (tex->twin3d_value != tex->upload_value || tex->twin3d_written != tex->last_written || tex->twin3d_layer != layer) {
if (!cmd) cmd = upload_cmd();
full_barrier(cmd, "barrier (a flat texture's 3D copy: before)");
std::vector<VkImageCopy> rs;
for (uint32_t l = 0; l < tex->levels; l++) {
VkImageCopy r{};
r.srcSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, l, layer, 1};
r.dstSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, l, 0, 1};
r.extent = {std::max(1u, tex->width >> l), std::max(1u, tex->height >> l), 1};
rs.push_back(r);
}
vkCmdCopyImage(cmd, tex->image, VK_IMAGE_LAYOUT_GENERAL, tex->twin3d, VK_IMAGE_LAYOUT_GENERAL, (uint32_t)rs.size(), rs.data());
full_barrier(cmd, "barrier (a flat texture's 3D copy: after)");
tex->twin3d_value = tex->upload_value, tex->twin3d_written = tex->last_written, tex->twin3d_layer = layer;
}
uint64_t key = mix((uint64_t)f, (uint64_t)level0 << 32 | nlevels);
key = mix(key, (uint64_t)cm.r << 24 | (uint64_t)cm.g << 16 | (uint64_t)cm.b << 8 | cm.a);
const auto it = tex->twin3d_views.find(key);
if (it != tex->twin3d_views.end()) return it->second;
VkImageViewCreateInfo vi{VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO};
vi.image = tex->twin3d;
vi.viewType = VK_IMAGE_VIEW_TYPE_3D;
vi.format = f;
vi.components = cm;
vi.subresourceRange = {VK_IMAGE_ASPECT_COLOR_BIT, level0, nlevels, 0, 1};
VkImageView view = VK_NULL_HANDLE;
if (vkCreateImageView(dev.device, &vi, nullptr, &view) != VK_SUCCESS) view = VK_NULL_HANDLE;
tex->twin3d_views[key] = view;
return view;
}
VkImageView texture_view(Texture* tex, const nvn_tex* t, const nvn_view* v, VkImageViewType type, ViewUse use,
VkFormat* format_out, VkFormat force_format) {
if (!tex) return VK_NULL_HANDLE;
@@ -2615,6 +2758,7 @@ VkImageView texture_view(Texture* tex, const nvn_tex* t, const nvn_view* v, VkIm
else if (type == VK_IMAGE_VIEW_TYPE_2D && nlayers > 1) nlayers = 1;
else if (type == VK_IMAGE_VIEW_TYPE_1D && nlayers > 1) nlayers = 1;
}
int twin_layer = -1; // (Triforce: a flat texture wanted as 3D: the layer its 3D copy is of)
// the shader's type over the image's: what Vulkan allows of them
switch (type) {
case VK_IMAGE_VIEW_TYPE_1D: case VK_IMAGE_VIEW_TYPE_1D_ARRAY:
@@ -2636,7 +2780,11 @@ VkImageView texture_view(Texture* tex, const nvn_tex* t, const nvn_view* v, VkIm
if (type == VK_IMAGE_VIEW_TYPE_2D) nlayers = 1;
break;
case VK_IMAGE_VIEW_TYPE_3D:
if (tex->type != VK_IMAGE_TYPE_3D) return VK_NULL_HANDLE;
if (tex->type != VK_IMAGE_TYPE_3D) {
// (Triforce: a flat texture sampled as a 3D one: its one-slice 3D copy, twin3d_view above)
if (use != ViewUse::Sample) return VK_NULL_HANDLE;
twin_layer = (int)layer0;
}
layer0 = 0, nlayers = 1;
break;
case VK_IMAGE_VIEW_TYPE_CUBE: case VK_IMAGE_VIEW_TYPE_CUBE_ARRAY:
@@ -2675,6 +2823,10 @@ VkImageView texture_view(Texture* tex, const nvn_tex* t, const nvn_view* v, VkIm
nf, (int)f, sw[0], sw[1], sw[2], sw[3], d.swizzle_r, d.swizzle_g, d.swizzle_b, d.swizzle_a,
v && v->set_swizzle ? 1 : 0);
}
if (twin_layer >= 0) {
if (format_out) *format_out = f;
return twin3d_view(tex, f, level0, nlevels, (uint32_t)twin_layer, cm);
}
uint64_t key = mix((uint64_t)use, (uint64_t)type);
key = mix(key, (uint64_t)f << 32 | aspect);
key = mix(key, (uint64_t)level0 << 48 | (uint64_t)nlevels << 32 | (uint64_t)layer0 << 16 | nlayers);
@@ -3945,9 +4097,13 @@ uint64_t preset_texture_budget(uint64_t preset_mb, bool preset_applies, bool vra
return integrated ? std::min(b, own) : b;
}
uint64_t dead_sweep_next = 0; // (the present of the next pass for the dead targets between texture_frame's looks; 0: none)
void texture_sweep(uint64_t completed, bool sweep, uint64_t budget, uint64_t evict_to, bool known, uint64_t used, uint64_t allowed,
bool dead_only);
void texture_frame(uint64_t completed) {
upload_frame_report();
frame_upload_ns.store(0, std::memory_order_relaxed);
frame_astc_admitted.store(0, std::memory_order_relaxed);
if (const int64_t d = st_deferred_frame.exchange(0, std::memory_order_relaxed)) {
// (the views and descriptor sets resolved to the placeholder this frame go stale, as when textures go: the next frame
// looks those textures up again, and uploads them)
@@ -4050,9 +4206,28 @@ void texture_frame(uint64_t completed) {
// may have (other programs use the GPU too). The least recently used go first, down to 7/8 of it; nothing the
// last 8 presents used, nothing recorded work may still read.
static uint64_t own = 0;
if (frame_no % 10) return;
static bool last_known; // (the driver's numbers of the last full look, for the passes between)
static uint64_t last_used, last_allowed;
{ // (and a present in which a target's image was made new, not taken from the pool, asks for a pass three presents on,
// when what it replaced can first be found dead (RX_VK_DEAD_MIN_UNUSED): by the passes' own findings alone the first
// one of a run of new targets came at the next look, and in those ten presents the targets' blocks went 11 -> 22)
static uint64_t made_seen;
static const bool dead_quick = !(getenv("RX_VK_DEAD_QUICK") && !atoi(getenv("RX_VK_DEAD_QUICK")));
if (st_pool_made != made_seen) {
made_seen = st_pool_made;
if (dead_quick && (!dead_sweep_next || dead_sweep_next > frame_no + 3)) dead_sweep_next = frame_no + 3;
}
}
if (frame_no % 10) {
// (Triforce, 11-11) between the looks, a pass for the dead transient targets alone while the game makes them fast
// (dead_sweep_next, set by texture_sweep): see there
if (dead_sweep_next && frame_no >= dead_sweep_next)
texture_sweep(completed, true, ~0ull, 0, last_known, last_used, last_allowed, true);
return;
}
uint64_t used = 0, allowed = 0, block_free = 0;
const bool known = vram_usage(&used, &allowed, &block_free);
last_known = known, last_used = used, last_allowed = allowed;
if (!own) // (RX_VK_VRAM_MB, else 70% of what the driver allows the process at the first look)
own = getenv("RX_VK_VRAM_MB") ? (uint64_t)atoi(getenv("RX_VK_VRAM_MB")) << 20
: known ? std::min<uint64_t>(allowed / 100 * (allowed <= (8704ull << 20) ? 45 : 70), 6144ull << 20)
@@ -4129,6 +4304,7 @@ void texture_frame(uint64_t completed) {
}
if (budget < evict_to) evict_to = 0; // (hookshot, RX_VK_COMPACT: the driver's limit asks for more: 7/8 of that, as before)
if (frame_no % 600 == 0) {
memory_trial_tick(); // (Triforce: the memory handling on trial on this card, vk_memory.cpp)
vma_log_heaps();
vma_log_image_blocks();
{
@@ -4211,9 +4387,22 @@ void texture_frame(uint64_t completed) {
logf("vk: verify %ux%u: %d textures, %d changed since uploaded, %d of them unnoticed", verify_dim().first,
verify_dim().second, checked, changed, missed);
}
texture_sweep(completed, total_bytes <= budget, budget, evict_to, known, used, allowed, false);
}
/// (texture_frame) What goes: over the budget the least recently used, and the render targets drawn over since.
/// (Triforce, 11-11) dead_only: a pass between texture_frame's looks (every 10 presents) for the dead transient targets alone.
/// While the camera moves the game makes them at new sizes all the time: 1.2-3 new images a present running across Hyrule
/// Field in the lab, up to 11 a present between areas in a player's session (6733 in 600 presents). A dead one waited for the
/// next look, 10 presents plus the 3 it must go unused: at ~20 MB each that is ~650 MB of dead images in the lab's run and
/// 2.6 GB in that session, on top of what is drawn, their pool's blocks 18 -> 31, and at the driver's limit each new block
/// took 110-170 ms (ten frames of 115-180 ms in a row). So when a pass finds them coming at one a present and more, the next
/// pass is the next present, at one in three presents three presents on, else the next look as before: the dead wait 4-6
/// presents, not 13. RX_VK_DEAD_QUICK=0: only at the looks, as before
void texture_sweep(uint64_t completed, bool sweep, uint64_t budget, uint64_t evict_to, bool known, uint64_t used, uint64_t allowed,
bool dead_only) {
static const bool no_evict = getenv("RX_VK_NO_EVICT") != nullptr; // (an experiment: nothing evicted)
// (hookshot) under the budget, every 600 presents: the render targets drawn over since (below), unused for 120
const bool sweep = total_bytes <= budget;
// (hookshot-amd) and every 60 presents for the render targets drawn over by one at another start (below): OoT makes a frame's
// transient targets (768 x 432 RGBA16F, 1536 x 864) in a scratch heap whose offsets shift with what came before them in the
// frame, so turning the camera made new ones each frame ~1.2 MB from the last, each a new image. Left for the 600-present
@@ -4225,7 +4414,14 @@ void texture_frame(uint64_t completed) {
// ordered in 0.3 ms, 251 chosen in 0.2 (RTX 5070, Ryzen 7 9800X3D)
static const uint64_t dead_every =
getenv("RX_VK_DEAD_EVERY") ? (uint64_t)std::max(1, atoi(getenv("RX_VK_DEAD_EVERY"))) : 10;
if (no_evict || (sweep && frame_no % (dead_targets ? dead_every : 600))) return;
if (no_evict || (sweep && !dead_only && frame_no % (dead_targets ? dead_every : 600))) return;
static const bool dead_quick = !(getenv("RX_VK_DEAD_QUICK") && !atoi(getenv("RX_VK_DEAD_QUICK")));
static uint64_t dead_pass_last, quick_passes, quick_ns, quick_logged_at;
const auto pass_t0 = std::chrono::steady_clock::now();
const uint64_t pass_presents = std::max<uint64_t>(1, frame_no - dead_pass_last);
dead_pass_last = frame_no;
dead_sweep_next = 0; // (this pass is the one asked for; the next: below, or a new image's, texture_frame)
size_t shifted_now = 0;
std::vector<std::pair<uint64_t, Texture*>> order;
order.reserve(textures.size());
for (auto& kv : textures) order.push_back({kv.second->last_frame, kv.second.get()});
@@ -4418,7 +4614,7 @@ void texture_frame(uint64_t completed) {
}
for (const Evict& e : evict) { // (and each one's bookkeeping and its end, as before)
Texture* tex = e.tex;
if (e.shifted) n_dead++, b_dead += tex->bytes;
if (e.shifted) n_dead++, b_dead += tex->bytes, shifted_now++;
watch_tex(tex, "evicted");
if (!tex->gpu_written && tex->uploaded && !remade_skip_old()) { // (above: made again, it is uploaded from memory)
std::lock_guard lk(from_memory_mu);
@@ -4450,7 +4646,21 @@ void texture_frame(uint64_t completed) {
(unsigned long long)(b_dead >> 20));
}
const size_t n = order.size() - textures.size();
if (n) logf("vk: %zu textures retired (%zu left, %llu MB)", n, textures.size(), (unsigned long long)(total_bytes >> 20));
if (n && !dead_only) logf("vk: %zu textures retired (%zu left, %llu MB)", n, textures.size(), (unsigned long long)(total_bytes >> 20));
// (the next pass for the dead targets: by how fast this one found them coming)
if (dead_quick && dead_targets) {
if (shifted_now >= pass_presents) dead_sweep_next = frame_no + 1;
else if (shifted_now * 3 >= pass_presents) dead_sweep_next = frame_no + 3;
}
if (dead_only) {
quick_passes++;
quick_ns += (uint64_t)std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now() - pass_t0).count();
}
if (quick_passes && frame_no >= quick_logged_at + 600) {
quick_logged_at = frame_no;
logf("vk: passes for the dead render targets between the looks: %llu so far, %.2f ms each", (unsigned long long)quick_passes,
quick_ns / 1e6 / (double)quick_passes);
}
}
void texture_log_guest(const Texture* t) {

View File

@@ -90,6 +90,13 @@ struct Texture {
uint64_t wb_value = 0; // (the submission whose own commands write its texels into its memory; 0: none)
uint64_t pool_key = 0; // (a render target's image: its create info's key in the pool it goes to when the texture does)
std::unordered_map<uint64_t, VkImageView> views;
// (Triforce) a flat texture a program samples as a 3D one: its one-slice 3D copy (vk_texture.cpp twin3d_view), the layer it
// was copied from and the texture's contents it was copied at
VkImage twin3d = VK_NULL_HANDLE;
VmaAllocation twin3d_alloc = nullptr;
uint32_t twin3d_layer = 0;
uint64_t twin3d_value = ~0ull, twin3d_written = ~0ull;
std::unordered_map<uint64_t, VkImageView> twin3d_views;
};
/// Makes texels [x, x + w) x [y, y + h) of level `level`, layers [first, first + count), of a sparse texture resident

View File

@@ -28,7 +28,29 @@ bool vma_init(PFN_vkGetInstanceProcAddr gipa) {
ci.vulkanApiVersion = VK_API_VERSION_1_3;
ci.pVulkanFunctions = &f;
ci.preferredLargeHeapBlockSize = 64ull << 20; // (64 MB blocks: Eglantine found 256 MB ones wasted VRAM)
return vmaCreateAllocator(&ci, &vma) == VK_SUCCESS;
if (vmaCreateAllocator(&ci, &vma) != VK_SUCCESS) return false;
// (Triforce, a lab tool) RX_VK_TEST_BALLAST_MB=<n>: that much VRAM taken and held for the session, in 256 MB device buffers
// of their own. RX_VK_TEST_ALLOWED_MB only changes the number the budget is told; this fills the card for real, as a card
// with less VRAM or another program's use does: the driver's own behaviour at its limit (allocations of 110-170 ms in a
// player's log) only shows then
if (const char* e = getenv("RX_VK_TEST_BALLAST_MB")) {
const uint64_t want = (uint64_t)std::max(0, atoi(e)) << 20, chunk = 256ull << 20;
uint64_t got = 0;
for (; got < want; got += chunk) {
VkBufferCreateInfo bi{VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO};
bi.size = std::min(chunk, want - got);
bi.usage = VK_BUFFER_USAGE_TRANSFER_DST_BIT;
VmaAllocationCreateInfo ai{};
ai.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE;
ai.requiredFlags = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
ai.flags = VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
VkBuffer b = VK_NULL_HANDLE;
VmaAllocation a = VK_NULL_HANDLE;
if (vmaCreateBuffer(vma, &bi, &ai, &b, &a, nullptr) != VK_SUCCESS) break; // (kept: never freed)
}
logf("vk: lab: %llu MB of VRAM taken as ballast (RX_VK_TEST_BALLAST_MB)", (unsigned long long)(got >> 20));
}
return true;
}
// Images come from a pool of 128 MB blocks: on their own, VMA gave most of them dedicated memory (the driver prefers it
@@ -47,6 +69,8 @@ const uint64_t kImageBlock = (getenv("RX_VK_IMAGE_BLOCK_MB") ? (uint64_t)std::ma
// image made there then took 30-70 ms (RTX 5070). RX_VK_TARGET_BLOCKS=0: in the textures' blocks, as before
VmaPool target_image_pool = VK_NULL_HANDLE;
bool target_pool_tried = false;
size_t target_floor_blocks = 0;
bool target_floor_kept() { return target_floor_blocks != 0; }
VmaPool images_pool();
VmaPool target_images_pool() {
if (target_pool_tried) return target_image_pool;
@@ -68,9 +92,22 @@ VmaPool target_images_pool() {
VmaPoolCreateInfo pi{};
pi.memoryTypeIndex = type;
pi.blockSize = kImageBlock;
if (vmaCreatePool(vma, &pi, &target_image_pool) != VK_SUCCESS) target_image_pool = VK_NULL_HANDLE;
logf("vk: render targets' images from blocks of their own (%llu MB, memory type %u)%s", (unsigned long long)(kImageBlock >> 20),
type, target_image_pool ? "" : ": not made");
// (Triforce, 11-11) A floor of blocks the pool keeps for the session (RX_VK_TARGET_FLOOR_MB; 0: none, as before). While the
// camera moves the game makes its transient targets at new sizes all the time (a player's session: up to 6733 new images in
// 600 presents, 11 a frame), their blocks emptied and filled again; VMA keeps one empty block and frees the rest, and at the
// driver's limit (its budget reached) not even that one: every block the pool needed again was a new 128 MB from the
// driver, 110-170 ms each there. That session had ten frames of 115-180 ms in a row from it, VRAM climbing 9.5 -> 10.8 GB
// of 10.5 allowed. Below the floor VMA never gives a block back, so nothing is asked of the driver in play
static const uint64_t floor_mb = getenv("RX_VK_TARGET_FLOOR_MB") ? (uint64_t)std::max(0, atoi(getenv("RX_VK_TARGET_FLOOR_MB"))) : 0;
pi.minBlockCount = (size_t)((floor_mb << 20) / kImageBlock);
target_floor_blocks = pi.minBlockCount;
if (vmaCreatePool(vma, &pi, &target_image_pool) != VK_SUCCESS) {
pi.minBlockCount = 0; // (no room for the floor now: the pool as before)
target_floor_blocks = 0;
if (vmaCreatePool(vma, &pi, &target_image_pool) != VK_SUCCESS) target_image_pool = VK_NULL_HANDLE;
}
logf("vk: render targets' images from blocks of their own (%llu MB, memory type %u; %zu blocks kept for the session)%s",
(unsigned long long)(kImageBlock >> 20), type, pi.minBlockCount, target_image_pool ? "" : ": not made");
return target_image_pool;
}
@@ -182,7 +219,58 @@ bool image_compact_pass_end(uint64_t* freed_bytes, uint32_t* freed_blocks) {
return compact_over(freed_bytes, freed_blocks);
}
bool create_image_inner(const VkImageCreateInfo& ci, VkImage& image, VmaAllocation& allocation, bool dedicated, bool target, bool textures);
thread_local int create_image_path = 0; // (the last create_image_inner: 0 its pool or a dedicated one, 1 the pool refused, 2 system memory)
/// (Triforce) An image that takes over 3 ms to make: what it was, how long, and whether its pool took a new block for it (60
/// lines). A player's session had frames of 50 to 150 ms in play that were all "texture lookups", 24 render targets made in one
bool create_image(const VkImageCreateInfo& ci, VkImage& image, VmaAllocation& allocation, bool dedicated, bool target, bool textures) {
const auto blocks = [&]() -> uint32_t {
VmaPool pool = textures ? texture_pool : target ? target_image_pool : image_pool;
if (!pool) return 0;
VmaStatistics st{};
vmaGetPoolStatistics(vma, pool, &st);
return st.blockCount;
};
const uint32_t before = blocks();
const auto t0 = std::chrono::steady_clock::now();
create_image_path = 0;
const bool ok = create_image_inner(ci, image, allocation, dedicated, target, textures);
const double ms = std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - t0).count();
{ // (each 600 target images: how many the pool refused, and what it holds)
static std::atomic<int64_t> targets{0}, refused{0}, new_blocks{0};
if (target) {
refused += create_image_path != 0;
new_blocks += blocks() > before;
if (++targets % 600 == 0 && target_image_pool) {
VmaStatistics st{};
vmaGetPoolStatistics(vma, target_image_pool, &st);
logf("vk: render targets' blocks: %u blocks %llu MB hold %u images %llu MB; of %lld target images so far %lld took a "
"new block and %lld were refused by the pool", st.blockCount, (unsigned long long)(st.blockBytes >> 20),
st.allocationCount, (unsigned long long)(st.allocationBytes >> 20), (long long)targets.load(),
(long long)new_blocks.load(), (long long)refused.load());
}
}
}
static std::atomic<int> told{0};
static std::atomic<int64_t> n{0}, slow{0};
static std::atomic<int64_t> slow_us{0};
n++;
if (ms > 3.0) {
slow++, slow_us += (int64_t)(ms * 1000);
if (told++ < 60) {
uint64_t used = 0, allowed = 0, block_free = 0;
vram_usage(&used, &allowed, &block_free);
logf("vk: an image took %.1f ms to make: %ux%u x%u levels %u format %d (%s)%s; its pool's blocks %u -> %u; VRAM %llu of %llu "
"MB, %llu MB free in the blocks; %lld of %lld images so far took over 3 ms (%.0f ms in all)", ms, ci.extent.width,
ci.extent.height, ci.arrayLayers, ci.mipLevels, (int)ci.format, textures ? "a texture" : target ? "a render target" : "other",
!ok ? ", NOT made" : create_image_path ? ", outside its pool" : "", before, blocks(), (unsigned long long)(used >> 20),
(unsigned long long)(allowed >> 20),
(unsigned long long)(block_free >> 20), (long long)slow.load(), (long long)n.load(), slow_us.load() / 1000.0);
}
}
return ok;
}
bool create_image_inner(const VkImageCreateInfo& ci, VkImage& image, VmaAllocation& allocation, bool dedicated, bool target, bool textures) {
VmaAllocationCreateInfo ai{};
ai.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE;
// (in VRAM or not at all: an image VMA puts in system memory when VRAM is full is read over PCIe every frame,
@@ -202,8 +290,10 @@ bool create_image(const VkImageCreateInfo& ci, VkImage& image, VmaAllocation& al
pai.pool = pool;
pai.usage = VMA_MEMORY_USAGE_UNKNOWN;
if (vmaCreateImage(vma, &ci, &pai, &image, &allocation, nullptr) == VK_SUCCESS) return true;
create_image_path = 1; // (the pool refused it: the allocator's own blocks, or a memory object of its own)
}
if (vmaCreateImage(vma, &ci, &ai, &image, &allocation, nullptr) == VK_SUCCESS) return true;
create_image_path = 2;
// (hookshot) VRAM is full (a 4 GB card at a high resolution, #96): an image left unmade is a black screen, so it goes
// to system memory instead, slow but drawn
if (getenv("RX_VK_IMAGES_VRAM_ONLY")) return false;
@@ -278,6 +368,37 @@ bool create_device_buffer(uint64_t size, VkBufferUsageFlags usage, VkBuffer& buf
if (dedicated) ai.flags |= VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
return vmaCreateBuffer(vma, &bi, &ai, &buffer, &allocation, nullptr) == VK_SUCCESS;
}
/// (Triforce) Device memory for a piece of a sparse buffer (vk_memory.cpp's mirrors in pieces): size bytes at that alignment,
/// of a type in type_bits, within the driver's budget; the memory and the offset in it to bind
bool vma_piece_alloc(uint64_t size, uint64_t alignment, uint32_t type_bits, VmaAllocation& allocation, VkDeviceMemory& memory,
uint64_t& offset) {
VkMemoryRequirements req{};
req.size = size, req.alignment = alignment, req.memoryTypeBits = type_bits;
VmaAllocationCreateInfo ai{};
ai.requiredFlags = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
if (dev.memory_budget) ai.flags = VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT;
VmaAllocationInfo info{};
const auto t0 = std::chrono::steady_clock::now();
const bool ok = vmaAllocateMemory(vma, &req, &ai, &allocation, &info) == VK_SUCCESS;
{ // (Triforce, 11-11: one that takes over 3 ms is named, 40 times: at the driver's limit an image's new block took
// 110-170 ms, and submissions beside those were 130-150 ms of "memory sync" with nothing said of them)
const double ms = std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - t0).count();
static std::atomic<int> told{0};
if (ms > 3.0 && told++ < 40) {
uint64_t used = 0, allowed = 0, block_free = 0;
vram_usage(&used, &allowed, &block_free);
logf("vk: a mirror's piece took %.1f ms to get its memory (%llu KB%s); VRAM %llu of %llu MB", ms,
(unsigned long long)(size >> 10), ok ? "" : ", NOT given", (unsigned long long)(used >> 20),
(unsigned long long)(allowed >> 20));
}
}
if (!ok) return false;
memory = info.deviceMemory, offset = info.offset;
return true;
}
void vma_piece_free(VmaAllocation allocation) {
vmaFreeMemory(vma, allocation);
}
/// (every heap: what VMA has in blocks and allocations there, what the process uses of it, its budget; one line)
/// The image pools: their blocks, the images in them, and the free ranges between them (every 600 presents, with the heaps)
void vma_log_image_blocks() {

47
host/frame_whole.h Normal file
View File

@@ -0,0 +1,47 @@
/* (hookshot) Whole frames out of a frame step that can be under one, for the game's counters that add the step, or the step
times a whole number, cut to whole frames (nx2/oot_patches.py patch_whole_frames). At the console's 60 fps the step is 1 and
they count frames. Under a Frame rate cap above 60 the step is the frame's own time (host/nn_vi.c, hos_frame_seconds_bits):
0.42 at 144 fps, cut to 0 every frame, and the counter never moves, so what waits on it never comes: the pick-up indicator
above Link's head counts the rupees shown up to the wallet's (f_80a87e50: the shown +864 moves by (int)(step * the
distance / 50, at least 1)) and fades only when they meet; it stayed at the old count until a door or a talk replaced it.
While the frames' own time is the step (carry), the amount is added to the counter's own remainder and the whole frames in
it are handed out, so the counter moves at the game's rate whatever the frame rate. Otherwise it is cut as the game cuts it:
60 fps as before. One remainder per counter address (or per object: each water surface keeps its own), reused oldest
first. 64 slots: a scene's water surfaces each take one every frame, and more of them than slots would evict each other in
turn, every one starting again from 0 and standing still. Not thread-safe by itself: the caller holds a lock. */
#pragma once
#include <stdint.h>
#include <string.h>
#define FRAME_WHOLE_SLOTS 64
typedef struct {
uint64_t at[FRAME_WHOLE_SLOTS];
float rem[FRAME_WHOLE_SLOTS];
uint32_t used[FRAME_WHOLE_SLOTS]; /* (the call count at the slot's last use: the oldest is reused) */
uint32_t calls;
} FrameWhole;
/* (as the lifted code cuts it: toward zero, NaN and below zero to 0, at most 0x7fffffff: the counts are small and positive) */
static inline uint32_t frame_whole_cut(float f) {
if (!(f > 0.0f)) return 0;
if (f >= 2147483648.0f) return 0x7fffffffu;
return (uint32_t)f;
}
static inline uint32_t frame_whole(FrameWhole *w, uint64_t at, uint32_t amount_bits, int carry) {
float amount;
memcpy(&amount, &amount_bits, 4);
if (!carry || !(amount > 0.0f) || amount >= 2147483648.0f) return frame_whole_cut(amount);
w->calls++;
int slot = -1, oldest = 0;
for (int i = 0; i < FRAME_WHOLE_SLOTS; i++) {
if (w->at[i] == at) { slot = i; break; }
if (w->used[i] < w->used[oldest]) oldest = i;
}
if (slot < 0) slot = oldest, w->at[slot] = at, w->rem[slot] = 0.0f;
w->used[slot] = w->calls;
const float sum = w->rem[slot] + amount;
const uint32_t whole = frame_whole_cut(sum);
w->rem[slot] = sum - (float)whole;
return whole;
}

View File

@@ -224,6 +224,59 @@ static float text_scale(const char *env, int menu) {
volatile uint64_t hos_step_probe = 0; /* (f_80706be0's object, set by the lifted code when the frame-time hook is in this build; its
+0xd58 is the game clock in 1/60 s: nx2/oot_patches.py) */
volatile uint32_t hos_frame_seconds_bits = 0; /* (the frame's seconds as a float's bits for that hook, 0: the game's own step) */
/* A movie's frame count while the frames' time is given (a Frame rate cap above 60): how many 60ths of a second this update of
that movie player is worth, the rest carried to its next one, so the count the cutscene's cues are placed by keeps the clock
as the movie's picture does (issue 1742: the narration ran ahead above 60 fps). 1, the game's own step, otherwise, and with
HOS_FPS_MOVIE=0 (for comparisons). Called by the lifted f_83b7c710 (nx2/oot_patches.py). */
uint32_t hos_movie_frames(uint64_t player) {
enum { PLACES = 16 };
const uint32_t bits = __atomic_load_n(&hos_frame_seconds_bits, __ATOMIC_RELAXED); /* (realtime_step writes it at each present) */
if (!bits) return 1;
static SRWLOCK lock = SRWLOCK_INIT;
static int timed = -1; /* (these four under the lock) */
static struct { uint64_t player, used; double part; } carry[PLACES];
static uint64_t calls;
static unsigned made;
float secs;
uint32_t n = 1;
memcpy(&secs, &bits, 4);
AcquireSRWLockExclusive(&lock);
if (timed < 0) timed = !(getenv("HOS_FPS_MOVIE") && !atoi(getenv("HOS_FPS_MOVIE")));
if (timed) {
int i = 0;
while (i < PLACES && carry[i].player != player) i++;
if (i == PLACES) {
/* (a player not seen yet takes the place used longest ago, so a movie that plays keeps its own while others come
and go: its count is exact while no more than PLACES players, itself among them, update in turn. It starts at a part
of a 60th drawn from a hash of how many places were made: with that many players at once every update is a first
one, and each count then still grows by the frames' time on average, whatever the number of players, instead of
standing still. A sequence with a fixed step, the golden ratio's multiples, stood still for one player in 233) */
uint32_t x = made++;
x ^= x >> 16, x *= 0x85ebca6bu, x ^= x >> 13, x *= 0xc2b2ae35u, x ^= x >> 16; /* (MurmurHash3's last step) */
i = 0;
for (int k = 1; k < PLACES; k++)
if (carry[k].used < carry[i].used) i = k;
carry[i].player = player, carry[i].part = x / 4294967296.0;
}
carry[i].used = ++calls;
carry[i].part += (double)secs * 60.0;
n = (uint32_t)carry[i].part;
carry[i].part -= n;
}
ReleaseSRWLockExclusive(&lock);
return n;
}
/* (nx2/oot_patches.py patch_whole_frames) the whole frames a game counter at `at` takes for an amount of the frame step (its
float's bits): with the frames' own time as the step, their fractions carried per counter; else cut as the game cuts it */
#include "frame_whole.h"
static FrameWhole frame_whole_state;
static SRWLOCK frame_whole_lock = SRWLOCK_INIT; /* (the game's update threads) */
uint32_t hos_frame_whole(uint64_t at, uint32_t amount_bits) {
AcquireSRWLockExclusive(&frame_whole_lock);
const uint32_t whole = frame_whole(&frame_whole_state, at, amount_bits, __atomic_load_n(&hos_frame_seconds_bits, __ATOMIC_RELAXED) != 0);
ReleaseSRWLockExclusive(&frame_whole_lock);
return whole;
}
volatile uint64_t hos_res_module = 0; /* (f_831f3200's object, whose +0xe5c is the 3D's scale: nx2/oot_patches.py) */
static void realtime_step(int64_t now) {
int rx_option(int id, int otherwise);
@@ -357,9 +410,9 @@ static void realtime_step(int64_t now) {
const float f = (float)(secs > 4.0 / 60.0 ? 4.0 / 60.0 : secs < 1.0 / 240.0 ? 1.0 / 240.0 : secs);
uint32_t bits;
memcpy(&bits, &f, 4);
hos_frame_seconds_bits = bits;
__atomic_store_n(&hos_frame_seconds_bits, bits, __ATOMIC_RELAXED); /* (read by the game's threads) */
} else {
hos_frame_seconds_bits = 0;
__atomic_store_n(&hos_frame_seconds_bits, 0, __ATOMIC_RELAXED);
}
const int top = timed ? 60 : fps_max();
double ratio = sum / count * 60.0 + carry;

66
host/oot_high_fps.c Normal file
View File

@@ -0,0 +1,66 @@
/* (hookshot) The sound manager's job at 60 Hz and the physics solver's steps rounded up, under a Frame rate cap above 60: the
lifted f_80435300, f_838ceb80 and f_838cfc10 call these (nx2/oot_patches.py); host/oot_high_fps.h has why and the logic.
HOS_FPS_LOG (set): what they do, every 600 calls. */
#include "hos.h"
#include "oot_high_fps.h"
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <stdlib.h>
extern volatile uint32_t hos_frame_seconds_bits; /* (host/nn_vi.c: the frame's seconds, 0 at a cap of 60) */
static int timed(void) { return __atomic_load_n(&hos_frame_seconds_bits, __ATOMIC_RELAXED) != 0; }
static int fps_log(void) {
static int on = -1; /* (read by the game's threads) */
int v = __atomic_load_n(&on, __ATOMIC_RELAXED);
if (v < 0) v = getenv("HOS_FPS_LOG") != NULL, __atomic_store_n(&on, v, __ATOMIC_RELAXED);
return v;
}
static OotSoundGate sound_gate = OOT_SOUND_GATE_INIT;
static long long sound_calls, sound_runs; /* (HOS_FPS_LOG's counts, under sound_lock) */
static SRWLOCK sound_lock = SRWLOCK_INIT; /* (the job's phases may come from different threads) */
/* f_80435300's head: phase w2, slot the guest address of the pointer to the object whose +0x10 is the game step (the job's own
load). 0: the job returns at once */
int hos_oot_sound_gate(Ctx *c, uint32_t phase, uint64_t slot) {
(void)c;
float step = 1.0f;
const int t = timed();
if (t && phase == 0) {
const uint64_t object = G64(slot);
const uint64_t time = object ? G64(object) : 0;
if (time) step = oot_bits_float(*(const uint32_t *)GP(time + 0x10));
}
long long ran = -1;
AcquireSRWLockExclusive(&sound_lock);
const int run = oot_sound_gate(&sound_gate, phase, step, t);
if (phase == 0 && fps_log()) {
sound_runs += run;
if (!(++sound_calls % 600)) ran = sound_runs, sound_runs = 0;
}
ReleaseSRWLockExclusive(&sound_lock);
if (ran >= 0)
hos_log("oot: sound job ran %lld of the last 600 frames (step %.3f%s)", ran, step,
oot_sound_partial(step, t) ? ", held to 60 Hz" : "");
return run;
}
/* The step f_80435300's phase 0 hands to the sound engine (its float's bits) */
uint32_t hos_oot_sound_step(uint32_t step_bits) {
return oot_sound_step(step_bits, timed());
}
/* The solver steps of f_838ceb80 / f_838cfc10: game, the game's count; scaled_bits, the float it cut; per_frame_bits, the steps
of a 60 fps frame (a float) */
uint32_t hos_oot_solver_steps(uint32_t game, uint32_t scaled_bits, uint32_t per_frame_bits) {
const int32_t steps = oot_solver_steps((int32_t)game, scaled_bits, per_frame_bits, timed());
if (fps_log()) {
static volatile LONG64 calls;
const LONG64 n = InterlockedIncrement64(&calls);
if (n % 600 == 1)
hos_log("oot: physics solver steps %d (the game's %d: frame step %.3f x %.0f steps a 60 fps frame)", steps, (int32_t)game,
(oot_bits_float(scaled_bits) - 0.001f) / oot_bits_float(per_frame_bits), oot_bits_float(per_frame_bits));
}
return (uint32_t)steps;
}

86
host/oot_high_fps.h Normal file
View File

@@ -0,0 +1,86 @@
/* (hookshot) Two places in the game that need whole 60 fps frames, under a Frame rate cap above 60, where the frame step is the
frame's own time (host/nn_vi.c, hos_frame_seconds_bits: 0.5 at 120 fps). nx2/oot_patches.py hooks them; host/oot_high_fps.c
holds the state. At a cap of 60 (no frame time given: timed 0) both return what the game computed, so nothing changes.
The sound manager's job (f_80435300, its phase in w2): phase 0 hands the game step to the sound engine, whose fade and delay
timers and parameter ramps count updates of at least one whole step. Fed less, some sounds are not released when an area
unloads, the voices in use climb until the game's voice budget is full, and new sounds (sword, voices), then the area music,
stop until a restart. Above 60 fps the job runs at 60 Hz: phase 0 adds the step to a remainder and runs once a whole step has
built up, with a step of 1 (sound_step), keeping the rest; the job's other phases follow that frame's choice. Sound keeps the
clock on average. A step of 1 or more runs as the game has it and keeps the remainder: around 60 fps (a cap above 60 with
the game near 60, or VRR) steps of 0.98 and 1.02 alternate, and the 0.98 that did not run comes back as a whole step later
(dropped, the sound clock ran at half the real one). A step of 0 or less (or NaN), and every phase at a cap of 60, run as the
game has them, and clear the remainder.
(After zortos293/Lichii PR #1289 by auz466, GPL-3.0: the same gate as a code patch for the emulator.)
The physics world's solver steps: the game sets them each update to (int)(frame step * the steps of a 60 fps frame + 0.001)
(f_838ceb80 and f_838cfc10, into the solver's +712; 8 a 60 fps frame in the places seen). A step under 1 cuts that short,
and as the measured step wanders around its mean the count changes from frame to frame (8 x 0.49 is 3, 8 x 0.5 is 4 at 120
fps); 4 steps a 60 fps frame would be 0 above 240. A moving platform (the stone one under the spikes in the Deku Tree)
then slides out from under Link (issue 1798). While the step is under 1 the count is rounded up instead, so a solver step
never covers more time than at 60 fps, give or take 10% (the frame time's jitter, so a steady rate keeps a steady count),
and it is at least 1 and never under the game's. A step of 1 or more (a frame slower than 60 fps) keeps the game's count. */
#pragma once
#include <math.h>
#include <stdint.h>
#include <string.h>
typedef struct {
float part; /* (the steps added up since the job last ran, under 1) */
int run; /* (phase 0's choice, for the other phases of the same frame) */
} OotSoundGate;
#define OOT_SOUND_GATE_INIT {0.0f, 1} /* (every phase runs until a phase 0 has chosen) */
static inline float oot_bits_float(uint32_t bits) {
float f;
memcpy(&f, &bits, 4);
return f;
}
static inline uint32_t oot_float_bits(float f) {
uint32_t bits;
memcpy(&bits, &f, 4);
return bits;
}
/* (whether the step is a part of a frame: the only case the gate and the feed change) */
static inline int oot_sound_partial(float step, int timed) {
return timed && step > 0.0f && step < 1.0f;
}
/* 1: run the job now, 0: return from it at once */
static inline int oot_sound_gate(OotSoundGate *g, uint32_t phase, float step, int timed) {
if (!timed) { /* (a cap of 60: every phase of every frame, whatever came before) */
g->part = 0.0f;
g->run = 1;
return 1;
}
if (phase != 0) return g->run;
if (!(step > 0.0f)) { /* (0, below 0, NaN: as the game has it) */
g->part = 0.0f;
g->run = 1;
return 1;
}
if (step >= 1.0f) { /* (a whole step or more: runs with it, the remainder kept for the next partial ones) */
g->run = 1;
return 1;
}
g->part += step;
g->run = g->part >= 1.0f;
if (g->run) g->part -= 1.0f;
return g->run;
}
/* The step phase 0 hands to the sound engine: a whole one while the gate holds the job to whole steps */
static inline uint32_t oot_sound_step(uint32_t step_bits, int timed) {
return oot_sound_partial(oot_bits_float(step_bits), timed) ? 0x3f800000u : step_bits;
}
/* The solver's steps: game (the game's own count), scaled_bits (frame step * steps a 60 fps frame + 0.001, the float it cut),
per_frame_bits (the steps of a 60 fps frame, a float) */
static inline int32_t oot_solver_steps(int32_t game, uint32_t scaled_bits, uint32_t per_frame_bits, int timed) {
const float scaled = oot_bits_float(scaled_bits) - 0.001f, per_frame = oot_bits_float(per_frame_bits);
if (!timed || !(scaled > 0.0f) || !(per_frame > 0.0f) || !(scaled < per_frame) || per_frame > 1000.0f) return game;
int32_t steps = (int32_t)ceilf(scaled - 0.1f);
if (steps < 1) steps = 1;
return steps > game ? steps : game;
}

View File

@@ -247,6 +247,179 @@ def unpatch_frame_seconds(s):
return s if STEP_MARK in s else s.replace(STEP_DECL, "", 1) # (a hook of another shape stays, with its declarations)
# A movie's frame count above a 60 cap. The game's movie player counts its updates while a movie plays (f_83b7c710: one more
# at +760 of the player each frame), and a cutscene that shows a movie takes that count as its timeline's frame (f_83b5ec10,
# f_83b5e9a0): the narration, the rumble and the cutscene's end are placed by it. The movie itself follows the clock, so above
# 60 fps those cues ran ahead of the picture, fps / 60 times too fast (issue 1742). The count now grows by the frames' time, as
# the game's clock does (hos_movie_frames, host/nn_vi.c: 1 as before while the host gives no frame time).
MOVIE_HEAD = "void f_83b7c710(Ctx *__restrict A64_NOESC c) {\n"
# (the load and the add alone: the store after them is rewritten by the page marks' pass, which runs after these edits)
MOVIE_OLD = (" c->x[8]=recomp_load32(c,c->x[20] + 760);\n"
" { uint64_t _a=c->x[8], _b=1ULL; uint64_t _r=_a+_b; _r&=0xFFFFFFFFULL; c->x[8]=_r; }\n")
MOVIE_NEW = MOVIE_OLD.replace("_b=1ULL", "_b=hos_movie_frames(c->x[20])")
MOVIE_DECL = "uint32_t hos_movie_frames(uint64_t player);\n"
def patch_movie_frames(s):
"""The movie player's count by the frames' time; None when the file has no such function, s unchanged when the function is
there but its code differs (another game version) or has the patch already."""
head = s.find(MOVIE_HEAD)
if head < 0:
return None
end = s.find("\nvoid f_", head + len(MOVIE_HEAD))
body = s[head:end if end >= 0 else len(s)]
if body.count(MOVIE_OLD) != 1:
return s
at = head + body.index(MOVIE_OLD)
s = s[:at] + MOVIE_NEW + s[at + len(MOVIE_OLD):]
return s if MOVIE_DECL in s else s.replace('#include "a64.h"\n', '#include "a64.h"\n' + MOVIE_DECL, 1)
# Counters that add the frame step, or the step times a whole number, cut to whole frames (host/frame_whole.h). At 60 fps the step
# is 1; under a Frame rate cap above 60 it is the frame's own time, under 1 (0.42 at 144 fps), and the cut is 0 every frame:
# - f_80a87e50, the pick-up indicator above Link's head (a rupee, a Deku Stick): the count it shows (+864) moves toward the
# wallet's (+860) by (int)(step * max(distance / 50, 1)) and the indicator fades when they meet. Cut to 0, it stayed at the old
# count until a door or a talk replaced it (issue 1832).
# - f_800672e0, the water renderer: its ripple simulation runs (int)step times the water's substeps (+76 of the pass) each frame.
# Cut to 0, the rings Link and the actors make in water stood still above 60 fps. Found by atk3001 for Lichii (zortos293/
# Lichii#1152), as ported to Fritchii (Frizti/Fritchii d389685a4, 063dbb741): its remainder is kept per water object (x21).
# - f_803edf70, the time of day: each frame it adds (int)(2 * step) game seconds (+60) to the seconds of the day (+40, a day of
# 86400, then the day count +44). Cut, it added 1 a frame below 120 fps (the day slower: 83% at 100) and 0 above 120, where
# the clock stood still and the sun swung back and forth over the frozen hour.
# hos_frame_whole carries the fractions per counter while the frames' own time is the step, so each moves at the game's rate;
# otherwise it cuts as the game does. A site is added here once it is seen to misbehave in game. (A site's function, its code to
# replace, and the counter's address for its remainder.)
WHOLE_SITES = (
# (the cut alone, the function's one: the multiply before it is a64_fpx_hot_mul32_fma32 in a lift where the function is
# among A64_HOT_FUNCS)
("void f_80a87e50(Ctx *__restrict A64_NOESC c) {@",
" c->x[12]=recomp_fp_to_int(c->vreg[0][0]&UINT64_C(0xffffffff),32,32,1,3,c->fpcr,&c->fpsr);@",
"c->x[19] + 864"),
# (the cut alone, the function's one)
("void f_800672e0(Ctx *__restrict A64_NOESC c) {@",
" c->x[9]=recomp_fp_to_int(c->vreg[0][0]&UINT64_C(0xffffffff),32,32,1,3,c->fpcr,&c->fpsr);@",
"c->x[21]"),
# (the cut with the store after it: the function cuts into x8 the same way once more, in the branch that sets
# the time from another source)
("void f_803edf70(Ctx *__restrict A64_NOESC c) {@",
" c->x[8]=recomp_fp_to_int(c->vreg[0][0]&UINT64_C(0xffffffff),32,32,1,3,c->fpcr,&c->fpsr);@"
" recomp_store32(c,c->x[19] + 60,c->x[8]);@",
"c->x[19] + 60"),
)
WHOLE_SITES = tuple((h.replace("@", "\n"), o.replace("@", "\n"), k) for h, o, k in WHOLE_SITES)
WHOLE_MARK = "/* (the step's whole frames: nx2/oot_patches.py) */"
WHOLE_DECL = "extern uint32_t hos_frame_whole(uint64_t at, uint32_t amount_bits);\n"
def whole_new(old, key):
"""old with its cut to whole frames (the one recomp_fp_to_int) made hos_frame_whole's for the counter at key"""
lines = old.split("\n")
for i, line in enumerate(lines):
if "recomp_fp_to_int(c->vreg[0][0]" in line:
reg = line.split("=", 1)[0]
lines[i] = "%s=hos_frame_whole(%s, (uint32_t)c->vreg[0][0]); %s" % (reg, key, WHOLE_MARK)
return "\n".join(lines)
def patch_whole_frames(s):
"""The counters of WHOLE_SITES take hos_frame_whole's whole frames: (s, the sites patched here). A file without the function,
one patched before, or one whose code differs (another game version) is left as it is."""
done = 0
for head_text, old, key in WHOLE_SITES:
head = s.find(head_text)
if head < 0:
continue
end = s.find("\n}\n", head)
body = s[head:end if end >= 0 else len(s)]
new = whole_new(old, key)
if new in body or body.count(old) != 1:
continue
at = head + body.index(old)
s = s[:at] + new + s[at + len(old):]
done += 1
if done and WHOLE_DECL not in s:
s = s.replace('#include "a64.h"\n', '#include "a64.h"\n' + WHOLE_DECL, 1)
return s, done
# Above a Frame rate cap of 60, two places that need whole 60 fps frames (host/oot_high_fps.h has why and the logic; at a cap of
# 60 the host hands back what the game computed):
# - the sound manager's job, f_80435300 (phase in w2; phase 0 loads the game step, [[slot]]+0x10, and hands it to the sound
# engine): its head asks hos_oot_sound_gate whether to run now (60 Hz above 60 fps, the other phases following phase 0), and
# the step it loads goes through hos_oot_sound_step (a whole one while the gate holds the job). After zortos293/Lichii PR #1289
# (auz466), which does the same as a code patch for the emulator.
# - the physics solver's step count, set each update from the frame step in f_838ceb80 and f_838cfc10 (the same code in both):
# hos_oot_solver_steps rounds it up instead of down, at least 1, so Link stays on moving platforms (issue 1798).
SOUND_HEAD = "void f_80435300(Ctx *__restrict A64_NOESC c) {\n"
SOUND_SLOT = "g_module_base + 0x80435000ULL + (int64_t)75927552" # (the page of the step's pointer, whose +2192 holds it)
SOUND_OLD = (" c->x[8]=" + SOUND_SLOT + ";\n"
" { uint64_t _r = ((uint64_t)0 | ((c->x[0]) << 0)); c->x[19] = _r; }\n"
" c->x[8]=recomp_load64(c,c->x[8] + 2192);\n"
" c->x[8]=recomp_load64(c,c->x[8] + 0);\n"
" { { uint64_t _v=recomp_load32(c,(c->x[8]+16ULL)); a64_vset64(c->vreg[0],_v); } }\n")
SOUND_NEW = SOUND_OLD.replace("uint64_t _v=recomp_load32(c,(c->x[8]+16ULL));",
"uint64_t _v=hos_oot_sound_step((uint32_t)recomp_load32(c,(c->x[8]+16ULL)));")
SOUND_GATE = (" if (!hos_oot_sound_gate(c, (uint32_t)c->x[2], " + SOUND_SLOT + " + 2192)) return;"
" /* (nx2/oot_patches.py: the sound job at 60 Hz above 60 fps) */\n")
SOUND_DECL = "extern int hos_oot_sound_gate(Ctx *c, uint32_t phase, uint64_t slot);\nextern uint32_t hos_oot_sound_step(uint32_t step_bits);\n"
SOLVER_HEADS = ("void f_838ceb80(Ctx *__restrict A64_NOESC c) {\n", "void f_838cfc10(Ctx *__restrict A64_NOESC c) {\n")
# (the cut of frame step * steps a frame + 0.001, then its store into the solver: vreg 0 the float cut, vreg 1 the steps a frame)
SOLVER_OLD = (" c->x[1]=recomp_fp_to_int(c->vreg[0][0]&UINT64_C(0xffffffff),32,32,1,3,c->fpcr,&c->fpsr);\n"
" recomp_store32(c,c->x[0] + 712,c->x[1]);\n")
SOLVER_NEW = SOLVER_OLD.replace(
" recomp_store32(", " c->x[1]=hos_oot_solver_steps((uint32_t)c->x[1], (uint32_t)c->vreg[0][0], (uint32_t)c->vreg[1][0]);"
" /* (nx2/oot_patches.py: solver steps rounded up above 60 fps) */\n recomp_store32(", 1)
SOLVER_DECL = "extern uint32_t hos_oot_solver_steps(uint32_t game, uint32_t scaled_bits, uint32_t per_frame_bits);\n"
def _function(s, head_text):
"""(start, end) of the function whose head is head_text, or None"""
head = s.find(head_text)
if head < 0:
return None
end = s.find("\nvoid f_", head + len(head_text))
return head, end if end >= 0 else len(s)
def patch_sound_job(s):
"""The sound job's gate and step: (s, 1) when the job is in s with them (made now or before), (s, 0) when it is not here or
its code differs (another game version: left as it is)"""
at = _function(s, SOUND_HEAD)
if at is None:
return s, 0
head, end = at
body = s[head:end]
if body.count(SOUND_NEW) == 1 and body.count(SOUND_GATE) == 1:
return s, 1
if body.count(SOUND_OLD) != 1 or SOUND_GATE in body:
return s, 0
body = body.replace(SOUND_OLD, SOUND_NEW, 1)
body = SOUND_HEAD + SOUND_GATE + body[len(SOUND_HEAD):]
s = s[:head] + body + s[end:]
if SOUND_DECL not in s:
s = s.replace('#include "a64.h"\n', '#include "a64.h"\n' + SOUND_DECL, 1)
return s, 1
def patch_solver_steps(s):
"""The solver step count's two places: (s, how many of them are in s with the hook, made now or before)"""
done = 0
for head_text in SOLVER_HEADS:
at = _function(s, head_text)
if at is None:
continue
head, end = at
body = s[head:end]
if body.count(SOLVER_NEW) == 1:
done += 1
elif body.count(SOLVER_OLD) == 1:
s = s[:head] + body.replace(SOLVER_OLD, SOLVER_NEW, 1) + s[end:]
done += 1
if done and SOLVER_DECL not in s:
s = s.replace('#include "a64.h"\n', '#include "a64.h"\n' + SOLVER_DECL, 1)
return s, done
def patch_text_speed(s):
"""The scale at the head of the letters' block of f_80b1e140; None when the file has no such function, s unchanged when the
function is there but its code differs (another game version)."""
@@ -274,7 +447,7 @@ def apply(gen, *, protect_hearts=False, required=False):
The other patches only warn when their place is not found once."""
import oot_native_resolution # (HOS_SCENE_NATIVE=1: the scene's base scale 1.0; off by default since #849's output resolution)
oot_native_resolution.apply(gen)
done = {"dynamic resolution branch": 0, "text speed block": 0, "fixed path's scale": 0}
done = {"dynamic resolution branch": 0, "text speed block": 0, "fixed path's scale": 0, "movie's frame count": 0}
if protect_hearts: # (the supported build's: these places are its functions)
done["menu focus hook"] = 0
done["menu tick hook"] = 0
@@ -284,6 +457,8 @@ def apply(gen, *, protect_hearts=False, required=False):
done["free camera"] = 0
done["mouse camera"] = 0
step_sites, step_files = 0, [] # (the divides with the hook, and the files that have it)
whole_sites = 0 # (the counters of WHOLE_SITES with the whole frames: patch_whole_frames)
sound_jobs = solver_sites = 0 # (patch_sound_job, patch_solver_steps: the places found with their hooks)
for path in sorted(glob.glob(os.path.join(glob.escape(gen), "code_*.c"))):
with open(path, encoding="utf-8", newline="") as f:
s = f.read()
@@ -308,6 +483,13 @@ def apply(gen, *, protect_hearts=False, required=False):
out = tick
done["menu tick hook"] += 1
print("oot_patches: the menus' update hook in", os.path.basename(path))
movie = patch_movie_frames(out)
if movie is not None:
if MOVIE_NEW in movie: # (made now, or by an earlier run over this file)
done["movie's frame count"] += 1
if movie != out:
print("oot_patches: the movie's frame count by the frames' time in", os.path.basename(path))
out = movie
out, n = fixed_scale(out)
if n:
done["fixed path's scale"] += 1
@@ -343,8 +525,26 @@ def apply(gen, *, protect_hearts=False, required=False):
step_files.append(path)
if sites:
print("oot_patches: the frame's time at %d step divide(s) in %s" % (sites, os.path.basename(path)))
out, whole = patch_whole_frames(out)
whole_sites += out.count(WHOLE_MARK) # (made now, or by an earlier run over this file)
if whole:
print("oot_patches: %d frame counter(s) with the step's whole frames in %s" % (whole, os.path.basename(path)))
sound, found = patch_sound_job(out)
sound_jobs += found
if sound != out:
print("oot_patches: the sound job at 60 Hz above 60 fps in", os.path.basename(path))
out = sound
solver, found = patch_solver_steps(out)
solver_sites += found
if solver != out:
print("oot_patches: the solver steps rounded up above 60 fps in", os.path.basename(path))
out = solver
if out != s:
whole_write.write_whole(path, out, newline="")
if sound_jobs != 1: # (without it, above 60 fps sounds and then the music can stop after a few loading zones)
print("oot_patches: WARNING: the sound job was found %d times, not once" % sound_jobs)
if solver_sites != len(SOLVER_HEADS): # (without them, above 60 fps moving platforms can drop Link)
print("oot_patches: WARNING: the solver step count was found at %d of %d places" % (solver_sites, len(SOLVER_HEADS)))
for what, n in done.items():
if n != 1:
print("oot_patches: WARNING: the %s was found %d times" % (what, n))
@@ -357,6 +557,9 @@ def apply(gen, *, protect_hearts=False, required=False):
whole_write.write_whole(path, unpatch_frame_seconds(s), newline="")
print("oot_patches: WARNING: the frame's time was found at %d of %d step divides, none patched (a Frame rate cap above 60 "
"keeps the game's own step)" % (step_sites, len(STEP_SITES)))
elif whole_sites != len(WHOLE_SITES): # (the frames' own time without them: the pick-up indicator stays above 60 fps)
print("oot_patches: WARNING: %d of %d frame counters found for the step's whole frames (above 60 fps a counter can stop: "
"the pick-up indicator, the water ripples, the time of day)" % (whole_sites, len(WHOLE_SITES)))
apply_output_size(gen, required=required)

View File

@@ -78,7 +78,7 @@ def main():
waits["%s: %s" % (th, guest_frame(frames))] += 1
if not is_game(th):
vk["%s %s: %s" % (th, "runs" if r else "waits", frames[0] if not r else
next((f for f in frames if f.startswith("hookshot_vk:")), frames[0]))] += 1
next((f for f in frames if f.startswith(("triforce_vk:", "hookshot_vk:"))), frames[0]))] += 1
for k, c in waits.most_common(top):
print(" %5.1f%% %s" % (100.0 * c / max(1, len(idle)), k))
print(" (renderer threads then)")

View File

@@ -29,8 +29,8 @@ def load_map(path, base):
# The modules whose frames are Hookshot's own code (the game and the runtime, the renderer): Windows, then Linux
GAME_MODS = ("hookshot.exe", "hookshot")
VK_MODS = ("hookshot_vk.dll", "hookshot_vk.so")
GAME_MODS = ("Triforce.exe", "hookshot.exe", "hookshot") # (Triforce's names first; upstream's kept for its samples)
VK_MODS = ("triforce_vk.dll", "hookshot_vk.dll", "hookshot_vk.so")
RUN_MODS = GAME_MODS + VK_MODS
@@ -53,7 +53,7 @@ def load_symbols(folder=None):
Windows build, the binaries of the Linux one"""
folder = folder or os.environ.get("RX_MAPS", BUILD)
maps = {}
for mod in ("hookshot_vk.dll", "hookshot.exe"):
for mod in ("triforce_vk.dll", "Triforce.exe", "hookshot_vk.dll", "hookshot.exe"):
mp = os.path.join(folder, mod[:-4] + ".map")
if os.path.exists(mp):
maps[mod] = load_map(mp, 0x180000000 if mod.endswith(".dll") else 0x140000000)

View File

@@ -7,7 +7,8 @@ param(
[Parameter(Mandatory = $true)][string]$Tag,
[string]$Deps = 'D:\src\release\triforce-deps',
[string]$Out = (Join-Path $PSScriptRoot 'out\release'),
[switch]$AllowNoCompiler
[switch]$AllowNoCompiler,
[switch]$AllowNoStateList
)
$ErrorActionPreference = 'Stop'
$repo = Split-Path $PSScriptRoot
@@ -45,6 +46,17 @@ New-Item -ItemType Directory -Force $work | Out-Null
foreach ($n in 'hot_funcs.txt', 'hot_order.txt', 'hookshot_pieces.prof') {
if (Test-Path (Join-Path $repo "work\$n")) { Copy-Item (Join-Path $repo "work\$n") $work }
}
# 3b. the pipeline state list (nx2/pipeline_states.py --out work\pipeline_states.bin <lists from play>): nx2/make_game.py puts it
# beside the game as triforce_pipeline_states_shipped.bin and the first start builds its pipelines before the title. Without it
# every player compiles each pipeline when it is first drawn: the stutter of every first visit (v0.2.0 shipped none: "0 shipped"
# in every log). Hashes and draw state only, no game code.
$states = Join-Path $repo 'work\pipeline_states.bin'
if (Test-Path $states) {
Copy-Item $states $work
"work: pipeline state list, {0:N0} pipelines" -f (((Get-Item $states).Length - 12) / 528)
} elseif (-not $AllowNoStateList) {
throw "no work\pipeline_states.bin: players would compile every pipeline in play. Make it with nx2/pipeline_states.py --out (or -AllowNoStateList)"
}
if (Test-Path (Join-Path $repo 'work\image')) { robocopy (Join-Path $repo 'work\image') (Join-Path $work 'image') /E /NFL /NDL /NJH /NJS /NP | Out-Null }
Add-Type -AssemblyName System.Drawing

View File

@@ -22,8 +22,8 @@ def main():
shader = (VK / "vk_shader.cpp").read_text(encoding="utf-8")
profile = shader[shader.index("Shader::Profile make_profile()"):shader.index("Shader::HostTranslateInfo host_info()")]
for needle, why in (
("dev.subgroup_size > 32 && (dev.sg32_stages & VK_SHADER_STAGE_ALL_GRAPHICS)",
"a graphics stage running wider than 32 lanes is not told to the translator"),
("p.warp_size_potentially_larger_than_guest = dev.subgroup_max > 32",
"an unpinned stage's maximum subgroup width is not told to the translator"),
("p.max_subgroup_size = p.warp_size_potentially_larger_than_guest ? dev.subgroup_max : 32",
"the broadcast fallbacks do not walk the host's upper lanes"),
("p.has_broken_spirv_fragment_shuffle = dev.driver_id == VK_DRIVER_ID_AMD_PROPRIETARY",

View File

@@ -0,0 +1,152 @@
"""Check 32- and 64-lane shader profiles and AMD's fragment shuffle path without a GPU."""
import argparse
import os
from pathlib import Path
import subprocess
import tempfile
RUN_ALL_ARGS = ["--cxx", "{cxx}"]
ROOT = Path(__file__).resolve().parents[1]
VK = ROOT / "gpu/vk"
FIXTURE = r'''
#include <algorithm>
#include <cassert>
#include <cstdint>
#include <cstdlib>
#include "shader_recompiler/profile.h"
#include "shader_recompiler/stage.h"
enum VkShaderStageFlagBits : uint32_t {
VK_SHADER_STAGE_VERTEX_BIT = 1,
VK_SHADER_STAGE_FRAGMENT_BIT = 16,
VK_SHADER_STAGE_COMPUTE_BIT = 32,
VK_SHADER_STAGE_TASK_BIT_EXT = 64,
VK_SHADER_STAGE_MESH_BIT_EXT = 128,
VK_SHADER_STAGE_ALL_GRAPHICS = 31,
VK_SHADER_STAGE_ALL = 0x7fffffff,
};
using VkShaderStageFlags = uint32_t;
constexpr uint32_t VK_DRIVER_ID_AMD_PROPRIETARY = 1;
struct {
uint32_t sg32_stages = 0, subgroup_size = 32, subgroup_max = 32, driver_id = 0;
struct { struct { uint64_t minStorageBufferOffsetAlignment = 16; } limits; } props;
} dev;
using Shader::Stage;
using Id = uint32_t;
struct EmitContext {
const Shader::Profile& profile;
Stage stage;
Id lane;
Id U32[2]{};
Id Const(Id value) { return value; }
Id OpShiftRightLogical(Id, Id a, Id b) { return a >> b; }
Id OpShiftLeftLogical(Id, Id a, Id b) { return a << b; }
Id OpIAdd(Id, Id a, Id b) { return a + b; }
};
Id GetThreadId(const EmitContext& ctx) { return ctx.lane; }
'''
CASES = r'''
void check(const Shader::Profile& p, uint32_t size) {
assert(p.max_subgroup_size == size);
assert(p.warp_size_potentially_larger_than_guest == (size > 32));
}
int main(int argc, char** argv) {
assert(argc == 12);
const auto number = [&](int i) { return uint32_t(std::strtoul(argv[i], nullptr, 10)); };
dev.subgroup_size = number(1);
dev.subgroup_max = number(2);
dev.sg32_stages = number(3);
dev.driver_id = number(4);
check(profile(), number(5));
check(profile_graphics(VK_SHADER_STAGE_VERTEX_BIT), number(6));
check(profile_graphics(VK_SHADER_STAGE_FRAGMENT_BIT), number(7));
check(profile_pinned(), number(8));
check(profile_graphics(VK_SHADER_STAGE_TASK_BIT_EXT), number(9));
check(profile_graphics(VK_SHADER_STAGE_MESH_BIT_EXT), number(10));
const bool workaround = number(11) != 0;
check_stamp(number(5), number(3), workaround, getenv("RX_VK_WARP64") != nullptr);
const auto& fragment = profile_graphics(VK_SHADER_STAGE_FRAGMENT_BIT);
const auto& compute = profile_pinned();
EmitContext lo{fragment, Stage::Fragment, 31};
EmitContext hi{fragment, Stage::Fragment, 32};
EmitContext cs{compute, Stage::Compute, 31};
assert(UseShuffleFallback(lo) == workaround && UseShuffleFallback(hi) == workaround);
assert(!UseShuffleFallback(cs));
assert(HostThreadId(lo, 7) == 7 && HostThreadId(cs, 7) == 7);
assert(HostThreadId(hi, 7) == (number(7) > 32 ? 39 : 7));
}
'''
def section(source, start, end):
first = source.index(start)
return source[first:source.index(end, first)]
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--cxx", default="clang++")
args = parser.parse_args()
shader = (VK / "vk_shader.cpp").read_text(encoding="utf-8")
warp = (VK / "eden/shader_recompiler/backend/spirv/emit_spirv_warp.cpp").read_text(encoding="utf-8")
stamp = section(shader, " const uint32_t lanes[]", " // (the records keep Shader::Info")
check_stamp = r'''
void check_stamp(uint32_t size, uint32_t pinned, bool workaround, bool forced) {
bool added = false;
const auto add = [&](const uint32_t* values, size_t bytes) {
assert(bytes == 5 * sizeof(uint32_t));
assert(values[0] == uint32_t(size > 32) && values[1] == size);
assert(values[2] == uint32_t(workaround) && values[3] == pinned && values[4] == uint32_t(forced));
added = true;
};
''' + stamp + r'''
assert(added);
}
'''
fixture = (FIXTURE + section(shader, "Shader::Profile make_profile(", "Shader::HostTranslateInfo host_info()")
+ section(shader, "const Shader::Profile& profile()", "const std::vector<Code>& codes_of(")
+ section(warp, "Id HostThreadId(", "Id GuestLane(")
+ section(warp, "bool UseShuffleFallback(", "Id ShuffleAbsolute(") + check_stamp + CASES)
with tempfile.TemporaryDirectory(prefix="hookshot-subgroup-profile-") as tmp:
cpp, exe = Path(tmp) / "fixture.cpp", Path(tmp) / "fixture.exe"
cpp.write_text(fixture, encoding="utf-8")
subprocess.run([args.cxx, "-std=c++20", "-I", str(VK / "eden"), "-I", str(ROOT / "gpu/third_party/suyu"),
str(cpp), "-o", str(exe)], check=True, timeout=60)
# A fresh process per device: the real profiles are cached once for the selected GPU.
# Inputs: default lanes, maximum lanes, pinned-stage mask, proprietary AMD driver.
# Expected widths: base, vertex, fragment, compute, task, mesh; then fragment workaround.
cases = [
((32, 32, 0, 0), (32, 32, 32, 32, 32, 32, 0), {}),
((64, 64, 32, 1), (64, 64, 64, 32, 64, 64, 1), {}),
((32, 64, 32, 1), (64, 64, 64, 32, 64, 64, 1), {}),
((32, 64, 0, 0), (64, 64, 64, 64, 64, 64, 0), {}),
((32, 64, 31, 1), (64, 32, 32, 64, 64, 64, 1), {}),
((64, 64, 240, 1), (64, 64, 32, 32, 32, 32, 1), {}),
((64, 64, 0, 0), (64, 64, 64, 64, 64, 64, 0), {}),
((64, 64, 32, 1), (32, 32, 32, 32, 32, 32, 1), {"RX_VK_WARP64": "0"}),
((64, 64, 240, 1), (64, 64, 64, 32, 64, 64, 1), {"RX_VK_WARP64": "1"}),
((64, 64, 32, 1), (64, 64, 64, 32, 64, 64, 0), {"RX_VK_FRAG_SHUFFLE_FALLBACK": "0"}),
((64, 64, 32, 0), (64, 64, 64, 32, 64, 64, 1), {"RX_VK_FRAG_SHUFFLE_FALLBACK": "1"}),
]
env = {k: v for k, v in os.environ.items() if k not in ("RX_VK_WARP64", "RX_VK_FRAG_SHUFFLE_FALLBACK")}
for device, expected, overrides in cases:
subprocess.run([str(exe), *map(str, device + expected)], env=env | overrides, check=True, timeout=10)
assert "EmitSPIRV(profile_pinned(), rt, t->program, bindings)" in shader
for stage in ("TASK", "MESH"):
assert f"fallback ? profile_pinned() : profile_graphics(VK_SHADER_STAGE_{stage}_BIT_EXT)" in shader
for start, end in (("Id ShuffleRelative(", "Id BroadcastLane("), ("Id EmitShuffleIndex(", "Id EmitShuffleUp("),
("Id EmitShuffleButterfly(", "Id EmitQuadBroadcast(")):
assert "UseShuffleFallback(ctx)" in section(warp, start, end)
quad = section(warp, "Id EmitQuadBroadcast(", "Id EmitQuadSwap(")
assert "ctx.profile.has_broken_spirv_subgroup_shuffle" in quad and "UseShuffleFallback" not in quad
print("stage subgroup widths, 32-lane partitions, AMD fragment shuffles, and cache identity ok")
if __name__ == "__main__":
main()

120
tests/test_frame_whole.py Normal file
View File

@@ -0,0 +1,120 @@
"""Game counters that add the frame step, or the step times a whole number, cut to whole frames keep counting above 60 fps
(host/frame_whole.h, nx2/oot_patches.py patch_whole_frames).
The pick-up indicator above Link's head (f_80a87e50) moves the count it shows toward the wallet's by (int)(step * per) and fades
when they meet; the time of day (f_803edf70) adds (int)(2 * step) game seconds a frame. Under a Frame rate cap above 60 the step is
the frame's own time, under 1 (0.42 at 144 fps): cut to 0 every frame, the count never moved and the indicator stayed, and the
clock stood still. With the frames' own time
as the step (carry), hos_frame_whole adds the amount to the counter's own remainder and hands out its whole frames: 144 frames of
60/144 are 60; an amount of 1 or more keeps its fraction too; two counters keep their own remainders; NaN and amounts below zero
give 0. Without it, the amount is cut as the game cuts it (60 fps as before). The water renderer (f_800672e0) runs its ripple
simulation (int)step times a frame the same way, its remainder kept per water object: forty surfaces stepped in turn each still
count their 60 frames a second. The patch replaces the one cut of each site, once,
declares the hook, and leaves a file patched before and a function whose code differs as they are.
python -I -B tests/test_frame_whole.py
"""
import shutil
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "nx2"))
import oot_patches # noqa: E402
FIXTURE = r"""
#include "frame_whole.h"
#include <math.h>
#include <stdio.h>
static int fails;
#define CHECK(c) do { if (!(c)) { printf("FAIL line %d: %s\n", __LINE__, #c); fails++; } } while (0)
static uint32_t bits(float f) { uint32_t b; memcpy(&b, &f, 4); return b; }
int main(void) {
static FrameWhole w;
const uint64_t a = 0x1000, b = 0x2000;
uint32_t n = 0;
for (int i = 0; i < 144; i++) n += frame_whole(&w, a, bits(60.0f / 144.0f), 1);
CHECK(n == 60 || n == 59); /* (a second at 144 fps: 60 frames, give or take the float's last one) */
n = 0;
for (int i = 0; i < 1440; i++) n += frame_whole(&w, a, bits(60.0f / 144.0f), 1);
CHECK(n >= 599 && n <= 601);
CHECK(frame_whole(&w, b, bits(0.42f), 1) == 0 && frame_whole(&w, b, bits(0.42f), 1) == 0 && frame_whole(&w, b, bits(0.42f), 1) == 1);
n = 0;
for (int i = 0; i < 4; i++) n += frame_whole(&w, b + 8, bits(1.25f), 1); /* (the step times 3: its fractions kept) */
CHECK(n == 5);
n = 0;
for (int i = 0; i < 144; i++) n += frame_whole(&w, b + 16, bits(2.0f * 60.0f / 144.0f), 1); /* (the time of day) */
CHECK(n == 120 || n == 119); /* (a second at 144 fps: the 120 game seconds of one at 60) */
CHECK(frame_whole(&w, a, bits(0.42f), 0) == 0 && frame_whole(&w, a, bits(1.5f), 0) == 1 && frame_whole(&w, a, bits(2.0f), 0) == 2);
CHECK(frame_whole(&w, a, bits(-1.0f), 1) == 0 && frame_whole(&w, a, bits(NAN), 1) == 0 && frame_whole(&w, a, bits(0.0f), 1) == 0);
for (uint64_t k = 0; k < 100; k++) frame_whole(&w, 0x9000 + k, bits(0.5f), 1); /* (more counters than slots: the oldest go) */
CHECK(frame_whole(&w, 0x9000 + 99, bits(0.5f), 1) == 1); /* (a slot still held: 0.5 + 0.5) */
CHECK(frame_whole(&w, 0x9000, bits(0.5f), 1) == 0); /* (the oldest's slot was reused: it starts again from 0) */
/* (a scene's water surfaces, each stepped once a frame at 144 fps: 40 of them, more than the old 16 slots, each still counts
its 60 frames a second) */
{
static FrameWhole water;
uint32_t passes[40] = {0};
for (int f = 0; f < 144; f++)
for (int k = 0; k < 40; k++) passes[k] += frame_whole(&water, 0x7000000 + (uint64_t)k * 0x400, bits(60.0f / 144.0f), 1);
for (int k = 0; k < 40; k++) CHECK(passes[k] == 60 || passes[k] == 59);
}
if (!fails) printf("PASS\n");
return fails != 0;
}
"""
LIFTED = ['#include "a64.h"\n' + head + " c->x[9] = recomp_load32(c,(c->x[19] + 864));\n" + old + "return;\n}\n"
for head, old, _ in oot_patches.WHOLE_SITES]
class FrameWholeTests(unittest.TestCase):
def test_counter_keeps_counting_under_one(self):
compiler = shutil.which("gcc") or shutil.which("clang")
if not compiler:
self.skipTest("no C compiler")
with tempfile.TemporaryDirectory(prefix="hookshot-frame-whole-") as tmp:
tmp = Path(tmp)
(tmp / "t.c").write_text(FIXTURE, encoding="utf-8")
binary = tmp / "t.exe"
build = subprocess.run([compiler, "-std=gnu11", "-Wall", "-O1", "-I", str(ROOT / "host"), str(tmp / "t.c"),
"-o", str(binary), "-lm"], text=True, capture_output=True)
self.assertEqual(build.returncode, 0, build.stderr)
run = subprocess.run([str(binary)], text=True, capture_output=True, timeout=30)
self.assertEqual(run.returncode, 0, run.stdout + run.stderr)
def test_patch_replaces_the_cut_once(self):
for lifted, (_, old, key) in zip(LIFTED, oot_patches.WHOLE_SITES):
out, n = oot_patches.patch_whole_frames(lifted)
self.assertEqual(n, 1)
self.assertEqual(out.count("=hos_frame_whole(%s, (uint32_t)c->vreg[0][0]);" % key), 1)
self.assertNotIn("recomp_fp_to_int", out)
self.assertIn(oot_patches.WHOLE_DECL, out)
self.assertEqual(oot_patches.patch_whole_frames(out), (out, 0)) # (patched before: as it is)
other = lifted.replace("32,32,", "32,31,") # (another game version's code)
self.assertEqual(oot_patches.patch_whole_frames(other), (other, 0))
none = lifted.replace("void f_8", "void f_9")
self.assertEqual(oot_patches.patch_whole_frames(none), (none, 0))
# (a lift where the function is hot: its float helpers are a64_fpx_hot_*, the cut the same)
_, old0, _ = oot_patches.WHOLE_SITES[0]
hot = LIFTED[0].replace(old0, " _v=a64_fpx_hot_mul32_fma32(c,_a,_b,_z);\n a64_vset64(c->vreg[0],_v);}\n" + old0)
self.assertEqual(oot_patches.patch_whole_frames(hot)[1], 1)
def test_clock_patches_its_seconds_alone(self):
# (f_803edf70 cuts into x8 the same way once more, in the branch that sets the time from another source: that cut
# stays the game's)
i = next(i for i, (h, _, _) in enumerate(oot_patches.WHOLE_SITES) if "f_803edf70" in h)
cut = " c->x[8]=recomp_fp_to_int(c->vreg[0][0]&UINT64_C(0xffffffff),32,32,1,3,c->fpcr,&c->fpsr);\n"
lifted = LIFTED[i].replace("return;\n", cut + " recomp_store32(c,c->x[19] + 12,c->x[8]);\nreturn;\n")
out, n = oot_patches.patch_whole_frames(lifted)
self.assertEqual(n, 1)
self.assertIn("=hos_frame_whole(c->x[19] + 60, (uint32_t)c->vreg[0][0]);", out)
self.assertEqual(out.count(cut), 1)
self.assertIn(cut + " recomp_store32(c,c->x[19] + 12,", out)
if __name__ == "__main__":
raise SystemExit(unittest.main(verbosity=1))

View File

@@ -0,0 +1,172 @@
"""Above a 60 fps cap the sound job runs at 60 Hz and the physics solver keeps whole-frame steps (host/oot_high_fps.h,
nx2/oot_patches.py patch_sound_job and patch_solver_steps).
Sound: under a Frame rate cap above 60 the frame step is the frame's own time (0.5 at 120 fps). The sound manager's job (f_80435300)
fed steps under 1 left sounds unreleased at area changes until the game's voice budget was full and sounds, then the music,
stopped. The gate runs phase 0 once a whole step has built up (120 frames of 0.5: 60 runs; 144 of 60/144: 60), the other phases
follow phase 0's choice, the engine is handed a whole step then, and a step of 1 or more, 0 or below, NaN, or no frame time at
all (a cap of 60) runs every frame with the game's own step. A step of 1 or more keeps the remainder, so around 60 fps (0.98 and
1.02 alternating, 0.9 and 1.1, or mean 1 with jitter) the sound clock keeps the real one; a cap of 60 runs every phase, before
the first phase 0 too and after a phase 0 that held the job.
Solver: the game cuts frame step * n + 0.001 to its solver steps (n a 60 fps frame: 8 at 60 fps, 3 or 4 at 120 as the step
wanders; with n = 4, 0 above 240). With the frames' own time and a step under 1 it is rounded up (10% leeway for the frame time's
jitter) and at least 1, never under the game's; a step of 1 or more, or no frame time (a cap of 60), keeps the game's count.
The patches put the gate at the job's head and the step through the host, hook both solver places, declare the hooks, and leave
a file patched before, or whose code differs, as it is.
python -I -B tests/test_high_fps_fixes.py
"""
import shutil
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "nx2"))
import oot_patches # noqa: E402
FIXTURE = r"""
#include "oot_high_fps.h"
#include <stdio.h>
static int fails;
#define CHECK(c) do { if (!(c)) { printf("FAIL line %d: %s\n", __LINE__, #c); fails++; } } while (0)
static uint32_t bits(float f) { return oot_float_bits(f); }
static int runs(float step, int frames, int timed) {
OotSoundGate g = OOT_SOUND_GATE_INIT;
int n = 0;
for (int i = 0; i < frames; i++) {
const int run = oot_sound_gate(&g, 0, step, timed);
CHECK(oot_sound_gate(&g, 1, step, timed) == run); /* (phase 1 follows phase 0) */
n += run;
}
return n;
}
/* (the sound engine's time over these frames' steps against the real time: what it is handed each frame the job runs) */
static double sound_clock(const float *steps, int n) {
OotSoundGate g = OOT_SOUND_GATE_INIT;
double real = 0, sound = 0;
for (int i = 0; i < n; i++) {
real += steps[i];
if (oot_sound_gate(&g, 0, steps[i], 1)) sound += oot_bits_float(oot_sound_step(oot_float_bits(steps[i]), 1));
}
return sound / real;
}
int main(void) {
CHECK(runs(0.5f, 120, 1) == 60);
const int n144 = runs(60.0f / 144.0f, 144, 1);
CHECK(n144 == 60 || n144 == 59);
CHECK(runs(60.0f / 1000.0f, 1000, 1) >= 59 && runs(60.0f / 1000.0f, 1000, 1) <= 60);
CHECK(runs(0.5f, 120, 0) == 120); /* (a cap of 60: every frame) */
CHECK(runs(1.0f, 60, 1) == 60 && runs(1.5f, 60, 1) == 60);
CHECK(runs(0.0f, 60, 1) == 60 && runs(-1.0f, 60, 1) == 60 && runs(NAN, 60, 1) == 60);
{ /* (a whole step after partial ones runs with the game's step and keeps the remainder) */
OotSoundGate g = OOT_SOUND_GATE_INIT;
CHECK(oot_sound_gate(&g, 0, 0.75f, 1) == 0 && oot_sound_gate(&g, 2, 0.75f, 1) == 0);
CHECK(oot_sound_gate(&g, 0, 1.0f, 1) == 1 && g.part == 0.75f && oot_sound_gate(&g, 1, 1.0f, 1) == 1);
CHECK(oot_sound_gate(&g, 0, 0.75f, 1) == 1 && g.part == 0.5f);
CHECK(oot_sound_gate(&g, 0, 0.0f, 1) == 1 && g.part == 0.0f); /* (a step of 0: as the game has it, the remainder cleared) */
}
{ /* (around 60 fps the sound clock keeps the real one: dropping the remainder at each whole step made it 0.51 of it) */
static float steps[20000];
for (int i = 0; i < 20000; i++) steps[i] = i & 1 ? 1.02f : 0.98f;
double r = sound_clock(steps, 20000);
CHECK(r > 0.999 && r < 1.001);
for (int i = 0; i < 20000; i++) steps[i] = i & 1 ? 1.1f : 0.9f;
r = sound_clock(steps, 20000);
CHECK(r > 0.999 && r < 1.001);
uint32_t x = 12345; /* (mean 1, about 0.03 apart: a sum of uniforms) */
for (int i = 0; i < 20000; i++) {
float u = 0;
for (int k = 0; k < 12; k++) x = x * 1664525u + 1013904223u, u += (float)(x >> 8) / 16777216.0f;
steps[i] = 1.0f + (u - 6.0f) * 0.03f;
}
r = sound_clock(steps, 20000);
CHECK(r > 0.999 && r < 1.001);
for (int i = 0; i < 20000; i++) steps[i] = 0.5f;
r = sound_clock(steps, 20000);
CHECK(r > 0.999 && r < 1.001);
}
{ /* (a cap of 60 runs every phase: before any phase 0, and after a phase 0 that held the job at a cap above 60) */
OotSoundGate g = OOT_SOUND_GATE_INIT;
CHECK(oot_sound_gate(&g, 1, 1.0f, 0) == 1 && oot_sound_gate(&g, 2, 1.0f, 0) == 1);
OotSoundGate h = OOT_SOUND_GATE_INIT;
CHECK(oot_sound_gate(&h, 1, 1.0f, 1) == 1); /* (above 60, before the first phase 0: it runs) */
CHECK(oot_sound_gate(&h, 0, 0.5f, 1) == 0 && oot_sound_gate(&h, 1, 0.5f, 0) == 1 && h.part == 0.0f); /* (the cap changed) */
CHECK(oot_sound_gate(&h, 2, 0.5f, 0) == 1 && oot_sound_gate(&h, 0, 0.5f, 0) == 1);
}
CHECK(oot_sound_step(bits(0.5f), 1) == bits(1.0f));
CHECK(oot_sound_step(bits(0.5f), 0) == bits(0.5f));
CHECK(oot_sound_step(bits(1.25f), 1) == bits(1.25f) && oot_sound_step(bits(0.0f), 1) == bits(0.0f));
/* (the game's: (int)(step * n + 0.001), n the steps of a 60 fps frame) */
#define GAME(step, n) ((int32_t)((step) * (n) + 0.001f))
#define OURS(step, n, timed) oot_solver_steps(GAME(step, n), bits((step) * (n) + 0.001f), bits(n), timed)
CHECK(OURS(1.0f, 4.0f, 1) == 4 && OURS(1.0f, 8.0f, 1) == 8 && OURS(1.0f, 8.0f, 0) == 8);
CHECK(OURS(0.5f, 4.0f, 1) == 2 && OURS(0.49f, 4.0f, 1) == 2 && OURS(0.52f, 4.0f, 1) == 2 && GAME(0.49f, 4.0f) == 1);
CHECK(OURS(0.49f, 8.0f, 1) == 4 && OURS(0.5f, 8.0f, 1) == 4 && GAME(0.49f, 8.0f) == 3);
CHECK(OURS(0.25f, 4.0f, 1) == 1 && OURS(0.24f, 4.0f, 1) == 1 && OURS(0.06f, 4.0f, 1) == 1 && GAME(0.24f, 4.0f) == 0);
CHECK(OURS(60.0f / 144.0f, 4.0f, 1) == 2 && OURS(60.0f / 90.0f, 4.0f, 1) == 3);
CHECK(OURS(0.49f, 4.0f, 0) == 1 && OURS(0.06f, 4.0f, 0) == 0); /* (a cap of 60: the game's count) */
CHECK(OURS(1.6f, 4.0f, 1) == 6 && OURS(1.708f, 8.0f, 1) == 13); /* (a slow frame: the game's count) */
CHECK(oot_solver_steps(3, bits(NAN), bits(4.0f), 1) == 3 && oot_solver_steps(0, bits(-1.0f), bits(4.0f), 1) == 0);
CHECK(oot_solver_steps(1, bits(2.0f), bits(0.0f), 1) == 1); /* (no steps a frame: the game's) */
if (!fails) printf("PASS\n");
return fails != 0;
}
"""
LIFTED_SOUND = ('#include "a64.h"\n' + oot_patches.SOUND_HEAD + " /* nop/hint */\nif ((uint32_t)c->x[2] != 0) { goto L_1; }\n"
+ oot_patches.SOUND_OLD + "L_1:;\n return;\n}\nvoid f_80435360(Ctx *__restrict A64_NOESC c) {\n"
+ oot_patches.SOUND_OLD + "}\n")
LIFTED_SOLVER = '#include "a64.h"\n' + "".join(head + " c->x[8]=1;\n" + oot_patches.SOLVER_OLD + "}\n"
for head in oot_patches.SOLVER_HEADS)
class HighFpsTests(unittest.TestCase):
def test_gate_and_steps(self):
compiler = shutil.which("gcc") or shutil.which("clang")
if not compiler:
self.skipTest("no C compiler")
with tempfile.TemporaryDirectory(prefix="hookshot-high-fps-") as tmp:
tmp = Path(tmp)
(tmp / "t.c").write_text(FIXTURE, encoding="utf-8")
binary = tmp / "t.exe"
build = subprocess.run([compiler, "-std=gnu11", "-Wall", "-O1", "-I", str(ROOT / "host"), str(tmp / "t.c"),
"-o", str(binary), "-lm"], text=True, capture_output=True)
self.assertEqual(build.returncode, 0, build.stderr)
run = subprocess.run([str(binary)], text=True, capture_output=True, timeout=30)
self.assertEqual(run.returncode, 0, run.stdout + run.stderr)
def test_sound_patch(self):
out, n = oot_patches.patch_sound_job(LIFTED_SOUND)
self.assertEqual(n, 1)
head = out.index(oot_patches.SOUND_HEAD) + len(oot_patches.SOUND_HEAD)
self.assertTrue(out[head:].startswith(oot_patches.SOUND_GATE)) # (before the job's own first instruction)
self.assertEqual(out.count("hos_oot_sound_gate(c,"), 1)
self.assertEqual(out.count(oot_patches.SOUND_NEW), 1) # (the job's own load only, not the next function's)
self.assertEqual(out.count(oot_patches.SOUND_OLD), 1)
self.assertIn(oot_patches.SOUND_DECL, out)
self.assertEqual(oot_patches.patch_sound_job(out), (out, 1)) # (patched before: as it is)
other = LIFTED_SOUND.replace("+ 2192)", "+ 2200)") # (another game version's code)
self.assertEqual(oot_patches.patch_sound_job(other), (other, 0))
none = LIFTED_SOUND.replace("void f_80435300", "void f_90435300")
self.assertEqual(oot_patches.patch_sound_job(none), (none, 0))
def test_solver_patch(self):
out, n = oot_patches.patch_solver_steps(LIFTED_SOLVER)
self.assertEqual(n, 2)
self.assertEqual(out.count(oot_patches.SOLVER_NEW), 2)
self.assertIn(oot_patches.SOLVER_DECL, out)
hook = out.index("hos_oot_solver_steps((uint32_t)c->x[1]")
self.assertLess(out.index("recomp_fp_to_int"), hook)
self.assertLess(hook, out.index("recomp_store32(c,c->x[0] + 712"))
self.assertEqual(oot_patches.patch_solver_steps(out), (out, 2))
other = LIFTED_SOLVER.replace("+ 712", "+ 716")
self.assertEqual(oot_patches.patch_solver_steps(other), (other, 0))
if __name__ == "__main__":
raise SystemExit(unittest.main(verbosity=1))

View File

@@ -0,0 +1,236 @@
"""Above a 60 cap a movie's frame count grows by the frames' time, so a cutscene's cues stay with its movie (issue 1742).
The game's movie player counts its updates while a movie plays, and a cutscene that shows a movie places its narration, its
rumble and its end by that count. The movie's picture and sound follow the clock, so at 120 fps the cues came twice too early:
narration lines started over each other. nx2/oot_patches.py makes the count's step hos_movie_frames(player), and host/nn_vi.c
answers 1 while the host gives no frame time (a cap of 60: as shipped) and the frames' time in 60ths, the rest carried to the
player's next update, while it does (HOS_FPS_MOVIE=0: 1 then too, for comparisons). A game version whose code differs keeps
its own code and the build goes on with a warning.
python -I -B tests/test_movie_frame_count.py [--cc gcc]
"""
import argparse
import contextlib
import io
import os
import re
import subprocess
import sys
import tempfile
from pathlib import Path
RUN_ALL_ARGS = ["--cc", "gcc"]
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "nx2"))
import oot_patches # noqa: E402
# (the shape of the lifted update: the count loaded, one added, stored; the same lines in another function stay as they are)
STORE = " recomp_store32(c,c->x[20] + 760,c->x[8]);\n"
LIFTED = ('#include "a64.h"\n'
"void f_83b7c710(Ctx *__restrict A64_NOESC c) {\n"
" c->x[8] = 0x1ULL;\n" + oot_patches.MOVIE_OLD + STORE + "}\n"
"void f_83b7c870(Ctx *__restrict A64_NOESC c) {\n" + oot_patches.MOVIE_OLD + STORE + "}\n")
HOST_ANCHOR = "uint32_t hos_movie_frames(uint64_t player) {"
def block(source, marker):
if marker not in source:
raise SystemExit("FAIL: host/nn_vi.c has no '%s'" % marker)
start = source.index(marker)
return source[start:start + re.search(r"^}", source[start:], re.M).end()]
def check(cond, what):
if not cond:
raise SystemExit("FAIL: " + what)
FIXTURE = r'''#define _CRT_SECURE_NO_WARNINGS
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define CHECK(x) do { if (!(x)) { fprintf(stderr, "FAIL:%d %s\n", __LINE__, #x); exit(1); } } while (0)
/* (the lock as a flag: taken once, given back once) */
typedef int SRWLOCK;
#define SRWLOCK_INIT 0
static void AcquireSRWLockExclusive(SRWLOCK *l) { CHECK(!*l); *l = 1; }
static void ReleaseSRWLockExclusive(SRWLOCK *l) { CHECK(*l); *l = 0; }
volatile uint32_t hos_frame_seconds_bits;
@HOST@
/* (the lifted update with the patch, on a small guest: the count at +760 of the player) */
typedef struct { uint64_t x[31]; } Ctx;
static uint32_t guest[2][256];
static uint32_t recomp_load32(Ctx *c, uint64_t a) { (void)c; return guest[a >> 12][(a & 0xfff) / 4]; }
static void recomp_store32(Ctx *c, uint64_t a, uint64_t v) { (void)c; guest[a >> 12][(a & 0xfff) / 4] = (uint32_t)v; }
static uint32_t update(int player) {
Ctx ctx, *c = &ctx;
memset(&ctx, 0, sizeof ctx);
c->x[20] = (uint64_t)player << 12;
@LIFTED@
return guest[player][760 / 4];
}
static void seconds(double s) {
const float f = (float)s;
uint32_t bits;
memcpy(&bits, &f, 4);
hos_frame_seconds_bits = s > 0 ? bits : 0;
}
static int within(uint32_t got, uint32_t want) { return got + 1 >= want && got <= want + 1; }
int main(int argc, char **argv) {
uint32_t n = 0, before;
if (argc > 1 && !strcmp(argv[1], "off")) { /* (HOS_FPS_MOVIE=0: the game's own count at any rate) */
seconds(1.0 / 120);
for (int i = 0; i < 600; i++) n = update(0);
CHECK(n == 600);
printf("PASS: off\n");
return 0;
}
/* no frame time (a cap of 60): one an update, as the game counts */
seconds(0);
for (int i = 0; i < 600; i++) n = update(0);
CHECK(n == 600);
CHECK(hos_movie_frames(0x5000) == 1 && hos_movie_frames(0x5000) == 1);
/* 120 fps: half a 60th an update, a whole one every second update */
seconds(1.0 / 120);
before = n;
for (int i = 0; i < 600; i++) {
const uint32_t was = n;
n = update(0);
CHECK(n - was <= 1);
}
CHECK(within(n - before, 300));
/* 240 fps and 40 fps: a quarter and one and a half */
seconds(1.0 / 240);
before = n;
for (int i = 0; i < 960; i++) n = update(0);
CHECK(within(n - before, 240));
seconds(1.0 / 40);
before = n;
for (int i = 0; i < 400; i++) {
const uint32_t was = n;
n = update(0);
CHECK(n - was == 1 || n - was == 2);
}
CHECK(within(n - before, 600));
/* two movies at once: each its own count (one carry for both would give each half) */
seconds(1.0 / 120);
before = n;
{
const uint32_t other = update(1) - 0; /* (its first update) */
uint32_t m = other;
for (int i = 0; i < 600; i++) n = update(0), m = update(1);
CHECK(within(n - before, 300) && within(m - other, 300));
}
/* a movie that plays while others come and go keeps its count: 14 more that play with it and a new player every update,
16 at a time, at 240 fps (places taken in the order they were made gave this one a new start every second update: a
quarter of a 60th an update then counted 0 or 1 every two, 1200 only on average) */
seconds(1.0 / 240);
{
uint32_t mine = 0, steady = 0;
for (int i = 0; i < 4800; i++) {
mine += hos_movie_frames(0x7000000);
for (int k = 0; k < 14; k++) steady += hos_movie_frames(0x7100000 + (uint64_t)k * 0x1000);
hos_movie_frames(0x8000000 + (uint64_t)i * 0x1000);
}
CHECK(within(mine, 1200) && steady + 14 >= 14 * 1200 && steady <= 14 * 1200 + 14);
}
seconds(1.0 / 120);
/* more movies at once than the host keeps places for, each updated in turn: every update finds its player's place taken and
starts from a drawn part of a 60th, so 480 updates of half a 60th give 240 on average, 11 either way as one standard
deviation. Every count within 5 of those, for any number of players: with a new place started at 0 every count stood
still, and with a fixed step between the starts one player's did among 233 (and 29 players gave 108 to 131 of 120) */
{
static const int counts[] = {9, 17, 29, 40, 233, 377};
static uint32_t sum[377];
uint32_t least = ~0u, most = 0;
for (unsigned t = 0; t < sizeof counts / sizeof counts[0]; t++) {
const int players = counts[t];
memset(sum, 0, sizeof sum);
for (int i = 0; i < 480; i++)
for (int k = 0; k < players; k++) sum[k] += hos_movie_frames(0x100000 + (uint64_t)t * 0x1000000 + (uint64_t)k * 0x1000);
for (int k = 0; k < players; k++) {
CHECK(sum[k] >= 240 - 55 && sum[k] <= 240 + 55);
least = sum[k] < least ? sum[k] : least, most = sum[k] > most ? sum[k] : most;
}
}
printf("more players than places: %u to %u of 240\n", least, most);
}
before = n;
for (int i = 0; i < 600; i++) n = update(0);
CHECK(within(n - before, 300));
/* back at a cap of 60: one an update again */
seconds(0);
before = n;
for (int i = 0; i < 10; i++) n = update(0);
CHECK(n - before == 10);
printf("PASS\n");
return 0;
}
'''
def main():
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("--cc", default="gcc")
a = p.parse_args()
out = oot_patches.patch_movie_frames(LIFTED)
check(out is not None and out.count("_b=hos_movie_frames(c->x[20])") == 1, "the count's step is not the host's, once")
update = out[out.index("void f_83b7c710"):out.index("void f_83b7c870")]
check("hos_movie_frames" in update and "hos_movie_frames" not in out[out.index("void f_83b7c870"):],
"the same lines in another function were patched")
check(out.startswith('#include "a64.h"\n' + oot_patches.MOVIE_DECL), "the lifted file does not declare the host's function")
back = out.replace(oot_patches.MOVIE_DECL, "", 1).replace(oot_patches.MOVIE_NEW, oot_patches.MOVIE_OLD)
check(back == LIFTED, "more than the count's step changed")
check(oot_patches.patch_movie_frames(out) == out, "a second run changed a patched file")
check(oot_patches.patch_movie_frames(LIFTED.replace("f_83b7c710", "f_83b7c714")) is None,
"a file without the function was not left to the caller")
for what, other in (("another step", LIFTED.replace("_b=1ULL", "_b=2ULL")),
("another field", LIFTED.replace("+ 760", "+ 764")),
("the lines twice", LIFTED.replace(STORE + "}\nvoid f_83b7c870", STORE + oot_patches.MOVIE_OLD + STORE + "}\nvoid f_83b7c870"))):
check(oot_patches.patch_movie_frames(other) == other, "a function with %s was patched (another game version)" % what)
with tempfile.TemporaryDirectory(prefix="movie-count-") as tmp:
f = Path(tmp) / "code_000.c"
f.write_text(LIFTED, encoding="utf-8", newline="")
for run in ("first", "second"): # (the second over the patched file: counted as made, nothing written)
log = io.StringIO()
with contextlib.redirect_stdout(log):
oot_patches.apply(tmp)
check("WARNING: the movie's frame count" not in log.getvalue(), "the %s run warns though the count is patched" % run)
check(f.read_text(encoding="utf-8") == out, "the %s run did not leave the patched file" % run)
other = LIFTED.replace("_b=1ULL", "_b=2ULL")
f.write_text(other, encoding="utf-8", newline="")
log = io.StringIO()
with contextlib.redirect_stdout(log):
oot_patches.apply(tmp)
check("WARNING: the movie's frame count was found 0 times" in log.getvalue(), "no warning for a count that was not found")
check(f.read_text(encoding="utf-8") == other, "another version's function was changed")
vi = (ROOT / "host" / "nn_vi.c").read_text(encoding="utf-8")
lifted = oot_patches.MOVIE_NEW + STORE
fixture = FIXTURE.replace("@HOST@", block(vi, HOST_ANCHOR)).replace("@LIFTED@", lifted)
with tempfile.TemporaryDirectory(prefix="movie-count-") as tmp:
c = Path(tmp) / "fixture.c"
exe = Path(tmp) / "fixture.exe"
c.write_text(fixture, encoding="utf-8")
subprocess.run([a.cc, "-std=c11", "-Wall", "-Wextra", "-Werror", str(c), "-o", str(exe)], check=True)
env = {k: v for k, v in os.environ.items() if not k.startswith("HOS_")}
subprocess.run([str(exe)], env=env, check=True)
subprocess.run([str(exe)], env=dict(env, HOS_FPS_MOVIE="1"), check=True) # (read by its value: 1 is on)
subprocess.run([str(exe), "off"], env=dict(env, HOS_FPS_MOVIE="0"), check=True)
r = subprocess.run([str(exe), "off"], env=env, capture_output=True, text=True)
check(r.returncode != 0 and "FAIL:" in r.stderr, "the count is the game's own without HOS_FPS_MOVIE=0")
# (the negative control: the game's own step of one an update fails the same checks at 120 fps)
plain = Path(tmp) / "plain.c"
plain.write_text(FIXTURE.replace("@HOST@", block(vi, HOST_ANCHOR)).replace("@LIFTED@", oot_patches.MOVIE_OLD + STORE),
encoding="utf-8")
subprocess.run([a.cc, "-std=c11", "-Wall", "-Wextra", "-Werror", "-Wno-unused-function", str(plain), "-o", str(exe)], check=True)
r = subprocess.run([str(exe)], env=env, capture_output=True, text=True)
check(r.returncode != 0 and "FAIL:" in r.stderr, "the unpatched count passes the checks: they prove nothing")
if __name__ == "__main__":
main()