mirror of
https://git.triforceproject.com/triforce/source
synced 2026-11-20 16:49:57 -04:00
Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
afbb0d140a | ||
|
|
362d576c7d | ||
|
|
a939e9d0d1 | ||
|
|
035977a061 | ||
|
|
b855d86841 | ||
|
|
a6bb7ffa09 |
321
CONTRIBUTING.md
321
CONTRIBUTING.md
@@ -1,12 +1,13 @@
|
||||
# Contributing to Hookshot
|
||||
# Contributing to Triforce
|
||||
|
||||
Hookshot is a static recompilation of the Switch 2 *The Legend of Zelda: Ocarina of Time* remake: the game's AArch64
|
||||
code is lifted to C, the Horizon OS and NVN2 are implemented natively, and the GPU work is replayed on Vulkan. No
|
||||
emulator is involved, and the game is built on your PC from your own copy.
|
||||
Triforce is a native PC port of the Switch 2 *The Legend of Zelda: Ocarina of Time* remake. The game's code is
|
||||
translated to C and built on your own PC from your own copy of the game; the system and the graphics are rebuilt
|
||||
natively, with the graphics replayed on Vulkan. No emulator is involved. Triforce builds on the open-source Hookshot
|
||||
project (GPL-3.0), and its code keeps that license.
|
||||
|
||||
This page covers what to send, how to build and test it, and the rules that keep the project out of trouble. The
|
||||
[development guide](docs/development.md) has the full pipeline and its commands; the [design notes](docs/design.md)
|
||||
have the architecture and the lessons the code follows.
|
||||
have the architecture.
|
||||
|
||||
## The rule with no exceptions
|
||||
|
||||
@@ -14,273 +15,135 @@ have the architecture and the lessons the code follows.
|
||||
never will. That includes:
|
||||
|
||||
- the `.nup`, `romfs`, `exefs`, `.nso`, `.nca`, `.nsp`, `.xci` or `.nsz`/`.xcz` files, and any title keys;
|
||||
- `work/` and everything in it: the extracted dump, `image.bin`, the lifted C, test builds, runs and screenshots;
|
||||
- `hookshot.exe`, `hookshot_vk.dll`, `hookshot_programs.bin` (and its `hookshot_programs.idx`), `hookshot_pipelines.bin`,
|
||||
`mkw_vk_pipelines.bin` and other build output;
|
||||
- saves (`user/`), `hookshot_options.ini`, `hookshot_env.txt` and `language.txt`.
|
||||
- `work/` and everything in it: the extracted dump, `image.bin`, the translated C, test builds, runs and screenshots;
|
||||
- `Triforce.exe`, `triforce_vk.dll`, `triforce_shaders.bin`, `triforce_programs.bin` (and `.idx`),
|
||||
`triforce_pipelines.bin`, `mkw_vk_pipelines.bin` and every other file a build or a run makes;
|
||||
- saves (`user/`), `triforce_options.ini`, `triforce_env.txt` and `language.txt`.
|
||||
|
||||
Do not open an issue or a PR asking where to get the game, how to decrypt it, or for someone else's files. A
|
||||
screenshot, a video and `hookshot_log.txt` are fine and often necessary; the log holds no game data.
|
||||
Do not open an issue or a pull request asking where to get the game, how to decrypt it, or for someone else's files.
|
||||
A screenshot, a video and `triforce_log.txt` are fine and often necessary; the log holds no game data.
|
||||
|
||||
`.gitignore` covers most of this, but check `git status` before you commit. If you are not sure whether a file is
|
||||
game data, leave it out.
|
||||
|
||||
## Ways to help
|
||||
## Reporting a problem
|
||||
|
||||
What to work on first, and the steps from claiming a problem to its release, are in the [workflow](docs/workflow.md):
|
||||
priorities by severity and by how many players run the affected hardware, and the hardware tiers a change must not
|
||||
break. Claim in Discord #dev-work before you start.
|
||||
Open an [issue](https://github.com/ruckus7382sfgsgdf/CBeta0.2/issues) and include:
|
||||
|
||||
- the Triforce version (the release name, or the first line of `triforce_log.txt`);
|
||||
- your graphics card, its driver version, and your Windows version;
|
||||
- where you were in the game and what you did;
|
||||
- the logs:
|
||||
- **a game problem:** `triforce_log.txt` from your game folder (the folder with `Triforce.exe`). If you started the
|
||||
game again after the problem, send `triforce_log.1.txt` too: each start keeps the four runs before it as
|
||||
`triforce_log.1.txt` (the last) to `triforce_log.4.txt`.
|
||||
- **a Setup or build problem:** `setup.log` from `%LOCALAPPDATA%\Programs\Triforce`, and for a failed build the text
|
||||
from **Show details** on the build page.
|
||||
|
||||
Before you report:
|
||||
|
||||
- Try the [newest release](https://github.com/ruckus7382sfgsgdf/CBeta0.2/releases) first.
|
||||
- Search the open issues; if yours is there, add your details to it.
|
||||
- For graphics or performance problems, test without ReShade, overlays, shader-cache tools and frame-generation
|
||||
injectors: they change what Triforce does.
|
||||
- If graphics look wrong after an update, clearing the shader caches often helps: close the game and delete
|
||||
`triforce_programs.bin`, `triforce_programs.idx`, `triforce_pipelines.bin`, `triforce_pipeline_states.bin`,
|
||||
`triforce_computes.bin`, `mkw_vk_pipelines.bin` and `triforce_prepared.bin` from the game folder (never
|
||||
`triforce_shaders.bin`). The next start prepares them again.
|
||||
|
||||
One topic per issue or pull request, in English, please.
|
||||
|
||||
## Ways to help
|
||||
|
||||
| | What |
|
||||
|---|---|
|
||||
| **Test on other hardware** | NVIDIA cards are the most tested; the README's [status table](readme.md#project-status) says what works where, and [tested configurations](docs/tested-configurations.md) lists the runs behind it (build, hardware, driver, where, fps). Reports from AMD and Intel cards, other NVIDIA drivers, other distributions and SteamOS are useful even when everything works: add a row there. |
|
||||
| **Play further** | The tested area is the start of the game. Reaching new areas, cutscenes, menus and items finds problems the developer has not seen. |
|
||||
| **Report what is broken** | Use the [issue forms](https://github.com/NotRexed/Hookshot/issues/new/choose): game problem, build problem or idea. Include the version, your hardware, where you were and `hookshot_log.txt`. |
|
||||
| **Send a fix** | Fork, fix, test, open a pull request. Small, focused changes are easiest to review and merge. |
|
||||
| **Improve the docs** | The README, the development guide and the design notes are part of the project. Corrections and clearer steps are welcome. |
|
||||
|
||||
## Before you open an issue or a PR
|
||||
|
||||
- Try the [newest release](https://github.com/NotRexed/Hookshot/releases/latest) first. It includes every fix before it.
|
||||
- Search the open issues. If yours is already there, add your details instead of opening another.
|
||||
- English, please.
|
||||
- One topic per issue or pull request. Do not mix a fix with a refactor or a batch of unrelated changes.
|
||||
- Report the version you actually ran: the release name, or the first line of `hookshot_log.txt`.
|
||||
- Test without ReShade, shader-cache tools, overlays and frame-generation injectors when you report a graphics or
|
||||
performance problem. They change what Hookshot does.
|
||||
| **Test on other hardware** | NVIDIA cards are the most tested. Reports from AMD and Intel cards and other drivers are useful even when everything works. |
|
||||
| **Play further** | Reaching new areas, cutscenes, menus and items finds problems nobody has seen yet. |
|
||||
| **Report what is broken** | As above, with the version, your hardware, where you were and the log. |
|
||||
| **Send a fix** | Fork, fix, test, open a pull request. Small, focused changes are easiest to review. |
|
||||
| **Improve the docs** | Corrections and clearer steps are welcome. |
|
||||
|
||||
## Development setup
|
||||
|
||||
You need your own supported dump: program `0400486024DEC000`, update v131072, build `e0f3ccfa`. Other versions do
|
||||
not work, and the build reads the dump where it lies without changing it. Builds need about 60 GB free and a run uses
|
||||
about 10 GB of RAM. Build on an SSD if you can: a hard drive works, but takes much longer, and the builder warns you
|
||||
before it starts.
|
||||
You need your own supported copy of the game: program `0400486024DEC000`, update v131072, build `e0f3ccfa`. Other
|
||||
versions do not work, and the build reads your files where they are without changing them. A build needs about 60 GB
|
||||
free and takes around 40 minutes; a run uses about 10 GB of RAM. Build on an SSD if you can.
|
||||
|
||||
### Windows, from the command line
|
||||
The easy way to get every tool a build needs is **Triforce Setup** from the
|
||||
[releases](https://github.com/ruckus7382sfgsgdf/CBeta0.2/releases): it installs the tools (Python with its packages,
|
||||
llvm-mingw, the Vulkan headers, FFmpeg, SDL2) into `%LOCALAPPDATA%\Programs\Triforce\tools`.
|
||||
|
||||
1. Install Python 3 from [python.org](https://www.python.org/downloads/) (not the Microsoft Store build: `python -I`
|
||||
does not see packages installed there) and the pipeline's packages:
|
||||
|
||||
```powershell
|
||||
python -m pip install -r requirements.txt
|
||||
```
|
||||
|
||||
(the second line: the button prompts' packages. Without them the game shows the Switch's icons)
|
||||
To build from a clone instead:
|
||||
|
||||
1. Install Python 3 from [python.org](https://www.python.org/downloads/) (not the Microsoft Store build) and the
|
||||
pipeline's packages: `python -m pip install -r requirements.txt`.
|
||||
2. Get [llvm-mingw](https://github.com/mstorsjo/llvm-mingw) (the `ucrt-x86_64` build) and set `HOOKSHOT_MINGW` to its
|
||||
folder. Without it the build falls back to emsdk's clang at `C:\emsdk` with Visual Studio's C++ tools and the
|
||||
Windows SDK (`HOOKSHOT_MSVC=1` forces that route).
|
||||
|
||||
3. Copy the pieces a clone does not carry from the `tools` folder Hookshot Builder installs. Run `HookshotBuilder.exe`
|
||||
from the latest release once (releases no longer carry a `HookshotBuilder.zip`): the folder is
|
||||
`%LOCALAPPDATA%\HookshotBuilder\versions\<version>\HookshotBuilder\tools`.
|
||||
|
||||
| From `tools` | To | Variable |
|
||||
|---|---|---|
|
||||
| `tools\cpm` | anywhere | `RX_CPM` points at it (Vulkan headers and FFmpeg; without it, system headers or `HOOKSHOT_*_HEADERS`, the standalone build stops) |
|
||||
| `tools\wgpu-native` | anywhere | `WGPU_NATIVE` points at it (the WebGPU fallback renderer; without it the game is built without one) |
|
||||
| `tools\third_party` | `third_party\` | (its `.dll`, `.exe`, `.lib` and `.sha1` files, to the same places: `a64emit`, `lld-link`, `zbic` and SDL2) |
|
||||
| `tools\work` | `work\` | (hot-function list, link order, profile and icon; without it the game builds but runs slower) |
|
||||
|
||||
folder.
|
||||
3. From the installed tools folder, point `RX_CPM` at `tools\cpm` and `WGPU_NATIVE` at `tools\wgpu-native`, and copy
|
||||
`tools\third_party` into `third_party\` and `tools\work` into `work\`.
|
||||
4. Run everything with `python -I` from the repository root.
|
||||
|
||||
### Linux
|
||||
(Some environment variables still carry the older `HOOKSHOT_` name; they work as documented.).
|
||||
|
||||
Either use the `Hookshot-<version>-x86_64.AppImage` from the Releases page, or build the environment and the AppImage
|
||||
yourself:
|
||||
### Triforce Setup
|
||||
|
||||
```sh
|
||||
scripts/linux/deps.sh # LLVM, Python packages, FFmpeg with dav1d, SDL2, Vulkan headers
|
||||
scripts/linux/appimage.sh # the AppImage, in build-linux/
|
||||
```
|
||||
|
||||
The AppImage needs glibc 2.35 or newer and a Vulkan 1.3 driver with mesh shaders. `HOOKSHOT_JOBS` sets how many files
|
||||
compile at once. The README's [Linux section](readme.md#on-linux) has the details and the runtime paths.
|
||||
|
||||
### NVIDIA DLSS (optional)
|
||||
|
||||
DLSS (`gpu/vk/vk_dlss.cpp`) is built in only with NVIDIA's DLSS SDK, which is NVIDIA's, under NVIDIA's own license, and
|
||||
is never committed, bundled or uploaded: not in a PR, not in a release, not in a CI artifact. Fetch it for your own
|
||||
builds with `scripts/ngx_sdk.py` (it prints NVIDIA's license notice and needs `--accept-nvidia-license` or a typed yes),
|
||||
then build with `HOOKSHOT_NGX_SDK=<sdk folder>`:
|
||||
|
||||
```powershell
|
||||
python -I scripts/ngx_sdk.py --platform windows C:\path\ngx-sdk # headers, nvsdk_ngx_s.lib, nvngx_dlss.dll
|
||||
$env:HOOKSHOT_NGX_SDK = "C:\path\ngx-sdk"
|
||||
python -I nx2/make_game.py --nup "C:\path\your-game.nup" --out "C:\Games\Hookshot-dlss"
|
||||
```
|
||||
|
||||
On Windows the llvm-mingw build links NVIDIA's static `nvsdk_ngx_s.lib` (built for Microsoft's static runtime) into
|
||||
`hookshot_vk.dll`, with `gpu/vk/vk_ngx_msvc_rt.cpp` and `host/mingw_msvc_shims.c` for what it needs of that runtime.
|
||||
`make_game.py` copies `nvngx_dlss.dll` into the game folder's `ngx` folder; never put NGX's core, `_nvngx.dll`, beside
|
||||
the game (it is the driver's). On Linux it is `--platform linux` (or `scripts/linux/ngx-sdk.sh`), with `libnvsdk_ngx.a`
|
||||
and `libnvidia-ngx-dlss.so`. A Windows game with DLSS can be built on Linux too, with `RX_TARGET=windows` as below.
|
||||
CI's `vulkan-dlss` workflow (`.github/workflows/vulkan-dlss.yml`) checks only that `hookshot_vk.dll` still links with the
|
||||
SDK. It runs by hand, and on a push or pull request only when a file DLSS's build depends on changes; it accepts NVIDIA's
|
||||
license for that run alone and uploads nothing.
|
||||
|
||||
Never share a game folder built with the SDK. A change to the pinned release updates the file table and SHA-256s in
|
||||
`scripts/ngx_sdk.py` (and `tests/test_ngx_sdk.py`), and the release named in the builder's license text
|
||||
(`builder/HookshotBuilder/MainWindow.xaml.cs`).
|
||||
|
||||
### Hookshot Builder
|
||||
|
||||
The builder is a .NET 8 WPF app. Open `builder/HookshotBuilder.sln` in Visual Studio 2022 with the .NET desktop
|
||||
workload, or build it from a developer prompt:
|
||||
|
||||
```powershell
|
||||
dotnet build builder/HookshotBuilder.sln
|
||||
```
|
||||
|
||||
You only need Visual Studio for the builder itself. The game build uses llvm-mingw.
|
||||
|
||||
The Windows app's **Build workers** choice applies to builds started from that window. **Automatic** keeps the existing
|
||||
`HOOKSHOT_JOBS` override, or the build tools' free-memory policy when no override is set. A number overrides it for that
|
||||
build's child process only; more workers need more memory and are not always faster. This choice is not saved: a new
|
||||
window, including one started to update a game, uses Automatic again.
|
||||
The launcher lives in `setup/`: a .NET Framework 4.8 WPF app, built with `powershell -File setup\build.ps1` (needs the
|
||||
C# compiler from Visual Studio Build Tools). `setup\pack_tools.ps1` packs the tools zip a release carries. Never run a
|
||||
development build's install, update or uninstall without `TRIFORCE_TEST_ROOT` set to a scratch folder: it would act
|
||||
on your real install.
|
||||
|
||||
## Building
|
||||
|
||||
| Command | What it does |
|
||||
|---|---|
|
||||
| `python -I nx2/make_game.py --nup "C:\path\your-game.nup" --out "C:\Games\Hookshot" --language en-US` | Build a complete game folder from a `.nup`. A build takes about 40 minutes. |
|
||||
| `python -I nx2/make_game.py --romfs "C:\path\romfs" --exefs "C:\path\exefs" --out "C:\Games\Hookshot"` | The same, from extracted folders. |
|
||||
| `python -I nx2/make_game.py --nup "C:\path\your-game.nup" --out "C:\Games\Triforce" --language en-US` | Build a complete game folder from a `.nup`. |
|
||||
| `python -I nx2/make_game.py --romfs "C:\path\romfs" --exefs "C:\path\exefs" --out "C:\Games\Triforce"` | The same, from extracted folders. |
|
||||
| `python -I nx2/make_game.py --check --nup "C:\path\your-game.nup"` | Check the game and its version without building. |
|
||||
| `python -I nx2/dump.py --dump <folder with exefs/ and romfs/> --set` | Record the dump in `work/dump.txt`. |
|
||||
| `python -I nx2/lift.py --standalone --write` | Lift the game to C in `work/sa/gen`. |
|
||||
| `python -I nx2/build.py --standalone --jobs 4` | Compile `work/sa/build/hookshot.exe` and `hookshot_vk.dll`. |
|
||||
| `powershell -File scripts/run-vk.ps1 -Window -Seconds 120` | Run the standalone build with the Vulkan back end and take screenshots in `work/runs/<name>/shots`. |
|
||||
| `python -I scripts/prof.py <stacks file>` | Read a profile sampled with `HOS_PROFILE`, `HOS_SAMPLE` and `HOS_SAMPLE_FILE`. |
|
||||
|
||||
Use a separate `--work` folder or `OOT_WORK` if you keep more than one dump around. The lifted C and every build are
|
||||
local artifacts; they are never committed.
|
||||
The translated C and every build are local files; they are never committed.
|
||||
|
||||
## Testing
|
||||
|
||||
### Game-free checks
|
||||
|
||||
These need no dump and are what CI runs, on every pull request and every push to `main`. One command runs them all:
|
||||
The game-free checks need no game files:
|
||||
|
||||
```powershell
|
||||
python -I -B tests/run_all.py
|
||||
```
|
||||
|
||||
It runs every `tests/test_*.py` and `nx2/test_*.py` in its own process, the C++ unit tests under `gpu/vk/tests` and a
|
||||
byte-compile of every tracked Python file, goes on after a failure, and ends with a table (slowest first) and every
|
||||
failure's output. Name a check to run only the ones whose path contains it (`python -I -B tests/run_all.py ub_`). A new
|
||||
`test_*.py` file is picked up by itself: no workflow to edit.
|
||||
It runs every `tests/test_*.py` and `nx2/test_*.py`, the C++ unit tests under `gpu/vk/tests`, and a byte-compile of
|
||||
every Python file, and ends with a table of results. Name a check to run only the ones whose path contains it. A new
|
||||
`test_*.py` file is picked up by itself.
|
||||
|
||||
Some checks compile a small C or C++ fixture and need a compiler (`g++`, `gcc` or `clang++` on PATH), the SPIR-V
|
||||
validator (`spirv-val`), the SPIR-V or Vulkan headers (`HOOKSHOT_SPIRV_HEADERS` / `HOOKSHOT_VULKAN_HEADERS`, or the CPM
|
||||
cache in `RX_CPM` or `build-linux/deps/cpm`), or the `zstandard` package. A check whose tool is missing is skipped and
|
||||
says so; CI passes `--strict`, which fails it instead. A check that cannot run in CI (a GPU, the game, a Visual Studio
|
||||
prompt), or runs on one OS only, says so in its own file with `RUN_ALL_SKIP`; `--no-skip` runs it anyway.
|
||||
When you add a fix, add a game-free test where the behavior can be tested without the game. For a renderer or runtime
|
||||
fix, say what you saw before and after, on which GPU and driver, and where in the game.
|
||||
|
||||
### Checks that need your dump or a run
|
||||
## Code conventions
|
||||
|
||||
These are slow and local, and they need `work/dump.txt` from `nx2/dump.py --set`:
|
||||
- Match the file you are editing: its indentation, comment style and naming. Do not reformat unrelated code.
|
||||
- Do not modify `third_party/`. Its parts are unmodified upstream copies, each with its notice in its own
|
||||
`PROVENANCE.md`.
|
||||
- New switches use the `RX_` prefix in the renderer and `HOS_` in the runtime, are off unless asked for, and cost
|
||||
nothing while off.
|
||||
- Rendering fixes must reproduce what the game itself does, not tuned factors that only look right on one screen.
|
||||
- Python runs with `-I`; tests stay on the standard library. Keep line endings LF.
|
||||
|
||||
| Command | What it checks |
|
||||
|---|---|
|
||||
| `python -I nx2/test_insn.py` | Every instruction form the game uses, Unicorn against the lifted C. |
|
||||
| `python -I nx2/test_leaf.py --calls` | Whole functions and call trees on the relocated image, native against Unicorn. |
|
||||
| `python -I nx2/census.py --nx2` | The instruction census per module. |
|
||||
| `python -I nx2/lower_test.py work/lowfail` | Shader programs that failed to lower. |
|
||||
| `powershell -File scripts/run-vk.ps1 -Window` | An in-game run: screenshots, the flash detector, frame dumps and the profiler. |
|
||||
## Pull requests
|
||||
|
||||
For a graphics fix, look at the screenshots in `work/runs/<name>/shots` and delete them afterwards; they reach
|
||||
gigabytes and are derived from the game. The switches for scripted input, screenshots, frame dumps and profiles are in
|
||||
the development guide's [reference](docs/development.md#switches-for-testing-and-debugging).
|
||||
|
||||
### When you add a fix or a check
|
||||
|
||||
- Add a game-free test where the behavior can be tested without the dump. The existing tests are small Python files
|
||||
under `tests/` or `nx2/` that read source files and run them, or compile a tiny C++ fixture.
|
||||
- Name it `tests/test_*.py` or `nx2/test_*.py`: `tests/run_all.py` picks it up by itself and CI runs it on every pull
|
||||
request, so there is no workflow to edit. What it needs from the runner it declares in its own file, after its
|
||||
docstring and imports, as plain literals `tests/run_all.py` reads without importing the test (its docstring has
|
||||
the details); there is no central list to add a line to:
|
||||
|
||||
```python
|
||||
RUN_ALL_ARGS = ["--cxx", "{cxx}", "--headers", "{vulkan_headers}"] # a compiler, headers, spirv-val ...
|
||||
RUN_ALL_MODULES = ["zstandard"] # packages it imports
|
||||
RUN_ALL_SKIP = {"windows": "POSIX signals: the Linux guard"} # cannot run in CI there (a GPU, the game,
|
||||
# one OS only), and why
|
||||
```
|
||||
|
||||
`{name}` is a tool the runner finds (`cxx`, `vulkan_headers`, `spirv_headers`, `spirv_val`, `glslang`, `dotnet`);
|
||||
a check whose tool is missing is skipped, or fails with `--strict`. A test that needs none of this declares nothing.
|
||||
- Keep the test fast: every check shares one CI job with a 20-minute limit. Make it pass on both Ubuntu and Windows
|
||||
where it can.
|
||||
- If the fix is in the renderer or the runtime, say what you saw before and after, on which GPU and driver, and in
|
||||
which part of the game.
|
||||
|
||||
## Code and data conventions
|
||||
|
||||
- Match the file you are editing: there is no formatter configuration, so use the indentation, comment style and
|
||||
naming around you. Do not reformat unrelated code in the same change.
|
||||
- Keep OoT-specific code in new files where possible. `nx2/`, `host/` and `gpu/` are kept close to mkw-recomp so
|
||||
upstream fixes can be applied as a patch; keeping the paths and the shared files recognizable is what makes that
|
||||
work. Each sync names its new mkw-recomp base commit in the commit message.
|
||||
- Do not modify `third_party/`. Its parts are unmodified upstream copies with their notices in `PROVENANCE.md`; add a
|
||||
new folder and a provenance note rather than editing one.
|
||||
- Treat the local mkw-recomp and Eglantine checkouts as read-only references: read from them, run them, never edit
|
||||
them.
|
||||
- New switches use the `RX_` prefix in the renderer and `HOS_` in the runtime. They are off, or neutral, unless
|
||||
asked for, and they do not cost anything while off. Document them where the surrounding switches are documented.
|
||||
- The GPU replay rules in [design notes, section 4](docs/design.md#4-the-lessons-from-mkw-as-rules-here) are not
|
||||
suggestions: nothing the replay reads may come from live guest memory, waits are honoured per queue, and texture
|
||||
and sampler registrations stay in submission order. A change that reads guest memory at replay time is a
|
||||
regression, even if it looks right on one frame.
|
||||
- Python runs with `-I`. Tests stay on the standard library so CI needs no install step; the pipeline's existing
|
||||
dependencies (lz4, numpy, capstone, unicorn, zstandard) are the exception, not a reason to add more. So are the
|
||||
button prompts' (fontTools, Pillow, astc-encoder-py, texture2ddecoder: `nx2/prompt_requirements.txt`), accepted for
|
||||
`nx2/prompt_assets.py` and `nx2/prompt_draw.py` alone, at build time. Maintainers' agents reviewing a change: check
|
||||
that these four are still used only there, and flag any other use. Keep line endings LF (`.gitattributes`).
|
||||
- New code has to be GPL-compatible. See below.
|
||||
|
||||
## Commits and pull requests
|
||||
|
||||
1. Fork the repository and branch from `main`. Name the branch for the change: `fix/...`, `linux/...`, `builder/...`,
|
||||
`docs/...`.
|
||||
2. Keep the pull request to one topic. Bug fixes do not carry refactors, and two unrelated fixes are two PRs.
|
||||
3. Write the title as a short imperative sentence, as the history does: "Stop scanning storage-buffer tables after
|
||||
one pass", "Linux: a crash report gives the whole host stack". Start the body with why the PR is needed, in one or
|
||||
two plain sentences a player could follow: the bug, the missing feature or the cost it removes ("The menu's text
|
||||
is cut off at 4K", "First starts spend 20 s waiting on a few shaders"). Then say what changed, how it was tested
|
||||
(GPU, driver, game area, fps before and after), and link the issue it fixes.
|
||||
4. Include screenshots when the change shows on screen: graphics, the menu, the start-up screen, an overlay, a
|
||||
message. Before and after for a fix, the new screen for a feature, and the sizes or settings it was checked at. A
|
||||
screenshot of the game is fine; the rule above is about game files, not pictures of the game.
|
||||
5. Run the relevant checks. CI must pass; a change that only works on your machine is not ready.
|
||||
6. Rebase or merge `main` when it has moved, and resolve conflicts in your branch rather than in the review.
|
||||
7. Do not include build output, dumps, generated files or a `.gitignore` bypass. Check `git status`.
|
||||
|
||||
Commit subjects decide the next release: write them as Conventional Commits (`feat:`, `fix:`, and `!` or
|
||||
`BREAKING CHANGE:` for a break). The version rules, branches and changelog format are in
|
||||
[docs/versioning.md](docs/versioning.md).
|
||||
|
||||
Maintainer commits record AI assistance with `Assisted-by:` and `Co-Authored-By:` trailers. If you used a tool, add
|
||||
the trailers that describe it; if you wrote the change yourself, there is nothing to add.
|
||||
1. Fork the repository and branch from `main`.
|
||||
2. One topic per pull request: a fix does not carry a refactor.
|
||||
3. Title: a short imperative sentence. Body: why it is needed, what changed, how it was tested (GPU, driver, game
|
||||
area, fps before and after), and the issue it fixes.
|
||||
4. Screenshots for anything that shows on screen, before and after.
|
||||
5. No build output, dumps or generated files. Check `git status`.
|
||||
|
||||
## Licensing
|
||||
|
||||
Hookshot is free software under the [GNU General Public License, version 3 or later](LICENSE). By submitting a change
|
||||
you agree to license it under those terms. Only send code you can license that way. Do not paste code from a project
|
||||
with an incompatible license, and do not paste decompiled game code or anything generated from the dump into a PR.
|
||||
Triforce is free software under the [GNU General Public License, version 3 or later](LICENSE), like the Hookshot code
|
||||
it builds on. By submitting a change you agree to license it under those terms. Do not paste code from a project with
|
||||
an incompatible license, and never paste decompiled game code or anything generated from the game files.
|
||||
|
||||
The third-party parts (Eden, suyu, Sirit, the Vulkan Memory Allocator, SDL, Opus, Dear ImGui, zstd, the Libertinus
|
||||
fonts, zbic and the a64-kit tools) keep their own notices. Preserve them, and do not relicense their code.
|
||||
fonts, zbic and the a64-kit tools) keep their own notices. Preserve them.
|
||||
|
||||
NVIDIA's DLSS SDK is not one of them: it is never in the repository, and nothing of it may be added (see
|
||||
[NVIDIA DLSS](#nvidia-dlss-optional)).
|
||||
|
||||
## Where to talk
|
||||
|
||||
- **Issues:** [use the forms](https://github.com/NotRexed/Hookshot/issues/new/choose).
|
||||
- **Discord:** https://discord.gg/EajnD2AJD9. Use the `hookshot-discussion` channel; the server is mainly for
|
||||
web ports, and that channel is where Hookshot talk belongs.
|
||||
- **Newest release:** https://github.com/NotRexed/Hookshot/releases/latest.
|
||||
|
||||
Thanks for testing, reporting and sending fixes. Every report from a different GPU and every area played further than
|
||||
before moves the port forward.
|
||||
Thanks for testing, reporting and sending fixes. Every report from a different GPU and every area played further
|
||||
moves the port forward.
|
||||
|
||||
@@ -96,8 +96,8 @@ bool UseShuffleFallback(EmitContext& ctx) {
|
||||
(ctx.profile.has_broken_spirv_fragment_shuffle && ctx.stage == Stage::Fragment);
|
||||
}
|
||||
|
||||
Id ShuffleAbsolute(EmitContext& ctx, Id value, Id src_thread_id) {
|
||||
if (!UseShuffleFallback(ctx)) {
|
||||
Id ShuffleAbsolute(EmitContext& ctx, Id value, Id src_thread_id, bool broadcast) {
|
||||
if (!broadcast) {
|
||||
return ctx.OpGroupNonUniformShuffle(ctx.U32[1], SubgroupScope(ctx), value, src_thread_id);
|
||||
}
|
||||
Id result{ctx.u32_zero_value};
|
||||
@@ -111,8 +111,9 @@ Id ShuffleAbsolute(EmitContext& ctx, Id value, Id src_thread_id) {
|
||||
}
|
||||
|
||||
Id ShuffleRelative(EmitContext& ctx, Id value, Id delta, Id src_thread_id, spv::Op op) {
|
||||
if (!ctx.profile.support_shuffle_relative || UseShuffleFallback(ctx)) {
|
||||
return ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id));
|
||||
const bool broadcast{UseShuffleFallback(ctx)};
|
||||
if (!ctx.profile.support_shuffle_relative || broadcast) {
|
||||
return ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id), broadcast);
|
||||
}
|
||||
const Id result{ctx.OpGroupNonUniformShuffleXor(ctx.U32[1], SubgroupScope(ctx), value, delta)};
|
||||
ctx.opcode_rewrites.emplace_back(result.value, op);
|
||||
@@ -260,7 +261,8 @@ Id EmitShuffleIndex(EmitContext& ctx, IR::Inst* inst, Id value, Id index, Id cla
|
||||
return ctx.OpSelect(ctx.U32[1], in_range, BroadcastLane(ctx, value, lane.U32() & 31),
|
||||
value);
|
||||
}
|
||||
const Id shuffled{ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id))};
|
||||
const Id shuffled{
|
||||
ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id), UseShuffleFallback(ctx))};
|
||||
return ctx.OpSelect(ctx.U32[1], in_range, shuffled, value);
|
||||
}
|
||||
|
||||
@@ -316,7 +318,7 @@ Id EmitShuffleButterfly(EmitContext& ctx, IR::Inst* inst, Id value, Id index, Id
|
||||
// (hookshot) through the broadcasts where shuffles are broken (UseShuffleFallback: AMD's fragment stages, Lichii #273),
|
||||
// as the absolute and relative shuffles are; #273 left SHFL.BFLY native, and some of Kokiri Forest's plants use it
|
||||
const Id shuffled{UseShuffleFallback(ctx)
|
||||
? ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id))
|
||||
? ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id), true)
|
||||
: ctx.OpGroupNonUniformShuffleXor(ctx.U32[1], SubgroupScope(ctx), value, mask)};
|
||||
return ctx.OpSelect(ctx.U32[1], in_range, shuffled, value);
|
||||
}
|
||||
@@ -328,7 +330,8 @@ Id EmitQuadBroadcast(EmitContext& ctx, Id value, Id lane) {
|
||||
const Id base{ctx.OpBitwiseAnd(ctx.U32[1], GetThreadId(ctx), ctx.Const(~3u))};
|
||||
const Id local_lane{ctx.OpBitwiseAnd(ctx.U32[1], lane, ctx.Const(3u))};
|
||||
const Id src_thread_id{ctx.OpBitwiseOr(ctx.U32[1], base, local_lane)};
|
||||
return ShuffleAbsolute(ctx, value, src_thread_id);
|
||||
return ShuffleAbsolute(ctx, value, src_thread_id,
|
||||
ctx.profile.has_broken_spirv_subgroup_shuffle);
|
||||
}
|
||||
|
||||
Id EmitQuadSwap(EmitContext& ctx, Id value, Id direction) {
|
||||
|
||||
@@ -1190,11 +1190,46 @@ void state_record(const Program& p, const NvnState& s, const Targets& at, VkPrim
|
||||
if (!state_file) state_open_locked();
|
||||
if (state_file && state_seen.insert(h).second) fwrite(&r, sizeof r, 1, state_file), fflush(state_file);
|
||||
}
|
||||
/// (perf push 11-11) Which shipped list this folder's notes were last made from, in triforce_states_applied.bin: the list's
|
||||
/// contents and the translator, hashed. The lists were gone through only inside a prepare (a first start, a new translator, a
|
||||
/// new seed): a folder prepared already that got a longer shipped list (an update with the same translator, a list copied in)
|
||||
/// never noted its pipelines, and built each in play when first drawn (a lab folder with a 12067-pipeline list: 2687 built in
|
||||
/// play, the list not read once). 0: there is no shipped list
|
||||
uint64_t state_shipped_id() {
|
||||
FILE* f = fopen(state_list_path("triforce_pipeline_states_shipped.bin").c_str(), "rb");
|
||||
if (!f) return 0;
|
||||
uint64_t h = 0xCBF29CE484222325ull ^ program_cache_stamp(), total = 0;
|
||||
std::vector<uint8_t> buf(1 << 16);
|
||||
for (size_t n; (n = fread(buf.data(), 1, buf.size(), f)) > 0; total += n)
|
||||
for (size_t i = 0; i < n; i++) h = (h ^ buf[i]) * 1099511628211ull;
|
||||
fclose(f);
|
||||
return total > 12 ? (h | 1) : 0;
|
||||
}
|
||||
/// Whether the shipped list is one the notes were not made from yet
|
||||
bool state_shipped_new() {
|
||||
if (getenv("RX_VK_NO_STATE_LIST")) return false;
|
||||
const uint64_t id = state_shipped_id();
|
||||
if (!id) return false;
|
||||
uint64_t had = 0;
|
||||
if (FILE* f = fopen(state_list_path("triforce_states_applied.bin").c_str(), "rb")) {
|
||||
if (fread(&had, 8, 1, f) != 1) had = 0;
|
||||
fclose(f);
|
||||
}
|
||||
return had != id;
|
||||
}
|
||||
/// (prepare_run, the seed open) every pipeline of the state lists (the player's own and the shipped one): its program translated
|
||||
/// when the program cache lacks it, then noted in triforce_pipelines.bin for precompile_run to build. RX_VK_NO_STATE_LIST=1: off
|
||||
void state_list_prepare(void (*progress)(size_t done, size_t total)) {
|
||||
if (getenv("RX_VK_NO_STATE_LIST")) return;
|
||||
const auto t0 = std::chrono::steady_clock::now();
|
||||
const uint64_t shipped_id = state_shipped_id();
|
||||
struct Applied { // (written when the lists were gone through, however this returns)
|
||||
uint64_t id;
|
||||
~Applied() {
|
||||
if (!id) return;
|
||||
if (FILE* f = fopen(state_list_path("triforce_states_applied.bin").c_str(), "wb")) fwrite(&id, 8, 1, f), fclose(f);
|
||||
}
|
||||
} applied{shipped_id};
|
||||
std::vector<StateRec> recs = state_list_read(state_list_path("triforce_pipeline_states_shipped.bin"));
|
||||
const size_t shipped = recs.size();
|
||||
for (const StateRec& r : state_list_read(state_list_path("triforce_pipeline_states.bin"))) recs.push_back(r);
|
||||
@@ -1220,6 +1255,9 @@ void state_list_prepare(void (*progress)(size_t done, size_t total)) {
|
||||
rec_program[i] = it->second;
|
||||
}
|
||||
std::atomic<size_t> next{0}, made{0}, missing{0};
|
||||
// (perf push 11-11: why a program of the list has no key. 0 a header-only stage, 1 mesh on a GPU without, 2 a stage the
|
||||
// seed lacks or has as another kind. Measured on OoT: all 2178 of 6150 are the last, vertex and fragment both absent)
|
||||
std::atomic<size_t> why_missing[3]{};
|
||||
const auto worker = [&] {
|
||||
ub_thread_off(); // (a background thread: see ub_thread_off; the pipeline state list's builds raced the replay's updates)
|
||||
SetThreadPriority(GetCurrentThread(), THREAD_PRIORITY_BELOW_NORMAL);
|
||||
@@ -1230,7 +1268,15 @@ void state_list_prepare(void (*progress)(size_t done, size_t total)) {
|
||||
std::unique_ptr<Program> p;
|
||||
// (a mesh program on a GPU without mesh shaders is not made; one with a header-only stage is made in play)
|
||||
if (!r.code[3] && (!r.mesh || dev.mesh_shader)) p = seed_program_of(r.code, r.mesh != 0, r.vattr, r.color_types, &want[i].key);
|
||||
if (!want[i].key) missing++;
|
||||
if (!want[i].key) {
|
||||
missing++;
|
||||
why_missing[r.code[3] ? 0 : r.mesh && !dev.mesh_shader ? 1 : 2]++;
|
||||
static std::atomic<int> told{0};
|
||||
if (getenv("RX_VK_SEED_CHECK") && told++ < 40)
|
||||
logf("vk: state list: no key for %016llx %016llx %016llx %016llx mesh %u",
|
||||
(unsigned long long)r.code[0], (unsigned long long)r.code[1], (unsigned long long)r.code[2],
|
||||
(unsigned long long)r.code[3], r.mesh);
|
||||
}
|
||||
if (p) {
|
||||
if (!p->failed) made++;
|
||||
for (Stage& st : p->stages)
|
||||
@@ -1311,6 +1357,10 @@ void state_list_prepare(void (*progress)(size_t done, size_t total)) {
|
||||
"by their textures (%zu made), %zu noted, %zu whose program variant was not made, in %.1f s", recs.size(), shipped,
|
||||
want.size(), made.load(), missing.load(), variant_of.size(), variants_made.load(), noted_n, no_variant,
|
||||
std::chrono::duration<double>(std::chrono::steady_clock::now() - t0).count());
|
||||
if (missing.load())
|
||||
logf("vk: state list: the %zu programs without a key: %zu with a header-only stage, %zu mesh (none on this GPU), %zu with a "
|
||||
"stage the game's shader list lacks (made in play)", missing.load(), why_missing[0].load(), why_missing[1].load(),
|
||||
why_missing[2].load());
|
||||
}
|
||||
|
||||
// ---- (hookshot) Stage libraries, as TotK Recomp's runtime/vk_pipeline.c: the part of a pipeline that costs is its shaders, and with
|
||||
@@ -2060,8 +2110,10 @@ struct HandleHit {
|
||||
uint64_t destroy_gen;
|
||||
bool sampling; // (its use reads the texture's contents)
|
||||
uint64_t tex_epoch; // (the texture epoch it was last touched in)
|
||||
bool known = true; // (Triforce: the slot is registered: with no view, a texture whose image is not there yet)
|
||||
};
|
||||
TLS_OBJECT(handle_hits, std::unordered_map<uint64_t, HandleHit>)
|
||||
thread_local bool view_slot_known = true; // (Triforce: the last texture_view_of's slot was a registered one)
|
||||
// (hookshot) RX_VK_TRACE_TEX=<w>x<h>,...: each draw sampling a texture of that size is logged with its state, every 30th frame, 600 lines: a
|
||||
// model that does not show is found by its textures, and whether the game draws it, and how, read from the line
|
||||
thread_local const Texture* trace_tex_hit = nullptr;
|
||||
@@ -2125,6 +2177,7 @@ VkImageView texture_view_of(uint32_t handle, VkImageViewType want, ViewUse use,
|
||||
}
|
||||
if (tex_out) *tex_out = it->second.tex;
|
||||
if (level_out) *level_out = it->second.level;
|
||||
view_slot_known = it->second.known;
|
||||
return it->second.view;
|
||||
}
|
||||
ParLock lk;
|
||||
@@ -2141,6 +2194,31 @@ VkImageView texture_view_of(uint32_t handle, VkImageViewType want, ViewUse use,
|
||||
if (tex) level = image_level(tex, has_view ? view_level(&v) : 0);
|
||||
}
|
||||
if (!view) { // (what a shader reads as the dummy instead: once per texture slot)
|
||||
{ // (Triforce, diagnostics) RX_VK_LOG_DUMMY=1: every 10 s, the slots that were resolved to the dummy in the last second
|
||||
static const bool log_dummy = getenv("RX_VK_LOG_DUMMY") != nullptr;
|
||||
if (log_dummy) {
|
||||
struct Seen { int64_t n = 0; std::chrono::steady_clock::time_point last; char desc[200]; };
|
||||
static std::unordered_map<uint32_t, Seen> seen;
|
||||
static auto next = std::chrono::steady_clock::now() + std::chrono::seconds(10);
|
||||
const auto now = std::chrono::steady_clock::now();
|
||||
Seen& d = seen[handle & 0xFFFFF];
|
||||
d.n++, d.last = now;
|
||||
if (!known) snprintf(d.desc, sizeof d.desc, "not registered");
|
||||
else snprintf(d.desc, sizeof d.desc, "%s; target %d format %#x %ux%ux%u levels %u gpu %#llx view %s (format %#x target %d) want %d use %d",
|
||||
tex ? "no view of it" : "no image for it", t.d.target, t.d.format, t.d.width, t.d.height, t.d.depth, t.d.levels,
|
||||
(unsigned long long)t.gpu, has_view ? "yes" : "no", has_view && v.set_format ? v.format : 0,
|
||||
has_view && v.set_target ? v.target : -1, (int)want, (int)use);
|
||||
if (now >= next) {
|
||||
next = now + std::chrono::seconds(10);
|
||||
int k = 0;
|
||||
for (auto& [slot, e] : seen)
|
||||
if (now - e.last < std::chrono::seconds(1) && k++ < 48)
|
||||
logf("vk: dummy: slot %#x resolved %lld times in 10 s: %s", slot, (long long)e.n, e.desc);
|
||||
logf("vk: dummy: %d slots resolved to the dummy in the last second", k);
|
||||
for (auto& [slot, e] : seen) e.n = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
static std::unordered_map<uint32_t, int> told;
|
||||
if (told.size() < 64 && !told[handle & 0xFFFFF]++) {
|
||||
if (!known)
|
||||
@@ -2153,7 +2231,8 @@ VkImageView texture_view_of(uint32_t handle, VkImageViewType want, ViewUse use,
|
||||
has_view && v.set_format ? v.format : 0, has_view && v.set_target ? v.target : -1, (int)want, (int)use);
|
||||
}
|
||||
}
|
||||
handle_hits()[key] = {view, tex, level, version, dgen, use == ViewUse::Sample || use == ViewUse::Storage, texture_epoch()};
|
||||
handle_hits()[key] = {view, tex, level, version, dgen, use == ViewUse::Sample || use == ViewUse::Storage, texture_epoch(), known};
|
||||
view_slot_known = known;
|
||||
if (tex_out) *tex_out = tex;
|
||||
if (level_out) *level_out = level;
|
||||
return view;
|
||||
@@ -2328,6 +2407,7 @@ bool dump_sampled_wanted(uint64_t key) {
|
||||
} // (RX_VK_DUMP: the storage images the descriptors let the program write)
|
||||
TLS_OBJECT(dump_inputs, std::string) // (and what it samples)
|
||||
int64_t st_dummy_textures; // (textures sampled as the dummy: no image for the handle)
|
||||
int64_t st_dummy_known; // (Triforce: of those, registered textures: their image not made or not uploaded yet)
|
||||
|
||||
void dump_input(char kind, uint32_t index, uint32_t handle, const Texture* tex) {
|
||||
char b[160];
|
||||
@@ -2877,7 +2957,7 @@ void build_descriptors(const NvnState& s, const Program& p) {
|
||||
tex ? tex->width : 0, tex ? tex->height : 0, tex ? tex->layers : 0, tex ? (void*)tex->image : nullptr,
|
||||
(void*)view, view ? "" : " (dummy)");
|
||||
if (view && tex && tex->gpu_written) draw_sampled().push_back(tex);
|
||||
if (!view) view = dummies.view[(int)vt], st_dummy_textures += handle != 0;
|
||||
if (!view) view = dummies.view[(int)vt], st_dummy_textures += handle != 0, st_dummy_known += handle != 0 && view_slot_known;
|
||||
dw().imgs.push_back({sampler_for((int)(handle >> 20), d.is_depth), view, VK_IMAGE_LAYOUT_GENERAL});
|
||||
}
|
||||
w.pImageInfo = &dw().imgs[first];
|
||||
@@ -2942,6 +3022,10 @@ TLS_OBJECT(pushed, std::array<Pushed, 2>) // [graphics, compute]
|
||||
/// to it)
|
||||
VkPipelineCache shared_pipeline_cache() { return pcache; }
|
||||
void draw_skip_report(double frames) { skip_report_impl(frames), fine_report(frames); }
|
||||
/// (Triforce) how many texture bindings have sampled the dummy so far (a texture with no image yet: left for a later frame, not
|
||||
/// made, evicted): RX_VK_FRAME_TRACE's last column, the measure of materials showing flat while their textures come in
|
||||
/// Registered textures only: OoT binds one handle of no registered slot (0x3126f) in ~18 places a frame, always the dummy
|
||||
int64_t draw_dummy_textures() { return st_dummy_known; }
|
||||
void descriptors_disturbed() {
|
||||
pushed()[0].cmd = pushed()[1].cmd = VK_NULL_HANDLE;
|
||||
}
|
||||
@@ -7102,6 +7186,15 @@ bool prepare_run(void (*progress)(size_t done, size_t total), bool background) {
|
||||
logf("vk: the shader list was hashed (%.0f MB in %.0f ms): written since the last check, or too close to it to go by its time",
|
||||
prepared_for.seed_size / 1048576.0, hash_ms);
|
||||
if (why == PrepareWhy::None && !getenv("RX_VK_PREPARE_AGAIN")) {
|
||||
// (perf push 11-11: a shipped state list the notes were not made from, in a folder prepared already: its pipelines are
|
||||
// noted now, for the precompile that follows, as a prepare would have. See state_shipped_id)
|
||||
if (!background && state_shipped_new()) {
|
||||
logf("vk: the shipped pipeline list is not the one this folder's notes were made from: its pipelines are noted now");
|
||||
if (seed_open()) {
|
||||
state_list_prepare(nullptr);
|
||||
seed_close();
|
||||
}
|
||||
}
|
||||
if (getenv("RX_VK_SEED_CHECK")) seed_open();
|
||||
if (prepared_for.warmed == warmed || background) return false;
|
||||
// (translated, and the driver's part is another driver's, or of before the libraries were wanted: that part alone, no
|
||||
@@ -7279,6 +7372,47 @@ void prepare_background() {
|
||||
}).detach();
|
||||
}
|
||||
|
||||
// ---- (perf push 11-11) The precompile on a PC with little RAM. Every pipeline built here is kept for the session, with its
|
||||
// program: measured at a fresh Market (RTX 4070 SUPER), the process holds 12.9 GB in RAM without the precompile, 13.4 GB with
|
||||
// 4704 graphics and 275 compute pipelines, 15.7 GB with 8550 and 1650; after a tour of ten areas 18.6 against 20.6 GB. What
|
||||
// the precompile is for is the compile, which the caches keep (ours, mkw_vk_pipelines.bin, and the driver's): with them warm,
|
||||
// the same runs without it built 1462 and 4046 pipelines in play in 357 and 1639 ms in all (0.2-0.4 ms each), with as many
|
||||
// frames over 40, 100 and 250 ms as with it. So where RAM is short (under 20 GB in the PC: the 16 GB ones, whose logs show the
|
||||
// system taking pages back all through play) a pipeline is built before the game once, on the first start that has its note,
|
||||
// and that start records it (triforce_pipelines.bin.built, for this translator and driver); later starts leave it to its first
|
||||
// draw, and neither it nor its program is in RAM until then.
|
||||
// The price, same tour: of ~4050 pipelines then built in play, ten or twelve submissions spend 10-65 ms on them (some in frames
|
||||
// that were slow anyway, an area's arrival; three or four make a 40 ms frame of their own), and one in two tours had a
|
||||
// pipeline the caches missed (508 ms). So it is not the default: RX_VK_PRECOMPILE_KEEP=auto: this rule where the PC has under
|
||||
// 20 GB of RAM; =0: on any PC; unset or 1: every start builds and keeps all, as before ----
|
||||
constexpr uint64_t kBuiltComputes = 0xC04D50555445D001ull; // (in the record: the compute programs' pipelines were built)
|
||||
bool precompile_keeps_all() {
|
||||
const char* e = getenv("RX_VK_PRECOMPILE_KEEP");
|
||||
if (!e) return true;
|
||||
if (strcmp(e, "auto")) return atoi(e) != 0;
|
||||
#ifdef _WIN32
|
||||
MEMORYSTATUSEX ms{};
|
||||
ms.dwLength = sizeof ms;
|
||||
if (GlobalMemoryStatusEx(&ms)) return ms.ullTotalPhys >= (20ull << 30);
|
||||
#endif
|
||||
return true;
|
||||
}
|
||||
std::unordered_set<uint64_t> built_read(uint64_t stamp) {
|
||||
std::unordered_set<uint64_t> keys;
|
||||
if (FILE* f = fopen((pipe_notes_path() + ".built").c_str(), "rb")) {
|
||||
uint64_t v = 0;
|
||||
if (fread(&v, 8, 1, f) == 1 && v == stamp)
|
||||
while (fread(&v, 8, 1, f) == 1) keys.insert(v);
|
||||
fclose(f);
|
||||
}
|
||||
return keys;
|
||||
}
|
||||
void built_write(uint64_t stamp, const std::unordered_set<uint64_t>& keys) {
|
||||
std::vector<uint64_t> all{stamp};
|
||||
all.insert(all.end(), keys.begin(), keys.end());
|
||||
if (FILE* f = fopen((pipe_notes_path() + ".built").c_str(), "wb")) fwrite(all.data(), 8, all.size(), f), fclose(f);
|
||||
}
|
||||
|
||||
void precompile_run(void (*progress)(size_t done, size_t total)) {
|
||||
const auto t0 = std::chrono::steady_clock::now();
|
||||
// (a start that dies in here would die in here every time, the notes being what it builds from: a player saw the console reach
|
||||
@@ -7312,12 +7446,29 @@ void precompile_run(void (*progress)(size_t done, size_t total)) {
|
||||
std::lock_guard lk(state_mu);
|
||||
if (!state_file) state_open_locked();
|
||||
}
|
||||
// (little RAM, see precompile_keeps_all: the pipelines an earlier start built for these caches are left to their first draw)
|
||||
const bool light = !precompile_keeps_all();
|
||||
const uint64_t built_stamp = warmed_stamp(gpl_wanted());
|
||||
std::unordered_set<uint64_t> built;
|
||||
const size_t noted_n = notes.size();
|
||||
bool computes_built = false;
|
||||
if (light) {
|
||||
built = built_read(built_stamp);
|
||||
computes_built = built.count(kBuiltComputes) != 0;
|
||||
std::erase_if(notes, [&](const std::unique_ptr<PipeNote>& n) { return built.count(n->key) != 0; });
|
||||
logf("vk: precompile: RAM spared (RX_VK_PRECOMPILE_KEEP=1: as before): %zu of %zu noted pipelines were built by "
|
||||
"an earlier start and are left to their first draw%s", noted_n - notes.size(), noted_n,
|
||||
computes_built ? ", the compute programs' too" : "");
|
||||
}
|
||||
std::vector<uint64_t> building; // (their keys, for the record once they are built)
|
||||
if (light)
|
||||
for (const auto& n : notes) building.push_back(n->key);
|
||||
std::unordered_set<uint64_t> wanted;
|
||||
for (const auto& n : notes) wanted.insert(n->prog);
|
||||
// (the compute programs: those play dispatched (triforce_computes.bin), or every kept one when there is no list yet)
|
||||
const char* which = getenv("RX_VK_PRECOMPILE_COMPUTES");
|
||||
const std::unordered_set<uint64_t> used = computes_read();
|
||||
const bool all_computes = which ? !strcmp(which, "all") : used.empty();
|
||||
const std::unordered_set<uint64_t> used = computes_built ? std::unordered_set<uint64_t>{} : computes_read();
|
||||
const bool all_computes = !computes_built && (which ? !strcmp(which, "all") : used.empty());
|
||||
if (!all_computes && !(which && !strcmp(which, "none"))) wanted.insert(used.begin(), used.end());
|
||||
std::vector<Program*> computes, fallbacks;
|
||||
const size_t programs_n = program_cache_load_some(wanted, [&](Program& p) {
|
||||
@@ -7431,6 +7582,13 @@ void precompile_run(void (*progress)(size_t done, size_t total)) {
|
||||
} else {
|
||||
for (std::thread& t : threads) t.join();
|
||||
delete pre;
|
||||
// (little RAM: what this start built is on disk before the record says so, and the next start leaves it out)
|
||||
if (light && !prepare_stopped() && (!building.empty() || !computes_built)) {
|
||||
built.insert(building.begin(), building.end());
|
||||
built.insert(kBuiltComputes);
|
||||
pipeline_cache_save(true);
|
||||
built_write(built_stamp, built);
|
||||
}
|
||||
}
|
||||
logf("vk: precompile: %zu programs, %zu graphics and %zu compute pipelines, %zu mesh fallback programs in %.0f ms (%.0f ms "
|
||||
"of it the caches opened, the notes and programs read)", programs_n, graphics_n, computes_n, fallbacks_n,
|
||||
|
||||
@@ -65,6 +65,7 @@ void dry_marks_begin(); // (at the start of each dry pass: the last targets it
|
||||
void draw_stats(int64_t* draws, int64_t* skipped, int64_t* dispatches, int64_t* pipelines);
|
||||
void draw_stats2(int64_t* passes, int64_t* mesh, double* pipeline_ms);
|
||||
void draw_stats3(int64_t* pending_skips, int64_t* waits, double* wait_ms);
|
||||
int64_t draw_dummy_textures(); // (Triforce: texture bindings that sampled the dummy so far: RX_VK_FRAME_TRACE)
|
||||
/// (hookshot) pipelines linked from stage libraries in the draw, and the libraries asked of the driver for them: had, and cold;
|
||||
/// of the cold, those a worker has built since, and the workers' time in them. False: no stage libraries (RX_VK_GPL)
|
||||
bool draw_stats4(int64_t* links, double* link_ms, int64_t* lib_hits, int64_t* lib_cold, int64_t* lib_built, double* lib_build_ms);
|
||||
|
||||
@@ -210,6 +210,9 @@ void memory_drop_shadow(uint64_t guest, uint64_t size); // (a write the captur
|
||||
/// when any write-back ever did
|
||||
uint64_t memory_writeback_value(uint64_t guest, uint64_t size, bool* ever);
|
||||
void memory_note_readback(uint64_t guest, uint64_t size); // (PR 430) a CPU read of GPU results: see memory_rb_refresh
|
||||
/// (Triforce) around the write of an NVN event's word by a command buffer's signal: its write-back is not one the captures of
|
||||
/// its page wait for (vk_memory.cpp). This thread's bindings while on
|
||||
void memory_bind_event_write(bool on);
|
||||
bool memory_rb_refresh(VkCommandBuffer up);
|
||||
void memory_rb_recording_begun(); // (hookshot) at a recording's start, on its thread: the submission it joins (memory_rb_refresh)
|
||||
void memory_rb_segments_begin(uint64_t base, uint64_t n); // (and a parallel replay's segments: their order, base on)
|
||||
@@ -219,11 +222,12 @@ void memory_note_gpu_write(uint64_t gpu, uint64_t size);
|
||||
uint64_t vk_timeline_completed();
|
||||
void vk_timeline_wait(uint64_t v);
|
||||
uint32_t memory_generation();
|
||||
uintptr_t memory_stamp_who(uint64_t guest, uint32_t* gen, uintptr_t* anchor); // (RX_VK_STAMP_SRC: who wrote that page last)
|
||||
/// 1: a page of [guest, guest + size) was written after generation gen; 0: none was
|
||||
bool memory_written_since(uint64_t guest, uint64_t size, uint32_t gen);
|
||||
void memory_stamp(uint64_t guest, uint64_t size); // a write the watch does not see (GPU writes, host writes)
|
||||
/// The page runs of [guest, guest + size) written after gen, as byte ranges from guest (appended to out)
|
||||
void memory_written_runs(uint64_t guest, uint64_t size, uint32_t gen, std::vector<std::pair<uint64_t, uint64_t>>& out);
|
||||
void memory_written_runs(uint64_t guest, uint64_t size, uint32_t gen, std::vector<std::pair<uint64_t, uint64_t>>& out, bool tex = false);
|
||||
uint64_t memory_pages_collected(); // (statistics)
|
||||
|
||||
/// An NVN format as Vulkan stores it (vk_formats.cpp): ASTC is decoded to RGBA8, block_w x block_h
|
||||
@@ -250,6 +254,7 @@ struct ImageMove {
|
||||
};
|
||||
/// The free space inside the textures' pool's blocks, and what the blocks hold in all; false without the pool
|
||||
bool textures_pool_usage(uint64_t* block_bytes, uint64_t* free_bytes);
|
||||
bool target_floor_kept(); // (the render targets' pool keeps a floor of blocks for the session: RX_VK_TARGET_FLOOR_MB)
|
||||
bool image_compact_begin(uint64_t max_bytes_a_pass, uint32_t max_moves_a_pass);
|
||||
bool image_compact_active();
|
||||
/// A pass begun: the moves VMA asks for. False: nothing left to move, the compaction is over (then *freed_bytes and
|
||||
@@ -271,6 +276,19 @@ bool create_buffer(uint64_t size, VkBufferUsageFlags usage, bool host_visible, V
|
||||
void destroy_buffer(VkBuffer buffer, VmaAllocation allocation);
|
||||
/// A buffer in VRAM only (within the driver's budget), or false
|
||||
bool create_device_buffer(uint64_t size, VkBufferUsageFlags usage, VkBuffer& buffer, VmaAllocation& allocation, bool dedicated = false);
|
||||
/// (Triforce) device memory for a piece of a sparse buffer, and its release (vk_vma.cpp)
|
||||
bool vma_piece_alloc(uint64_t size, uint64_t alignment, uint32_t type_bits, VmaAllocation& allocation, VkDeviceMemory& memory,
|
||||
uint64_t& offset);
|
||||
void vma_piece_free(VmaAllocation allocation);
|
||||
/// (Triforce) the mirrors' pieces made resident in this recording, bound before it is submitted (vk_memory.cpp)
|
||||
void memory_mirror_binds_flush();
|
||||
bool memory_mirror_bind_wait(VkSemaphoreSubmitInfo* wait); // (and the wait a queue submission makes for those binds)
|
||||
/// (Triforce, vk_memory.cpp) The memory handling played on NVIDIA cards only (mirrors in pieces, compacted textures): on there,
|
||||
/// and on trial on a card of another make until triforce_memtrial.txt says it passed or failed. tick: at the 600th presents;
|
||||
/// device_lost: from device_lost()
|
||||
bool memory_trial_on();
|
||||
void memory_trial_tick();
|
||||
void memory_trial_device_lost();
|
||||
/// Device-local memory this process uses and what the driver allows it now (VK_EXT_memory_budget), summed over the
|
||||
/// device-local heaps; false when unknown
|
||||
bool vram_usage(uint64_t* used, uint64_t* budget, uint64_t* block_free = nullptr);
|
||||
|
||||
@@ -885,6 +885,7 @@ void prepare_with_screen() {
|
||||
std::thread worker([] {
|
||||
const bool update = prepare_is_update(); // (an update, RX_VK_PREPARE_BACKGROUND=1: what the notes name now, the rest in play)
|
||||
const bool prepared = !update && prepare_run(prep_progress, false);
|
||||
if (update) state_list_run(); // (as the start without a screen does, below: the state lists' pipelines before the precompile)
|
||||
if (prepared && !prepare_stopped()) { // (a first start only: the pipelines the notes predict, into the driver's cache)
|
||||
prep_done.store(0), prep_total.store(0);
|
||||
prep_stage.store(1);
|
||||
|
||||
1581
gpu/vk/vk_memory.cpp
1581
gpu/vk/vk_memory.cpp
File diff suppressed because it is too large
Load Diff
@@ -180,6 +180,8 @@ void queue_flush_locked() { // (dev.present_mu held)
|
||||
si.pCommandBufferInfos = pend.cbs.data();
|
||||
si.signalSemaphoreInfoCount = n;
|
||||
si.pSignalSemaphoreInfos = sig;
|
||||
VkSemaphoreSubmitInfo bind_wait; // (Triforce: after the memory of the mirrors' new pieces is bound, vk_memory.cpp)
|
||||
if (memory_mirror_bind_wait(&bind_wait)) si.waitSemaphoreInfoCount = 1, si.pWaitSemaphoreInfos = &bind_wait;
|
||||
PROF(PROF_QSUBMIT);
|
||||
const VkResult res = vkQueueSubmit2(dev.queue, 1, &si, VK_NULL_HANDLE);
|
||||
static int failures;
|
||||
@@ -355,6 +357,8 @@ void submit_recording(Recording& r, bool signal_value = true, uint64_t signal_at
|
||||
return;
|
||||
}
|
||||
PROF(PROF_QSUBMIT);
|
||||
VkSemaphoreSubmitInfo bind_wait; // (Triforce: after the memory of the mirrors' new pieces is bound, vk_memory.cpp)
|
||||
if (memory_mirror_bind_wait(&bind_wait)) si.waitSemaphoreInfoCount = 1, si.pWaitSemaphoreInfos = &bind_wait;
|
||||
const VkResult res = vkQueueSubmit2(dev.queue, 1, &si, r.fence);
|
||||
r.submitted = res == VK_SUCCESS;
|
||||
static int failures;
|
||||
@@ -732,7 +736,9 @@ void replay(VkCommandBuffer& cmd, NvnState& s, const uint8_t* p, const uint8_t*
|
||||
end_pass(cmd);
|
||||
GprofSpan gp(cmd, "event signal (barrier, write, barrier)");
|
||||
full_barrier(cmd);
|
||||
memory_bind_event_write(true); // (Triforce: not a write-back the captures of its page wait for)
|
||||
gpu_write(cmd, e.gpu, &v, 4);
|
||||
memory_bind_event_write(false);
|
||||
break;
|
||||
}
|
||||
case NOP_WAIT_EVENT: { // (a wait for an event another queue's work signals: submit_job orders them)
|
||||
@@ -1282,10 +1288,76 @@ void maybe_split(VkCommandBuffer& cmd) {
|
||||
// (hookshot) One cause of those is gone: the dry pass did not mark the targets of a clear (NOP_CLEAR_COLOR in replay()). With
|
||||
// that, Castle Town's market and eight other places showed no black frame on 3 to 8 threads; the Kakariko walk was not run
|
||||
// again, so it stays off. On a 9950X3D the replay's wall time in the market goes from 10.9 ms a frame to 7.75 on 3 threads.
|
||||
int par_threads() {
|
||||
static const int n = std::clamp(getenv("RX_VK_PARALLEL") ? atoi(getenv("RX_VK_PARALLEL")) : 0, 0, 8);
|
||||
// (Triforce) RX_VK_PARALLEL_AUTO=<n> (2 to 8): the parallel replay on <n> threads where it is the faster of the two, measured
|
||||
// as the game runs. On three threads it is faster where the replay is the frame's limit and slower where the GPU is: at the
|
||||
// Castle Town save 67.1 fps against 58.8 (RTX 4070 SUPER, 2560 x 1440, windowed), on a tour of Hyrule Field, Kakariko and
|
||||
// Lake Hylia 59.5 and 61.9 against 64.8 and 66.7 (its recordings done sooner, the game's thread only waits longer for the
|
||||
// GPU, and the dry pass and the hand-over are not free); the flicker counts of the frame log were the same either way.
|
||||
// par_tune_present tries the other way for a moment every so often and keeps the one whose frames are shorter.
|
||||
// RX_VK_PARALLEL=<n> still means always, as before.
|
||||
// EXPERIMENTAL, off unless asked for: its first runs (2026-11-11) gained nothing, 54.7 fps against 58.9 standing in Castle Town
|
||||
// and 60.4 against 61.7 on the tour. The frame time of a scene with a crowd moves by 2 ms within seconds (one trial read 18.59
|
||||
// and then 16.44 ms for the same way), more than the difference being measured, so the trials decide on noise
|
||||
int par_threads_max() {
|
||||
static const int n = [] {
|
||||
if (const char* e = getenv("RX_VK_PARALLEL")) return std::clamp(atoi(e), 0, 8);
|
||||
if (const char* e = getenv("RX_VK_PARALLEL_AUTO")) return std::clamp(atoi(e), 0, 8);
|
||||
return 0;
|
||||
}();
|
||||
return n;
|
||||
}
|
||||
bool par_auto() {
|
||||
static const bool v = !getenv("RX_VK_PARALLEL") && par_threads_max() >= 2;
|
||||
return v;
|
||||
}
|
||||
bool par_auto_on = false; // (the replay thread's: the parallel replay is the one in use now)
|
||||
int par_threads() {
|
||||
const int n = par_threads_max();
|
||||
return par_auto() && !par_auto_on ? 0 : n;
|
||||
}
|
||||
/// At each present (the replay thread): the tuner. Every kEvery presents it times kTake frames as they are, kTake the other way
|
||||
/// (after kSettle to settle) and kTake as they were again, and changes over when the other way's mean frame was at least 4%
|
||||
/// shorter than both: a scene that changed under the trial does not pass for a gain. Frames over 60 ms (a load) are left out
|
||||
void par_tune_present() {
|
||||
if (!par_auto()) return;
|
||||
constexpr int kEvery = 900, kTake = 80, kSettle = 24;
|
||||
static auto last = std::chrono::steady_clock::now();
|
||||
const auto now = std::chrono::steady_clock::now();
|
||||
const double ms = std::chrono::duration<double, std::milli>(now - last).count();
|
||||
last = now;
|
||||
static int phase = 0, n = 0, cnt[3];
|
||||
static double sum[3];
|
||||
static int64_t frames = 0, next_trial = 600, st_trials = 0, st_changes = 0;
|
||||
frames++;
|
||||
if (phase == 0) {
|
||||
if (frames < next_trial) return;
|
||||
phase = 1, n = 0;
|
||||
sum[0] = sum[1] = sum[2] = 0, cnt[0] = cnt[1] = cnt[2] = 0;
|
||||
return;
|
||||
}
|
||||
const int k = phase - 1;
|
||||
n++;
|
||||
if (n > (phase == 1 ? 0 : kSettle) && ms <= 60.0) sum[k] += ms, cnt[k]++;
|
||||
if (n < (phase == 1 ? kTake : kSettle + kTake)) return;
|
||||
n = 0;
|
||||
if (phase == 1) {
|
||||
par_auto_on = !par_auto_on, phase = 2;
|
||||
} else if (phase == 2) {
|
||||
par_auto_on = !par_auto_on, phase = 3;
|
||||
} else {
|
||||
phase = 0, st_trials++;
|
||||
const bool ok = cnt[0] >= kTake * 3 / 4 && cnt[1] >= kTake * 3 / 4 && cnt[2] >= kTake * 3 / 4;
|
||||
const double a = cnt[0] ? sum[0] / cnt[0] : 0, b = cnt[1] ? sum[1] / cnt[1] : 0, c = cnt[2] ? sum[2] / cnt[2] : 0;
|
||||
const bool better = ok && b < std::min(a, c) * 0.96;
|
||||
if (better) par_auto_on = !par_auto_on, st_changes++;
|
||||
next_trial = frames + (better ? kEvery : kEvery * 2);
|
||||
if (better || st_trials <= 8 || st_trials % 20 == 0)
|
||||
logf("vk: parallel replay, trial %lld: %s %.2f ms a frame, the other way %.2f, %s again %.2f: %s (%lld changes so far)",
|
||||
(long long)st_trials, par_auto_on != better ? "parallel" : "serial", a, b, par_auto_on != better ? "parallel" : "serial", c,
|
||||
better ? (par_auto_on ? "parallel from now on" : "serial from now on") : (ok ? "as it was" : "not measured (a load)"),
|
||||
(long long)st_changes);
|
||||
}
|
||||
}
|
||||
struct ParPool {
|
||||
std::mutex mu;
|
||||
std::condition_variable cv, done;
|
||||
@@ -1531,6 +1603,8 @@ const uint8_t* replay_handle(VkCommandBuffer& cmd, const uint8_t* h, bool may_st
|
||||
si.pCommandBufferInfos = cb;
|
||||
std::lock_guard lk(dev.present_mu);
|
||||
queue_flush_locked();
|
||||
VkSemaphoreSubmitInfo bind_wait; // (Triforce: after the memory of the mirrors' new pieces is bound, vk_memory.cpp)
|
||||
if (memory_mirror_bind_wait(&bind_wait)) si.waitSemaphoreInfoCount = 1, si.pWaitSemaphoreInfos = &bind_wait;
|
||||
const VkResult res = vkQueueSubmit2(dev.queue, 1, &si, j->r->fence);
|
||||
j->r->submitted = res == VK_SUCCESS;
|
||||
if (res != VK_SUCCESS) logf("vk: segment submit %llu failed (%d)", (unsigned long long)j->r->value, (int)res);
|
||||
@@ -2559,6 +2633,7 @@ uint32_t crc32(const uint8_t* p, size_t n, uint32_t c = 0xFFFFFFFFu) {
|
||||
/// closed (issue 1275: captures kept 25 -> 6527, staging 256 -> 1408 MB). Now the report once (a thread that sees the loss
|
||||
/// too waits for it, instead of ending the process before it is written), then the process ends as replay_fatal does
|
||||
[[noreturn]] void device_lost() {
|
||||
memory_trial_device_lost(); // (Triforce: with the memory handling on trial on this card: off from the next start)
|
||||
static std::once_flag once;
|
||||
std::call_once(once, checkpoints_report);
|
||||
replay_fatal("the GPU device was lost (VK_ERROR_DEVICE_LOST): the driver stopped or reset the GPU, and nothing sent "
|
||||
@@ -2837,7 +2912,8 @@ void status_tick() {
|
||||
std::string dir(exe, n);
|
||||
FILE* f = fopen((dir.substr(0, dir.find_last_of("\\/") + 1) + "triforce_frames.csv").c_str(), "w");
|
||||
if (f) fprintf(f, "frame,ms,pipe_waits,pipe_wait_ms,left_out,translate_ms,uploads,upload_mb,textures,memsync_ms,pipelines_ms,"
|
||||
"uploads_ms,programs_ms,slot_wait_ms,gpu_behind_ms,draw_ms,desc_ms,cmds_ms,tex_lookup_ms,textures_gone\n");
|
||||
"uploads_ms,programs_ms,slot_wait_ms,gpu_behind_ms,draw_ms,desc_ms,cmds_ms,tex_lookup_ms,textures_gone,"
|
||||
"dummies,draws\n");
|
||||
return f;
|
||||
}();
|
||||
if (trace && last_ns) {
|
||||
@@ -2851,12 +2927,17 @@ void status_tick() {
|
||||
const auto dp = [](int i) { return (prof_ns[i] - l_prof[i]) / 1e6; };
|
||||
static uint64_t l_gone;
|
||||
const uint64_t gone = texture_destroy_generation();
|
||||
fprintf(trace, "%llu,%.2f,%lld,%.2f,%lld,%.1f,%lld,%.2f,%zu,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%llu\n",
|
||||
static int64_t l_dummy, l_draws; // (Triforce: the bindings that sampled the dummy, and the draws, this frame)
|
||||
const int64_t dummy = draw_dummy_textures();
|
||||
int64_t draws_now = 0, sk_now = 0, di_now = 0, pl_now = 0;
|
||||
draw_stats(&draws_now, &sk_now, &di_now, &pl_now);
|
||||
fprintf(trace, "%llu,%.2f,%lld,%.2f,%lld,%.1f,%lld,%.2f,%zu,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%llu,%lld,%lld\n",
|
||||
(unsigned long long)frames, (now_ns - last_ns) / 1e6, (long long)(pw - l_pw), pwms - l_pwms, (long long)(ps - l_ps),
|
||||
tr - l_tr, (long long)(up - l_up), (upb - l_upb) / 1048576.0, texture_count(), dp(PROF_MEMSYNC), dp(PROF_PIPE),
|
||||
dp(PROF_UPLOAD), dp(PROF_PROGRAM), dp(PROF_SLOTWAIT), (st_wait_ns - l_wait) / 1e6, dp(PROF_DRAW), dp(PROF_DESC),
|
||||
dp(PROF_CMDS), dp(PROF_TEXGET), (unsigned long long)(gone - l_gone));
|
||||
l_gone = gone;
|
||||
dp(PROF_CMDS), dp(PROF_TEXGET), (unsigned long long)(gone - l_gone), (long long)(dummy - l_dummy),
|
||||
(long long)(draws_now - l_draws));
|
||||
l_gone = gone, l_dummy = dummy, l_draws = draws_now;
|
||||
if (frames % 60 == 0) fflush(trace);
|
||||
l_ps = ps, l_pw = pw, l_pwms = pwms, l_up = up, l_upb = upb, l_tr = tr, l_wait = st_wait_ns;
|
||||
std::memcpy(l_prof, prof_ns, sizeof l_prof);
|
||||
@@ -2964,6 +3045,7 @@ void do_present(const nvn_tex* t, const int crop[4]) {
|
||||
full_barrier(r.cmd);
|
||||
presents++;
|
||||
presents_now = presents;
|
||||
par_tune_present(); // (Triforce: RX_VK_PARALLEL_AUTO)
|
||||
wide_sides_tick();
|
||||
if (presents == 1) hang_watch_start();
|
||||
{ // (EvilBoyce, diagnostics) RX_VK_WATCH_CPU=<present>:<gpu address>[,...] (hex addresses): from that present on, the
|
||||
@@ -4178,7 +4260,7 @@ bool replay_start() {
|
||||
VkSemaphoreCreateInfo si{VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO};
|
||||
vkCreateSemaphore(dev.device, &si, nullptr, &f.acquired);
|
||||
}
|
||||
if (par_threads() >= 2 && !init_pools_make(pi)) return false; // (last: nothing after it fails and leaves them made)
|
||||
if (par_threads_max() >= 2 && !init_pools_make(pi)) return false; // (last: nothing after it fails and leaves them made)
|
||||
logf("vk: replay ready (%s; push descriptors %s (%u), extended dynamic state 3 %s, custom border colours %s)",
|
||||
dev.offscreen ? "offscreen" : "window", dev.push_descriptor ? "yes" : "no", dev.max_push_descriptors,
|
||||
dev.eds3_blend ? "yes" : "no", dev.custom_border_color ? "yes" : "no");
|
||||
|
||||
@@ -566,11 +566,9 @@ Shader::Profile make_profile() {
|
||||
p.support_scaled_attributes = false; // (*_UI2F / *_I2F read as integers, converted in the shader: R32 has no scaled format)
|
||||
p.support_multi_viewport = true;
|
||||
p.support_sampled_image_array_nonuniform_indexing = true;
|
||||
// (hookshot) a graphics stage the GPU cannot pin to 32 lanes runs at its default: 64 on AMD, where only compute is pinned
|
||||
// (vk_main.cpp). The translator then keeps each 32-lane guest warp to its half of the host's, as Eden does on AMD; the
|
||||
// long path is right at 32 lanes too. The ballots and shuffles of the game's fragment shaders ran across both halves
|
||||
p.warp_size_potentially_larger_than_guest =
|
||||
dev.subgroup_size > 32 && (dev.sg32_stages & VK_SHADER_STAGE_ALL_GRAPHICS) != VK_SHADER_STAGE_ALL_GRAPHICS;
|
||||
// Unpinned SPIR-V 1.6 stages can use any supported subgroup size, even when the default is 32.
|
||||
// The stage-specific profiles below keep the short path only where pin_subgroup actually pins 32 lanes.
|
||||
p.warp_size_potentially_larger_than_guest = dev.subgroup_max > 32;
|
||||
// (RX_VK_WARP64=0 or 1, a test: the graphics stages' long warp path off or on, whatever the GPU; in the cache's stamp)
|
||||
if (const char* e = getenv("RX_VK_WARP64")) p.warp_size_potentially_larger_than_guest = e[0] == '1';
|
||||
p.lower_left_origin_mode = false;
|
||||
@@ -1225,9 +1223,11 @@ uint64_t disk_stamp() {
|
||||
const uint32_t mesh = (dev.mesh_shader ? 1u : 0u) | (dev.mesh_fallback ? 2u : 0u) | (dev.task_mesh_fallback ? 4u : 0u);
|
||||
add(&mesh, sizeof mesh);
|
||||
// (and what of the GPU the translator is told (make_profile): its warps' lanes and the driver's shuffle workaround, and
|
||||
// which stages it pins to 32 lanes, which keep the short path: profile_graphics)
|
||||
// which stages it pins to 32 lanes, which keep the short path: profile_graphics. An explicit warp override bypasses
|
||||
// the pinned graphics profile even if the base profile already has the requested width)
|
||||
const uint32_t lanes[] = {profile().warp_size_potentially_larger_than_guest ? 1u : 0u, profile().max_subgroup_size,
|
||||
profile().has_broken_spirv_fragment_shuffle ? 1u : 0u, (uint32_t)dev.sg32_stages};
|
||||
profile().has_broken_spirv_fragment_shuffle ? 1u : 0u, (uint32_t)dev.sg32_stages,
|
||||
getenv("RX_VK_WARP64") ? 1u : 0u};
|
||||
add(lanes, sizeof lanes);
|
||||
// (the records keep Shader::Info as it lies in memory: a cache written by another C++ library (the MSVC build's, then the
|
||||
// llvm-mingw one's libc++) has other layouts, and reading it crashed at start. The library and the sizes, so it starts again)
|
||||
|
||||
@@ -241,7 +241,21 @@ std::unordered_map<uint64_t, std::vector<Pooled>> target_pool; // by pool_key_
|
||||
uint64_t pool_bytes, pool_n, st_pool_reused, st_pool_made, st_pool_destroyed;
|
||||
bool pool_pressed; // (VRAM near what the driver allows: texture_frame's budget)
|
||||
bool compact_on() { // (hookshot, RX_VK_COMPACT=1: the textures in a pool of their own, compacted: compact_step)
|
||||
static const bool v = getenv("RX_VK_COMPACT") && atoi(getenv("RX_VK_COMPACT")) != 0;
|
||||
// (Triforce) on by default on NVIDIA cards (RX_VK_COMPACT=0: off; =1: on, any card). Measured there: an RTX 4070 SUPER
|
||||
// played with it since 2026-11-10 (a 584,000-present session, 132 compactions, no fault; 85 -> 80.6 fps after a six-area
|
||||
// tour against 81 -> 64 without), and with the allowance held to 7300 MB (RX_VK_TEST_ALLOWED_MB, an 8 GB card's) a tour of
|
||||
// Hyrule Field, Kakariko, Lake Hylia and Kokiri peaked at 7165 MB of VRAM against 8565 without (over the allowance in 2 of
|
||||
// 12 reports against 11) at 69 fps against ~64. On an 8 GB card what is over the allowance is paged to system memory, and
|
||||
// the textures and render targets evicted to get back under it are made and uploaded again, 58 to 190 ms a frame (an
|
||||
// RX 5700 XT's and an RTX 3060's logs, 2026-11-11). Not measured on AMD or Intel: as it was there until someone does
|
||||
// (Triforce, later that day) and on trial on the other makes' cards: memory_trial_on (vk_memory.cpp), which turns it off by
|
||||
// itself on a PC where a session with it loses the GPU or two end in a crash
|
||||
static const bool v = [] {
|
||||
if (const char* e = getenv("RX_VK_COMPACT")) return atoi(e) != 0;
|
||||
const bool on = memory_trial_on();
|
||||
if (on) logf("vk: the textures' VRAM is compacted (RX_VK_COMPACT=0: off)");
|
||||
return on;
|
||||
}();
|
||||
return v;
|
||||
}
|
||||
uint64_t pool_cap() {
|
||||
@@ -284,8 +298,11 @@ void pool_put(Dead& d) {
|
||||
/// presents: destroyed, at most 16 a present (64 while VRAM is short). Returns how many
|
||||
size_t pool_trim() {
|
||||
std::lock_guard lk(pool_mu);
|
||||
const uint64_t cap = pool_pressed ? 0 : pool_cap();
|
||||
const size_t most = pool_pressed ? 64 : 16;
|
||||
// (Triforce, 11-11: with a floor of blocks kept (target_floor_kept) the pool is not emptied while VRAM is short: what it gave
|
||||
// back went into blocks the floor keeps, the driver's count did not move, and every target of the next frames was made new)
|
||||
const bool give_back = pool_pressed && !target_floor_kept();
|
||||
const uint64_t cap = give_back ? 0 : pool_cap();
|
||||
const size_t most = give_back ? 64 : 16;
|
||||
size_t n = 0;
|
||||
for (auto it = target_pool.begin(); it != target_pool.end() && n < most;) {
|
||||
auto& v = it->second;
|
||||
@@ -1297,6 +1314,7 @@ struct UpSlow { int64_t ns; uint32_t w, h, layers, nvn; int route; uint64_t byte
|
||||
std::vector<UpSlow> up_slowest;
|
||||
std::mutex up_mu; // (uploads may come from a parallel replay's threads)
|
||||
std::atomic<int64_t> frame_upload_ns{0}; // (this frame's time in upload(), every thread's)
|
||||
std::atomic<int> frame_astc_admitted{0}; // (Triforce: this frame's new ASTC textures asked about so far: upload_defer)
|
||||
std::atomic<int64_t> st_deferred{0}, st_deferred_frame{0};
|
||||
void upload_frame_report() {
|
||||
std::lock_guard lk(up_mu);
|
||||
@@ -1589,11 +1607,13 @@ bool upload_inner(Texture& tex, const Ranges* dirty) {
|
||||
|
||||
/// What the CPU wrote of a texture's storage since gen, as byte ranges from its start. A texture in a virtual
|
||||
/// (sparse) pool is looked at page by page through its mapping, and is all dirty when its mapping changed.
|
||||
bool texture_dirty(Texture& tex, uint32_t gen, Ranges& out) {
|
||||
/// same_ok (Triforce): a capture's stamp of a page with the bytes of its last copy is no write (vk_memory.cpp page_gen_tex): only
|
||||
/// for an uploaded texture's "written since my upload?"
|
||||
bool texture_dirty(Texture& tex, uint32_t gen, Ranges& out, bool same_ok = false) {
|
||||
out.clear();
|
||||
constexpr uint64_t kVirtual = 1ull << 40, kVPage = 1ull << 16;
|
||||
if (tex.gpu < kVirtual) {
|
||||
memory_written_runs(tex.guest, tex.guest_size, gen, out);
|
||||
memory_written_runs(tex.guest, tex.guest_size, gen, out, same_ok);
|
||||
return !out.empty();
|
||||
}
|
||||
// (the page walk only when the GPU mappings changed: the terrain's sparse arrays span over 1 GB, 64 KB pages,
|
||||
@@ -1661,7 +1681,7 @@ bool texture_dirty(Texture& tex, uint32_t gen, Ranges& out) {
|
||||
const size_t remaps = out.size();
|
||||
for (const Texture::Run& r : tex.runs) {
|
||||
in_run.clear();
|
||||
memory_written_runs(r.guest, r.n, gen, in_run);
|
||||
memory_written_runs(r.guest, r.n, gen, in_run, same_ok);
|
||||
for (const auto& [lo, hi] : in_run) {
|
||||
if (out.size() > remaps && out.back().second == r.off + lo) out.back().second = r.off + hi;
|
||||
else out.push_back({r.off + lo, r.off + hi});
|
||||
@@ -1693,6 +1713,14 @@ void destroy_later(Texture& tex) {
|
||||
}
|
||||
d.memory = tex.sparse_chunks;
|
||||
dead.push_back(std::move(d));
|
||||
if (tex.twin3d) { // (Triforce: its one-slice 3D copy goes with it)
|
||||
Dead t{tex.last_value, tex.twin3d, tex.twin3d_alloc, {}, {}};
|
||||
for (auto& kv : tex.twin3d_views)
|
||||
if (kv.second) t.views.push_back(kv.second);
|
||||
dead.push_back(std::move(t));
|
||||
tex.twin3d = VK_NULL_HANDLE, tex.twin3d_alloc = nullptr;
|
||||
tex.twin3d_views.clear();
|
||||
}
|
||||
total_bytes -= std::min(total_bytes, tex.bytes + tex.pad);
|
||||
}
|
||||
|
||||
@@ -2175,6 +2203,7 @@ VkCommandBuffer upload_command_buffer() {
|
||||
}
|
||||
bool texture_uploads_recorded() {
|
||||
if (memory_rb_refresh(up_cmd)) up_used = true; // (PR 430: readback pages from guest memory, last of the uploads)
|
||||
memory_mirror_binds_flush(); // (Triforce: the memory of the mirrors' new pieces, bound before this recording is submitted)
|
||||
memory_table_flush(); // (vk_memory.cpp: the page table's changes, after this recording's mirror copies)
|
||||
if (up_used) full_barrier(up_cmd); // (before the recording that reads them)
|
||||
return up_used;
|
||||
@@ -2315,6 +2344,13 @@ namespace {
|
||||
bool upload_defer(const Texture& tex) {
|
||||
static const int64_t budget_ns = (int64_t)(getenv("RX_VK_UPLOAD_BUDGET_MS") ? atof(getenv("RX_VK_UPLOAD_BUDGET_MS")) : 40.0) * 1000000;
|
||||
if (!budget_ns || !upload_defer_ok || tex.uploaded || !tex.astc || tex.gpu_written) return false;
|
||||
// (Triforce) and the first RX_VK_UPLOAD_MIN_ASTC of a frame (24) are never left for later. The budget counts every upload of
|
||||
// the frame, and a frame whose other uploads took it all left every new ASTC texture on the placeholder: on a card at its
|
||||
// VRAM allowance the render targets evicted and made again are uploaded from memory on the CPU, 58 to 190 ms a frame for
|
||||
// frames on end (an RX 5700 XT's log, 2026-11-11: 8,586 textures left for later in one session), and the materials stayed
|
||||
// flat white or blue for as long as that went on. An ASTC upload is decoded on the GPU: its part of such a frame is small
|
||||
static const int min_astc = getenv("RX_VK_UPLOAD_MIN_ASTC") ? std::max(0, atoi(getenv("RX_VK_UPLOAD_MIN_ASTC"))) : 24;
|
||||
if (frame_astc_admitted.fetch_add(1, std::memory_order_relaxed) < min_astc) return false;
|
||||
if (frame_upload_ns.load(std::memory_order_relaxed) < budget_ns) return false;
|
||||
st_deferred.fetch_add(1, std::memory_order_relaxed), st_deferred_frame.fetch_add(1, std::memory_order_relaxed);
|
||||
return true;
|
||||
@@ -2402,7 +2438,7 @@ Texture* texture_touch(Texture* tex, bool for_sampling) {
|
||||
"%llu)", tex->id, (unsigned long long)tex->gpu, tex->width, tex->height, tex->layers, tex->nvn_format,
|
||||
(unsigned long long)frame_no, (unsigned long long)tex->last_written);
|
||||
}
|
||||
if (tex->uploaded && !gpu_dirty && !texture_dirty(*tex, since, dirty)) return tex;
|
||||
if (tex->uploaded && !gpu_dirty && !texture_dirty(*tex, since, dirty, !tex->gpu_written)) return tex;
|
||||
const uint32_t gen = memory_generation();
|
||||
// (storage render targets took last, which the CPU has not written since: nothing to upload, the memory holds
|
||||
// nothing the GPU's images did not replace. The game's transient heaps move with the dynamic resolution, and a
|
||||
@@ -2488,6 +2524,35 @@ Texture* texture_touch(Texture* tex, bool for_sampling) {
|
||||
// (hookshot) a BC3 image whose last upload failed waits before the next try: its upload stages the whole ASTC source (some
|
||||
// 200 MB for a terrain array), not to be repeated every submission while the memory it needs is not there
|
||||
if (tex->bc && !tex->uploaded && tex->bc_retry_frame > frame_no) return tex;
|
||||
// (Triforce, RX_VK_STAMP_SRC=1, a diagnostic: a texture of 1 MB or more uploaded again: who wrote its first dirty page (the
|
||||
// write watch 1, a capture 2, else a caller of memory_stamp, placed by memory_stamp's own address), and whether its memory
|
||||
// holds other bytes than at its last upload at all. 400 lines)
|
||||
static const bool stamp_src = getenv("RX_VK_STAMP_SRC") != nullptr;
|
||||
if (stamp_src && tex->uploaded && tex->guest_size >= (1u << 20)) {
|
||||
static int told;
|
||||
static std::unordered_map<uint32_t, int> times; // (RX_VK_STAMP_SRC=<n>: only a texture's n-th such upload and later)
|
||||
static const int from_nth = std::max(1, atoi(getenv("RX_VK_STAMP_SRC")));
|
||||
const uint64_t h = guest_hash(*tex);
|
||||
if (++times[tex->id] >= from_nth && told++ < 400) {
|
||||
uint64_t part = 0;
|
||||
for (const auto& r : dirty) part += r.second - r.first;
|
||||
const uint64_t off = dirty.empty() ? 0 : dirty[0].first;
|
||||
const uint64_t guest = tex->gpu < (1ull << 40) ? tex->guest + off : host->gpu_guest(tex->gpu + off);
|
||||
uint32_t pg = 0;
|
||||
uintptr_t anchor = 0;
|
||||
const uintptr_t who = guest ? memory_stamp_who(guest, &pg, &anchor) : 0;
|
||||
logf("vk: stamp src: #%u %#llx %ux%u x%u NVN %#x uploaded again (%d) at frame %llu: %llu of %llu KB dirty in %zu ranges%s; bytes %s; "
|
||||
"page %#llx gen %u (uploaded at %u) by %s%+lld", tex->id, (unsigned long long)tex->gpu, tex->width, tex->height,
|
||||
tex->layers, tex->nvn_format, times[tex->id], (unsigned long long)frame_no, (unsigned long long)(part >> 10),
|
||||
(unsigned long long)(tex->guest_size >> 10), dirty.size(), gpu_dirty ? " (a GPU buffer wrote it)" : "",
|
||||
!tex->verify_hash ? "not hashed before" : h == tex->verify_hash ? "THE SAME as at its last upload" : "changed",
|
||||
(unsigned long long)guest, pg, since, who == 1 ? "the write watch" : who == 2 ? "a capture" : who ? "memory_stamp" : "unknown",
|
||||
who > 2 ? (long long)(who - anchor) : 0ll);
|
||||
}
|
||||
tex->verify_hash = h;
|
||||
} else if (stamp_src && tex->guest_size >= (1u << 20)) {
|
||||
tex->verify_hash = guest_hash(*tex);
|
||||
}
|
||||
const auto up0 = std::chrono::steady_clock::now();
|
||||
const bool up_ok = upload(*tex, partial || sparse_written ? &dirty : nullptr);
|
||||
{ // (an upload over 15 ms: what it was, 60 of them)
|
||||
@@ -2565,6 +2630,84 @@ Texture* texture_touch(Texture* tex, bool for_sampling) {
|
||||
return tex;
|
||||
}
|
||||
|
||||
/// (Triforce, 2026-11-11) A flat texture sampled by a program whose instruction is a 3D sample: a one-slice 3D image with its
|
||||
/// texels, and a 3D view of that. Vulkan has no 3D view of a 2D image, so such a binding had no view and read zeros (a null
|
||||
/// descriptor). The console's GPU samples by the texture's own header whatever the instruction says: a flat texture's texel at
|
||||
/// (u, v), the third coordinate unused, which a one-slice 3D image gives for any w. OoT's water in Zora's River binds one of
|
||||
/// the game's 4 x 4 textures so (slot 0x102, 44 lookups in 10 s; its ripples ran in slow motion). Small plain textures only:
|
||||
/// one layer copied, up to 1024 x 1024, not depth, not kept as BC3. Copied again at a lookup after the texture's contents
|
||||
/// changed (an upload, or the GPU writing it): the lookups are cached by slot, so one the GPU draws every frame is a frame or
|
||||
/// more behind here. RX_VK_NO_TWIN3D=1: no such view, as before
|
||||
VkImageView twin3d_view(Texture* tex, VkFormat f, uint32_t level0, uint32_t nlevels, uint32_t layer, const VkComponentMapping& cm) {
|
||||
static const bool off = getenv("RX_VK_NO_TWIN3D") != nullptr;
|
||||
if (off || tex->type != VK_IMAGE_TYPE_2D || tex->sparse || tex->bc || tex->samples != VK_SAMPLE_COUNT_1_BIT ||
|
||||
tex->aspect != VK_IMAGE_ASPECT_COLOR_BIT || is_bc(tex->format) || is_astc(tex->format) ||
|
||||
(uint64_t)tex->width * tex->height > 1024ull * 1024 || !tex->image)
|
||||
return VK_NULL_HANDLE;
|
||||
VkCommandBuffer cmd = VK_NULL_HANDLE;
|
||||
if (!tex->twin3d) {
|
||||
VkImageCreateInfo ci{VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO};
|
||||
ci.imageType = VK_IMAGE_TYPE_3D;
|
||||
ci.format = tex->format;
|
||||
ci.extent = {tex->width, tex->height, 1};
|
||||
ci.mipLevels = tex->levels;
|
||||
ci.arrayLayers = 1;
|
||||
ci.samples = VK_SAMPLE_COUNT_1_BIT;
|
||||
ci.tiling = VK_IMAGE_TILING_OPTIMAL;
|
||||
ci.usage = VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT;
|
||||
ci.flags = VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT; // (the view's format may be another of its class: sRGB or not)
|
||||
if (!create_image(ci, tex->twin3d, tex->twin3d_alloc, false, false, false)) {
|
||||
tex->twin3d = VK_NULL_HANDLE, tex->twin3d_alloc = nullptr;
|
||||
return VK_NULL_HANDLE;
|
||||
}
|
||||
cmd = upload_cmd();
|
||||
VkImageMemoryBarrier2 b{VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER_2};
|
||||
b.srcStageMask = b.dstStageMask = VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT;
|
||||
b.dstAccessMask = VK_ACCESS_2_MEMORY_READ_BIT | VK_ACCESS_2_MEMORY_WRITE_BIT;
|
||||
b.oldLayout = VK_IMAGE_LAYOUT_UNDEFINED, b.newLayout = VK_IMAGE_LAYOUT_GENERAL;
|
||||
b.srcQueueFamilyIndex = b.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
|
||||
b.image = tex->twin3d;
|
||||
b.subresourceRange = {VK_IMAGE_ASPECT_COLOR_BIT, 0, VK_REMAINING_MIP_LEVELS, 0, 1};
|
||||
VkDependencyInfo di{VK_STRUCTURE_TYPE_DEPENDENCY_INFO};
|
||||
di.imageMemoryBarrierCount = 1, di.pImageMemoryBarriers = &b;
|
||||
vkCmdPipelineBarrier2(cmd, &di);
|
||||
tex->twin3d_value = ~0ull, tex->twin3d_written = ~0ull;
|
||||
static int told;
|
||||
if (told++ < 12)
|
||||
logf("vk: texture #%u %#llx %ux%u levels %u NVN %#x is sampled as a 3D texture: a one-slice 3D copy of it made", tex->id,
|
||||
(unsigned long long)tex->gpu, tex->width, tex->height, tex->levels, tex->nvn_format);
|
||||
}
|
||||
if (tex->twin3d_value != tex->upload_value || tex->twin3d_written != tex->last_written || tex->twin3d_layer != layer) {
|
||||
if (!cmd) cmd = upload_cmd();
|
||||
full_barrier(cmd, "barrier (a flat texture's 3D copy: before)");
|
||||
std::vector<VkImageCopy> rs;
|
||||
for (uint32_t l = 0; l < tex->levels; l++) {
|
||||
VkImageCopy r{};
|
||||
r.srcSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, l, layer, 1};
|
||||
r.dstSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, l, 0, 1};
|
||||
r.extent = {std::max(1u, tex->width >> l), std::max(1u, tex->height >> l), 1};
|
||||
rs.push_back(r);
|
||||
}
|
||||
vkCmdCopyImage(cmd, tex->image, VK_IMAGE_LAYOUT_GENERAL, tex->twin3d, VK_IMAGE_LAYOUT_GENERAL, (uint32_t)rs.size(), rs.data());
|
||||
full_barrier(cmd, "barrier (a flat texture's 3D copy: after)");
|
||||
tex->twin3d_value = tex->upload_value, tex->twin3d_written = tex->last_written, tex->twin3d_layer = layer;
|
||||
}
|
||||
uint64_t key = mix((uint64_t)f, (uint64_t)level0 << 32 | nlevels);
|
||||
key = mix(key, (uint64_t)cm.r << 24 | (uint64_t)cm.g << 16 | (uint64_t)cm.b << 8 | cm.a);
|
||||
const auto it = tex->twin3d_views.find(key);
|
||||
if (it != tex->twin3d_views.end()) return it->second;
|
||||
VkImageViewCreateInfo vi{VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO};
|
||||
vi.image = tex->twin3d;
|
||||
vi.viewType = VK_IMAGE_VIEW_TYPE_3D;
|
||||
vi.format = f;
|
||||
vi.components = cm;
|
||||
vi.subresourceRange = {VK_IMAGE_ASPECT_COLOR_BIT, level0, nlevels, 0, 1};
|
||||
VkImageView view = VK_NULL_HANDLE;
|
||||
if (vkCreateImageView(dev.device, &vi, nullptr, &view) != VK_SUCCESS) view = VK_NULL_HANDLE;
|
||||
tex->twin3d_views[key] = view;
|
||||
return view;
|
||||
}
|
||||
|
||||
VkImageView texture_view(Texture* tex, const nvn_tex* t, const nvn_view* v, VkImageViewType type, ViewUse use,
|
||||
VkFormat* format_out, VkFormat force_format) {
|
||||
if (!tex) return VK_NULL_HANDLE;
|
||||
@@ -2615,6 +2758,7 @@ VkImageView texture_view(Texture* tex, const nvn_tex* t, const nvn_view* v, VkIm
|
||||
else if (type == VK_IMAGE_VIEW_TYPE_2D && nlayers > 1) nlayers = 1;
|
||||
else if (type == VK_IMAGE_VIEW_TYPE_1D && nlayers > 1) nlayers = 1;
|
||||
}
|
||||
int twin_layer = -1; // (Triforce: a flat texture wanted as 3D: the layer its 3D copy is of)
|
||||
// the shader's type over the image's: what Vulkan allows of them
|
||||
switch (type) {
|
||||
case VK_IMAGE_VIEW_TYPE_1D: case VK_IMAGE_VIEW_TYPE_1D_ARRAY:
|
||||
@@ -2636,7 +2780,11 @@ VkImageView texture_view(Texture* tex, const nvn_tex* t, const nvn_view* v, VkIm
|
||||
if (type == VK_IMAGE_VIEW_TYPE_2D) nlayers = 1;
|
||||
break;
|
||||
case VK_IMAGE_VIEW_TYPE_3D:
|
||||
if (tex->type != VK_IMAGE_TYPE_3D) return VK_NULL_HANDLE;
|
||||
if (tex->type != VK_IMAGE_TYPE_3D) {
|
||||
// (Triforce: a flat texture sampled as a 3D one: its one-slice 3D copy, twin3d_view above)
|
||||
if (use != ViewUse::Sample) return VK_NULL_HANDLE;
|
||||
twin_layer = (int)layer0;
|
||||
}
|
||||
layer0 = 0, nlayers = 1;
|
||||
break;
|
||||
case VK_IMAGE_VIEW_TYPE_CUBE: case VK_IMAGE_VIEW_TYPE_CUBE_ARRAY:
|
||||
@@ -2675,6 +2823,10 @@ VkImageView texture_view(Texture* tex, const nvn_tex* t, const nvn_view* v, VkIm
|
||||
nf, (int)f, sw[0], sw[1], sw[2], sw[3], d.swizzle_r, d.swizzle_g, d.swizzle_b, d.swizzle_a,
|
||||
v && v->set_swizzle ? 1 : 0);
|
||||
}
|
||||
if (twin_layer >= 0) {
|
||||
if (format_out) *format_out = f;
|
||||
return twin3d_view(tex, f, level0, nlevels, (uint32_t)twin_layer, cm);
|
||||
}
|
||||
uint64_t key = mix((uint64_t)use, (uint64_t)type);
|
||||
key = mix(key, (uint64_t)f << 32 | aspect);
|
||||
key = mix(key, (uint64_t)level0 << 48 | (uint64_t)nlevels << 32 | (uint64_t)layer0 << 16 | nlayers);
|
||||
@@ -3945,9 +4097,13 @@ uint64_t preset_texture_budget(uint64_t preset_mb, bool preset_applies, bool vra
|
||||
return integrated ? std::min(b, own) : b;
|
||||
}
|
||||
|
||||
uint64_t dead_sweep_next = 0; // (the present of the next pass for the dead targets between texture_frame's looks; 0: none)
|
||||
void texture_sweep(uint64_t completed, bool sweep, uint64_t budget, uint64_t evict_to, bool known, uint64_t used, uint64_t allowed,
|
||||
bool dead_only);
|
||||
void texture_frame(uint64_t completed) {
|
||||
upload_frame_report();
|
||||
frame_upload_ns.store(0, std::memory_order_relaxed);
|
||||
frame_astc_admitted.store(0, std::memory_order_relaxed);
|
||||
if (const int64_t d = st_deferred_frame.exchange(0, std::memory_order_relaxed)) {
|
||||
// (the views and descriptor sets resolved to the placeholder this frame go stale, as when textures go: the next frame
|
||||
// looks those textures up again, and uploads them)
|
||||
@@ -4050,9 +4206,28 @@ void texture_frame(uint64_t completed) {
|
||||
// may have (other programs use the GPU too). The least recently used go first, down to 7/8 of it; nothing the
|
||||
// last 8 presents used, nothing recorded work may still read.
|
||||
static uint64_t own = 0;
|
||||
if (frame_no % 10) return;
|
||||
static bool last_known; // (the driver's numbers of the last full look, for the passes between)
|
||||
static uint64_t last_used, last_allowed;
|
||||
{ // (and a present in which a target's image was made new, not taken from the pool, asks for a pass three presents on,
|
||||
// when what it replaced can first be found dead (RX_VK_DEAD_MIN_UNUSED): by the passes' own findings alone the first
|
||||
// one of a run of new targets came at the next look, and in those ten presents the targets' blocks went 11 -> 22)
|
||||
static uint64_t made_seen;
|
||||
static const bool dead_quick = !(getenv("RX_VK_DEAD_QUICK") && !atoi(getenv("RX_VK_DEAD_QUICK")));
|
||||
if (st_pool_made != made_seen) {
|
||||
made_seen = st_pool_made;
|
||||
if (dead_quick && (!dead_sweep_next || dead_sweep_next > frame_no + 3)) dead_sweep_next = frame_no + 3;
|
||||
}
|
||||
}
|
||||
if (frame_no % 10) {
|
||||
// (Triforce, 11-11) between the looks, a pass for the dead transient targets alone while the game makes them fast
|
||||
// (dead_sweep_next, set by texture_sweep): see there
|
||||
if (dead_sweep_next && frame_no >= dead_sweep_next)
|
||||
texture_sweep(completed, true, ~0ull, 0, last_known, last_used, last_allowed, true);
|
||||
return;
|
||||
}
|
||||
uint64_t used = 0, allowed = 0, block_free = 0;
|
||||
const bool known = vram_usage(&used, &allowed, &block_free);
|
||||
last_known = known, last_used = used, last_allowed = allowed;
|
||||
if (!own) // (RX_VK_VRAM_MB, else 70% of what the driver allows the process at the first look)
|
||||
own = getenv("RX_VK_VRAM_MB") ? (uint64_t)atoi(getenv("RX_VK_VRAM_MB")) << 20
|
||||
: known ? std::min<uint64_t>(allowed / 100 * (allowed <= (8704ull << 20) ? 45 : 70), 6144ull << 20)
|
||||
@@ -4129,6 +4304,7 @@ void texture_frame(uint64_t completed) {
|
||||
}
|
||||
if (budget < evict_to) evict_to = 0; // (hookshot, RX_VK_COMPACT: the driver's limit asks for more: 7/8 of that, as before)
|
||||
if (frame_no % 600 == 0) {
|
||||
memory_trial_tick(); // (Triforce: the memory handling on trial on this card, vk_memory.cpp)
|
||||
vma_log_heaps();
|
||||
vma_log_image_blocks();
|
||||
{
|
||||
@@ -4211,9 +4387,22 @@ void texture_frame(uint64_t completed) {
|
||||
logf("vk: verify %ux%u: %d textures, %d changed since uploaded, %d of them unnoticed", verify_dim().first,
|
||||
verify_dim().second, checked, changed, missed);
|
||||
}
|
||||
texture_sweep(completed, total_bytes <= budget, budget, evict_to, known, used, allowed, false);
|
||||
}
|
||||
|
||||
/// (texture_frame) What goes: over the budget the least recently used, and the render targets drawn over since.
|
||||
/// (Triforce, 11-11) dead_only: a pass between texture_frame's looks (every 10 presents) for the dead transient targets alone.
|
||||
/// While the camera moves the game makes them at new sizes all the time: 1.2-3 new images a present running across Hyrule
|
||||
/// Field in the lab, up to 11 a present between areas in a player's session (6733 in 600 presents). A dead one waited for the
|
||||
/// next look, 10 presents plus the 3 it must go unused: at ~20 MB each that is ~650 MB of dead images in the lab's run and
|
||||
/// 2.6 GB in that session, on top of what is drawn, their pool's blocks 18 -> 31, and at the driver's limit each new block
|
||||
/// took 110-170 ms (ten frames of 115-180 ms in a row). So when a pass finds them coming at one a present and more, the next
|
||||
/// pass is the next present, at one in three presents three presents on, else the next look as before: the dead wait 4-6
|
||||
/// presents, not 13. RX_VK_DEAD_QUICK=0: only at the looks, as before
|
||||
void texture_sweep(uint64_t completed, bool sweep, uint64_t budget, uint64_t evict_to, bool known, uint64_t used, uint64_t allowed,
|
||||
bool dead_only) {
|
||||
static const bool no_evict = getenv("RX_VK_NO_EVICT") != nullptr; // (an experiment: nothing evicted)
|
||||
// (hookshot) under the budget, every 600 presents: the render targets drawn over since (below), unused for 120
|
||||
const bool sweep = total_bytes <= budget;
|
||||
// (hookshot-amd) and every 60 presents for the render targets drawn over by one at another start (below): OoT makes a frame's
|
||||
// transient targets (768 x 432 RGBA16F, 1536 x 864) in a scratch heap whose offsets shift with what came before them in the
|
||||
// frame, so turning the camera made new ones each frame ~1.2 MB from the last, each a new image. Left for the 600-present
|
||||
@@ -4225,7 +4414,14 @@ void texture_frame(uint64_t completed) {
|
||||
// ordered in 0.3 ms, 251 chosen in 0.2 (RTX 5070, Ryzen 7 9800X3D)
|
||||
static const uint64_t dead_every =
|
||||
getenv("RX_VK_DEAD_EVERY") ? (uint64_t)std::max(1, atoi(getenv("RX_VK_DEAD_EVERY"))) : 10;
|
||||
if (no_evict || (sweep && frame_no % (dead_targets ? dead_every : 600))) return;
|
||||
if (no_evict || (sweep && !dead_only && frame_no % (dead_targets ? dead_every : 600))) return;
|
||||
static const bool dead_quick = !(getenv("RX_VK_DEAD_QUICK") && !atoi(getenv("RX_VK_DEAD_QUICK")));
|
||||
static uint64_t dead_pass_last, quick_passes, quick_ns, quick_logged_at;
|
||||
const auto pass_t0 = std::chrono::steady_clock::now();
|
||||
const uint64_t pass_presents = std::max<uint64_t>(1, frame_no - dead_pass_last);
|
||||
dead_pass_last = frame_no;
|
||||
dead_sweep_next = 0; // (this pass is the one asked for; the next: below, or a new image's, texture_frame)
|
||||
size_t shifted_now = 0;
|
||||
std::vector<std::pair<uint64_t, Texture*>> order;
|
||||
order.reserve(textures.size());
|
||||
for (auto& kv : textures) order.push_back({kv.second->last_frame, kv.second.get()});
|
||||
@@ -4418,7 +4614,7 @@ void texture_frame(uint64_t completed) {
|
||||
}
|
||||
for (const Evict& e : evict) { // (and each one's bookkeeping and its end, as before)
|
||||
Texture* tex = e.tex;
|
||||
if (e.shifted) n_dead++, b_dead += tex->bytes;
|
||||
if (e.shifted) n_dead++, b_dead += tex->bytes, shifted_now++;
|
||||
watch_tex(tex, "evicted");
|
||||
if (!tex->gpu_written && tex->uploaded && !remade_skip_old()) { // (above: made again, it is uploaded from memory)
|
||||
std::lock_guard lk(from_memory_mu);
|
||||
@@ -4450,7 +4646,21 @@ void texture_frame(uint64_t completed) {
|
||||
(unsigned long long)(b_dead >> 20));
|
||||
}
|
||||
const size_t n = order.size() - textures.size();
|
||||
if (n) logf("vk: %zu textures retired (%zu left, %llu MB)", n, textures.size(), (unsigned long long)(total_bytes >> 20));
|
||||
if (n && !dead_only) logf("vk: %zu textures retired (%zu left, %llu MB)", n, textures.size(), (unsigned long long)(total_bytes >> 20));
|
||||
// (the next pass for the dead targets: by how fast this one found them coming)
|
||||
if (dead_quick && dead_targets) {
|
||||
if (shifted_now >= pass_presents) dead_sweep_next = frame_no + 1;
|
||||
else if (shifted_now * 3 >= pass_presents) dead_sweep_next = frame_no + 3;
|
||||
}
|
||||
if (dead_only) {
|
||||
quick_passes++;
|
||||
quick_ns += (uint64_t)std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now() - pass_t0).count();
|
||||
}
|
||||
if (quick_passes && frame_no >= quick_logged_at + 600) {
|
||||
quick_logged_at = frame_no;
|
||||
logf("vk: passes for the dead render targets between the looks: %llu so far, %.2f ms each", (unsigned long long)quick_passes,
|
||||
quick_ns / 1e6 / (double)quick_passes);
|
||||
}
|
||||
}
|
||||
|
||||
void texture_log_guest(const Texture* t) {
|
||||
|
||||
@@ -90,6 +90,13 @@ struct Texture {
|
||||
uint64_t wb_value = 0; // (the submission whose own commands write its texels into its memory; 0: none)
|
||||
uint64_t pool_key = 0; // (a render target's image: its create info's key in the pool it goes to when the texture does)
|
||||
std::unordered_map<uint64_t, VkImageView> views;
|
||||
// (Triforce) a flat texture a program samples as a 3D one: its one-slice 3D copy (vk_texture.cpp twin3d_view), the layer it
|
||||
// was copied from and the texture's contents it was copied at
|
||||
VkImage twin3d = VK_NULL_HANDLE;
|
||||
VmaAllocation twin3d_alloc = nullptr;
|
||||
uint32_t twin3d_layer = 0;
|
||||
uint64_t twin3d_value = ~0ull, twin3d_written = ~0ull;
|
||||
std::unordered_map<uint64_t, VkImageView> twin3d_views;
|
||||
};
|
||||
|
||||
/// Makes texels [x, x + w) x [y, y + h) of level `level`, layers [first, first + count), of a sparse texture resident
|
||||
|
||||
@@ -28,7 +28,29 @@ bool vma_init(PFN_vkGetInstanceProcAddr gipa) {
|
||||
ci.vulkanApiVersion = VK_API_VERSION_1_3;
|
||||
ci.pVulkanFunctions = &f;
|
||||
ci.preferredLargeHeapBlockSize = 64ull << 20; // (64 MB blocks: Eglantine found 256 MB ones wasted VRAM)
|
||||
return vmaCreateAllocator(&ci, &vma) == VK_SUCCESS;
|
||||
if (vmaCreateAllocator(&ci, &vma) != VK_SUCCESS) return false;
|
||||
// (Triforce, a lab tool) RX_VK_TEST_BALLAST_MB=<n>: that much VRAM taken and held for the session, in 256 MB device buffers
|
||||
// of their own. RX_VK_TEST_ALLOWED_MB only changes the number the budget is told; this fills the card for real, as a card
|
||||
// with less VRAM or another program's use does: the driver's own behaviour at its limit (allocations of 110-170 ms in a
|
||||
// player's log) only shows then
|
||||
if (const char* e = getenv("RX_VK_TEST_BALLAST_MB")) {
|
||||
const uint64_t want = (uint64_t)std::max(0, atoi(e)) << 20, chunk = 256ull << 20;
|
||||
uint64_t got = 0;
|
||||
for (; got < want; got += chunk) {
|
||||
VkBufferCreateInfo bi{VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO};
|
||||
bi.size = std::min(chunk, want - got);
|
||||
bi.usage = VK_BUFFER_USAGE_TRANSFER_DST_BIT;
|
||||
VmaAllocationCreateInfo ai{};
|
||||
ai.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE;
|
||||
ai.requiredFlags = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
|
||||
ai.flags = VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
|
||||
VkBuffer b = VK_NULL_HANDLE;
|
||||
VmaAllocation a = VK_NULL_HANDLE;
|
||||
if (vmaCreateBuffer(vma, &bi, &ai, &b, &a, nullptr) != VK_SUCCESS) break; // (kept: never freed)
|
||||
}
|
||||
logf("vk: lab: %llu MB of VRAM taken as ballast (RX_VK_TEST_BALLAST_MB)", (unsigned long long)(got >> 20));
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// Images come from a pool of 128 MB blocks: on their own, VMA gave most of them dedicated memory (the driver prefers it
|
||||
@@ -47,6 +69,8 @@ const uint64_t kImageBlock = (getenv("RX_VK_IMAGE_BLOCK_MB") ? (uint64_t)std::ma
|
||||
// image made there then took 30-70 ms (RTX 5070). RX_VK_TARGET_BLOCKS=0: in the textures' blocks, as before
|
||||
VmaPool target_image_pool = VK_NULL_HANDLE;
|
||||
bool target_pool_tried = false;
|
||||
size_t target_floor_blocks = 0;
|
||||
bool target_floor_kept() { return target_floor_blocks != 0; }
|
||||
VmaPool images_pool();
|
||||
VmaPool target_images_pool() {
|
||||
if (target_pool_tried) return target_image_pool;
|
||||
@@ -68,9 +92,22 @@ VmaPool target_images_pool() {
|
||||
VmaPoolCreateInfo pi{};
|
||||
pi.memoryTypeIndex = type;
|
||||
pi.blockSize = kImageBlock;
|
||||
if (vmaCreatePool(vma, &pi, &target_image_pool) != VK_SUCCESS) target_image_pool = VK_NULL_HANDLE;
|
||||
logf("vk: render targets' images from blocks of their own (%llu MB, memory type %u)%s", (unsigned long long)(kImageBlock >> 20),
|
||||
type, target_image_pool ? "" : ": not made");
|
||||
// (Triforce, 11-11) A floor of blocks the pool keeps for the session (RX_VK_TARGET_FLOOR_MB; 0: none, as before). While the
|
||||
// camera moves the game makes its transient targets at new sizes all the time (a player's session: up to 6733 new images in
|
||||
// 600 presents, 11 a frame), their blocks emptied and filled again; VMA keeps one empty block and frees the rest, and at the
|
||||
// driver's limit (its budget reached) not even that one: every block the pool needed again was a new 128 MB from the
|
||||
// driver, 110-170 ms each there. That session had ten frames of 115-180 ms in a row from it, VRAM climbing 9.5 -> 10.8 GB
|
||||
// of 10.5 allowed. Below the floor VMA never gives a block back, so nothing is asked of the driver in play
|
||||
static const uint64_t floor_mb = getenv("RX_VK_TARGET_FLOOR_MB") ? (uint64_t)std::max(0, atoi(getenv("RX_VK_TARGET_FLOOR_MB"))) : 0;
|
||||
pi.minBlockCount = (size_t)((floor_mb << 20) / kImageBlock);
|
||||
target_floor_blocks = pi.minBlockCount;
|
||||
if (vmaCreatePool(vma, &pi, &target_image_pool) != VK_SUCCESS) {
|
||||
pi.minBlockCount = 0; // (no room for the floor now: the pool as before)
|
||||
target_floor_blocks = 0;
|
||||
if (vmaCreatePool(vma, &pi, &target_image_pool) != VK_SUCCESS) target_image_pool = VK_NULL_HANDLE;
|
||||
}
|
||||
logf("vk: render targets' images from blocks of their own (%llu MB, memory type %u; %zu blocks kept for the session)%s",
|
||||
(unsigned long long)(kImageBlock >> 20), type, pi.minBlockCount, target_image_pool ? "" : ": not made");
|
||||
return target_image_pool;
|
||||
}
|
||||
|
||||
@@ -182,7 +219,58 @@ bool image_compact_pass_end(uint64_t* freed_bytes, uint32_t* freed_blocks) {
|
||||
return compact_over(freed_bytes, freed_blocks);
|
||||
}
|
||||
|
||||
bool create_image_inner(const VkImageCreateInfo& ci, VkImage& image, VmaAllocation& allocation, bool dedicated, bool target, bool textures);
|
||||
thread_local int create_image_path = 0; // (the last create_image_inner: 0 its pool or a dedicated one, 1 the pool refused, 2 system memory)
|
||||
/// (Triforce) An image that takes over 3 ms to make: what it was, how long, and whether its pool took a new block for it (60
|
||||
/// lines). A player's session had frames of 50 to 150 ms in play that were all "texture lookups", 24 render targets made in one
|
||||
bool create_image(const VkImageCreateInfo& ci, VkImage& image, VmaAllocation& allocation, bool dedicated, bool target, bool textures) {
|
||||
const auto blocks = [&]() -> uint32_t {
|
||||
VmaPool pool = textures ? texture_pool : target ? target_image_pool : image_pool;
|
||||
if (!pool) return 0;
|
||||
VmaStatistics st{};
|
||||
vmaGetPoolStatistics(vma, pool, &st);
|
||||
return st.blockCount;
|
||||
};
|
||||
const uint32_t before = blocks();
|
||||
const auto t0 = std::chrono::steady_clock::now();
|
||||
create_image_path = 0;
|
||||
const bool ok = create_image_inner(ci, image, allocation, dedicated, target, textures);
|
||||
const double ms = std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - t0).count();
|
||||
{ // (each 600 target images: how many the pool refused, and what it holds)
|
||||
static std::atomic<int64_t> targets{0}, refused{0}, new_blocks{0};
|
||||
if (target) {
|
||||
refused += create_image_path != 0;
|
||||
new_blocks += blocks() > before;
|
||||
if (++targets % 600 == 0 && target_image_pool) {
|
||||
VmaStatistics st{};
|
||||
vmaGetPoolStatistics(vma, target_image_pool, &st);
|
||||
logf("vk: render targets' blocks: %u blocks %llu MB hold %u images %llu MB; of %lld target images so far %lld took a "
|
||||
"new block and %lld were refused by the pool", st.blockCount, (unsigned long long)(st.blockBytes >> 20),
|
||||
st.allocationCount, (unsigned long long)(st.allocationBytes >> 20), (long long)targets.load(),
|
||||
(long long)new_blocks.load(), (long long)refused.load());
|
||||
}
|
||||
}
|
||||
}
|
||||
static std::atomic<int> told{0};
|
||||
static std::atomic<int64_t> n{0}, slow{0};
|
||||
static std::atomic<int64_t> slow_us{0};
|
||||
n++;
|
||||
if (ms > 3.0) {
|
||||
slow++, slow_us += (int64_t)(ms * 1000);
|
||||
if (told++ < 60) {
|
||||
uint64_t used = 0, allowed = 0, block_free = 0;
|
||||
vram_usage(&used, &allowed, &block_free);
|
||||
logf("vk: an image took %.1f ms to make: %ux%u x%u levels %u format %d (%s)%s; its pool's blocks %u -> %u; VRAM %llu of %llu "
|
||||
"MB, %llu MB free in the blocks; %lld of %lld images so far took over 3 ms (%.0f ms in all)", ms, ci.extent.width,
|
||||
ci.extent.height, ci.arrayLayers, ci.mipLevels, (int)ci.format, textures ? "a texture" : target ? "a render target" : "other",
|
||||
!ok ? ", NOT made" : create_image_path ? ", outside its pool" : "", before, blocks(), (unsigned long long)(used >> 20),
|
||||
(unsigned long long)(allowed >> 20),
|
||||
(unsigned long long)(block_free >> 20), (long long)slow.load(), (long long)n.load(), slow_us.load() / 1000.0);
|
||||
}
|
||||
}
|
||||
return ok;
|
||||
}
|
||||
bool create_image_inner(const VkImageCreateInfo& ci, VkImage& image, VmaAllocation& allocation, bool dedicated, bool target, bool textures) {
|
||||
VmaAllocationCreateInfo ai{};
|
||||
ai.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE;
|
||||
// (in VRAM or not at all: an image VMA puts in system memory when VRAM is full is read over PCIe every frame,
|
||||
@@ -202,8 +290,10 @@ bool create_image(const VkImageCreateInfo& ci, VkImage& image, VmaAllocation& al
|
||||
pai.pool = pool;
|
||||
pai.usage = VMA_MEMORY_USAGE_UNKNOWN;
|
||||
if (vmaCreateImage(vma, &ci, &pai, &image, &allocation, nullptr) == VK_SUCCESS) return true;
|
||||
create_image_path = 1; // (the pool refused it: the allocator's own blocks, or a memory object of its own)
|
||||
}
|
||||
if (vmaCreateImage(vma, &ci, &ai, &image, &allocation, nullptr) == VK_SUCCESS) return true;
|
||||
create_image_path = 2;
|
||||
// (hookshot) VRAM is full (a 4 GB card at a high resolution, #96): an image left unmade is a black screen, so it goes
|
||||
// to system memory instead, slow but drawn
|
||||
if (getenv("RX_VK_IMAGES_VRAM_ONLY")) return false;
|
||||
@@ -278,6 +368,37 @@ bool create_device_buffer(uint64_t size, VkBufferUsageFlags usage, VkBuffer& buf
|
||||
if (dedicated) ai.flags |= VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
|
||||
return vmaCreateBuffer(vma, &bi, &ai, &buffer, &allocation, nullptr) == VK_SUCCESS;
|
||||
}
|
||||
/// (Triforce) Device memory for a piece of a sparse buffer (vk_memory.cpp's mirrors in pieces): size bytes at that alignment,
|
||||
/// of a type in type_bits, within the driver's budget; the memory and the offset in it to bind
|
||||
bool vma_piece_alloc(uint64_t size, uint64_t alignment, uint32_t type_bits, VmaAllocation& allocation, VkDeviceMemory& memory,
|
||||
uint64_t& offset) {
|
||||
VkMemoryRequirements req{};
|
||||
req.size = size, req.alignment = alignment, req.memoryTypeBits = type_bits;
|
||||
VmaAllocationCreateInfo ai{};
|
||||
ai.requiredFlags = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
|
||||
if (dev.memory_budget) ai.flags = VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT;
|
||||
VmaAllocationInfo info{};
|
||||
const auto t0 = std::chrono::steady_clock::now();
|
||||
const bool ok = vmaAllocateMemory(vma, &req, &ai, &allocation, &info) == VK_SUCCESS;
|
||||
{ // (Triforce, 11-11: one that takes over 3 ms is named, 40 times: at the driver's limit an image's new block took
|
||||
// 110-170 ms, and submissions beside those were 130-150 ms of "memory sync" with nothing said of them)
|
||||
const double ms = std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - t0).count();
|
||||
static std::atomic<int> told{0};
|
||||
if (ms > 3.0 && told++ < 40) {
|
||||
uint64_t used = 0, allowed = 0, block_free = 0;
|
||||
vram_usage(&used, &allowed, &block_free);
|
||||
logf("vk: a mirror's piece took %.1f ms to get its memory (%llu KB%s); VRAM %llu of %llu MB", ms,
|
||||
(unsigned long long)(size >> 10), ok ? "" : ", NOT given", (unsigned long long)(used >> 20),
|
||||
(unsigned long long)(allowed >> 20));
|
||||
}
|
||||
}
|
||||
if (!ok) return false;
|
||||
memory = info.deviceMemory, offset = info.offset;
|
||||
return true;
|
||||
}
|
||||
void vma_piece_free(VmaAllocation allocation) {
|
||||
vmaFreeMemory(vma, allocation);
|
||||
}
|
||||
/// (every heap: what VMA has in blocks and allocations there, what the process uses of it, its budget; one line)
|
||||
/// The image pools: their blocks, the images in them, and the free ranges between them (every 600 presents, with the heaps)
|
||||
void vma_log_image_blocks() {
|
||||
|
||||
47
host/frame_whole.h
Normal file
47
host/frame_whole.h
Normal file
@@ -0,0 +1,47 @@
|
||||
/* (hookshot) Whole frames out of a frame step that can be under one, for the game's counters that add the step, or the step
|
||||
times a whole number, cut to whole frames (nx2/oot_patches.py patch_whole_frames). At the console's 60 fps the step is 1 and
|
||||
they count frames. Under a Frame rate cap above 60 the step is the frame's own time (host/nn_vi.c, hos_frame_seconds_bits):
|
||||
0.42 at 144 fps, cut to 0 every frame, and the counter never moves, so what waits on it never comes: the pick-up indicator
|
||||
above Link's head counts the rupees shown up to the wallet's (f_80a87e50: the shown +864 moves by (int)(step * the
|
||||
distance / 50, at least 1)) and fades only when they meet; it stayed at the old count until a door or a talk replaced it.
|
||||
While the frames' own time is the step (carry), the amount is added to the counter's own remainder and the whole frames in
|
||||
it are handed out, so the counter moves at the game's rate whatever the frame rate. Otherwise it is cut as the game cuts it:
|
||||
60 fps as before. One remainder per counter address (or per object: each water surface keeps its own), reused oldest
|
||||
first. 64 slots: a scene's water surfaces each take one every frame, and more of them than slots would evict each other in
|
||||
turn, every one starting again from 0 and standing still. Not thread-safe by itself: the caller holds a lock. */
|
||||
#pragma once
|
||||
#include <stdint.h>
|
||||
#include <string.h>
|
||||
|
||||
#define FRAME_WHOLE_SLOTS 64
|
||||
typedef struct {
|
||||
uint64_t at[FRAME_WHOLE_SLOTS];
|
||||
float rem[FRAME_WHOLE_SLOTS];
|
||||
uint32_t used[FRAME_WHOLE_SLOTS]; /* (the call count at the slot's last use: the oldest is reused) */
|
||||
uint32_t calls;
|
||||
} FrameWhole;
|
||||
|
||||
/* (as the lifted code cuts it: toward zero, NaN and below zero to 0, at most 0x7fffffff: the counts are small and positive) */
|
||||
static inline uint32_t frame_whole_cut(float f) {
|
||||
if (!(f > 0.0f)) return 0;
|
||||
if (f >= 2147483648.0f) return 0x7fffffffu;
|
||||
return (uint32_t)f;
|
||||
}
|
||||
|
||||
static inline uint32_t frame_whole(FrameWhole *w, uint64_t at, uint32_t amount_bits, int carry) {
|
||||
float amount;
|
||||
memcpy(&amount, &amount_bits, 4);
|
||||
if (!carry || !(amount > 0.0f) || amount >= 2147483648.0f) return frame_whole_cut(amount);
|
||||
w->calls++;
|
||||
int slot = -1, oldest = 0;
|
||||
for (int i = 0; i < FRAME_WHOLE_SLOTS; i++) {
|
||||
if (w->at[i] == at) { slot = i; break; }
|
||||
if (w->used[i] < w->used[oldest]) oldest = i;
|
||||
}
|
||||
if (slot < 0) slot = oldest, w->at[slot] = at, w->rem[slot] = 0.0f;
|
||||
w->used[slot] = w->calls;
|
||||
const float sum = w->rem[slot] + amount;
|
||||
const uint32_t whole = frame_whole_cut(sum);
|
||||
w->rem[slot] = sum - (float)whole;
|
||||
return whole;
|
||||
}
|
||||
57
host/nn_vi.c
57
host/nn_vi.c
@@ -224,6 +224,59 @@ static float text_scale(const char *env, int menu) {
|
||||
volatile uint64_t hos_step_probe = 0; /* (f_80706be0's object, set by the lifted code when the frame-time hook is in this build; its
|
||||
+0xd58 is the game clock in 1/60 s: nx2/oot_patches.py) */
|
||||
volatile uint32_t hos_frame_seconds_bits = 0; /* (the frame's seconds as a float's bits for that hook, 0: the game's own step) */
|
||||
/* A movie's frame count while the frames' time is given (a Frame rate cap above 60): how many 60ths of a second this update of
|
||||
that movie player is worth, the rest carried to its next one, so the count the cutscene's cues are placed by keeps the clock
|
||||
as the movie's picture does (issue 1742: the narration ran ahead above 60 fps). 1, the game's own step, otherwise, and with
|
||||
HOS_FPS_MOVIE=0 (for comparisons). Called by the lifted f_83b7c710 (nx2/oot_patches.py). */
|
||||
uint32_t hos_movie_frames(uint64_t player) {
|
||||
enum { PLACES = 16 };
|
||||
const uint32_t bits = __atomic_load_n(&hos_frame_seconds_bits, __ATOMIC_RELAXED); /* (realtime_step writes it at each present) */
|
||||
if (!bits) return 1;
|
||||
static SRWLOCK lock = SRWLOCK_INIT;
|
||||
static int timed = -1; /* (these four under the lock) */
|
||||
static struct { uint64_t player, used; double part; } carry[PLACES];
|
||||
static uint64_t calls;
|
||||
static unsigned made;
|
||||
float secs;
|
||||
uint32_t n = 1;
|
||||
memcpy(&secs, &bits, 4);
|
||||
AcquireSRWLockExclusive(&lock);
|
||||
if (timed < 0) timed = !(getenv("HOS_FPS_MOVIE") && !atoi(getenv("HOS_FPS_MOVIE")));
|
||||
if (timed) {
|
||||
int i = 0;
|
||||
while (i < PLACES && carry[i].player != player) i++;
|
||||
if (i == PLACES) {
|
||||
/* (a player not seen yet takes the place used longest ago, so a movie that plays keeps its own while others come
|
||||
and go: its count is exact while no more than PLACES players, itself among them, update in turn. It starts at a part
|
||||
of a 60th drawn from a hash of how many places were made: with that many players at once every update is a first
|
||||
one, and each count then still grows by the frames' time on average, whatever the number of players, instead of
|
||||
standing still. A sequence with a fixed step, the golden ratio's multiples, stood still for one player in 233) */
|
||||
uint32_t x = made++;
|
||||
x ^= x >> 16, x *= 0x85ebca6bu, x ^= x >> 13, x *= 0xc2b2ae35u, x ^= x >> 16; /* (MurmurHash3's last step) */
|
||||
i = 0;
|
||||
for (int k = 1; k < PLACES; k++)
|
||||
if (carry[k].used < carry[i].used) i = k;
|
||||
carry[i].player = player, carry[i].part = x / 4294967296.0;
|
||||
}
|
||||
carry[i].used = ++calls;
|
||||
carry[i].part += (double)secs * 60.0;
|
||||
n = (uint32_t)carry[i].part;
|
||||
carry[i].part -= n;
|
||||
}
|
||||
ReleaseSRWLockExclusive(&lock);
|
||||
return n;
|
||||
}
|
||||
/* (nx2/oot_patches.py patch_whole_frames) the whole frames a game counter at `at` takes for an amount of the frame step (its
|
||||
float's bits): with the frames' own time as the step, their fractions carried per counter; else cut as the game cuts it */
|
||||
#include "frame_whole.h"
|
||||
static FrameWhole frame_whole_state;
|
||||
static SRWLOCK frame_whole_lock = SRWLOCK_INIT; /* (the game's update threads) */
|
||||
uint32_t hos_frame_whole(uint64_t at, uint32_t amount_bits) {
|
||||
AcquireSRWLockExclusive(&frame_whole_lock);
|
||||
const uint32_t whole = frame_whole(&frame_whole_state, at, amount_bits, __atomic_load_n(&hos_frame_seconds_bits, __ATOMIC_RELAXED) != 0);
|
||||
ReleaseSRWLockExclusive(&frame_whole_lock);
|
||||
return whole;
|
||||
}
|
||||
volatile uint64_t hos_res_module = 0; /* (f_831f3200's object, whose +0xe5c is the 3D's scale: nx2/oot_patches.py) */
|
||||
static void realtime_step(int64_t now) {
|
||||
int rx_option(int id, int otherwise);
|
||||
@@ -357,9 +410,9 @@ static void realtime_step(int64_t now) {
|
||||
const float f = (float)(secs > 4.0 / 60.0 ? 4.0 / 60.0 : secs < 1.0 / 240.0 ? 1.0 / 240.0 : secs);
|
||||
uint32_t bits;
|
||||
memcpy(&bits, &f, 4);
|
||||
hos_frame_seconds_bits = bits;
|
||||
__atomic_store_n(&hos_frame_seconds_bits, bits, __ATOMIC_RELAXED); /* (read by the game's threads) */
|
||||
} else {
|
||||
hos_frame_seconds_bits = 0;
|
||||
__atomic_store_n(&hos_frame_seconds_bits, 0, __ATOMIC_RELAXED);
|
||||
}
|
||||
const int top = timed ? 60 : fps_max();
|
||||
double ratio = sum / count * 60.0 + carry;
|
||||
|
||||
66
host/oot_high_fps.c
Normal file
66
host/oot_high_fps.c
Normal file
@@ -0,0 +1,66 @@
|
||||
/* (hookshot) The sound manager's job at 60 Hz and the physics solver's steps rounded up, under a Frame rate cap above 60: the
|
||||
lifted f_80435300, f_838ceb80 and f_838cfc10 call these (nx2/oot_patches.py); host/oot_high_fps.h has why and the logic.
|
||||
HOS_FPS_LOG (set): what they do, every 600 calls. */
|
||||
#include "hos.h"
|
||||
#include "oot_high_fps.h"
|
||||
#define WIN32_LEAN_AND_MEAN
|
||||
#include <windows.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
extern volatile uint32_t hos_frame_seconds_bits; /* (host/nn_vi.c: the frame's seconds, 0 at a cap of 60) */
|
||||
|
||||
static int timed(void) { return __atomic_load_n(&hos_frame_seconds_bits, __ATOMIC_RELAXED) != 0; }
|
||||
static int fps_log(void) {
|
||||
static int on = -1; /* (read by the game's threads) */
|
||||
int v = __atomic_load_n(&on, __ATOMIC_RELAXED);
|
||||
if (v < 0) v = getenv("HOS_FPS_LOG") != NULL, __atomic_store_n(&on, v, __ATOMIC_RELAXED);
|
||||
return v;
|
||||
}
|
||||
|
||||
static OotSoundGate sound_gate = OOT_SOUND_GATE_INIT;
|
||||
static long long sound_calls, sound_runs; /* (HOS_FPS_LOG's counts, under sound_lock) */
|
||||
static SRWLOCK sound_lock = SRWLOCK_INIT; /* (the job's phases may come from different threads) */
|
||||
|
||||
/* f_80435300's head: phase w2, slot the guest address of the pointer to the object whose +0x10 is the game step (the job's own
|
||||
load). 0: the job returns at once */
|
||||
int hos_oot_sound_gate(Ctx *c, uint32_t phase, uint64_t slot) {
|
||||
(void)c;
|
||||
float step = 1.0f;
|
||||
const int t = timed();
|
||||
if (t && phase == 0) {
|
||||
const uint64_t object = G64(slot);
|
||||
const uint64_t time = object ? G64(object) : 0;
|
||||
if (time) step = oot_bits_float(*(const uint32_t *)GP(time + 0x10));
|
||||
}
|
||||
long long ran = -1;
|
||||
AcquireSRWLockExclusive(&sound_lock);
|
||||
const int run = oot_sound_gate(&sound_gate, phase, step, t);
|
||||
if (phase == 0 && fps_log()) {
|
||||
sound_runs += run;
|
||||
if (!(++sound_calls % 600)) ran = sound_runs, sound_runs = 0;
|
||||
}
|
||||
ReleaseSRWLockExclusive(&sound_lock);
|
||||
if (ran >= 0)
|
||||
hos_log("oot: sound job ran %lld of the last 600 frames (step %.3f%s)", ran, step,
|
||||
oot_sound_partial(step, t) ? ", held to 60 Hz" : "");
|
||||
return run;
|
||||
}
|
||||
|
||||
/* The step f_80435300's phase 0 hands to the sound engine (its float's bits) */
|
||||
uint32_t hos_oot_sound_step(uint32_t step_bits) {
|
||||
return oot_sound_step(step_bits, timed());
|
||||
}
|
||||
|
||||
/* The solver steps of f_838ceb80 / f_838cfc10: game, the game's count; scaled_bits, the float it cut; per_frame_bits, the steps
|
||||
of a 60 fps frame (a float) */
|
||||
uint32_t hos_oot_solver_steps(uint32_t game, uint32_t scaled_bits, uint32_t per_frame_bits) {
|
||||
const int32_t steps = oot_solver_steps((int32_t)game, scaled_bits, per_frame_bits, timed());
|
||||
if (fps_log()) {
|
||||
static volatile LONG64 calls;
|
||||
const LONG64 n = InterlockedIncrement64(&calls);
|
||||
if (n % 600 == 1)
|
||||
hos_log("oot: physics solver steps %d (the game's %d: frame step %.3f x %.0f steps a 60 fps frame)", steps, (int32_t)game,
|
||||
(oot_bits_float(scaled_bits) - 0.001f) / oot_bits_float(per_frame_bits), oot_bits_float(per_frame_bits));
|
||||
}
|
||||
return (uint32_t)steps;
|
||||
}
|
||||
86
host/oot_high_fps.h
Normal file
86
host/oot_high_fps.h
Normal file
@@ -0,0 +1,86 @@
|
||||
/* (hookshot) Two places in the game that need whole 60 fps frames, under a Frame rate cap above 60, where the frame step is the
|
||||
frame's own time (host/nn_vi.c, hos_frame_seconds_bits: 0.5 at 120 fps). nx2/oot_patches.py hooks them; host/oot_high_fps.c
|
||||
holds the state. At a cap of 60 (no frame time given: timed 0) both return what the game computed, so nothing changes.
|
||||
|
||||
The sound manager's job (f_80435300, its phase in w2): phase 0 hands the game step to the sound engine, whose fade and delay
|
||||
timers and parameter ramps count updates of at least one whole step. Fed less, some sounds are not released when an area
|
||||
unloads, the voices in use climb until the game's voice budget is full, and new sounds (sword, voices), then the area music,
|
||||
stop until a restart. Above 60 fps the job runs at 60 Hz: phase 0 adds the step to a remainder and runs once a whole step has
|
||||
built up, with a step of 1 (sound_step), keeping the rest; the job's other phases follow that frame's choice. Sound keeps the
|
||||
clock on average. A step of 1 or more runs as the game has it and keeps the remainder: around 60 fps (a cap above 60 with
|
||||
the game near 60, or VRR) steps of 0.98 and 1.02 alternate, and the 0.98 that did not run comes back as a whole step later
|
||||
(dropped, the sound clock ran at half the real one). A step of 0 or less (or NaN), and every phase at a cap of 60, run as the
|
||||
game has them, and clear the remainder.
|
||||
(After zortos293/Lichii PR #1289 by auz466, GPL-3.0: the same gate as a code patch for the emulator.)
|
||||
|
||||
The physics world's solver steps: the game sets them each update to (int)(frame step * the steps of a 60 fps frame + 0.001)
|
||||
(f_838ceb80 and f_838cfc10, into the solver's +712; 8 a 60 fps frame in the places seen). A step under 1 cuts that short,
|
||||
and as the measured step wanders around its mean the count changes from frame to frame (8 x 0.49 is 3, 8 x 0.5 is 4 at 120
|
||||
fps); 4 steps a 60 fps frame would be 0 above 240. A moving platform (the stone one under the spikes in the Deku Tree)
|
||||
then slides out from under Link (issue 1798). While the step is under 1 the count is rounded up instead, so a solver step
|
||||
never covers more time than at 60 fps, give or take 10% (the frame time's jitter, so a steady rate keeps a steady count),
|
||||
and it is at least 1 and never under the game's. A step of 1 or more (a frame slower than 60 fps) keeps the game's count. */
|
||||
#pragma once
|
||||
#include <math.h>
|
||||
#include <stdint.h>
|
||||
#include <string.h>
|
||||
|
||||
typedef struct {
|
||||
float part; /* (the steps added up since the job last ran, under 1) */
|
||||
int run; /* (phase 0's choice, for the other phases of the same frame) */
|
||||
} OotSoundGate;
|
||||
#define OOT_SOUND_GATE_INIT {0.0f, 1} /* (every phase runs until a phase 0 has chosen) */
|
||||
|
||||
static inline float oot_bits_float(uint32_t bits) {
|
||||
float f;
|
||||
memcpy(&f, &bits, 4);
|
||||
return f;
|
||||
}
|
||||
static inline uint32_t oot_float_bits(float f) {
|
||||
uint32_t bits;
|
||||
memcpy(&bits, &f, 4);
|
||||
return bits;
|
||||
}
|
||||
|
||||
/* (whether the step is a part of a frame: the only case the gate and the feed change) */
|
||||
static inline int oot_sound_partial(float step, int timed) {
|
||||
return timed && step > 0.0f && step < 1.0f;
|
||||
}
|
||||
|
||||
/* 1: run the job now, 0: return from it at once */
|
||||
static inline int oot_sound_gate(OotSoundGate *g, uint32_t phase, float step, int timed) {
|
||||
if (!timed) { /* (a cap of 60: every phase of every frame, whatever came before) */
|
||||
g->part = 0.0f;
|
||||
g->run = 1;
|
||||
return 1;
|
||||
}
|
||||
if (phase != 0) return g->run;
|
||||
if (!(step > 0.0f)) { /* (0, below 0, NaN: as the game has it) */
|
||||
g->part = 0.0f;
|
||||
g->run = 1;
|
||||
return 1;
|
||||
}
|
||||
if (step >= 1.0f) { /* (a whole step or more: runs with it, the remainder kept for the next partial ones) */
|
||||
g->run = 1;
|
||||
return 1;
|
||||
}
|
||||
g->part += step;
|
||||
g->run = g->part >= 1.0f;
|
||||
if (g->run) g->part -= 1.0f;
|
||||
return g->run;
|
||||
}
|
||||
|
||||
/* The step phase 0 hands to the sound engine: a whole one while the gate holds the job to whole steps */
|
||||
static inline uint32_t oot_sound_step(uint32_t step_bits, int timed) {
|
||||
return oot_sound_partial(oot_bits_float(step_bits), timed) ? 0x3f800000u : step_bits;
|
||||
}
|
||||
|
||||
/* The solver's steps: game (the game's own count), scaled_bits (frame step * steps a 60 fps frame + 0.001, the float it cut),
|
||||
per_frame_bits (the steps of a 60 fps frame, a float) */
|
||||
static inline int32_t oot_solver_steps(int32_t game, uint32_t scaled_bits, uint32_t per_frame_bits, int timed) {
|
||||
const float scaled = oot_bits_float(scaled_bits) - 0.001f, per_frame = oot_bits_float(per_frame_bits);
|
||||
if (!timed || !(scaled > 0.0f) || !(per_frame > 0.0f) || !(scaled < per_frame) || per_frame > 1000.0f) return game;
|
||||
int32_t steps = (int32_t)ceilf(scaled - 0.1f);
|
||||
if (steps < 1) steps = 1;
|
||||
return steps > game ? steps : game;
|
||||
}
|
||||
@@ -247,6 +247,179 @@ def unpatch_frame_seconds(s):
|
||||
return s if STEP_MARK in s else s.replace(STEP_DECL, "", 1) # (a hook of another shape stays, with its declarations)
|
||||
|
||||
|
||||
# A movie's frame count above a 60 cap. The game's movie player counts its updates while a movie plays (f_83b7c710: one more
|
||||
# at +760 of the player each frame), and a cutscene that shows a movie takes that count as its timeline's frame (f_83b5ec10,
|
||||
# f_83b5e9a0): the narration, the rumble and the cutscene's end are placed by it. The movie itself follows the clock, so above
|
||||
# 60 fps those cues ran ahead of the picture, fps / 60 times too fast (issue 1742). The count now grows by the frames' time, as
|
||||
# the game's clock does (hos_movie_frames, host/nn_vi.c: 1 as before while the host gives no frame time).
|
||||
MOVIE_HEAD = "void f_83b7c710(Ctx *__restrict A64_NOESC c) {\n"
|
||||
# (the load and the add alone: the store after them is rewritten by the page marks' pass, which runs after these edits)
|
||||
MOVIE_OLD = (" c->x[8]=recomp_load32(c,c->x[20] + 760);\n"
|
||||
" { uint64_t _a=c->x[8], _b=1ULL; uint64_t _r=_a+_b; _r&=0xFFFFFFFFULL; c->x[8]=_r; }\n")
|
||||
MOVIE_NEW = MOVIE_OLD.replace("_b=1ULL", "_b=hos_movie_frames(c->x[20])")
|
||||
MOVIE_DECL = "uint32_t hos_movie_frames(uint64_t player);\n"
|
||||
|
||||
|
||||
def patch_movie_frames(s):
|
||||
"""The movie player's count by the frames' time; None when the file has no such function, s unchanged when the function is
|
||||
there but its code differs (another game version) or has the patch already."""
|
||||
head = s.find(MOVIE_HEAD)
|
||||
if head < 0:
|
||||
return None
|
||||
end = s.find("\nvoid f_", head + len(MOVIE_HEAD))
|
||||
body = s[head:end if end >= 0 else len(s)]
|
||||
if body.count(MOVIE_OLD) != 1:
|
||||
return s
|
||||
at = head + body.index(MOVIE_OLD)
|
||||
s = s[:at] + MOVIE_NEW + s[at + len(MOVIE_OLD):]
|
||||
return s if MOVIE_DECL in s else s.replace('#include "a64.h"\n', '#include "a64.h"\n' + MOVIE_DECL, 1)
|
||||
|
||||
|
||||
# Counters that add the frame step, or the step times a whole number, cut to whole frames (host/frame_whole.h). At 60 fps the step
|
||||
# is 1; under a Frame rate cap above 60 it is the frame's own time, under 1 (0.42 at 144 fps), and the cut is 0 every frame:
|
||||
# - f_80a87e50, the pick-up indicator above Link's head (a rupee, a Deku Stick): the count it shows (+864) moves toward the
|
||||
# wallet's (+860) by (int)(step * max(distance / 50, 1)) and the indicator fades when they meet. Cut to 0, it stayed at the old
|
||||
# count until a door or a talk replaced it (issue 1832).
|
||||
# - f_800672e0, the water renderer: its ripple simulation runs (int)step times the water's substeps (+76 of the pass) each frame.
|
||||
# Cut to 0, the rings Link and the actors make in water stood still above 60 fps. Found by atk3001 for Lichii (zortos293/
|
||||
# Lichii#1152), as ported to Fritchii (Frizti/Fritchii d389685a4, 063dbb741): its remainder is kept per water object (x21).
|
||||
# - f_803edf70, the time of day: each frame it adds (int)(2 * step) game seconds (+60) to the seconds of the day (+40, a day of
|
||||
# 86400, then the day count +44). Cut, it added 1 a frame below 120 fps (the day slower: 83% at 100) and 0 above 120, where
|
||||
# the clock stood still and the sun swung back and forth over the frozen hour.
|
||||
# hos_frame_whole carries the fractions per counter while the frames' own time is the step, so each moves at the game's rate;
|
||||
# otherwise it cuts as the game does. A site is added here once it is seen to misbehave in game. (A site's function, its code to
|
||||
# replace, and the counter's address for its remainder.)
|
||||
WHOLE_SITES = (
|
||||
# (the cut alone, the function's one: the multiply before it is a64_fpx_hot_mul32_fma32 in a lift where the function is
|
||||
# among A64_HOT_FUNCS)
|
||||
("void f_80a87e50(Ctx *__restrict A64_NOESC c) {@",
|
||||
" c->x[12]=recomp_fp_to_int(c->vreg[0][0]&UINT64_C(0xffffffff),32,32,1,3,c->fpcr,&c->fpsr);@",
|
||||
"c->x[19] + 864"),
|
||||
# (the cut alone, the function's one)
|
||||
("void f_800672e0(Ctx *__restrict A64_NOESC c) {@",
|
||||
" c->x[9]=recomp_fp_to_int(c->vreg[0][0]&UINT64_C(0xffffffff),32,32,1,3,c->fpcr,&c->fpsr);@",
|
||||
"c->x[21]"),
|
||||
# (the cut with the store after it: the function cuts into x8 the same way once more, in the branch that sets
|
||||
# the time from another source)
|
||||
("void f_803edf70(Ctx *__restrict A64_NOESC c) {@",
|
||||
" c->x[8]=recomp_fp_to_int(c->vreg[0][0]&UINT64_C(0xffffffff),32,32,1,3,c->fpcr,&c->fpsr);@"
|
||||
" recomp_store32(c,c->x[19] + 60,c->x[8]);@",
|
||||
"c->x[19] + 60"),
|
||||
)
|
||||
WHOLE_SITES = tuple((h.replace("@", "\n"), o.replace("@", "\n"), k) for h, o, k in WHOLE_SITES)
|
||||
WHOLE_MARK = "/* (the step's whole frames: nx2/oot_patches.py) */"
|
||||
WHOLE_DECL = "extern uint32_t hos_frame_whole(uint64_t at, uint32_t amount_bits);\n"
|
||||
|
||||
|
||||
def whole_new(old, key):
|
||||
"""old with its cut to whole frames (the one recomp_fp_to_int) made hos_frame_whole's for the counter at key"""
|
||||
lines = old.split("\n")
|
||||
for i, line in enumerate(lines):
|
||||
if "recomp_fp_to_int(c->vreg[0][0]" in line:
|
||||
reg = line.split("=", 1)[0]
|
||||
lines[i] = "%s=hos_frame_whole(%s, (uint32_t)c->vreg[0][0]); %s" % (reg, key, WHOLE_MARK)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def patch_whole_frames(s):
|
||||
"""The counters of WHOLE_SITES take hos_frame_whole's whole frames: (s, the sites patched here). A file without the function,
|
||||
one patched before, or one whose code differs (another game version) is left as it is."""
|
||||
done = 0
|
||||
for head_text, old, key in WHOLE_SITES:
|
||||
head = s.find(head_text)
|
||||
if head < 0:
|
||||
continue
|
||||
end = s.find("\n}\n", head)
|
||||
body = s[head:end if end >= 0 else len(s)]
|
||||
new = whole_new(old, key)
|
||||
if new in body or body.count(old) != 1:
|
||||
continue
|
||||
at = head + body.index(old)
|
||||
s = s[:at] + new + s[at + len(old):]
|
||||
done += 1
|
||||
if done and WHOLE_DECL not in s:
|
||||
s = s.replace('#include "a64.h"\n', '#include "a64.h"\n' + WHOLE_DECL, 1)
|
||||
return s, done
|
||||
|
||||
|
||||
# Above a Frame rate cap of 60, two places that need whole 60 fps frames (host/oot_high_fps.h has why and the logic; at a cap of
|
||||
# 60 the host hands back what the game computed):
|
||||
# - the sound manager's job, f_80435300 (phase in w2; phase 0 loads the game step, [[slot]]+0x10, and hands it to the sound
|
||||
# engine): its head asks hos_oot_sound_gate whether to run now (60 Hz above 60 fps, the other phases following phase 0), and
|
||||
# the step it loads goes through hos_oot_sound_step (a whole one while the gate holds the job). After zortos293/Lichii PR #1289
|
||||
# (auz466), which does the same as a code patch for the emulator.
|
||||
# - the physics solver's step count, set each update from the frame step in f_838ceb80 and f_838cfc10 (the same code in both):
|
||||
# hos_oot_solver_steps rounds it up instead of down, at least 1, so Link stays on moving platforms (issue 1798).
|
||||
SOUND_HEAD = "void f_80435300(Ctx *__restrict A64_NOESC c) {\n"
|
||||
SOUND_SLOT = "g_module_base + 0x80435000ULL + (int64_t)75927552" # (the page of the step's pointer, whose +2192 holds it)
|
||||
SOUND_OLD = (" c->x[8]=" + SOUND_SLOT + ";\n"
|
||||
" { uint64_t _r = ((uint64_t)0 | ((c->x[0]) << 0)); c->x[19] = _r; }\n"
|
||||
" c->x[8]=recomp_load64(c,c->x[8] + 2192);\n"
|
||||
" c->x[8]=recomp_load64(c,c->x[8] + 0);\n"
|
||||
" { { uint64_t _v=recomp_load32(c,(c->x[8]+16ULL)); a64_vset64(c->vreg[0],_v); } }\n")
|
||||
SOUND_NEW = SOUND_OLD.replace("uint64_t _v=recomp_load32(c,(c->x[8]+16ULL));",
|
||||
"uint64_t _v=hos_oot_sound_step((uint32_t)recomp_load32(c,(c->x[8]+16ULL)));")
|
||||
SOUND_GATE = (" if (!hos_oot_sound_gate(c, (uint32_t)c->x[2], " + SOUND_SLOT + " + 2192)) return;"
|
||||
" /* (nx2/oot_patches.py: the sound job at 60 Hz above 60 fps) */\n")
|
||||
SOUND_DECL = "extern int hos_oot_sound_gate(Ctx *c, uint32_t phase, uint64_t slot);\nextern uint32_t hos_oot_sound_step(uint32_t step_bits);\n"
|
||||
SOLVER_HEADS = ("void f_838ceb80(Ctx *__restrict A64_NOESC c) {\n", "void f_838cfc10(Ctx *__restrict A64_NOESC c) {\n")
|
||||
# (the cut of frame step * steps a frame + 0.001, then its store into the solver: vreg 0 the float cut, vreg 1 the steps a frame)
|
||||
SOLVER_OLD = (" c->x[1]=recomp_fp_to_int(c->vreg[0][0]&UINT64_C(0xffffffff),32,32,1,3,c->fpcr,&c->fpsr);\n"
|
||||
" recomp_store32(c,c->x[0] + 712,c->x[1]);\n")
|
||||
SOLVER_NEW = SOLVER_OLD.replace(
|
||||
" recomp_store32(", " c->x[1]=hos_oot_solver_steps((uint32_t)c->x[1], (uint32_t)c->vreg[0][0], (uint32_t)c->vreg[1][0]);"
|
||||
" /* (nx2/oot_patches.py: solver steps rounded up above 60 fps) */\n recomp_store32(", 1)
|
||||
SOLVER_DECL = "extern uint32_t hos_oot_solver_steps(uint32_t game, uint32_t scaled_bits, uint32_t per_frame_bits);\n"
|
||||
|
||||
|
||||
def _function(s, head_text):
|
||||
"""(start, end) of the function whose head is head_text, or None"""
|
||||
head = s.find(head_text)
|
||||
if head < 0:
|
||||
return None
|
||||
end = s.find("\nvoid f_", head + len(head_text))
|
||||
return head, end if end >= 0 else len(s)
|
||||
|
||||
|
||||
def patch_sound_job(s):
|
||||
"""The sound job's gate and step: (s, 1) when the job is in s with them (made now or before), (s, 0) when it is not here or
|
||||
its code differs (another game version: left as it is)"""
|
||||
at = _function(s, SOUND_HEAD)
|
||||
if at is None:
|
||||
return s, 0
|
||||
head, end = at
|
||||
body = s[head:end]
|
||||
if body.count(SOUND_NEW) == 1 and body.count(SOUND_GATE) == 1:
|
||||
return s, 1
|
||||
if body.count(SOUND_OLD) != 1 or SOUND_GATE in body:
|
||||
return s, 0
|
||||
body = body.replace(SOUND_OLD, SOUND_NEW, 1)
|
||||
body = SOUND_HEAD + SOUND_GATE + body[len(SOUND_HEAD):]
|
||||
s = s[:head] + body + s[end:]
|
||||
if SOUND_DECL not in s:
|
||||
s = s.replace('#include "a64.h"\n', '#include "a64.h"\n' + SOUND_DECL, 1)
|
||||
return s, 1
|
||||
|
||||
|
||||
def patch_solver_steps(s):
|
||||
"""The solver step count's two places: (s, how many of them are in s with the hook, made now or before)"""
|
||||
done = 0
|
||||
for head_text in SOLVER_HEADS:
|
||||
at = _function(s, head_text)
|
||||
if at is None:
|
||||
continue
|
||||
head, end = at
|
||||
body = s[head:end]
|
||||
if body.count(SOLVER_NEW) == 1:
|
||||
done += 1
|
||||
elif body.count(SOLVER_OLD) == 1:
|
||||
s = s[:head] + body.replace(SOLVER_OLD, SOLVER_NEW, 1) + s[end:]
|
||||
done += 1
|
||||
if done and SOLVER_DECL not in s:
|
||||
s = s.replace('#include "a64.h"\n', '#include "a64.h"\n' + SOLVER_DECL, 1)
|
||||
return s, done
|
||||
|
||||
|
||||
def patch_text_speed(s):
|
||||
"""The scale at the head of the letters' block of f_80b1e140; None when the file has no such function, s unchanged when the
|
||||
function is there but its code differs (another game version)."""
|
||||
@@ -274,7 +447,7 @@ def apply(gen, *, protect_hearts=False, required=False):
|
||||
The other patches only warn when their place is not found once."""
|
||||
import oot_native_resolution # (HOS_SCENE_NATIVE=1: the scene's base scale 1.0; off by default since #849's output resolution)
|
||||
oot_native_resolution.apply(gen)
|
||||
done = {"dynamic resolution branch": 0, "text speed block": 0, "fixed path's scale": 0}
|
||||
done = {"dynamic resolution branch": 0, "text speed block": 0, "fixed path's scale": 0, "movie's frame count": 0}
|
||||
if protect_hearts: # (the supported build's: these places are its functions)
|
||||
done["menu focus hook"] = 0
|
||||
done["menu tick hook"] = 0
|
||||
@@ -284,6 +457,8 @@ def apply(gen, *, protect_hearts=False, required=False):
|
||||
done["free camera"] = 0
|
||||
done["mouse camera"] = 0
|
||||
step_sites, step_files = 0, [] # (the divides with the hook, and the files that have it)
|
||||
whole_sites = 0 # (the counters of WHOLE_SITES with the whole frames: patch_whole_frames)
|
||||
sound_jobs = solver_sites = 0 # (patch_sound_job, patch_solver_steps: the places found with their hooks)
|
||||
for path in sorted(glob.glob(os.path.join(glob.escape(gen), "code_*.c"))):
|
||||
with open(path, encoding="utf-8", newline="") as f:
|
||||
s = f.read()
|
||||
@@ -308,6 +483,13 @@ def apply(gen, *, protect_hearts=False, required=False):
|
||||
out = tick
|
||||
done["menu tick hook"] += 1
|
||||
print("oot_patches: the menus' update hook in", os.path.basename(path))
|
||||
movie = patch_movie_frames(out)
|
||||
if movie is not None:
|
||||
if MOVIE_NEW in movie: # (made now, or by an earlier run over this file)
|
||||
done["movie's frame count"] += 1
|
||||
if movie != out:
|
||||
print("oot_patches: the movie's frame count by the frames' time in", os.path.basename(path))
|
||||
out = movie
|
||||
out, n = fixed_scale(out)
|
||||
if n:
|
||||
done["fixed path's scale"] += 1
|
||||
@@ -343,8 +525,26 @@ def apply(gen, *, protect_hearts=False, required=False):
|
||||
step_files.append(path)
|
||||
if sites:
|
||||
print("oot_patches: the frame's time at %d step divide(s) in %s" % (sites, os.path.basename(path)))
|
||||
out, whole = patch_whole_frames(out)
|
||||
whole_sites += out.count(WHOLE_MARK) # (made now, or by an earlier run over this file)
|
||||
if whole:
|
||||
print("oot_patches: %d frame counter(s) with the step's whole frames in %s" % (whole, os.path.basename(path)))
|
||||
sound, found = patch_sound_job(out)
|
||||
sound_jobs += found
|
||||
if sound != out:
|
||||
print("oot_patches: the sound job at 60 Hz above 60 fps in", os.path.basename(path))
|
||||
out = sound
|
||||
solver, found = patch_solver_steps(out)
|
||||
solver_sites += found
|
||||
if solver != out:
|
||||
print("oot_patches: the solver steps rounded up above 60 fps in", os.path.basename(path))
|
||||
out = solver
|
||||
if out != s:
|
||||
whole_write.write_whole(path, out, newline="")
|
||||
if sound_jobs != 1: # (without it, above 60 fps sounds and then the music can stop after a few loading zones)
|
||||
print("oot_patches: WARNING: the sound job was found %d times, not once" % sound_jobs)
|
||||
if solver_sites != len(SOLVER_HEADS): # (without them, above 60 fps moving platforms can drop Link)
|
||||
print("oot_patches: WARNING: the solver step count was found at %d of %d places" % (solver_sites, len(SOLVER_HEADS)))
|
||||
for what, n in done.items():
|
||||
if n != 1:
|
||||
print("oot_patches: WARNING: the %s was found %d times" % (what, n))
|
||||
@@ -357,6 +557,9 @@ def apply(gen, *, protect_hearts=False, required=False):
|
||||
whole_write.write_whole(path, unpatch_frame_seconds(s), newline="")
|
||||
print("oot_patches: WARNING: the frame's time was found at %d of %d step divides, none patched (a Frame rate cap above 60 "
|
||||
"keeps the game's own step)" % (step_sites, len(STEP_SITES)))
|
||||
elif whole_sites != len(WHOLE_SITES): # (the frames' own time without them: the pick-up indicator stays above 60 fps)
|
||||
print("oot_patches: WARNING: %d of %d frame counters found for the step's whole frames (above 60 fps a counter can stop: "
|
||||
"the pick-up indicator, the water ripples, the time of day)" % (whole_sites, len(WHOLE_SITES)))
|
||||
apply_output_size(gen, required=required)
|
||||
|
||||
|
||||
|
||||
@@ -78,7 +78,7 @@ def main():
|
||||
waits["%s: %s" % (th, guest_frame(frames))] += 1
|
||||
if not is_game(th):
|
||||
vk["%s %s: %s" % (th, "runs" if r else "waits", frames[0] if not r else
|
||||
next((f for f in frames if f.startswith("hookshot_vk:")), frames[0]))] += 1
|
||||
next((f for f in frames if f.startswith(("triforce_vk:", "hookshot_vk:"))), frames[0]))] += 1
|
||||
for k, c in waits.most_common(top):
|
||||
print(" %5.1f%% %s" % (100.0 * c / max(1, len(idle)), k))
|
||||
print(" (renderer threads then)")
|
||||
|
||||
@@ -29,8 +29,8 @@ def load_map(path, base):
|
||||
|
||||
|
||||
# The modules whose frames are Hookshot's own code (the game and the runtime, the renderer): Windows, then Linux
|
||||
GAME_MODS = ("hookshot.exe", "hookshot")
|
||||
VK_MODS = ("hookshot_vk.dll", "hookshot_vk.so")
|
||||
GAME_MODS = ("Triforce.exe", "hookshot.exe", "hookshot") # (Triforce's names first; upstream's kept for its samples)
|
||||
VK_MODS = ("triforce_vk.dll", "hookshot_vk.dll", "hookshot_vk.so")
|
||||
RUN_MODS = GAME_MODS + VK_MODS
|
||||
|
||||
|
||||
@@ -53,7 +53,7 @@ def load_symbols(folder=None):
|
||||
Windows build, the binaries of the Linux one"""
|
||||
folder = folder or os.environ.get("RX_MAPS", BUILD)
|
||||
maps = {}
|
||||
for mod in ("hookshot_vk.dll", "hookshot.exe"):
|
||||
for mod in ("triforce_vk.dll", "Triforce.exe", "hookshot_vk.dll", "hookshot.exe"):
|
||||
mp = os.path.join(folder, mod[:-4] + ".map")
|
||||
if os.path.exists(mp):
|
||||
maps[mod] = load_map(mp, 0x180000000 if mod.endswith(".dll") else 0x140000000)
|
||||
|
||||
@@ -7,7 +7,8 @@ param(
|
||||
[Parameter(Mandatory = $true)][string]$Tag,
|
||||
[string]$Deps = 'D:\src\release\triforce-deps',
|
||||
[string]$Out = (Join-Path $PSScriptRoot 'out\release'),
|
||||
[switch]$AllowNoCompiler
|
||||
[switch]$AllowNoCompiler,
|
||||
[switch]$AllowNoStateList
|
||||
)
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$repo = Split-Path $PSScriptRoot
|
||||
@@ -45,6 +46,17 @@ New-Item -ItemType Directory -Force $work | Out-Null
|
||||
foreach ($n in 'hot_funcs.txt', 'hot_order.txt', 'hookshot_pieces.prof') {
|
||||
if (Test-Path (Join-Path $repo "work\$n")) { Copy-Item (Join-Path $repo "work\$n") $work }
|
||||
}
|
||||
# 3b. the pipeline state list (nx2/pipeline_states.py --out work\pipeline_states.bin <lists from play>): nx2/make_game.py puts it
|
||||
# beside the game as triforce_pipeline_states_shipped.bin and the first start builds its pipelines before the title. Without it
|
||||
# every player compiles each pipeline when it is first drawn: the stutter of every first visit (v0.2.0 shipped none: "0 shipped"
|
||||
# in every log). Hashes and draw state only, no game code.
|
||||
$states = Join-Path $repo 'work\pipeline_states.bin'
|
||||
if (Test-Path $states) {
|
||||
Copy-Item $states $work
|
||||
"work: pipeline state list, {0:N0} pipelines" -f (((Get-Item $states).Length - 12) / 528)
|
||||
} elseif (-not $AllowNoStateList) {
|
||||
throw "no work\pipeline_states.bin: players would compile every pipeline in play. Make it with nx2/pipeline_states.py --out (or -AllowNoStateList)"
|
||||
}
|
||||
if (Test-Path (Join-Path $repo 'work\image')) { robocopy (Join-Path $repo 'work\image') (Join-Path $work 'image') /E /NFL /NDL /NJH /NJS /NP | Out-Null }
|
||||
|
||||
Add-Type -AssemblyName System.Drawing
|
||||
|
||||
@@ -22,8 +22,8 @@ def main():
|
||||
shader = (VK / "vk_shader.cpp").read_text(encoding="utf-8")
|
||||
profile = shader[shader.index("Shader::Profile make_profile()"):shader.index("Shader::HostTranslateInfo host_info()")]
|
||||
for needle, why in (
|
||||
("dev.subgroup_size > 32 && (dev.sg32_stages & VK_SHADER_STAGE_ALL_GRAPHICS)",
|
||||
"a graphics stage running wider than 32 lanes is not told to the translator"),
|
||||
("p.warp_size_potentially_larger_than_guest = dev.subgroup_max > 32",
|
||||
"an unpinned stage's maximum subgroup width is not told to the translator"),
|
||||
("p.max_subgroup_size = p.warp_size_potentially_larger_than_guest ? dev.subgroup_max : 32",
|
||||
"the broadcast fallbacks do not walk the host's upper lanes"),
|
||||
("p.has_broken_spirv_fragment_shuffle = dev.driver_id == VK_DRIVER_ID_AMD_PROPRIETARY",
|
||||
|
||||
152
tests/test_amd_subgroup_profiles.py
Normal file
152
tests/test_amd_subgroup_profiles.py
Normal file
@@ -0,0 +1,152 @@
|
||||
"""Check 32- and 64-lane shader profiles and AMD's fragment shuffle path without a GPU."""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import tempfile
|
||||
|
||||
RUN_ALL_ARGS = ["--cxx", "{cxx}"]
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
VK = ROOT / "gpu/vk"
|
||||
|
||||
FIXTURE = r'''
|
||||
#include <algorithm>
|
||||
#include <cassert>
|
||||
#include <cstdint>
|
||||
#include <cstdlib>
|
||||
#include "shader_recompiler/profile.h"
|
||||
#include "shader_recompiler/stage.h"
|
||||
|
||||
enum VkShaderStageFlagBits : uint32_t {
|
||||
VK_SHADER_STAGE_VERTEX_BIT = 1,
|
||||
VK_SHADER_STAGE_FRAGMENT_BIT = 16,
|
||||
VK_SHADER_STAGE_COMPUTE_BIT = 32,
|
||||
VK_SHADER_STAGE_TASK_BIT_EXT = 64,
|
||||
VK_SHADER_STAGE_MESH_BIT_EXT = 128,
|
||||
VK_SHADER_STAGE_ALL_GRAPHICS = 31,
|
||||
VK_SHADER_STAGE_ALL = 0x7fffffff,
|
||||
};
|
||||
using VkShaderStageFlags = uint32_t;
|
||||
constexpr uint32_t VK_DRIVER_ID_AMD_PROPRIETARY = 1;
|
||||
struct {
|
||||
uint32_t sg32_stages = 0, subgroup_size = 32, subgroup_max = 32, driver_id = 0;
|
||||
struct { struct { uint64_t minStorageBufferOffsetAlignment = 16; } limits; } props;
|
||||
} dev;
|
||||
|
||||
using Shader::Stage;
|
||||
using Id = uint32_t;
|
||||
struct EmitContext {
|
||||
const Shader::Profile& profile;
|
||||
Stage stage;
|
||||
Id lane;
|
||||
Id U32[2]{};
|
||||
Id Const(Id value) { return value; }
|
||||
Id OpShiftRightLogical(Id, Id a, Id b) { return a >> b; }
|
||||
Id OpShiftLeftLogical(Id, Id a, Id b) { return a << b; }
|
||||
Id OpIAdd(Id, Id a, Id b) { return a + b; }
|
||||
};
|
||||
Id GetThreadId(const EmitContext& ctx) { return ctx.lane; }
|
||||
'''
|
||||
|
||||
CASES = r'''
|
||||
void check(const Shader::Profile& p, uint32_t size) {
|
||||
assert(p.max_subgroup_size == size);
|
||||
assert(p.warp_size_potentially_larger_than_guest == (size > 32));
|
||||
}
|
||||
|
||||
int main(int argc, char** argv) {
|
||||
assert(argc == 12);
|
||||
const auto number = [&](int i) { return uint32_t(std::strtoul(argv[i], nullptr, 10)); };
|
||||
dev.subgroup_size = number(1);
|
||||
dev.subgroup_max = number(2);
|
||||
dev.sg32_stages = number(3);
|
||||
dev.driver_id = number(4);
|
||||
check(profile(), number(5));
|
||||
check(profile_graphics(VK_SHADER_STAGE_VERTEX_BIT), number(6));
|
||||
check(profile_graphics(VK_SHADER_STAGE_FRAGMENT_BIT), number(7));
|
||||
check(profile_pinned(), number(8));
|
||||
check(profile_graphics(VK_SHADER_STAGE_TASK_BIT_EXT), number(9));
|
||||
check(profile_graphics(VK_SHADER_STAGE_MESH_BIT_EXT), number(10));
|
||||
const bool workaround = number(11) != 0;
|
||||
check_stamp(number(5), number(3), workaround, getenv("RX_VK_WARP64") != nullptr);
|
||||
const auto& fragment = profile_graphics(VK_SHADER_STAGE_FRAGMENT_BIT);
|
||||
const auto& compute = profile_pinned();
|
||||
EmitContext lo{fragment, Stage::Fragment, 31};
|
||||
EmitContext hi{fragment, Stage::Fragment, 32};
|
||||
EmitContext cs{compute, Stage::Compute, 31};
|
||||
assert(UseShuffleFallback(lo) == workaround && UseShuffleFallback(hi) == workaround);
|
||||
assert(!UseShuffleFallback(cs));
|
||||
assert(HostThreadId(lo, 7) == 7 && HostThreadId(cs, 7) == 7);
|
||||
assert(HostThreadId(hi, 7) == (number(7) > 32 ? 39 : 7));
|
||||
}
|
||||
'''
|
||||
|
||||
|
||||
def section(source, start, end):
|
||||
first = source.index(start)
|
||||
return source[first:source.index(end, first)]
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--cxx", default="clang++")
|
||||
args = parser.parse_args()
|
||||
shader = (VK / "vk_shader.cpp").read_text(encoding="utf-8")
|
||||
warp = (VK / "eden/shader_recompiler/backend/spirv/emit_spirv_warp.cpp").read_text(encoding="utf-8")
|
||||
stamp = section(shader, " const uint32_t lanes[]", " // (the records keep Shader::Info")
|
||||
check_stamp = r'''
|
||||
void check_stamp(uint32_t size, uint32_t pinned, bool workaround, bool forced) {
|
||||
bool added = false;
|
||||
const auto add = [&](const uint32_t* values, size_t bytes) {
|
||||
assert(bytes == 5 * sizeof(uint32_t));
|
||||
assert(values[0] == uint32_t(size > 32) && values[1] == size);
|
||||
assert(values[2] == uint32_t(workaround) && values[3] == pinned && values[4] == uint32_t(forced));
|
||||
added = true;
|
||||
};
|
||||
''' + stamp + r'''
|
||||
assert(added);
|
||||
}
|
||||
'''
|
||||
fixture = (FIXTURE + section(shader, "Shader::Profile make_profile(", "Shader::HostTranslateInfo host_info()")
|
||||
+ section(shader, "const Shader::Profile& profile()", "const std::vector<Code>& codes_of(")
|
||||
+ section(warp, "Id HostThreadId(", "Id GuestLane(")
|
||||
+ section(warp, "bool UseShuffleFallback(", "Id ShuffleAbsolute(") + check_stamp + CASES)
|
||||
with tempfile.TemporaryDirectory(prefix="hookshot-subgroup-profile-") as tmp:
|
||||
cpp, exe = Path(tmp) / "fixture.cpp", Path(tmp) / "fixture.exe"
|
||||
cpp.write_text(fixture, encoding="utf-8")
|
||||
subprocess.run([args.cxx, "-std=c++20", "-I", str(VK / "eden"), "-I", str(ROOT / "gpu/third_party/suyu"),
|
||||
str(cpp), "-o", str(exe)], check=True, timeout=60)
|
||||
# A fresh process per device: the real profiles are cached once for the selected GPU.
|
||||
# Inputs: default lanes, maximum lanes, pinned-stage mask, proprietary AMD driver.
|
||||
# Expected widths: base, vertex, fragment, compute, task, mesh; then fragment workaround.
|
||||
cases = [
|
||||
((32, 32, 0, 0), (32, 32, 32, 32, 32, 32, 0), {}),
|
||||
((64, 64, 32, 1), (64, 64, 64, 32, 64, 64, 1), {}),
|
||||
((32, 64, 32, 1), (64, 64, 64, 32, 64, 64, 1), {}),
|
||||
((32, 64, 0, 0), (64, 64, 64, 64, 64, 64, 0), {}),
|
||||
((32, 64, 31, 1), (64, 32, 32, 64, 64, 64, 1), {}),
|
||||
((64, 64, 240, 1), (64, 64, 32, 32, 32, 32, 1), {}),
|
||||
((64, 64, 0, 0), (64, 64, 64, 64, 64, 64, 0), {}),
|
||||
((64, 64, 32, 1), (32, 32, 32, 32, 32, 32, 1), {"RX_VK_WARP64": "0"}),
|
||||
((64, 64, 240, 1), (64, 64, 64, 32, 64, 64, 1), {"RX_VK_WARP64": "1"}),
|
||||
((64, 64, 32, 1), (64, 64, 64, 32, 64, 64, 0), {"RX_VK_FRAG_SHUFFLE_FALLBACK": "0"}),
|
||||
((64, 64, 32, 0), (64, 64, 64, 32, 64, 64, 1), {"RX_VK_FRAG_SHUFFLE_FALLBACK": "1"}),
|
||||
]
|
||||
env = {k: v for k, v in os.environ.items() if k not in ("RX_VK_WARP64", "RX_VK_FRAG_SHUFFLE_FALLBACK")}
|
||||
for device, expected, overrides in cases:
|
||||
subprocess.run([str(exe), *map(str, device + expected)], env=env | overrides, check=True, timeout=10)
|
||||
assert "EmitSPIRV(profile_pinned(), rt, t->program, bindings)" in shader
|
||||
for stage in ("TASK", "MESH"):
|
||||
assert f"fallback ? profile_pinned() : profile_graphics(VK_SHADER_STAGE_{stage}_BIT_EXT)" in shader
|
||||
for start, end in (("Id ShuffleRelative(", "Id BroadcastLane("), ("Id EmitShuffleIndex(", "Id EmitShuffleUp("),
|
||||
("Id EmitShuffleButterfly(", "Id EmitQuadBroadcast(")):
|
||||
assert "UseShuffleFallback(ctx)" in section(warp, start, end)
|
||||
quad = section(warp, "Id EmitQuadBroadcast(", "Id EmitQuadSwap(")
|
||||
assert "ctx.profile.has_broken_spirv_subgroup_shuffle" in quad and "UseShuffleFallback" not in quad
|
||||
print("stage subgroup widths, 32-lane partitions, AMD fragment shuffles, and cache identity ok")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
120
tests/test_frame_whole.py
Normal file
120
tests/test_frame_whole.py
Normal file
@@ -0,0 +1,120 @@
|
||||
"""Game counters that add the frame step, or the step times a whole number, cut to whole frames keep counting above 60 fps
|
||||
(host/frame_whole.h, nx2/oot_patches.py patch_whole_frames).
|
||||
|
||||
The pick-up indicator above Link's head (f_80a87e50) moves the count it shows toward the wallet's by (int)(step * per) and fades
|
||||
when they meet; the time of day (f_803edf70) adds (int)(2 * step) game seconds a frame. Under a Frame rate cap above 60 the step is
|
||||
the frame's own time, under 1 (0.42 at 144 fps): cut to 0 every frame, the count never moved and the indicator stayed, and the
|
||||
clock stood still. With the frames' own time
|
||||
as the step (carry), hos_frame_whole adds the amount to the counter's own remainder and hands out its whole frames: 144 frames of
|
||||
60/144 are 60; an amount of 1 or more keeps its fraction too; two counters keep their own remainders; NaN and amounts below zero
|
||||
give 0. Without it, the amount is cut as the game cuts it (60 fps as before). The water renderer (f_800672e0) runs its ripple
|
||||
simulation (int)step times a frame the same way, its remainder kept per water object: forty surfaces stepped in turn each still
|
||||
count their 60 frames a second. The patch replaces the one cut of each site, once,
|
||||
declares the hook, and leaves a file patched before and a function whose code differs as they are.
|
||||
|
||||
python -I -B tests/test_frame_whole.py
|
||||
"""
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "nx2"))
|
||||
import oot_patches # noqa: E402
|
||||
|
||||
FIXTURE = r"""
|
||||
#include "frame_whole.h"
|
||||
#include <math.h>
|
||||
#include <stdio.h>
|
||||
static int fails;
|
||||
#define CHECK(c) do { if (!(c)) { printf("FAIL line %d: %s\n", __LINE__, #c); fails++; } } while (0)
|
||||
static uint32_t bits(float f) { uint32_t b; memcpy(&b, &f, 4); return b; }
|
||||
int main(void) {
|
||||
static FrameWhole w;
|
||||
const uint64_t a = 0x1000, b = 0x2000;
|
||||
uint32_t n = 0;
|
||||
for (int i = 0; i < 144; i++) n += frame_whole(&w, a, bits(60.0f / 144.0f), 1);
|
||||
CHECK(n == 60 || n == 59); /* (a second at 144 fps: 60 frames, give or take the float's last one) */
|
||||
n = 0;
|
||||
for (int i = 0; i < 1440; i++) n += frame_whole(&w, a, bits(60.0f / 144.0f), 1);
|
||||
CHECK(n >= 599 && n <= 601);
|
||||
CHECK(frame_whole(&w, b, bits(0.42f), 1) == 0 && frame_whole(&w, b, bits(0.42f), 1) == 0 && frame_whole(&w, b, bits(0.42f), 1) == 1);
|
||||
n = 0;
|
||||
for (int i = 0; i < 4; i++) n += frame_whole(&w, b + 8, bits(1.25f), 1); /* (the step times 3: its fractions kept) */
|
||||
CHECK(n == 5);
|
||||
n = 0;
|
||||
for (int i = 0; i < 144; i++) n += frame_whole(&w, b + 16, bits(2.0f * 60.0f / 144.0f), 1); /* (the time of day) */
|
||||
CHECK(n == 120 || n == 119); /* (a second at 144 fps: the 120 game seconds of one at 60) */
|
||||
CHECK(frame_whole(&w, a, bits(0.42f), 0) == 0 && frame_whole(&w, a, bits(1.5f), 0) == 1 && frame_whole(&w, a, bits(2.0f), 0) == 2);
|
||||
CHECK(frame_whole(&w, a, bits(-1.0f), 1) == 0 && frame_whole(&w, a, bits(NAN), 1) == 0 && frame_whole(&w, a, bits(0.0f), 1) == 0);
|
||||
for (uint64_t k = 0; k < 100; k++) frame_whole(&w, 0x9000 + k, bits(0.5f), 1); /* (more counters than slots: the oldest go) */
|
||||
CHECK(frame_whole(&w, 0x9000 + 99, bits(0.5f), 1) == 1); /* (a slot still held: 0.5 + 0.5) */
|
||||
CHECK(frame_whole(&w, 0x9000, bits(0.5f), 1) == 0); /* (the oldest's slot was reused: it starts again from 0) */
|
||||
/* (a scene's water surfaces, each stepped once a frame at 144 fps: 40 of them, more than the old 16 slots, each still counts
|
||||
its 60 frames a second) */
|
||||
{
|
||||
static FrameWhole water;
|
||||
uint32_t passes[40] = {0};
|
||||
for (int f = 0; f < 144; f++)
|
||||
for (int k = 0; k < 40; k++) passes[k] += frame_whole(&water, 0x7000000 + (uint64_t)k * 0x400, bits(60.0f / 144.0f), 1);
|
||||
for (int k = 0; k < 40; k++) CHECK(passes[k] == 60 || passes[k] == 59);
|
||||
}
|
||||
if (!fails) printf("PASS\n");
|
||||
return fails != 0;
|
||||
}
|
||||
"""
|
||||
|
||||
LIFTED = ['#include "a64.h"\n' + head + " c->x[9] = recomp_load32(c,(c->x[19] + 864));\n" + old + "return;\n}\n"
|
||||
for head, old, _ in oot_patches.WHOLE_SITES]
|
||||
|
||||
|
||||
class FrameWholeTests(unittest.TestCase):
|
||||
def test_counter_keeps_counting_under_one(self):
|
||||
compiler = shutil.which("gcc") or shutil.which("clang")
|
||||
if not compiler:
|
||||
self.skipTest("no C compiler")
|
||||
with tempfile.TemporaryDirectory(prefix="hookshot-frame-whole-") as tmp:
|
||||
tmp = Path(tmp)
|
||||
(tmp / "t.c").write_text(FIXTURE, encoding="utf-8")
|
||||
binary = tmp / "t.exe"
|
||||
build = subprocess.run([compiler, "-std=gnu11", "-Wall", "-O1", "-I", str(ROOT / "host"), str(tmp / "t.c"),
|
||||
"-o", str(binary), "-lm"], text=True, capture_output=True)
|
||||
self.assertEqual(build.returncode, 0, build.stderr)
|
||||
run = subprocess.run([str(binary)], text=True, capture_output=True, timeout=30)
|
||||
self.assertEqual(run.returncode, 0, run.stdout + run.stderr)
|
||||
|
||||
def test_patch_replaces_the_cut_once(self):
|
||||
for lifted, (_, old, key) in zip(LIFTED, oot_patches.WHOLE_SITES):
|
||||
out, n = oot_patches.patch_whole_frames(lifted)
|
||||
self.assertEqual(n, 1)
|
||||
self.assertEqual(out.count("=hos_frame_whole(%s, (uint32_t)c->vreg[0][0]);" % key), 1)
|
||||
self.assertNotIn("recomp_fp_to_int", out)
|
||||
self.assertIn(oot_patches.WHOLE_DECL, out)
|
||||
self.assertEqual(oot_patches.patch_whole_frames(out), (out, 0)) # (patched before: as it is)
|
||||
other = lifted.replace("32,32,", "32,31,") # (another game version's code)
|
||||
self.assertEqual(oot_patches.patch_whole_frames(other), (other, 0))
|
||||
none = lifted.replace("void f_8", "void f_9")
|
||||
self.assertEqual(oot_patches.patch_whole_frames(none), (none, 0))
|
||||
# (a lift where the function is hot: its float helpers are a64_fpx_hot_*, the cut the same)
|
||||
_, old0, _ = oot_patches.WHOLE_SITES[0]
|
||||
hot = LIFTED[0].replace(old0, " _v=a64_fpx_hot_mul32_fma32(c,_a,_b,_z);\n a64_vset64(c->vreg[0],_v);}\n" + old0)
|
||||
self.assertEqual(oot_patches.patch_whole_frames(hot)[1], 1)
|
||||
|
||||
def test_clock_patches_its_seconds_alone(self):
|
||||
# (f_803edf70 cuts into x8 the same way once more, in the branch that sets the time from another source: that cut
|
||||
# stays the game's)
|
||||
i = next(i for i, (h, _, _) in enumerate(oot_patches.WHOLE_SITES) if "f_803edf70" in h)
|
||||
cut = " c->x[8]=recomp_fp_to_int(c->vreg[0][0]&UINT64_C(0xffffffff),32,32,1,3,c->fpcr,&c->fpsr);\n"
|
||||
lifted = LIFTED[i].replace("return;\n", cut + " recomp_store32(c,c->x[19] + 12,c->x[8]);\nreturn;\n")
|
||||
out, n = oot_patches.patch_whole_frames(lifted)
|
||||
self.assertEqual(n, 1)
|
||||
self.assertIn("=hos_frame_whole(c->x[19] + 60, (uint32_t)c->vreg[0][0]);", out)
|
||||
self.assertEqual(out.count(cut), 1)
|
||||
self.assertIn(cut + " recomp_store32(c,c->x[19] + 12,", out)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(unittest.main(verbosity=1))
|
||||
172
tests/test_high_fps_fixes.py
Normal file
172
tests/test_high_fps_fixes.py
Normal file
@@ -0,0 +1,172 @@
|
||||
"""Above a 60 fps cap the sound job runs at 60 Hz and the physics solver keeps whole-frame steps (host/oot_high_fps.h,
|
||||
nx2/oot_patches.py patch_sound_job and patch_solver_steps).
|
||||
|
||||
Sound: under a Frame rate cap above 60 the frame step is the frame's own time (0.5 at 120 fps). The sound manager's job (f_80435300)
|
||||
fed steps under 1 left sounds unreleased at area changes until the game's voice budget was full and sounds, then the music,
|
||||
stopped. The gate runs phase 0 once a whole step has built up (120 frames of 0.5: 60 runs; 144 of 60/144: 60), the other phases
|
||||
follow phase 0's choice, the engine is handed a whole step then, and a step of 1 or more, 0 or below, NaN, or no frame time at
|
||||
all (a cap of 60) runs every frame with the game's own step. A step of 1 or more keeps the remainder, so around 60 fps (0.98 and
|
||||
1.02 alternating, 0.9 and 1.1, or mean 1 with jitter) the sound clock keeps the real one; a cap of 60 runs every phase, before
|
||||
the first phase 0 too and after a phase 0 that held the job.
|
||||
|
||||
Solver: the game cuts frame step * n + 0.001 to its solver steps (n a 60 fps frame: 8 at 60 fps, 3 or 4 at 120 as the step
|
||||
wanders; with n = 4, 0 above 240). With the frames' own time and a step under 1 it is rounded up (10% leeway for the frame time's
|
||||
jitter) and at least 1, never under the game's; a step of 1 or more, or no frame time (a cap of 60), keeps the game's count.
|
||||
|
||||
The patches put the gate at the job's head and the step through the host, hook both solver places, declare the hooks, and leave
|
||||
a file patched before, or whose code differs, as it is.
|
||||
|
||||
python -I -B tests/test_high_fps_fixes.py
|
||||
"""
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "nx2"))
|
||||
import oot_patches # noqa: E402
|
||||
|
||||
FIXTURE = r"""
|
||||
#include "oot_high_fps.h"
|
||||
#include <stdio.h>
|
||||
static int fails;
|
||||
#define CHECK(c) do { if (!(c)) { printf("FAIL line %d: %s\n", __LINE__, #c); fails++; } } while (0)
|
||||
static uint32_t bits(float f) { return oot_float_bits(f); }
|
||||
static int runs(float step, int frames, int timed) {
|
||||
OotSoundGate g = OOT_SOUND_GATE_INIT;
|
||||
int n = 0;
|
||||
for (int i = 0; i < frames; i++) {
|
||||
const int run = oot_sound_gate(&g, 0, step, timed);
|
||||
CHECK(oot_sound_gate(&g, 1, step, timed) == run); /* (phase 1 follows phase 0) */
|
||||
n += run;
|
||||
}
|
||||
return n;
|
||||
}
|
||||
/* (the sound engine's time over these frames' steps against the real time: what it is handed each frame the job runs) */
|
||||
static double sound_clock(const float *steps, int n) {
|
||||
OotSoundGate g = OOT_SOUND_GATE_INIT;
|
||||
double real = 0, sound = 0;
|
||||
for (int i = 0; i < n; i++) {
|
||||
real += steps[i];
|
||||
if (oot_sound_gate(&g, 0, steps[i], 1)) sound += oot_bits_float(oot_sound_step(oot_float_bits(steps[i]), 1));
|
||||
}
|
||||
return sound / real;
|
||||
}
|
||||
int main(void) {
|
||||
CHECK(runs(0.5f, 120, 1) == 60);
|
||||
const int n144 = runs(60.0f / 144.0f, 144, 1);
|
||||
CHECK(n144 == 60 || n144 == 59);
|
||||
CHECK(runs(60.0f / 1000.0f, 1000, 1) >= 59 && runs(60.0f / 1000.0f, 1000, 1) <= 60);
|
||||
CHECK(runs(0.5f, 120, 0) == 120); /* (a cap of 60: every frame) */
|
||||
CHECK(runs(1.0f, 60, 1) == 60 && runs(1.5f, 60, 1) == 60);
|
||||
CHECK(runs(0.0f, 60, 1) == 60 && runs(-1.0f, 60, 1) == 60 && runs(NAN, 60, 1) == 60);
|
||||
{ /* (a whole step after partial ones runs with the game's step and keeps the remainder) */
|
||||
OotSoundGate g = OOT_SOUND_GATE_INIT;
|
||||
CHECK(oot_sound_gate(&g, 0, 0.75f, 1) == 0 && oot_sound_gate(&g, 2, 0.75f, 1) == 0);
|
||||
CHECK(oot_sound_gate(&g, 0, 1.0f, 1) == 1 && g.part == 0.75f && oot_sound_gate(&g, 1, 1.0f, 1) == 1);
|
||||
CHECK(oot_sound_gate(&g, 0, 0.75f, 1) == 1 && g.part == 0.5f);
|
||||
CHECK(oot_sound_gate(&g, 0, 0.0f, 1) == 1 && g.part == 0.0f); /* (a step of 0: as the game has it, the remainder cleared) */
|
||||
}
|
||||
{ /* (around 60 fps the sound clock keeps the real one: dropping the remainder at each whole step made it 0.51 of it) */
|
||||
static float steps[20000];
|
||||
for (int i = 0; i < 20000; i++) steps[i] = i & 1 ? 1.02f : 0.98f;
|
||||
double r = sound_clock(steps, 20000);
|
||||
CHECK(r > 0.999 && r < 1.001);
|
||||
for (int i = 0; i < 20000; i++) steps[i] = i & 1 ? 1.1f : 0.9f;
|
||||
r = sound_clock(steps, 20000);
|
||||
CHECK(r > 0.999 && r < 1.001);
|
||||
uint32_t x = 12345; /* (mean 1, about 0.03 apart: a sum of uniforms) */
|
||||
for (int i = 0; i < 20000; i++) {
|
||||
float u = 0;
|
||||
for (int k = 0; k < 12; k++) x = x * 1664525u + 1013904223u, u += (float)(x >> 8) / 16777216.0f;
|
||||
steps[i] = 1.0f + (u - 6.0f) * 0.03f;
|
||||
}
|
||||
r = sound_clock(steps, 20000);
|
||||
CHECK(r > 0.999 && r < 1.001);
|
||||
for (int i = 0; i < 20000; i++) steps[i] = 0.5f;
|
||||
r = sound_clock(steps, 20000);
|
||||
CHECK(r > 0.999 && r < 1.001);
|
||||
}
|
||||
{ /* (a cap of 60 runs every phase: before any phase 0, and after a phase 0 that held the job at a cap above 60) */
|
||||
OotSoundGate g = OOT_SOUND_GATE_INIT;
|
||||
CHECK(oot_sound_gate(&g, 1, 1.0f, 0) == 1 && oot_sound_gate(&g, 2, 1.0f, 0) == 1);
|
||||
OotSoundGate h = OOT_SOUND_GATE_INIT;
|
||||
CHECK(oot_sound_gate(&h, 1, 1.0f, 1) == 1); /* (above 60, before the first phase 0: it runs) */
|
||||
CHECK(oot_sound_gate(&h, 0, 0.5f, 1) == 0 && oot_sound_gate(&h, 1, 0.5f, 0) == 1 && h.part == 0.0f); /* (the cap changed) */
|
||||
CHECK(oot_sound_gate(&h, 2, 0.5f, 0) == 1 && oot_sound_gate(&h, 0, 0.5f, 0) == 1);
|
||||
}
|
||||
CHECK(oot_sound_step(bits(0.5f), 1) == bits(1.0f));
|
||||
CHECK(oot_sound_step(bits(0.5f), 0) == bits(0.5f));
|
||||
CHECK(oot_sound_step(bits(1.25f), 1) == bits(1.25f) && oot_sound_step(bits(0.0f), 1) == bits(0.0f));
|
||||
/* (the game's: (int)(step * n + 0.001), n the steps of a 60 fps frame) */
|
||||
#define GAME(step, n) ((int32_t)((step) * (n) + 0.001f))
|
||||
#define OURS(step, n, timed) oot_solver_steps(GAME(step, n), bits((step) * (n) + 0.001f), bits(n), timed)
|
||||
CHECK(OURS(1.0f, 4.0f, 1) == 4 && OURS(1.0f, 8.0f, 1) == 8 && OURS(1.0f, 8.0f, 0) == 8);
|
||||
CHECK(OURS(0.5f, 4.0f, 1) == 2 && OURS(0.49f, 4.0f, 1) == 2 && OURS(0.52f, 4.0f, 1) == 2 && GAME(0.49f, 4.0f) == 1);
|
||||
CHECK(OURS(0.49f, 8.0f, 1) == 4 && OURS(0.5f, 8.0f, 1) == 4 && GAME(0.49f, 8.0f) == 3);
|
||||
CHECK(OURS(0.25f, 4.0f, 1) == 1 && OURS(0.24f, 4.0f, 1) == 1 && OURS(0.06f, 4.0f, 1) == 1 && GAME(0.24f, 4.0f) == 0);
|
||||
CHECK(OURS(60.0f / 144.0f, 4.0f, 1) == 2 && OURS(60.0f / 90.0f, 4.0f, 1) == 3);
|
||||
CHECK(OURS(0.49f, 4.0f, 0) == 1 && OURS(0.06f, 4.0f, 0) == 0); /* (a cap of 60: the game's count) */
|
||||
CHECK(OURS(1.6f, 4.0f, 1) == 6 && OURS(1.708f, 8.0f, 1) == 13); /* (a slow frame: the game's count) */
|
||||
CHECK(oot_solver_steps(3, bits(NAN), bits(4.0f), 1) == 3 && oot_solver_steps(0, bits(-1.0f), bits(4.0f), 1) == 0);
|
||||
CHECK(oot_solver_steps(1, bits(2.0f), bits(0.0f), 1) == 1); /* (no steps a frame: the game's) */
|
||||
if (!fails) printf("PASS\n");
|
||||
return fails != 0;
|
||||
}
|
||||
"""
|
||||
|
||||
LIFTED_SOUND = ('#include "a64.h"\n' + oot_patches.SOUND_HEAD + " /* nop/hint */\nif ((uint32_t)c->x[2] != 0) { goto L_1; }\n"
|
||||
+ oot_patches.SOUND_OLD + "L_1:;\n return;\n}\nvoid f_80435360(Ctx *__restrict A64_NOESC c) {\n"
|
||||
+ oot_patches.SOUND_OLD + "}\n")
|
||||
LIFTED_SOLVER = '#include "a64.h"\n' + "".join(head + " c->x[8]=1;\n" + oot_patches.SOLVER_OLD + "}\n"
|
||||
for head in oot_patches.SOLVER_HEADS)
|
||||
|
||||
|
||||
class HighFpsTests(unittest.TestCase):
|
||||
def test_gate_and_steps(self):
|
||||
compiler = shutil.which("gcc") or shutil.which("clang")
|
||||
if not compiler:
|
||||
self.skipTest("no C compiler")
|
||||
with tempfile.TemporaryDirectory(prefix="hookshot-high-fps-") as tmp:
|
||||
tmp = Path(tmp)
|
||||
(tmp / "t.c").write_text(FIXTURE, encoding="utf-8")
|
||||
binary = tmp / "t.exe"
|
||||
build = subprocess.run([compiler, "-std=gnu11", "-Wall", "-O1", "-I", str(ROOT / "host"), str(tmp / "t.c"),
|
||||
"-o", str(binary), "-lm"], text=True, capture_output=True)
|
||||
self.assertEqual(build.returncode, 0, build.stderr)
|
||||
run = subprocess.run([str(binary)], text=True, capture_output=True, timeout=30)
|
||||
self.assertEqual(run.returncode, 0, run.stdout + run.stderr)
|
||||
|
||||
def test_sound_patch(self):
|
||||
out, n = oot_patches.patch_sound_job(LIFTED_SOUND)
|
||||
self.assertEqual(n, 1)
|
||||
head = out.index(oot_patches.SOUND_HEAD) + len(oot_patches.SOUND_HEAD)
|
||||
self.assertTrue(out[head:].startswith(oot_patches.SOUND_GATE)) # (before the job's own first instruction)
|
||||
self.assertEqual(out.count("hos_oot_sound_gate(c,"), 1)
|
||||
self.assertEqual(out.count(oot_patches.SOUND_NEW), 1) # (the job's own load only, not the next function's)
|
||||
self.assertEqual(out.count(oot_patches.SOUND_OLD), 1)
|
||||
self.assertIn(oot_patches.SOUND_DECL, out)
|
||||
self.assertEqual(oot_patches.patch_sound_job(out), (out, 1)) # (patched before: as it is)
|
||||
other = LIFTED_SOUND.replace("+ 2192)", "+ 2200)") # (another game version's code)
|
||||
self.assertEqual(oot_patches.patch_sound_job(other), (other, 0))
|
||||
none = LIFTED_SOUND.replace("void f_80435300", "void f_90435300")
|
||||
self.assertEqual(oot_patches.patch_sound_job(none), (none, 0))
|
||||
|
||||
def test_solver_patch(self):
|
||||
out, n = oot_patches.patch_solver_steps(LIFTED_SOLVER)
|
||||
self.assertEqual(n, 2)
|
||||
self.assertEqual(out.count(oot_patches.SOLVER_NEW), 2)
|
||||
self.assertIn(oot_patches.SOLVER_DECL, out)
|
||||
hook = out.index("hos_oot_solver_steps((uint32_t)c->x[1]")
|
||||
self.assertLess(out.index("recomp_fp_to_int"), hook)
|
||||
self.assertLess(hook, out.index("recomp_store32(c,c->x[0] + 712"))
|
||||
self.assertEqual(oot_patches.patch_solver_steps(out), (out, 2))
|
||||
other = LIFTED_SOLVER.replace("+ 712", "+ 716")
|
||||
self.assertEqual(oot_patches.patch_solver_steps(other), (other, 0))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(unittest.main(verbosity=1))
|
||||
236
tests/test_movie_frame_count.py
Normal file
236
tests/test_movie_frame_count.py
Normal file
@@ -0,0 +1,236 @@
|
||||
"""Above a 60 cap a movie's frame count grows by the frames' time, so a cutscene's cues stay with its movie (issue 1742).
|
||||
|
||||
The game's movie player counts its updates while a movie plays, and a cutscene that shows a movie places its narration, its
|
||||
rumble and its end by that count. The movie's picture and sound follow the clock, so at 120 fps the cues came twice too early:
|
||||
narration lines started over each other. nx2/oot_patches.py makes the count's step hos_movie_frames(player), and host/nn_vi.c
|
||||
answers 1 while the host gives no frame time (a cap of 60: as shipped) and the frames' time in 60ths, the rest carried to the
|
||||
player's next update, while it does (HOS_FPS_MOVIE=0: 1 then too, for comparisons). A game version whose code differs keeps
|
||||
its own code and the build goes on with a warning.
|
||||
|
||||
python -I -B tests/test_movie_frame_count.py [--cc gcc]
|
||||
"""
|
||||
import argparse
|
||||
import contextlib
|
||||
import io
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
RUN_ALL_ARGS = ["--cc", "gcc"]
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "nx2"))
|
||||
import oot_patches # noqa: E402
|
||||
|
||||
# (the shape of the lifted update: the count loaded, one added, stored; the same lines in another function stay as they are)
|
||||
STORE = " recomp_store32(c,c->x[20] + 760,c->x[8]);\n"
|
||||
LIFTED = ('#include "a64.h"\n'
|
||||
"void f_83b7c710(Ctx *__restrict A64_NOESC c) {\n"
|
||||
" c->x[8] = 0x1ULL;\n" + oot_patches.MOVIE_OLD + STORE + "}\n"
|
||||
"void f_83b7c870(Ctx *__restrict A64_NOESC c) {\n" + oot_patches.MOVIE_OLD + STORE + "}\n")
|
||||
HOST_ANCHOR = "uint32_t hos_movie_frames(uint64_t player) {"
|
||||
|
||||
|
||||
def block(source, marker):
|
||||
if marker not in source:
|
||||
raise SystemExit("FAIL: host/nn_vi.c has no '%s'" % marker)
|
||||
start = source.index(marker)
|
||||
return source[start:start + re.search(r"^}", source[start:], re.M).end()]
|
||||
|
||||
|
||||
def check(cond, what):
|
||||
if not cond:
|
||||
raise SystemExit("FAIL: " + what)
|
||||
|
||||
|
||||
FIXTURE = r'''#define _CRT_SECURE_NO_WARNINGS
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#define CHECK(x) do { if (!(x)) { fprintf(stderr, "FAIL:%d %s\n", __LINE__, #x); exit(1); } } while (0)
|
||||
/* (the lock as a flag: taken once, given back once) */
|
||||
typedef int SRWLOCK;
|
||||
#define SRWLOCK_INIT 0
|
||||
static void AcquireSRWLockExclusive(SRWLOCK *l) { CHECK(!*l); *l = 1; }
|
||||
static void ReleaseSRWLockExclusive(SRWLOCK *l) { CHECK(*l); *l = 0; }
|
||||
volatile uint32_t hos_frame_seconds_bits;
|
||||
@HOST@
|
||||
/* (the lifted update with the patch, on a small guest: the count at +760 of the player) */
|
||||
typedef struct { uint64_t x[31]; } Ctx;
|
||||
static uint32_t guest[2][256];
|
||||
static uint32_t recomp_load32(Ctx *c, uint64_t a) { (void)c; return guest[a >> 12][(a & 0xfff) / 4]; }
|
||||
static void recomp_store32(Ctx *c, uint64_t a, uint64_t v) { (void)c; guest[a >> 12][(a & 0xfff) / 4] = (uint32_t)v; }
|
||||
static uint32_t update(int player) {
|
||||
Ctx ctx, *c = &ctx;
|
||||
memset(&ctx, 0, sizeof ctx);
|
||||
c->x[20] = (uint64_t)player << 12;
|
||||
@LIFTED@
|
||||
return guest[player][760 / 4];
|
||||
}
|
||||
static void seconds(double s) {
|
||||
const float f = (float)s;
|
||||
uint32_t bits;
|
||||
memcpy(&bits, &f, 4);
|
||||
hos_frame_seconds_bits = s > 0 ? bits : 0;
|
||||
}
|
||||
static int within(uint32_t got, uint32_t want) { return got + 1 >= want && got <= want + 1; }
|
||||
int main(int argc, char **argv) {
|
||||
uint32_t n = 0, before;
|
||||
if (argc > 1 && !strcmp(argv[1], "off")) { /* (HOS_FPS_MOVIE=0: the game's own count at any rate) */
|
||||
seconds(1.0 / 120);
|
||||
for (int i = 0; i < 600; i++) n = update(0);
|
||||
CHECK(n == 600);
|
||||
printf("PASS: off\n");
|
||||
return 0;
|
||||
}
|
||||
/* no frame time (a cap of 60): one an update, as the game counts */
|
||||
seconds(0);
|
||||
for (int i = 0; i < 600; i++) n = update(0);
|
||||
CHECK(n == 600);
|
||||
CHECK(hos_movie_frames(0x5000) == 1 && hos_movie_frames(0x5000) == 1);
|
||||
/* 120 fps: half a 60th an update, a whole one every second update */
|
||||
seconds(1.0 / 120);
|
||||
before = n;
|
||||
for (int i = 0; i < 600; i++) {
|
||||
const uint32_t was = n;
|
||||
n = update(0);
|
||||
CHECK(n - was <= 1);
|
||||
}
|
||||
CHECK(within(n - before, 300));
|
||||
/* 240 fps and 40 fps: a quarter and one and a half */
|
||||
seconds(1.0 / 240);
|
||||
before = n;
|
||||
for (int i = 0; i < 960; i++) n = update(0);
|
||||
CHECK(within(n - before, 240));
|
||||
seconds(1.0 / 40);
|
||||
before = n;
|
||||
for (int i = 0; i < 400; i++) {
|
||||
const uint32_t was = n;
|
||||
n = update(0);
|
||||
CHECK(n - was == 1 || n - was == 2);
|
||||
}
|
||||
CHECK(within(n - before, 600));
|
||||
/* two movies at once: each its own count (one carry for both would give each half) */
|
||||
seconds(1.0 / 120);
|
||||
before = n;
|
||||
{
|
||||
const uint32_t other = update(1) - 0; /* (its first update) */
|
||||
uint32_t m = other;
|
||||
for (int i = 0; i < 600; i++) n = update(0), m = update(1);
|
||||
CHECK(within(n - before, 300) && within(m - other, 300));
|
||||
}
|
||||
/* a movie that plays while others come and go keeps its count: 14 more that play with it and a new player every update,
|
||||
16 at a time, at 240 fps (places taken in the order they were made gave this one a new start every second update: a
|
||||
quarter of a 60th an update then counted 0 or 1 every two, 1200 only on average) */
|
||||
seconds(1.0 / 240);
|
||||
{
|
||||
uint32_t mine = 0, steady = 0;
|
||||
for (int i = 0; i < 4800; i++) {
|
||||
mine += hos_movie_frames(0x7000000);
|
||||
for (int k = 0; k < 14; k++) steady += hos_movie_frames(0x7100000 + (uint64_t)k * 0x1000);
|
||||
hos_movie_frames(0x8000000 + (uint64_t)i * 0x1000);
|
||||
}
|
||||
CHECK(within(mine, 1200) && steady + 14 >= 14 * 1200 && steady <= 14 * 1200 + 14);
|
||||
}
|
||||
seconds(1.0 / 120);
|
||||
/* more movies at once than the host keeps places for, each updated in turn: every update finds its player's place taken and
|
||||
starts from a drawn part of a 60th, so 480 updates of half a 60th give 240 on average, 11 either way as one standard
|
||||
deviation. Every count within 5 of those, for any number of players: with a new place started at 0 every count stood
|
||||
still, and with a fixed step between the starts one player's did among 233 (and 29 players gave 108 to 131 of 120) */
|
||||
{
|
||||
static const int counts[] = {9, 17, 29, 40, 233, 377};
|
||||
static uint32_t sum[377];
|
||||
uint32_t least = ~0u, most = 0;
|
||||
for (unsigned t = 0; t < sizeof counts / sizeof counts[0]; t++) {
|
||||
const int players = counts[t];
|
||||
memset(sum, 0, sizeof sum);
|
||||
for (int i = 0; i < 480; i++)
|
||||
for (int k = 0; k < players; k++) sum[k] += hos_movie_frames(0x100000 + (uint64_t)t * 0x1000000 + (uint64_t)k * 0x1000);
|
||||
for (int k = 0; k < players; k++) {
|
||||
CHECK(sum[k] >= 240 - 55 && sum[k] <= 240 + 55);
|
||||
least = sum[k] < least ? sum[k] : least, most = sum[k] > most ? sum[k] : most;
|
||||
}
|
||||
}
|
||||
printf("more players than places: %u to %u of 240\n", least, most);
|
||||
}
|
||||
before = n;
|
||||
for (int i = 0; i < 600; i++) n = update(0);
|
||||
CHECK(within(n - before, 300));
|
||||
/* back at a cap of 60: one an update again */
|
||||
seconds(0);
|
||||
before = n;
|
||||
for (int i = 0; i < 10; i++) n = update(0);
|
||||
CHECK(n - before == 10);
|
||||
printf("PASS\n");
|
||||
return 0;
|
||||
}
|
||||
'''
|
||||
|
||||
|
||||
def main():
|
||||
p = argparse.ArgumentParser(description=__doc__)
|
||||
p.add_argument("--cc", default="gcc")
|
||||
a = p.parse_args()
|
||||
|
||||
out = oot_patches.patch_movie_frames(LIFTED)
|
||||
check(out is not None and out.count("_b=hos_movie_frames(c->x[20])") == 1, "the count's step is not the host's, once")
|
||||
update = out[out.index("void f_83b7c710"):out.index("void f_83b7c870")]
|
||||
check("hos_movie_frames" in update and "hos_movie_frames" not in out[out.index("void f_83b7c870"):],
|
||||
"the same lines in another function were patched")
|
||||
check(out.startswith('#include "a64.h"\n' + oot_patches.MOVIE_DECL), "the lifted file does not declare the host's function")
|
||||
back = out.replace(oot_patches.MOVIE_DECL, "", 1).replace(oot_patches.MOVIE_NEW, oot_patches.MOVIE_OLD)
|
||||
check(back == LIFTED, "more than the count's step changed")
|
||||
check(oot_patches.patch_movie_frames(out) == out, "a second run changed a patched file")
|
||||
check(oot_patches.patch_movie_frames(LIFTED.replace("f_83b7c710", "f_83b7c714")) is None,
|
||||
"a file without the function was not left to the caller")
|
||||
for what, other in (("another step", LIFTED.replace("_b=1ULL", "_b=2ULL")),
|
||||
("another field", LIFTED.replace("+ 760", "+ 764")),
|
||||
("the lines twice", LIFTED.replace(STORE + "}\nvoid f_83b7c870", STORE + oot_patches.MOVIE_OLD + STORE + "}\nvoid f_83b7c870"))):
|
||||
check(oot_patches.patch_movie_frames(other) == other, "a function with %s was patched (another game version)" % what)
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix="movie-count-") as tmp:
|
||||
f = Path(tmp) / "code_000.c"
|
||||
f.write_text(LIFTED, encoding="utf-8", newline="")
|
||||
for run in ("first", "second"): # (the second over the patched file: counted as made, nothing written)
|
||||
log = io.StringIO()
|
||||
with contextlib.redirect_stdout(log):
|
||||
oot_patches.apply(tmp)
|
||||
check("WARNING: the movie's frame count" not in log.getvalue(), "the %s run warns though the count is patched" % run)
|
||||
check(f.read_text(encoding="utf-8") == out, "the %s run did not leave the patched file" % run)
|
||||
other = LIFTED.replace("_b=1ULL", "_b=2ULL")
|
||||
f.write_text(other, encoding="utf-8", newline="")
|
||||
log = io.StringIO()
|
||||
with contextlib.redirect_stdout(log):
|
||||
oot_patches.apply(tmp)
|
||||
check("WARNING: the movie's frame count was found 0 times" in log.getvalue(), "no warning for a count that was not found")
|
||||
check(f.read_text(encoding="utf-8") == other, "another version's function was changed")
|
||||
|
||||
vi = (ROOT / "host" / "nn_vi.c").read_text(encoding="utf-8")
|
||||
lifted = oot_patches.MOVIE_NEW + STORE
|
||||
fixture = FIXTURE.replace("@HOST@", block(vi, HOST_ANCHOR)).replace("@LIFTED@", lifted)
|
||||
with tempfile.TemporaryDirectory(prefix="movie-count-") as tmp:
|
||||
c = Path(tmp) / "fixture.c"
|
||||
exe = Path(tmp) / "fixture.exe"
|
||||
c.write_text(fixture, encoding="utf-8")
|
||||
subprocess.run([a.cc, "-std=c11", "-Wall", "-Wextra", "-Werror", str(c), "-o", str(exe)], check=True)
|
||||
env = {k: v for k, v in os.environ.items() if not k.startswith("HOS_")}
|
||||
subprocess.run([str(exe)], env=env, check=True)
|
||||
subprocess.run([str(exe)], env=dict(env, HOS_FPS_MOVIE="1"), check=True) # (read by its value: 1 is on)
|
||||
subprocess.run([str(exe), "off"], env=dict(env, HOS_FPS_MOVIE="0"), check=True)
|
||||
r = subprocess.run([str(exe), "off"], env=env, capture_output=True, text=True)
|
||||
check(r.returncode != 0 and "FAIL:" in r.stderr, "the count is the game's own without HOS_FPS_MOVIE=0")
|
||||
# (the negative control: the game's own step of one an update fails the same checks at 120 fps)
|
||||
plain = Path(tmp) / "plain.c"
|
||||
plain.write_text(FIXTURE.replace("@HOST@", block(vi, HOST_ANCHOR)).replace("@LIFTED@", oot_patches.MOVIE_OLD + STORE),
|
||||
encoding="utf-8")
|
||||
subprocess.run([a.cc, "-std=c11", "-Wall", "-Wextra", "-Werror", "-Wno-unused-function", str(plain), "-o", str(exe)], check=True)
|
||||
r = subprocess.run([str(exe)], env=env, capture_output=True, text=True)
|
||||
check(r.returncode != 0 and "FAIL:" in r.stderr, "the unpatched count passes the checks: they prove nothing")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user