mirror of
https://git.triforceproject.com/triforce/source
synced 2026-11-19 16:20:01 -04:00
Triforce v0.2.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
29
docs/capture-dimensions.md
Normal file
29
docs/capture-dimensions.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# Screenshot dimension validation
|
||||
|
||||
`RX_VK_SHOT_W` and `RX_VK_SHOT_H` accept positive decimal integers. Missing variables
|
||||
retain the1280x720 default. Empty, signed, whitespace-containing, partially numeric,
|
||||
zero and overflowing values disable screenshots for that process and report the
|
||||
invalid request once. They do not fall back to a different requested capture size.
|
||||
|
||||
Before any screenshot allocation, the dimensions must fit the device's
|
||||
`maxImageDimension2D`, the RGBA readback's CPU addressable size, and the existing
|
||||
PNG encoder. That encoder writes one IDAT with stored deflate blocks: the checked
|
||||
size includes its zlib header, five bytes per block and Adler32. The
|
||||
[PNG chunk limit](https://www.w3.org/TR/png-3/#5Chunk-layout) is2^31-1 bytes.
|
||||
This is a format/capability bound; it does not add a memory budget or guarantee
|
||||
that a valid request can be allocated on a particular GPU.
|
||||
|
||||
Rejected requests never enter the capture producer. Window presentation continues;
|
||||
headless presentation keeps its existing offscreen policy and produces no capture
|
||||
for an invalid size. The fixed-size binary frame logger is independent. Valid
|
||||
standard, ultrawide, odd and small dimensions retain their requested sizes.
|
||||
|
||||
`tests/test_capture_dimensions.py` compiles production setup, the real capture
|
||||
entry/presentation conditions, and the PNG writer. A Vulkan allocation model checks
|
||||
rejection before calls, reports once, and preserves window/headless routing. A
|
||||
small real PNG validates chunk CRCs, decoded scanlines and stored-block sizing.
|
||||
Large dimension/format boundaries use handles only, never large allocations.
|
||||
|
||||
Resource readiness in #1472 remains separate. In a combined source tree its sliced
|
||||
setup test must include the actual dimension guard and initialized physical-device
|
||||
limits before exercising allocation failures; it must not stub the guard away.
|
||||
35
docs/capture-readback.md
Normal file
35
docs/capture-readback.md
Normal file
@@ -0,0 +1,35 @@
|
||||
# Capture copies and CPU visibility
|
||||
|
||||
Normal screenshots, generated-frame screenshots and binary frame-log slots are
|
||||
GPU-written mapped staging buffers. Their contents reach CPU consumers through
|
||||
three distinct requirements:
|
||||
|
||||
1. Record a copy-write to host-read dependency for the copied buffer range.
|
||||
2. Wait for its owning GPU submission or presentation-slot fence.
|
||||
3. Invalidate that allocation range before reading a non-coherent host cache.
|
||||
|
||||
`capture_host_read_barrier` uses COPY/TRANSFER_WRITE to HOST/HOST_READ, offset0
|
||||
and the complete copied byte count, with no queue ownership transfer. Each
|
||||
capture producer records it immediately after its copy. Existing timeline/fence
|
||||
waits remain in place. `invalidate_mapped` delegates cache atom alignment and
|
||||
coherent-memory handling to VMA.
|
||||
|
||||
`shot_pixels` owns normal screenshot bytes after the timeline wait. A pending
|
||||
screenshot already copied to CPU memory keeps using that snapshot if its parity
|
||||
slot is reused; it must not invalidate or reread the reused GPU slot. Generated
|
||||
screenshots invalidate their own allocation after the same completed value.
|
||||
Frame-log collection invalidates only a used slot after its fence wait and
|
||||
preserves timestamp, sequence and kind metadata.
|
||||
|
||||
The source-sliced regression executes the actual three copy/dependency tails
|
||||
and the CPU consumers against authored device bytes, host-domain availability
|
||||
and coherent/non-coherent caches. Four phases (normal, owned snapshot, made,
|
||||
binary log), two cache modes and two host-visibility modes give16 scenarios.
|
||||
Permissive host visibility remains a positive control; explicit visibility
|
||||
detects a missing producer dependency even on coherent memory. These models
|
||||
verify protocol and ownership, not physical non-coherent GPU memory or driver
|
||||
behavior. PNG encoding and generic device-loss/wait-error handling are outside
|
||||
this test; no performance or intended visual change is claimed.
|
||||
|
||||
References: [Khronos host readback](https://docs.vulkan.org/guide/latest/synchronization_examples.html#cpu-read-back-of-data-written-by-a-compute-shader)
|
||||
and [VMA cache management](https://gpuopen-librariesandsdks.github.io/VulkanMemoryAllocator/html/memory_mapping.html).
|
||||
33
docs/capture-slot-ownership.md
Normal file
33
docs/capture-slot-ownership.md
Normal file
@@ -0,0 +1,33 @@
|
||||
# Pending screenshot slot ownership
|
||||
|
||||
Normal debug screenshots use two readback buffers, selected by present parity.
|
||||
A pending capture may outlive the next non-shot present when the GPU is behind.
|
||||
Another shot two presents later selects the same buffer. Submitting that new copy
|
||||
before reading the old capture lets a fast GPU replace the pixels, even though
|
||||
the later read waits for the older timeline value. The filename/flash analysis
|
||||
then identifies the old frame while consuming the new frame's bytes.
|
||||
|
||||
Before recording a new copy into a pending capture's slot, snapshot that capture
|
||||
into CPU-owned pixels. Its existing consumer moves the snapshot instead of reading
|
||||
the reused buffer. Another slot still follows the asynchronous path, and no
|
||||
pending capture adds no wait. Re-arming starts with no saved snapshot, so the old
|
||||
pixels cannot be attached to the new frame. The generated-shot path and capture
|
||||
format/metadata remain unchanged.
|
||||
|
||||
`do_present` is serialized by the replay job consumer; synchronous replay calls
|
||||
hold `sync_order_mu`. The pending state remains in that producer/consumer flow.
|
||||
There is at most one pending snapshot, with the same pixel allocation the normal
|
||||
consumer already makes, moved earlier only when slot reuse requires it.
|
||||
|
||||
`tests/test_capture_slot_ownership.py` compiles the actual copy-selection, snapshot,
|
||||
read and pending-consume statements with a queued GPU model. It covers a deferred
|
||||
capture across a non-shot frame, same/opposite parity, fast/late new completion,
|
||||
no-pending captures and re-arming across sparse intervals. It checks pixels and
|
||||
metadata together, and verifies that different slots do not wait before submit.
|
||||
This models legal GPU ordering; it does not claim to reproduce a hardware race.
|
||||
|
||||
This change concerns buffer lifetime. Host-cache visibility is the separate
|
||||
`invalidate_mapped` contract tracked in PR1502; if that change lands first, retain
|
||||
its normal-shot invalidation inside `shot_pixels`, after the wait and before the
|
||||
copy into CPU-owned storage. PR1025's on-demand path still needs the same lifetime
|
||||
rule when integrated; request paths do not change readback ownership.
|
||||
19
docs/compiler-memory-accounting.md
Normal file
19
docs/compiler-memory-accounting.md
Normal file
@@ -0,0 +1,19 @@
|
||||
# Compiler memory accounting
|
||||
|
||||
The build memory gate counts a live compiler driver and its descendants, rather
|
||||
than just the small driver process. `rss_mb` returns zero for a process not yet
|
||||
attached, unavailable counters, or a root whose `Popen.poll()` already reports
|
||||
completion. Retaining a Popen/OS handle must not let stale process-tree counters
|
||||
or a subsequently reused PID contribute memory after that root has ended.
|
||||
|
||||
This check precedes the platform counter query. It does not change compiler peak
|
||||
estimates, worker floors, reserves, lease release or live child accounting. A
|
||||
process can still exit during a later process-tree snapshot; this is not an atomic
|
||||
snapshot or a complete PID-ancestry race solution.
|
||||
|
||||
`tests/test_rss_liveness.py` retains an actual completed Popen and models a stale
|
||||
positive backend sample. The known exit takes precedence without querying that
|
||||
backend; live-tree sampling and unavailable-counter fallback are also checked.
|
||||
`test_mem_gate_children.py` separately checks a real child holding300MiB, and the
|
||||
existing memory-gate/lease regressions cover scheduling behavior. These tests need
|
||||
no compiler, game dump or game execution.
|
||||
265
docs/design.md
Normal file
265
docs/design.md
Normal file
@@ -0,0 +1,265 @@
|
||||
# Hookshot (oot-recomp): design
|
||||
|
||||
A native x86-64 Windows and Linux build of the Switch 2 *The Legend of Zelda: Ocarina of Time* remake: the game's AArch64
|
||||
code translated to C ahead of time from the user's own decrypted dump and compiled with clang, the Horizon OS and
|
||||
NVN2 implemented natively, and the GPU work replayed on Vulkan. No CPU or GPU emulation at any point: the same
|
||||
model as mkw-recomp (Mario Kart World), which this project forks.
|
||||
|
||||
This document is the plan. The numbers in it that depend on the binaries are filled in from the dump (section 9);
|
||||
until it exists they are marked *dump*.
|
||||
|
||||
## 1. Short answer
|
||||
|
||||
**The game is a Switch 2 title, so the base is mkw-recomp, not TOTKPort.** Public reports before its release
|
||||
(2026-11-05) describe a Switch 2 exclusive, by Nintendo, 1080p to 1440p docked and 720p handheld, with the ocarina
|
||||
playable by humming into the console's microphone. A Switch 2 executable means:
|
||||
|
||||
| | Switch 2 (this game, expected) | Switch 1 (TOTK, TOTKPort's base) |
|
||||
|---|---|---|
|
||||
| CPU code | ARMv8.2+ (Cortex-A78C): PAC returns, LSE atomics, CAS, LDAPR | ARMv8.0 (Cortex-A57) |
|
||||
| Relocations | DT_RELR packed + RELA | RELA |
|
||||
| GPU API | NVN2 (API 231.x): native images, DLSS, mesh/task programs | NVN 55.x |
|
||||
| Shaders | Ampere SASS (SM86) in `Nin_Ounce_NVN2` packages | Maxwell SASS in `Nin_NX_NVN` packages |
|
||||
| Services | nx2 gs/gpu/vi2 sessions, no nvdrv in the npdm | nvdrv, vi:u |
|
||||
|
||||
mkw-recomp already handles every row of the left column, from the lifter (`nx2/insn.py`, `nx2/nso2.py`) to the
|
||||
Vulkan back end with Eden's SM86 recompiler (`gpu/vk`), and its standalone runtime plays MKW races at 45-55 fps.
|
||||
TOTKPort's NVN1 layer and Maxwell path would have to be rebuilt for Switch 2 first.
|
||||
|
||||
`nx2/dump.py` checks this on the real dump: six independent signals (program id prefix, ACID layout, nvdrv in the
|
||||
services, DT_RELR, ARMv8.1+ instructions, NVN2 proc names) must all say Switch 2. If they say Switch 1, section 3
|
||||
changes: TOTKPort's `hos/nvn_native.c` (NVN 55.15) would sit under this repository's Vulkan back end instead.
|
||||
|
||||
## 2. What is reused (checked 2026-11-04)
|
||||
|
||||
| From | What | Where here |
|
||||
|---|---|---|
|
||||
| **mkw-recomp** `7336f0a` (`..\mkw-recomp`, local) | lift driver and its fixes, Switch 2 forms, DT_RELR, Eden's image layout, 64-bit guest window, loop-header polls, stable chunks, Unicorn tests, build; the standalone runtime; the native NVN2 layer; the Vulkan back end | `nx2/`, `host/`, `gpu/`, `third_party/` (commit `86871a1`, byte-identical) |
|
||||
| **a64-kit** (Rexcomp's AArch64 kit, TOTKPort `60de030`, via mkw-recomp) | NSO loader, image builder, lifter, suyu's translator, instruction harness | `third_party/a64-kit` (unmodified) |
|
||||
| **Eden's shader recompiler** (Eglantine build 181, via mkw-recomp) | SM86 -> IR -> SPIR-V, task/mesh programs | `gpu/vk/eden` |
|
||||
| **TOTKPort** (`Desktop\Ports\TOTK\TOTKPort`) | the original HLE runtime and NVN layer that mkw-recomp's are forks of | reference only |
|
||||
| **Eglantine** (`..\Eglantine`, read and run only) | Switch 2 service protocols, the reference renderer, an A/B oracle | reference only; its CPM cache supplies the Vulkan/SPIR-V headers, VMA, FFmpeg |
|
||||
|
||||
Keeping the fork's paths identical to mkw-recomp's is deliberate: MKW is the reference port and gets fixes every
|
||||
day (the queue/event ordering and pool captures of 2026-11-04 among them). A later mkw-recomp commit comes over with
|
||||
|
||||
```
|
||||
git -C ../mkw-recomp diff 7336f0a <new> -- nx2 host gpu third_party > mkw.patch
|
||||
git apply -3 mkw.patch
|
||||
```
|
||||
|
||||
which works because the base blobs are in this repository. Each sync names its new mkw-recomp base in its commit
|
||||
message. OoT-specific code goes into new files where it can, so the shared files stay easy to patch.
|
||||
|
||||
## 3. Architecture
|
||||
|
||||
```
|
||||
exefs/{rtld,main,subsdk*,sdk} + main.npdm (the user's dump, read in place: work/dump.txt)
|
||||
│ a64-kit tools/nso.py + nx2/nso2.py (DT_RELR)
|
||||
▼
|
||||
modules: LZ4 segments, MOD0, dynamic section, symbols, RELA + RELR, .eh_frame_hdr starts
|
||||
│ nx2/image.py (Eden's layout from 0x80000000: rtld, main, subsdk*, sdk; we are rtld)
|
||||
▼
|
||||
image: relocated, imports bound across modules; the HLE set hooked (nx2/standalone.py)
|
||||
│ a64-kit lift/a64_lift.py + nx2/insn.py + nx2/lift_fixes.py + nx2/poll.py + nx2/chunks.py
|
||||
▼
|
||||
one C function per guest function (work/sa/gen, ~1 GB, local), 64-bit guest window (nx2/rtgen.py)
|
||||
│ clang (llvm-mingw, emsdk or the system's) -O1, the hot functions -O2 with the play profile, lld (nx2/build.py --standalone)
|
||||
▼
|
||||
hookshot.exe = lifted game + host/ (Horizon: threads, SVCs, memory, fs, hid, audio, vi, gs, NVN2)
|
||||
hookshot_vk.dll = the Vulkan back end (gpu/vk), loaded by host/rx_backend.c unless RX_BACKEND names another renderer
|
||||
```
|
||||
|
||||
- **Standalone from the start.** MKW went emulator-hosted first (Eglantine with a recompiled CPU backend), then
|
||||
standalone. That detour is not needed any more: the standalone runtime exists and is the user's rule anyway
|
||||
("100% native"). Eglantine stays the reference to compare against, never part of the process.
|
||||
- **The SDK is replaced at the nn:: boundary** (TOTKPort's model): `HLE(<mangled name>)` functions in `host/` stand
|
||||
in for the SDK's OS-facing functions; the rest of the SDK (libc, nn::util, nn::mem, the NVN2 driver's front end)
|
||||
runs as lifted code. Unimplemented OS-facing imports get generated stubs that log their first call.
|
||||
- **Memory.** A 512 GB reservation; the image at Eden's `0x80000000` (the lift's addresses), the heap from 64 GB,
|
||||
address regions growing down from 512 GB; committed as handed out or on first touch (`host/hos.h`).
|
||||
- **GPU.** `host/nvn_native.c` implements the NVN2 procs over guest memory in place: objects live where the game
|
||||
puts them, command buffers record an op stream (`gpu/render/rx_nvn.h`), queue submits hand it to the back end.
|
||||
`gpu/vk` replays it on Vulkan 1.3: pools imported as host memory, SM86 programs through Eden's recompiler to
|
||||
SPIR-V, mesh/task programs with `VK_EXT_mesh_shader`. The WebGPU renderer (`gpu/render`) stays in the Windows exe as the
|
||||
fallback when Vulkan is missing, as in mkw-recomp, when the build has wgpu-native (`WGPU_NATIVE`).
|
||||
|
||||
## 4. The lessons from MKW, as rules here
|
||||
|
||||
Each of these cost days in mkw-recomp; the fork has the fixes, and these are the checks that keep them true for a
|
||||
second game.
|
||||
|
||||
1. **Cross-queue `nvnCommandBufferWaitEvent` is honoured.** MKW's queues pass stages back and forth with events;
|
||||
replaying them in arrival order on one Vulkan queue flashed. `gpu/vk/vk_replay.cpp` `submit_job` replays each
|
||||
submission as far as its waits allow, with per-queue state (`RX_VK_EVENT_WAITS=0` turns it off, for A/B only).
|
||||
*Check early:* log OoT's queues and event waits on the first boot (which queues, how many waits per frame).
|
||||
2. **Nothing the replay reads comes from live guest memory.** Games write per-frame data ahead of the GPU: every
|
||||
read is from submit-time captures, and pools are captured from their creation, not their first GPU use
|
||||
(`gpu/vk/vk_memory.cpp`, `RX_VK_CAPTURE_POOL_MB`). *Check:* the mirror sync log's live-sync count stays ~0.
|
||||
3. **Texture and sampler registrations are queued in order with the submits**, never applied out of band.
|
||||
4. **GPU timestamps drive dynamic resolution.** OoT is reported at 1080p-1440p docked, so it has a resolution
|
||||
controller fed by timestamp reports: the fake GPU time (the counter reports the replay writes, `gpu/vk/vk_replay.cpp`
|
||||
`NOP_REPORT_COUNTER`: the 31.25 MHz tick clock since the frame's present, times `RX_VK_TS_SCALE`, 0.1 by default) must be
|
||||
scaled so the controller settles where the PC is, not at its floor. *Check:* the render size it picks over time.
|
||||
5. **Clears after draws, and draws that sample a target their pass already wrote (or write one an earlier draw of the
|
||||
pass sampled), get passes of their own** (`vk_draw.cpp` `clear_after_draws`, `draw_hazards`). A draw that samples
|
||||
a target it writes itself stays in the pass.
|
||||
6. **2D views of 3D textures need `VK_EXT_image_2d_view_of_3d`** (the RTX 4060 Ti has it).
|
||||
7. **The CPU saturates first.** Ryzen 5 7600X, 6 cores / 12 threads: the game's threads, the replay, captures and
|
||||
pipeline compiles share it, and extra helper threads have lowered fps before. Thread counts are measured, never
|
||||
raised by default (`HOS_VCORES`, `RX_VK_CAPTURE_THREADS`, `HOS_THREAD_TIMES=<seconds>`, the `HOS_SAMPLE` profiler
|
||||
with `scripts/prof.py`).
|
||||
|
||||
Also inherited, and game-specific in MKW: MKW's frame-pacer hooks at fixed guest addresses are gated on MKW's build
|
||||
id (`hos_game_is_mkw`); OoT gets its own once its pacer is found, never by address coincidence.
|
||||
|
||||
## 5. What OoT needs that MKW did not (checked on the dump, 2026-11-04)
|
||||
|
||||
| | Found | Plan |
|
||||
|---|---|---|
|
||||
| Dump format | one `.nup` package (base v65536 + update v131072, each an exefs/ and romfs/ tree, data stored as is) | `nx2/nup.py` extracts the newest title to `work/dump` |
|
||||
| NSO compression | `main` is ZBIC (NSO flags bit 7, firmware 22.0.0+: zstd with its own magic and BIC-coded tables); rtld, sdk, subsdk0 are LZ4 | `nx2/unpack.py` rewrites ZBIC NSOs as plain ones (hashes checked), decoder `third_party/zbic` |
|
||||
| SDK / NVN2 version | 22.2.8 (MKW: 21.4.0); same 46 services and 49 SVCs as MKW | 26 NVN proc names MKW never asked for, 14 not in `host/nvn_native.c` yet (stubs that stop the game only when called; DLSS parameter procs, window acquire timeout, HDR metadata, `nvnProgramBuilderGetShaders`, ...); the device limits in `nvn_native.c` are MKW sdk's (`devinfo[]`) and may need OoT's |
|
||||
| Instructions | 4 UDOT (FEAT_DotProd) sites, the only forms suyu's translator lacked | `nx2/insn.py` `dot()` |
|
||||
| Engine | romfs laid out like TOTK's (agl, gsys, Phive, Banc, Bake), Havok 2025.2, NintendoWare G3d2/Vfx3/Ui, SMAA, `.zs` (zstd) files | romfs read from `work/dump/romfs` |
|
||||
| Microphone (the ocarina by humming) | to check at runtime (nn::audio input; `audsmx:u`, `auddmg:u` are in the services, as in MKW) | first a stub that hears silence; then a host microphone (WASAPI) or a key that plays the note |
|
||||
| Cutscene video / voice | `Movie/` in the romfs | MKW's paths; `HOS_NO_MOVIES=1` to skip while booting |
|
||||
| DLSS | NVN2 DLSS procs present (66 DLSS / mesh-task names) | a bilinear stand-in first (as Eglantine), FSR or NVIDIA's own DLSS later |
|
||||
|
||||
## 6. Stages and milestones
|
||||
|
||||
| | What | Done when |
|
||||
|---|---|---|
|
||||
| **S0** | the dump located and characterized (`nx2/dump.py --set`) | the six signals agree; program id, build ids, module sizes, SDK and NVN versions, romfs size recorded in section 9 |
|
||||
| **S1** | loader + RELR, image in Eden's layout, discovery, coverage, census; instruction test (every form the game uses) and function tests (self-contained and call trees) against Unicorn | 100 % of discovered instructions translated (UDF aside), no BR without a jump table, every form bit-exact, every comparable function case identical |
|
||||
| **S2** | the standalone lift (`lift.py --standalone --write`) and a full build (`build.py --standalone`): hookshot.exe and hookshot_vk.dll link | the boot runs `_init_libc*`, `nnosInitialize`, `nninitStartup`, the modules' DT_INIT, `nnMain` and reaches `nvnBootstrapLoader` |
|
||||
| **S3** | NVN2 for this game: the procs it asks for, device limits, its window; the Vulkan replay | **the first frame**: a `run-vk.ps1` screenshot shows what the game draws first (a logo or loading screen) |
|
||||
| **S4** | title screen and menus: romfs, save data, hid, audio (muted in tests), movies, the stubs it hits | the title screen and the file select are reachable by scripted input (`HOS_INPUT_SCRIPT` / `HOS_INPUT_LIVE`) |
|
||||
| **S5** | gameplay: correctness (the flash detector `RX_VK_FLASH=1`, frame dumps) and speed (the profiler) | Kokiri Forest plays without flashes at the game's own frame rate |
|
||||
| **S6** | PC: resolution, frame rate, controls, the microphone, DLSS/FSR | the user's call |
|
||||
|
||||
## 7. Testing
|
||||
|
||||
- **Instruction forms** (`nx2/test_insn.py`): every form the game's code contains, random states, Unicorn's
|
||||
ARMv8.x "max" CPU against the lifted C: X0-X30, SP, NZCV, V0-V31, FPSR and memory compared.
|
||||
- **Whole functions** (`nx2/test_leaf.py`, `--calls` for call trees): random functions of every module on the real
|
||||
relocated image, natively and in Unicorn, on the 64-bit window with polls firing; registers, V registers and
|
||||
every changed page compared.
|
||||
- **The production build** (`nx2/test_dll.py`, when a `hookshot_game.dll` is built).
|
||||
- **In the game**: `scripts/run-vk.ps1` screenshots, the flash detector, frame dumps (`RX_VK_DUMP*`), the sampled
|
||||
profiler (`HOS_SAMPLE` + `scripts/prof.py`), and Eglantine as the A/B reference for what a frame should look like.
|
||||
|
||||
## 8. Rules
|
||||
|
||||
- No game files in the repository: decompressed modules, the image, lifted C, builds, romfs copies, dumps and keys
|
||||
stay in `work/` (ignored) or in the dump's own folder. Nothing is pushed anywhere.
|
||||
- Commits end with `Assisted-by: Claude Opus 5.5` and `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`.
|
||||
- `..\Eglantine` is read and run, never modified; mkw-recomp and the other ports are read and copied from, never
|
||||
edited.
|
||||
- One game run at a time on this PC (mkw-recomp's runs take ~10 GB): `run-vk.ps1` refuses while `mkw.exe` or
|
||||
another `hookshot.exe` runs, and while Rainbow Six Siege is open; no heavy builds while Siege is open either. Runs are
|
||||
muted (`HOS_AUDIO=0`), at the priority `-Priority` gives (High by default), and screenshot folders are deleted once looked at.
|
||||
- The lifted C and every build are GPL (suyu's translator is GPL-2.0-or-later; Eden's code is GPL-3.0).
|
||||
|
||||
## 9. Status
|
||||
|
||||
2026-11-04 (before the dump): fork base committed (`86871a1` = mkw-recomp `7336f0a`), the tools made to take any
|
||||
dump, the runtime made game-neutral, the stage-1 tools checked on MKW's dump as a stand-in.
|
||||
|
||||
2026-11-04, the dump (`gamefile/...[0400486024dec000][v131072].nup`, 25.6 GB; extracted to `work/dump`, 26.2 GB):
|
||||
- **S0 done.** Switch 2 by all seven signals. Program id 0400486024DEC000, update v131072 (base v65536).
|
||||
SDK and NVN2 22.2.8. 39-bit address space, main thread priority 44 core 0, 1 MB stack, system resource 48 MB.
|
||||
|
||||
| module | build id | .text | .rodata | .data | .bss | eh starts |
|
||||
|---|---|---|---|---|---|---|
|
||||
| rtld | 8e213f228d12c7b0f8c21c3ce81b95e214417369 | 10,048 | 2,532 | 49,160 | 65,528 | 42 |
|
||||
| main | e0f3ccfaf50065ae4ea01eb5aeb13b1a45da3348 | 63,201,744 | 12,603,520 | 5,217,152 | 1,414,272 | 247,520 |
|
||||
| subsdk0 | b0110087a7b339ef74588d3bd0b4dd5d5c7abd0f | 542,576 | 234,548 | 69,648 | 65,520 | 1,039 |
|
||||
| sdk | 1643a838814f9c10c696f930ed681e7fa3fb9af8 | 8,906,832 | 6,093,948 | 435,448 | 1,862,408 | 35,786 |
|
||||
|
||||
romfs: 186,583 files, 24.4 GiB (Tex 8.8, TerrainArc 5.6, Model 2.6, Sound 2.0 GiB), 10,154 `Nin_Ounce_NVN2`
|
||||
shader files.
|
||||
- **S1:** the lift: 295,178 functions, 17,668,191 instructions, 100.0000 % translated (after UDOT), 1,475 jump
|
||||
tables, no BR without one. Instruction test: 742 forms, 44,172 cases, all pass.
|
||||
- **S2:** the standalone lift: 294,450 functions in 57 chunks (1.4 GB of C), 485 HLE functions, 243 OS imports
|
||||
stubbed, 22 sdk NVN functions routed to `host/nvn2.c`.
|
||||
The build links (`hookshot.exe`, 426 MB; ~25 min for a full build with 4 jobs, a few minutes after a relift).
|
||||
- **S3 done (2026-11-04, boot13).** The title screen renders at 60 fps: black for ~23 s (~1,400 presents) while it
|
||||
loads, then the logo, smoke and the menu (New Game / Options / ZELDA NOTES Setting), through the Vulkan back end.
|
||||
It runs the 150 s out with no crash. What it took, in boot order:
|
||||
- TLS: the owner handle at TLR+0x110 too (SDK 22). NVN API version 231.40 (main 0x8348c0f0 checks minor >= 40).
|
||||
- svcGetInfo 20 (UserExceptionContextAddress); the address arbiter (svc 0x34/0x35, WaitIfEqual64 too).
|
||||
- StandardSteadyClock::now returns in x0. ADR / ADRP+ADD code addresses as entries, but only outside the
|
||||
functions .eh_frame covers (all of them: 1.68 M duplicated instructions and a 122 MB chunk; now 79 entries),
|
||||
for the formatter callback 0x824c5560 that nothing else reaches.
|
||||
- SDK 22 overloads left to the guest SDK, which reached kernel calls the runtime does not have:
|
||||
CreateThread(..., prio, idealCore), ReleaseLightSemaphore(sem, count), MountRom(name),
|
||||
GetDirectoryEntryCount.
|
||||
- nn::os fibers on Win32 fibers, each with its own guest context (nn::async's FiberTaskCoordinator on the
|
||||
nn.npln threads); a fiber function's NULL return goes back to the thread.
|
||||
- Offline answers: ConnectToNamedPort gives a session whose requests are answered empty (sm:);
|
||||
nsd::GetEnvironmentIdentifier "lp1"; socket::detail::Poll nothing ready (gRPC calls it through a pointer);
|
||||
EnsureNetworkServiceAccountIdTokenCacheAsync fails.
|
||||
- nvnQueueWaitSync (an ordered no-op), SetCopyImageStride; no virtual core held through host file I/O
|
||||
(mkw-recomp 16bc3bc: the ARes loaders' 250 ms stalls are gone, longest wait now 0.7 ms).
|
||||
- **Open at S3:** nvnCommandBufferPushComputeConstants is not implemented; one NVN2 program (e409733c45530d75,
|
||||
stage 5) does not lower (an unsupported native opcode at +0x4330); 70 imports answered by the generic stub
|
||||
(audio effects, prepo, nifm, socket/ssl setup, vi refresh rate, hid gyro drift mode, pl::RequestLoadNxFont).
|
||||
Next: input to get past the title (S4: New Game into Kokiri Forest), then the open items above.
|
||||
- **S4 (2026-11-05): gameplay.** New Game → the movies (AV1 through Media Foundation) → Kokiri Forest, with the
|
||||
HUD, at 30-36 fps. What it took (commits through 7e88949 and the ones after):
|
||||
- Push constants (nvnCommandBufferPushComputeConstants) in c0 after the grid; ClearBuffer; prepo stubs.
|
||||
- setjmp/longjmp: the lifter wraps every call of a function that calls setjmp in a host setjmp of its own
|
||||
(host/hos_setjmp.c, assembly, returns_twice: clang's `__builtin_longjmp` restored rbp to the frame address,
|
||||
128 bytes off Win64's frame pointer), and longjmp's BR comes back through a64_ijump_hook.
|
||||
- HMMA on the host's tensor cores: HMMA_nx2 → IR MatrixMulAddF16 → OpCooperativeMatrixMulAddKHR 16x8x16 f16.
|
||||
gpu/vk/vk_coopmat.cpp measures the device's element order (NVIDIA: A's registers 1 and 2 swapped) and checks
|
||||
a multiply. OoT's networks (the movie upscaler, the gameplay TAAU) run at full speed (the movie at 60 fps).
|
||||
- Write-backs of GPU-written textures keep the pages the CPU wrote since (SceneLowPriority read texels as
|
||||
pointers). Storage buffer tables: GetCbuf(i, x*16 + base) addresses get a 16-entry descriptor array that the
|
||||
global memory fallbacks search (the froxel fog); the 32-bit global atomics got fallbacks too.
|
||||
- Program variants remember the storage images' integer-ness too: OoT writes R11G11B10 and RGBA8 targets
|
||||
through R32UI views, and a variant made while the slot held a float texture wrote packed words as floats
|
||||
(the yellow cast: NaN scene before the TAAU).
|
||||
- Extended registers: programs using ~249 GPRs (vertex and mesh) keep their high GPRs in IR registers 256 + n
|
||||
(REGMAP_nx2 prefixes, lower_to_maxwell.h RegisterMapPrefix; R246+ and R239 renamed to markers first).
|
||||
222 of the 240 dumped failing programs lower now (the offline harness: `nx2/lower_test.py work/lowfail`).
|
||||
- **Open at S4:** an intermittent Vulkan device loss (nvlddmkm event 153) shortly after the first gameplay frames,
|
||||
never with RX_VK_CHECKPOINTS=1 (five runs) and every time without (four); mesh output clamps did not change it.
|
||||
Six programs still do not lower (CALL linkage 0x944, a texture gather mode, an LDG mode). Gameplay is CPU-bound
|
||||
at 30-36 fps (the replay thread ~9 ms a frame, the GPU ~10 ms): the game's threads next.
|
||||
|
||||
## 10. Findings, 2026-11-06
|
||||
|
||||
- White flash: one NaN in the scene target goes through the bloom's averages to the whole picture for a frame, the HUD over it.
|
||||
`GuardFragmentOutputs` keeps float colour outputs finite (RX_VK_NAN_GUARD=0: off). Where the NaN comes from is not known.
|
||||
- Discards: a nested early return after a demote wrote its colours (`structured_control_flow.cpp`, from Lichii). That was the box
|
||||
around Link (`1e26755d`, the per-character shadow projector, drawn as a box that discards outside) and most of the milky look.
|
||||
- The OoT fog (0.25) and exposure (0.65) factors are still needed: with them off the fog is far too thick. Fog's cause is open.
|
||||
- Fps is CPU-bound: RX_VK_NO_WRITEBACK cut GPU time from 10 to 3.6 ms and left fps alone. HOS_PIN_BY_CORE 35 fps, HOS_SPIN_US=200
|
||||
slower, narrow barriers no change, RX_VK_WAIT_GPU no help.
|
||||
- Tools: RX_VK_FRAMELOG + scripts/framelog.py (every frame shown), RX_VK_PROBE=2 (the passes at speed), RX_VK_SLOW_LOG, HOS_WATCH.
|
||||
- A release build (`RX_RELEASE=1` at build time) precompiles before the game starts; test builds do not unless RX_VK_PRECOMPILE=1.
|
||||
hookshot_programs.bin (with hookshot_programs.idx, the list of its records), hookshot_pipelines.bin and mkw_vk_pipelines.bin go
|
||||
beside the executable.
|
||||
|
||||
## 11. Findings, 2026-11-06, day
|
||||
|
||||
- Builds: `nx2/split_hot.py` puts code_hot.c in eight files that compile side by side, 79 minutes to about 6. `RX_PGO=gen` counts branches
|
||||
(host/pgo_rt.c writes LLVM raw format 11, the toolchain has no compiler-rt profile library), `HOS_PGO=<path>` with `<path>.reset` and
|
||||
`<path>.dump` files drives a run, `nx2/pgo_pieces.py` fits the merged profile to the pieces, `RX_PGO=<file>` uses it. `RX_VK_ONLY=1`: the renderer alone.
|
||||
- Dark frames, open: lit surfaces come out black for one frame, geometry and depth intact. 5% with `RX_VK_FRAMEGEN=0`, 2.9% with the menu's Off, about 0.1% at 4x.
|
||||
Not the cause: render conditions, clear flags, ClearBuffer order, GPU lag, write-back, the real-time step, the ambient cubemap.
|
||||
`HOS_COND_RECORD` and `RX_VK_COND_ALWAYS` look like fixes by count and are not: the picture goes white. Tools: the frame log's dark trap dumps probes, passes and conditions; `RX_VK_PROBE_MIN`, `RX_VK_PROBE_UBO_HEX`.
|
||||
Also not the cause: colour write masks (`wm` in the pass trace: f on every lit pass), constants (`RX_VK_PROBE_UBO_HEX`), queue order (`RX_VK_EVENT_WAITS=0`), and the capture in
|
||||
synchronous mode (`RX_VK_SYNC_CAPTURE=1`, which does take 380,000 pages per 600 submits). `RX_VK_QUEUE_STATE=0` stops the game at the Load menu, and `RX_VK_MIRROR=0` is 8 fps: neither says anything.
|
||||
What holds: geometry and depth are right (black shapes against the fog), the prepass and the lit passes run with the same draw counts, and the lit passes output black.
|
||||
The lit materials' four storage buffers (c0+0x510..0x540) keep their addresses, sizes and non-zero counts in dark frames as seen in guest memory; the 720 KB one is all zeros in good
|
||||
frames too, so it is no cause, though its compute shader (4b03b304) runs every frame. Guest memory is not what the GPU reads, so these need comparing on the GPU's side.
|
||||
Next: their mirrors' contents read back on the GPU's timeline, and the bound textures past the centre texel. `RX_VK_SUBMIT_DELAY_US=3000` shows none, but at 33 fps, where they never show:
|
||||
repeat with a delay that keeps ~45 fps. The MKW session's mirror page skip fired 0 times here and was removed.
|
||||
Not an instrument's doing: with nothing attached, photographed from outside the window (37 s, ~2200 photographs): 16 with `RX_VK_FRAMEGEN=0`, none at the user's 4x. With
|
||||
`RX_VK_SUBMIT_DELAY_US`: 7 at 300, 0 at 1000; a repeat at 1000 with shots still had 7 events against 50-60. So it is a race with the game's threads: less overlap, fewer. The delay is a test, not a fix.
|
||||
- The builder's first full build: 43 minutes at below-normal priority, 0 failures; the folder starts from a double click and reaches the title at 60 fps.
|
||||
- Render conditions here gate single lights and read reports earlier in the same stream, so they cannot be read at record time.
|
||||
|
||||
221
docs/development.md
Normal file
221
docs/development.md
Normal file
@@ -0,0 +1,221 @@
|
||||
# Hookshot (oot-recomp)
|
||||
|
||||
**Hookshot** is the port's name (as Skybound is TOTKPort's and BlueShell is mkw-recomp's); the maintainer's repository
|
||||
folder stays `oot-recomp`. The game builds as `hookshot.exe` + `hookshot_vk.dll` (`hookshot` + `hookshot_vk.so` on Linux).
|
||||
|
||||
A static recompilation of the Switch 2 *The Legend of Zelda: Ocarina of Time* remake to native x86-64 Windows and Linux:
|
||||
the game's code lifted to C (NotRexed's a64-kit, Rexcomp's AArch64 kit), the Horizon OS and NVN2 implemented
|
||||
natively, the GPU work replayed on Vulkan. It is a fork of mkw-recomp (`..\mkw-recomp`, Mario Kart World), whose
|
||||
paths it keeps so MKW's fixes apply here.
|
||||
|
||||
**No game files are in this repository.** The dump is read where it lies (`work/dump.txt`); everything made from it
|
||||
(modules, image, lifted C, builds, runs) goes to `work/`, which git ignores.
|
||||
|
||||
Status: the game plays; the README's [status table](../readme.md#project-status) says how far. The design, its stages and
|
||||
the lessons carried over from MKW: [design.md](design.md).
|
||||
|
||||
## Use
|
||||
|
||||
Requirements: Python 3 with `pip install -r requirements.txt` (`lz4`, `numpy`, `capstone`, `unicorn` 2.1,
|
||||
`zstandard`, and the button-prompt packages); a compiler: on Windows llvm-mingw
|
||||
(`HOOKSHOT_MINGW`, or an `llvm-mingw` folder in the clone), else emsdk's clang at `C:\emsdk` (or `EMSDK`) with Visual
|
||||
Studio's C++ tools; on Linux clang and lld. For the Vulkan back end: distro Vulkan / SPIR-V / VMA headers under
|
||||
`/usr`, or `HOOKSHOT_VULKAN_HEADERS` / `HOOKSHOT_SPIRV_HEADERS` / `HOOKSHOT_VMA_HEADERS`, or a CPM package cache
|
||||
(`RX_CPM`, else `cpm` in the clone, else the old caches in mkw-recomp's `work/eg` or `..\Eglantine\.cache\cpm`).
|
||||
Without those the standalone build stops. [CONTRIBUTING.md](../CONTRIBUTING.md#development-setup) has the rest.
|
||||
|
||||
Run everything with `python -I` from this folder.
|
||||
|
||||
| Command | What |
|
||||
|---|---|
|
||||
| `python -I nx2/nup.py <file.nup> --extract work/dump` | the `.nup` dump package to `work/dump/exefs` + `romfs` (newest title; ZBIC NSOs made plain by `nx2/unpack.py`) |
|
||||
| `python -I nx2/dump.py --dump <folder with exefs/ and romfs/> --set` | what the dump is (Switch 1 or 2, ids, modules, SDK, NVN); `--set` records it in `work/dump.txt` |
|
||||
| `python -I nx2/lift.py [--write]` | image + discovery + coverage (`work/image/coverage.txt`); `--write` emits the C to `work/gen` |
|
||||
| `python -I nx2/census.py --nx2` | instruction census per module, after the Switch 2 forms |
|
||||
| `python -I nx2/test_insn.py [--per 6 --states 12]` | every instruction form the game uses, Unicorn vs lifted C |
|
||||
| `python -I nx2/test_leaf.py [--count 3000 --states 4] [--calls]` | random whole functions (or call trees), Unicorn vs recompiled native code on the real image |
|
||||
| `python -I nx2/lift.py --standalone --write` then `python -I nx2/build.py --standalone --jobs 4` | the standalone game: `work/sa/build/hookshot.exe` + `hookshot_vk.dll` |
|
||||
| `powershell -File scripts/run-vk.ps1 [-Window] [-Seconds 120] [-Name x]` | a muted test run on Vulkan, with screenshots in `work/runs/<name>/shots` |
|
||||
| `python -I scripts/prof.py <stacks file>` | a profile sampled with `HOS_PROFILE=<seconds>` + `HOS_SAMPLE=<thread>` + `HOS_SAMPLE_FILE=<stacks file>` (the sampler runs only with `HOS_PROFILE` set), named from the linker maps |
|
||||
|
||||
The dump can also come from `OOT_DUMP` (its folder) or `OOT_EXEFS`; `OOT_WORK` moves the work folder.
|
||||
|
||||
The built game's own arguments (`image.bin`, `--romfs`, `--user`, `--trace`, `--prepare-shaders`) are in the README's
|
||||
[Running the game from the command line](../readme.md#running-the-game-from-the-command-line).
|
||||
|
||||
Scripted test input (`HOS_INPUT_SCRIPT` and `HOS_INPUT_LIVE`) is applied before the window-focus check. A log saying
|
||||
the game window is not in front does not mean those presses were discarded. `HOS_HEADLESS` and
|
||||
`HOS_WINDOW_NOACTIVATE` change window creation, not this input routing. The game must still reach a state that reads
|
||||
and responds to the supplied controller input.
|
||||
|
||||
Physical keyboard and mouse input stays with the foreground window. Physical controllers can also play in the
|
||||
background when the existing "Controller in background" option or `HOS_INPUT_BACKGROUND=1` enables them;
|
||||
`HOS_INPUT_BACKGROUND=0` overrides that option. With `HOS_INPUT_EXCLUSIVE` set (to any value, `0` too) the script
|
||||
alone drives the game until 5 s after its last press; without it the player's devices add to the script's presses. It
|
||||
is not a focus override.
|
||||
|
||||
## Switches for testing and debugging
|
||||
|
||||
The running game reads its switches from the environment or from `hookshot_env.txt` beside `image.bin` (one
|
||||
`NAME=VALUE` a line, `#` for notes; `HOS_ENV_FILE=<path>` reads that file instead). The build reads its own from the
|
||||
environment. The README has the ones players are asked to try (`RX_VK_ASTC_RGBA8`, `RX_VK_LOW_SPEC`,
|
||||
`RX_VK_MOVIE_UPSCALE`, `RX_VK_SKIP_CODE`, `HOS_OOT_CHEATS`). There are several hundred more, most of them one-off
|
||||
experiments: `grep -rn 'getenv("' host gpu/vk` lists them, and the comment where each is read says what it does.
|
||||
"Set" below means any value, `0` included.
|
||||
|
||||
`RX_VK_MESH_NOCULL=<prefix,...>` turns off the culling in the task shaders whose native code key starts with one of
|
||||
the prefixes. On a Steam Deck, `RX_VK_MESH_NOCULL=2c56dc1006942f74` avoids the same hang without leaving anything out:
|
||||
that task shader walks a coarse depth grid cell by cell for every meshlet and ends with
|
||||
`visible = (result & 1) | (c3[2320] & 4)`, bit 4 being the game's own "draw everything". The switch makes that answer
|
||||
true, and dead code elimination removes the grid walk, so every meshlet is drawn and the rasterizer clips what is off
|
||||
screen. It costs some GPU time on cards that cull quickly, so it is off unless set. The log says
|
||||
"culling off, every meshlet drawn" for each task shader it changed.
|
||||
|
||||
### Running a test
|
||||
|
||||
| Switch | What it does |
|
||||
|---|---|
|
||||
| `HOS_HEADLESS` (set) | No window: the renderer draws offscreen, for screenshots and frame dumps |
|
||||
| `HOS_WINDOW_SIZE=<w>x<h>` | The window's size in pixels (1280x720 by default, wider for an ultrawide picture) |
|
||||
| `HOS_WINDOW_NOACTIVATE` (set) | The window opens behind the others and does not take the focus (Windows) |
|
||||
| `HOS_FULLSCREEN=1` | Starts in fullscreen; the Insert menu's Fullscreen row is then left alone |
|
||||
| `HOS_AUDIO=0` | No sound device is opened (the mixer still runs); `scripts/run-vk.ps1` sets it unless `-Sound` |
|
||||
| `HOS_LANGUAGE=<code>` | The game's language for this run, a code from `language.txt` |
|
||||
| `HOS_OUTPUT_RES=<0..3>`, `HOS_HANDHELD=1`, `HOS_ASPECT=<ratio>` | Over the options file: the output resolution (0 2048x1152, 1 2560x1440, 2 3840x2160, 3 1920x1080), the handheld profile, a picture wider than 16:9 (2.3889 for 3440x1440) |
|
||||
| `HOS_DYNRES=0`, `HOS_REALTIME_STEP=0`, `HOS_TEXT_SPEED=<0..3>` | Over the options file: Resolution 100% (non-zero: Auto), Game speed Per frame (1: Real time), Text speed |
|
||||
| `RX_VK_FRAMEGEN=0`, `RX_VK_FRAMEGEN_MULT=<2..8>`, `RX_VK_PACE=0`, `RX_VK_UPSCALE=<0..2>` | No frame generation whatever the menu says; over the options file: its multiplier, frame pacing, the upscaling filter |
|
||||
| `RX_VK_DEVICE=<n>` | The graphics card listed as `vk: device <n>` in the log, over the menu's GPU device |
|
||||
| `HOOKSHOT_NO_UPDATE_CHECK=1` | No look for a newer release at the start |
|
||||
| `HOS_TRACE=1` | Every Horizon call logged, as `--trace` does |
|
||||
| `RX_BACKEND=<name>` | Unset or `vulkan`: `hookshot_vk.dll`; any other value but `eden`: the WebGPU renderer, when the exe was built with one |
|
||||
|
||||
### Input for scripted runs
|
||||
|
||||
| Switch | What it does |
|
||||
|---|---|
|
||||
| `HOS_INPUT_SCRIPT=<presses>` | Presses timed from the game's first controller read |
|
||||
| `HOS_INPUT_LIVE=<file>` | A file the test appends lines of presses to, read every 30 ms; each line's times count from when it is read. A line `!cheat <n>` sends a Cheats page action, `!synclog` logs the NVN syncs (`!dump`, `!shots`, `!rdoc` and `!flasharm` are the WebGPU renderer's) |
|
||||
| `HOS_INPUT_FRAME_CLOCK=1` | The script's time runs at 60 game frames a second instead of the wall clock |
|
||||
| `HOS_INPUT_EXCLUSIVE`, `HOS_INPUT_BACKGROUND`, `HOS_INPUT_SCRIPT_BACKGROUND` (set) | As above; the last lets every device count while the window is behind another one |
|
||||
| `RX_VK_MENU_SCRIPT=<seconds>:<key>,...` | Timed Insert menu keys: `INS`, `UP`, `DOWN`, `LEFT`, `RIGHT`, `ESC`, `ENTER`, `TABL`, `TABR` |
|
||||
| `RX_VK_UI_PROMPTS=pad` or `keys` | The menu's button prompts for a controller or the keys, whatever was used last (screenshots) |
|
||||
| `RX_VK_UI_GAME_FONT=0` | The menu, the counter and the start-up screen in the included Libertinus, not the game's own font |
|
||||
|
||||
The presses are `<seconds>:<button>[+<button>...][/<ms>]`, separated by commas, for example `2:A,5.5:LUP/1500,9:ZL+R`.
|
||||
The buttons are `A B X Y L R ZL ZR PLUS MINUS LS RS LEFT UP RIGHT DOWN`, and the sticks pushed to their edge `LUP
|
||||
LDOWN LLEFT LRIGHT RUP RDOWN RLEFT RRIGHT`. A press is held for 200 ms unless `/<ms>` says otherwise. The names are
|
||||
case-sensitive, and an unknown one is ignored.
|
||||
|
||||
### Pictures and frames
|
||||
|
||||
| Switch | What it does |
|
||||
|---|---|
|
||||
| `RX_VK_SHOTS=<dir>` | A PNG every `RX_VK_SHOT_EVERY` presents (300) as `vk_<present>.png`, at `RX_VK_SHOT_W` x `RX_VK_SHOT_H` (1280x720) |
|
||||
| `RX_VK_FLASH=1` | The flash detector, with `RX_VK_SHOTS` and `RX_VK_SHOT_EVERY=1`: writes the frames around a flash and dumps the next one |
|
||||
| `RX_VK_FRAMELOG=<file>` | Every frame shown, at 320x180, while `<file>.go` exists; `python -I scripts/framelog.py <file>` finds the flashes |
|
||||
| `RX_VK_DUMP=<present>[+<count>][,...]` | Every step of the frame after that present as PNGs, in `dump_<present>` under `RX_VK_DUMP_DIR` (or `RX_VK_SHOTS`, or the current folder); `RX_VK_DUMP_STEPS=<lo>-<hi>` and `RX_VK_DUMP_PROGS=<key>,...` narrow it |
|
||||
| `RX_VK_PASS_TRACE=1` | The passes of the last frames, logged at a flash or a dark frame (`RX_VK_PASS_TRACE_AT=<n>`: every n-th present) |
|
||||
| `RX_VK_PROBE=1` or `2` | The centre texel after every pass: `probe_<frame>.txt` at a flash (1), or `probe_all_<n>.txt` at speed (2); `python -I scripts/probe.py` reads them |
|
||||
| `RX_VK_DUMP_FAILED=<dir>` | The native code of every program that does not translate, for `nx2/lower_test.py` (game data: keep it local) |
|
||||
|
||||
### Time and profiles
|
||||
|
||||
| Switch | What it does |
|
||||
|---|---|
|
||||
| `HOS_PROFILE=<seconds>` | Every interval, the busiest threads in the log; it also starts the samplers below |
|
||||
| `HOS_SAMPLE=<thread>` + `HOS_SAMPLE_FILE=<file>` | One thread's stacks, sampled every millisecond (needs `HOS_PROFILE`); `python -I scripts/prof.py <file>` reads them |
|
||||
| `HOS_SAMPLE_ALL=<file>` | Every game and renderer thread every 2 ms while `<file>.go` exists (needs `HOS_PROFILE`); `scripts/crit.py`, `scripts/vkprof.py` and `scripts/hotlist.py` read it (on Linux with `RX_MAPS=<game folder>`: they name frames from its `hookshot` and `hookshot_vk.so`) |
|
||||
| `HOS_THREAD_TIMES=<seconds>` | Every interval, the threads that used over 5% of a core |
|
||||
| `HOS_FPS_LOG` (set), `HOS_PRESENT_TRACE=<csv>` | The game's frame limiter, VRR flags and nominal rate every 300 presents; a CSV line per game present |
|
||||
| `RX_VK_FRAME_LOG=<csv>` | A CSV line per present: frame, seconds, frame time in ms |
|
||||
| `RX_VK_PASS_TIMING=1` or `2` | GPU time per pass (2: per program too), logged every 600 presents |
|
||||
| `RX_VK_SLOW_LOG=<ms>` | The replay's times for each frame that took longer than this |
|
||||
| `HOS_PGO=<file>` | In a run of an `RX_PGO=gen` build: `<file>.reset` clears the counters, `<file>.dump` writes them to `<file>`. It is read before `hookshot_env.txt`, so set it in the environment |
|
||||
|
||||
### The GPU
|
||||
|
||||
| Switch | What it does |
|
||||
|---|---|
|
||||
| `RX_VK_VALIDATION` (set) | The Khronos validation layer |
|
||||
| `RX_VK_CHECKPOINTS=0` | No NVIDIA checkpoints (on by default; after a lost device they name the draw it stopped at) |
|
||||
| `RX_VK_HANG_DUMP=<file>` | Every thread's stack to the file once no frame was shown for `RX_VK_HANG_SEC` seconds (20) (Windows) |
|
||||
| `RX_VK_NOOPT` (set) | Pipelines built without the driver's optimisations |
|
||||
| `RX_VK_SKIP_PROG=<prefix>,...` | Leaves out the draws of these programs (default `53509044401d78d2`; `-` leaves out none) |
|
||||
| `RX_VK_PIPE_SKIP=0` | A draw whose pipeline is not ready waits for it, instead of being left out when its targets hold an earlier frame |
|
||||
| `RX_VK_PRECOMPILE=0` or `1` | The shader preparation before the title off or on (on in builds made with `RX_RELEASE=1`, as the builders' are) |
|
||||
| `RX_VK_NO_PROGRAM_CACHE` (set) | Neither reads nor writes `hookshot_programs.bin` and the pipeline notes |
|
||||
| `RX_VK_MIRROR=0`, `RX_VK_MIRROR_MB=<MB>` | No VRAM mirrors of the game's memory pools (on for discrete cards); their budget |
|
||||
| `RX_VK_VRAM_MB=<MB>` | The texture cache's budget (default 70% of what the driver allows, at most 6 GB) |
|
||||
| `RX_VK_COMPACT=1`, `RX_VK_COMPACT_PCT=<50..90>`, `RX_VK_COMPACT_MB=<MB>` | VRAM compaction (off by default): the textures that are not render targets in a pool of their own, moved out of sparsely used blocks a few a present so that the blocks go back to the driver, and the texture budget holding the driver's count at that share of what it allows (85); the free space that starts a run (512; 192 while the count is over 90%) |
|
||||
| `RX_VK_NO_WRITEBACK` (set) | GPU writes to mirrored pools are not copied back to guest memory (an experiment, for A/B) |
|
||||
| `RX_VK_WB_DIFF=0` | Write-backs copy every byte again instead of only what changed against a VRAM shadow (the diff is on by default; v2.0.9 and v2.0.10 had it off, issue 1161) |
|
||||
| `RX_VK_DLSS=0/1/2`, `RX_VK_DLSS_PRESET=<letter>`, `RX_VK_DLSS_LOG=1`, `RX_VK_REFLEX`, `RX_VK_REFLEX_LOG=1`, `RX_VK_ANTILAG=0/1` | NVIDIA DLSS and Reflex, and AMD Anti-Lag 2, for one run (`=-` on Reflex or Anti-Lag 2: its extensions not even enabled): the readme's NVIDIA DLSS section. `RX_VK_DLSS_TRACE=<frames>` logs one frame's large draws every that many frames, `RX_VK_DLSS_DEBUG=motion` shows the motion vectors, `=alternate` the game's own upscale every other frame |
|
||||
| `RX_VK_FSR4=0/1/2`, `RX_VK_FSR_DIR=<folder>` | AMD FSR 4 for one run (0 never, 1 FSR 4, 2 its Native AA), and another folder for AMD's DLLs (not from `hookshot_env.txt`): the readme's AMD FSR 4 section. `RX_VK_FSR4=1/2` and `RX_VK_DLSS=1/2` go over the Upscaler row, a saved FSR 3.1 too where the engine they ask for runs; `RX_VK_FSR=1` goes over them (one upscaler at a time). The DLSS trace and debug switches above apply to FSR 4 too, on the frame chain they share |
|
||||
| `RX_VK_ASYNC=1` | The replay on a thread of its own instead of the game's submitting thread |
|
||||
| `HOS_VCORES=0`, `HOS_CORES=<1..8>`, `HOS_RESERVE_CORE=0` | No virtual cores; the cores the game sees (6); no core kept for the replay (one is, with 8 or more logical processors) |
|
||||
|
||||
### Building
|
||||
|
||||
A build reads and writes tens of gigabytes. When the game folder or the build files folder is on a hard drive,
|
||||
`make_game.py` warns that an SSD is much faster and builds anyway (the builders ask before they start).
|
||||
|
||||
| Switch | What it does |
|
||||
|---|---|
|
||||
| `HOOKSHOT_JOBS=<n>` (or `--jobs`) | Compilers at once (default one per CPU thread, each started when the free memory holds it; `HOOKSHOT_MEM_GATE=0` turns that check off) |
|
||||
| `HOOKSHOT_LIFT_JOBS=<n>` | Processes that write the lifted C (default the cores, at most 16) |
|
||||
| `HOOKSHOT_KEEP_BUILD_FILES=1` | Keeps the lifted C and the objects after a build, so that the next one compiles only what changed |
|
||||
| `HOOKSHOT_MINGW=<dir>`, `HOOKSHOT_MSVC=1` | The llvm-mingw to build with; the Visual Studio route instead |
|
||||
| `RX_CPM=<dir>`, `WGPU_NATIVE=<dir>` | The CPM cache (Vulkan and SPIR-V headers, VMA, FFmpeg) and wgpu-native |
|
||||
| `RX_TARGET=windows` | On Linux: the Windows game, with llvm-mingw's Linux build in `HOOKSHOT_MINGW` |
|
||||
| `RX_RELEASE=1` | A release build (`make_game.py` sets it): no console window, the shaders prepared before the title |
|
||||
| `RX_PGO=gen` or `RX_PGO=<file>` | The hot code counting its branches for a profile, or built with one (`make_game.py` passes the shipped profile) |
|
||||
| `RX_MARCH=<level>` | The CPU level to build for (default this PC's: `x86-64-v3`, else `x86-64-v2`, else `x86-64`) |
|
||||
| `RX_VK_ONLY=1` | Only the Vulkan back end, with no lift |
|
||||
| `OOT_WORK`, `OOT_DUMP`, `OOT_EXEFS`, `OOT_ROMFS` | The work folder and the dump's folders, over `work/dump.txt` |
|
||||
|
||||
## The options file
|
||||
|
||||
`hookshot_options.ini`, in the game folder, holds the Insert menu's rows as `<key>=<number>`: the row's name in lower
|
||||
case with `_` for spaces (`frame_generation=2`, `invert_camera_x=1`), except Output resolution, which is `output_res`.
|
||||
`gpu/vk/vk_menu.cpp` writes the whole file at every change (`save`), with `options_version=4`,
|
||||
`microphone_device=<name>` when a microphone is chosen by name, and the controls (`pad_<button>` and `key_<button>`:
|
||||
the README's [Controls](../readme.md#controls)). A value outside a row's range is clamped when it is read. Where the
|
||||
number is not the obvious one:
|
||||
|
||||
| Key | Values |
|
||||
|---|---|
|
||||
| `quality_preset` | 0 Custom, 1 Minimum (4GB), 2 Low (6GB), 3 Balanced (8GB), 4 High (12GB), 5 Ultra (16GB+) |
|
||||
| `gpu_device` | 0 Auto, n the n-th card with Vulkan 1.3 as the row lists them |
|
||||
| `frame_generation` | 0 Off, 2 to 8 the multiplier |
|
||||
| `resolution` | 0 Auto, 1 100%, 2 75%, 3 67%, 4 50% |
|
||||
| `output_res` | 0 2048x1152, 1 2560x1440, 2 3840x2160, 3 1920x1080, 4 5120x1440, 5 5120x2880 |
|
||||
| `upscaling` | 0 Off, 1 Catmull-Rom, 2 Lanczos |
|
||||
| `sharpness`, `rumble_strength`, `motion_sensitivity_x`, `motion_sensitivity_y` | percent |
|
||||
| `mouse_sensitivity` | hundredths: 100 is 1.00 |
|
||||
| `robust_buffers` | 0 Auto (on), 1 Off, 2 On |
|
||||
| `game_speed` | 0 Per frame, 1 Real time |
|
||||
| `text_speed` | 0 Normal, 1 2x, 2 4x, 3 Instant |
|
||||
| `microphone` | 0 Off, 1 Default, 2 and up a microphone (the name in `microphone_device` decides which) |
|
||||
| `mouse_camera` | 0 As the stick, 1 Native |
|
||||
| `layout_preset` | 0 Nintendo, 1 Xbox / PC, 2 Swap Face, 3 Custom |
|
||||
|
||||
The other rows are 0 Off and 1 On. Some are read at the start, before the menu exists, by the code that needs them:
|
||||
`handheld_profile` and `output_res` (`host/nn_applet.c`), `ultrawide` (`host/hos_main.c`), `robust_buffers`
|
||||
(`gpu/vk/vk_main.cpp`) and the controls (`host/nn_hid.c`).
|
||||
|
||||
## Layout
|
||||
|
||||
| Path | What |
|
||||
|---|---|
|
||||
| `docs/design.md` | why mkw-recomp's pipeline, the architecture, MKW's lessons as rules, stages, testing |
|
||||
| `nx2/` | the lift pipeline and its tests (mkw-recomp's, made game-neutral; `kit.py` has the paths), and `make_game.py`, which the builders run |
|
||||
| `host/` | the standalone Horizon runtime (TOTKPort's `hos/` as adapted by mkw-recomp), with the native NVN2 layer `nvn_native.c`; `host/posix` has the Win32 calls on Linux |
|
||||
| `gpu/vk/` | the Vulkan back end (`hookshot_vk.dll`, `hookshot_vk.so`), with Eden's shader recompiler and sirit |
|
||||
| `gpu/render/`, `gpu/third_party/` | the WebGPU renderer the Windows exe falls back to when it was built with wgpu-native, its op stream (`rx_nvn.h`), suyu's decoders |
|
||||
| `builder/` | Hookshot Builder: the Windows app (.NET 8 WPF), its setup and MSI, and the Linux builder and launcher scripts |
|
||||
| `scripts/` | test runs, the profile and log readers, the save copier (`oot_saves.py`), release tooling, `scripts/linux` (dependencies, AppImage) |
|
||||
| `tests/` | the game-free checks, all run by `tests/run_all.py` |
|
||||
| `third_party/` | a64-kit (Hookshot's few changes marked `(hookshot)`), libopus, zbic (Atmosphere's zstd with ZBIC, for unpacking only), SDL2's licence, Dear ImGui and its Vulkan backend, zstd's decoder, the Libertinus fonts |
|
||||
|
||||
License: GPL-3.0-or-later, as mkw-recomp: a64-kit's translator is suyu's (GPL-2.0-or-later) and Eden's code is
|
||||
GPL-3.0.
|
||||
26
docs/draw-stream-visibility.md
Normal file
26
docs/draw-stream-visibility.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# Draw-stream visibility
|
||||
|
||||
A recording's CPU-written uniform, vertex/index conversion and buffer-copy streams
|
||||
are published through VMA before their commands reach the GPU. Host-visible
|
||||
sequential-write allocations need not be coherent; queue submission alone does not
|
||||
flush their CPU caches.
|
||||
|
||||
The flush names the recording slot, including slots filled by worker threads. It
|
||||
covers each stream used before growth and the current used prefix. Earlier stream
|
||||
capacities are flushed in full (including unused padding); later retained streams
|
||||
from an earlier recording are left alone. Existing VMA cache-range alignment and
|
||||
coherent-memory handling apply. No per-draw flush or new allocation policy is added.
|
||||
|
||||
Both ordinary/batched submissions and successful parallel segments publish their
|
||||
streams. Failed segments keep their existing no-submit policy. Descriptor lifetime,
|
||||
slot completion/reuse and batching behavior remain unchanged. The internal draw
|
||||
helper does not change the host/renderer API.
|
||||
|
||||
`tests/test_draw_stream_visibility.py` runs production growth/reservation/copy,
|
||||
recording reset and complete serial submit code, plus the actual parallel segment
|
||||
submission tail, with independent expected GPU input bytes and separate host
|
||||
caches. Worker-to-main slot identity, retained-stream reuse, failed growth, empty
|
||||
recordings, batching and skipped segments are covered. Vulkan queue/fence behavior
|
||||
is modeled; this is not physical noncoherent hardware, a concurrency sanitizer,
|
||||
shader execution or campaign coverage. Global fallback buffers and mixed CPU/GPU
|
||||
argument rings have separate lifetimes and are outside this stream contract.
|
||||
38
docs/dump-readback.md
Normal file
38
docs/dump-readback.md
Normal file
@@ -0,0 +1,38 @@
|
||||
# Frame dump readback visibility
|
||||
|
||||
Frame dumps are diagnostic inputs: their statistics, images and raw float data
|
||||
must describe the completed GPU copy. `vk_dump.cpp` allocates mapped staging
|
||||
buffers with `create_buffer`, which does not require a host-coherent memory type.
|
||||
Waiting for completion alone does not establish every memory-domain/cache step.
|
||||
|
||||
Each dump copy records a COPY/TRANSFER_WRITE to HOST/HOST_READ dependency. The
|
||||
existing replay caller flushes/waits its submission outside an open burst before
|
||||
calling `dump_write`. The consumer then invalidates the copied allocation range
|
||||
before reading pixels or storage-buffer words. VMA handles non-coherent atom
|
||||
alignment and makes invalidation a no-op where host coherence permits it.
|
||||
|
||||
Typed color copies contain RGBA32 float, unsigned or signed integer texels.
|
||||
Depth and RGB9E5 copies retain their original dimensions and two/four-byte packed
|
||||
texels until invalidation; depth conversion may then downscale or replace the
|
||||
mapped pointer with temporary float output. Invalidating after that conversion
|
||||
would already be too late and could use the wrong range. Buffer dumps invalidate
|
||||
their actual capped, word-aligned copied size, including the final word.
|
||||
|
||||
This changes only the opt-in dump path. Existing GPU barriers, submission waits,
|
||||
allocation policy, file layout, numeric conversions and cleanup are preserved.
|
||||
It does not make PNG/raw files a single atomic transaction or change generic
|
||||
device-loss/wait-error handling. The readback contract assumes successful GPU
|
||||
completion, as the existing dump caller does.
|
||||
|
||||
`tests/test_dump_readback.py` compiles the actual dump translation unit, actual
|
||||
full-barrier implementation and dump consumer call into an authored CPU model.
|
||||
Its 56 scenarios cover coherent/non-coherent caches, explicit/permissive host
|
||||
visibility, typed and packed formats, downscaled depth, buffer tail, raw float
|
||||
data, statistics-only output, open-burst deferral and cleanup. Copy data, waits,
|
||||
cache behavior and Vulkan calls are modeled; PNG encoding is outside this test.
|
||||
This is protocol/numeric regression coverage, not proof of physical non-coherent
|
||||
hardware, driver behavior, campaign rendering or a performance improvement.
|
||||
|
||||
References: [Khronos readback example](https://docs.vulkan.org/guide/latest/synchronization_examples.html#cpu-read-back-of-data-written-by-a-compute-shader),
|
||||
[Vulkan memory domains](https://docs.vulkan.org/spec/latest/appendices/memorymodel.html),
|
||||
[VMA mapped-memory cache management](https://gpuopen-librariesandsdks.github.io/VulkanMemoryAllocator/html/memory_mapping.html).
|
||||
22
docs/flash-sampling.md
Normal file
22
docs/flash-sampling.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# PNG flash sampling
|
||||
|
||||
`RX_VK_FLASH` compares a normal screenshot against its two neighbors using a64x36
|
||||
RGB grid. Each cell covers its normalized source rectangle, including remainder
|
||||
rows and columns when the capture dimensions do not divide the grid evenly.
|
||||
Sampling retains the existing four-pixel stride. Its mean divides by the actual
|
||||
number of visited samples, so a partial stride cannot brighten or wrap a channel.
|
||||
Captures smaller than the grid repeat a valid source pixel for an otherwise empty
|
||||
cell. Valid capture dimensions are positive; this does not change allocation or
|
||||
dimension-validation policy.
|
||||
|
||||
The cell/region detector, thresholds and output filenames are unchanged. This is
|
||||
a correction to diagnostic brightness, not a change to game rendering, FOV or HUD.
|
||||
For example, a constant white3440x1440 capture used to become18 in the first grid
|
||||
cell after overflow into uint8; its sampled channels now stay255. A below-threshold
|
||||
brightness variation must not become a flash through that wrap.
|
||||
|
||||
`tests/test_flash_sampling.py` compiles the actual sampling and downstream detector
|
||||
code with synthetic RGBA images. It checks constant channels at standard, wide,
|
||||
odd and small sizes, labels source regions independently to detect partition drift,
|
||||
and exercises stable images, a moving edge, a real returning brightness pulse and
|
||||
a below-threshold negative. No game, GPU or image library is needed.
|
||||
31
docs/framegen-resource-failure.md
Normal file
31
docs/framegen-resource-failure.md
Normal file
@@ -0,0 +1,31 @@
|
||||
# Frame interpolation setup failures
|
||||
|
||||
Frame interpolation is optional. A failed pipeline, sampler, image or image-view
|
||||
creation must leave it disabled for the session, with its unused resources released
|
||||
and ordinary presentation still available. An unchecked sampler result must never
|
||||
be advertised as successful initialization.
|
||||
|
||||
`vk_framegen.cpp` constructs each pipeline and image locally, publishing the complete
|
||||
object only on success. Failure releases its intermediate resources. Interpolation
|
||||
initialization also releases earlier successful pipelines/samplers when a later
|
||||
stage fails. The separate window-scaling path uses the same pipeline constructor
|
||||
but retains its own state.
|
||||
|
||||
Resize keeps the existing display-job drain and queue/device-idle sequence before
|
||||
replacing images. If replacement fails, all newly allocated interpolation images,
|
||||
its setup objects and the published size/multiplier/history are cleared. Size and
|
||||
multiplier are committed only after every required image/view exists. Interpolation
|
||||
keeps the existing permanent-failure policy; this adds no per-frame retry loop,
|
||||
new setting, allocation budget or change to interpolation quality.
|
||||
The disabled flag is atomic: the replay thread writes it, while Reflex's
|
||||
game-thread eligibility check can read it through `framegen_on()`.
|
||||
|
||||
`tests/test_framegen_resources.py` compiles the real constructors and lifecycle
|
||||
functions with CPU Vulkan stubs and checks each creation failure, ownership order,
|
||||
no duplicate destruction, stale metadata, resize draining and the direct-presentation
|
||||
gate. It covers fixed2x and max8x image sets and the independent scale initialization.
|
||||
It needs Vulkan headers but no GPU, SDK vendor runtime or game assets.
|
||||
|
||||
Injected failures in these tests do not prove actual driver OOM recovery or
|
||||
device-loss handling. A successful game smoke validates integration, not a fault
|
||||
injection into the player's GPU.
|
||||
25
docs/framelog-sink-failures.md
Normal file
25
docs/framelog-sink-failures.md
Normal file
@@ -0,0 +1,25 @@
|
||||
# Frame-log sink failures
|
||||
|
||||
The optional binary frame log is ready only after its image, three mapped readback
|
||||
buffers, output file and writer thread are available. A failed startup disables
|
||||
logging for that process, reports the failing stage and releases the newly created
|
||||
resources and file. It does not announce successful recording or retry every frame.
|
||||
|
||||
The worker checks both record writes and flushes. A short write or failed flush
|
||||
disables further capture, discards queued CPU records, reports the error and closes
|
||||
the file once. Enqueue checks the same state under the queue lock, so a producer
|
||||
arriving after failure cannot leave records behind a worker that has exited.
|
||||
Readbacks already in flight retain their GPU resources for the existing process
|
||||
lifetime; they are not destroyed underneath submitted work.
|
||||
|
||||
Successful output keeps the existing record format, per-record flush and512record
|
||||
queue bound. After a failed write the last record may be incomplete: the strict
|
||||
reader rejects truncation rather than treating the file as a complete comparison.
|
||||
This does not add renderer shutdown/draining or guarantee durability at process
|
||||
exit; the healthy writer retains its existing process-lifetime behavior.
|
||||
|
||||
`tests/test_framelog_sink.py` executes production startup/cleanup, worker and enqueue
|
||||
code. Only GPU/IO calls and the startup thread factory are stand-ins. Cases cover
|
||||
allocation and missing-map failures, file/thread creation failure, repeated ready
|
||||
queries, real CPU-thread short-write/flush failures, late producers, unchanged
|
||||
record bytes and queue bounds. Tests require no game or GPU.
|
||||
18
docs/linux-update-sysroot.md
Normal file
18
docs/linux-update-sysroot.md
Normal file
@@ -0,0 +1,18 @@
|
||||
# Sysroot library aliases in Linux update parts
|
||||
|
||||
The AppImage's selected system libraries are copied by value into its sysroot.
|
||||
Some host distributions make a development alias such as libmvec.so an absolute
|
||||
symlink. Keeping that target would either depend on the player's host filesystem
|
||||
or be rejected by a strict updater. Dereferencing the
|
||||
source alias while copying preserves its bytes and attributes inside the part.
|
||||
|
||||
The internal relative lib -> usr/lib sysroot layout remains a link. Existing
|
||||
parts carrying absolute links are not retroactively rewritten. For legacy
|
||||
sysroot parts only, the updater rebases absolute links to relative links when
|
||||
their targets exist and resolve inside that sysroot. Absolute links in other
|
||||
parts, missing or circular targets, and paths that escape the sysroot remain
|
||||
errors.
|
||||
|
||||
Tests execute the production copy loop with synthetic library inputs, then
|
||||
package and unpack the resulting sysroot, alongside legacy-link and
|
||||
hostile-archive regression cases.
|
||||
27
docs/pipeline-cache-save-check.md
Normal file
27
docs/pipeline-cache-save-check.md
Normal file
@@ -0,0 +1,27 @@
|
||||
# Pipeline cache save failures
|
||||
|
||||
`pipeline_cache_save` writes a temporary file before replacing the existing cache. Allocation failures in its
|
||||
background worker must stay in that worker, release its busy flag and leave later saves possible. Starting the
|
||||
worker can fail too. The synchronous prepare-only save follows the same exception handling.
|
||||
|
||||
Run the game-free check with a C++20 compiler and its runtime libraries available:
|
||||
|
||||
```sh
|
||||
python -I -B tests/test_pipeline_cache_save.py --cxx clang++
|
||||
```
|
||||
|
||||
The fixture compiles the actual function with Vulkan API spies, real C++ containers/threads, and authored temporary
|
||||
file bytes. It injects allocation failures into the vector, temporary path and thread startup; an oversized vector
|
||||
checks `length_error`. Each fault case checks that the old file survives and a subsequent save completes. Success,
|
||||
the 32-pipeline threshold, a single forced-waiter scheduling smoke, Vulkan empty/error returns and a failed write
|
||||
are covered too. The waiter smoke is not a deterministic concurrency proof.
|
||||
|
||||
`--baseline-source /path/to/original/vk_draw.cpp` additionally compares the original success path and runs six
|
||||
original-failure negatives in separate, bounded processes. Uncaught worker exceptions terminate those children;
|
||||
an independent authored thread control identifies the exact MSVC fail-fast result where its custom terminate
|
||||
handler is bypassed. Fixed children must always exit successfully.
|
||||
|
||||
These checks use no driver or real cache and do not reproduce issue #1022's reported session. They do not establish
|
||||
its root cause, validate saving a multi-gigabyte driver cache, force an OS thread-resource failure, or change cache
|
||||
size/retry policy. The existing wait-then-store protocol for concurrent forced callers is unchanged; the current
|
||||
force call is in prepare-only startup. Driver/native faults and thread-detach failures are outside this change.
|
||||
34
docs/png-publication.md
Normal file
34
docs/png-publication.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Complete PNG publication
|
||||
|
||||
Screenshot, flash and frame-dump writers can run concurrently. The PNG writer
|
||||
previously built a shared CRC table without synchronization, ignored write/close
|
||||
errors, and opened the destination directly. A failed capture could replace an
|
||||
older complete file, while the normal capture still announced success.
|
||||
|
||||
The CRC table is now immutable after thread-safe local initialization. PNG sizes
|
||||
are checked before encoding. The encoded data is written to an exclusively
|
||||
created temporary in the destination directory; every write and the final close
|
||||
must succeed before replacement. Failure preserves the prior destination, closes
|
||||
the owned handle and attempts to remove only this call's temporary. Temporary
|
||||
name collisions are skipped without truncating other files. Diagnostics identify
|
||||
the phase, error code and path, with a bounded report count. A later capture can
|
||||
retry normally. Only successful normal captures announce that they were written.
|
||||
|
||||
Each PNG is published independently; a flash triplet is not a multi-file
|
||||
transaction. Same-directory replacement targets complete-file visibility on
|
||||
supported local filesystems, not power-loss durability or network-share
|
||||
transaction guarantees. The encoder's pixels, alpha omission, compression and
|
||||
capture scheduling are unchanged. Host/renderer ABI is unchanged.
|
||||
|
||||
The game-free test compiles the real CRC/writer and normal success-announcement
|
||||
body. Authored CRT/file replacement failures cover each write, close, publication,
|
||||
cleanup and open; old destinations and unrelated colliding temporaries must
|
||||
survive. Concurrent first use and competing writers produce independently decoded
|
||||
PNG chunks, CRCs, zlib data and RGB pixels, including multiple stored blocks.
|
||||
The existing capture dimension/PNG consumer check is retained with its new return
|
||||
type and required headers. Synthetic failures are not a physical disk-fault claim.
|
||||
|
||||
References: [PNG specification](https://www.w3.org/TR/png-3/),
|
||||
[exclusive fopen mode](https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/fopen-wfopen?view=msvc-170),
|
||||
[Windows replacement API](https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-movefileexa).
|
||||
Windows narrow paths continue to use the game's existing UTF-8 manifest policy.
|
||||
26
docs/presentation-device-loss.md
Normal file
26
docs/presentation-device-loss.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# Device loss during native presentation
|
||||
|
||||
The replay already treats `VK_ERROR_DEVICE_LOST` as terminal: `device_lost()`
|
||||
writes its checkpoint report once, then `replay_fatal()` flushes the host log
|
||||
before exiting. Continuing cannot complete GPU work and can retain captures and
|
||||
staging allocations indefinitely.
|
||||
|
||||
Native presentation uses the same policy for the frame-fence wait/reset,
|
||||
command-pool reset, command-buffer begin/end, image acquisition, blit submission
|
||||
and presentation. The check precedes dependent work: a failed fence wait cannot
|
||||
lead to readback, a failed submission cannot enqueue a present waiting for its
|
||||
unsignaled semaphore, and a failed present cannot be counted as completed by
|
||||
Reflex. Both the shared and dedicated presentation queues are covered.
|
||||
|
||||
This does not recover a lost device, change successful presentation, or replace
|
||||
the existing out-of-date/suboptimal/Wayland-timeout handling. Other error classes
|
||||
retain their existing behavior, including allocation errors; this is not a
|
||||
general submission-recovery mechanism.
|
||||
|
||||
`tests/test_present_device_loss.py` executes the actual call/guard sites with
|
||||
injected return values. It covers nine sites (two presentation queues), checks
|
||||
that loss reports before the site continues, and supplies non-loss controls.
|
||||
It tests routing rather than simulating an entire Vulkan implementation. The
|
||||
existing `test_device_lost_exit.py` separately exercises the real concurrent
|
||||
report-once and host-log-flush/exit contract. Hardware-reset recovery is not
|
||||
claimed, and no physical driver reset is needed to run these tests.
|
||||
36
docs/presentation-geometry-trace.md
Normal file
36
docs/presentation-geometry-trace.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# Effective presentation geometry diagnostics
|
||||
|
||||
Set `RX_VK_PRESENT_LOG=1` before launching to log presentation geometry changes.
|
||||
Unset it or set it to `0` to disable it. This writes ordinary diagnostic lines;
|
||||
it does not enable frame-log image readback or change rendering/settings.
|
||||
|
||||
Each sample reports texture size, supplied crop request, clipped content size,
|
||||
swapchain output size, the actual destination rectangle, observed host aspect,
|
||||
sampled-view availability, and the route selected by presentation:
|
||||
|
||||
- `direct`: the source blit was selected, including fallback when filtering returned no image.
|
||||
- `scaled`: the scaling function returned an image and its result was selected.
|
||||
- `clear`: presentation had no source and selected its black clear.
|
||||
- `hdr`: the HDR conversion function returned an image selected for presentation.
|
||||
A failed converter falls back to the scaled/direct route. Native PQ input bypasses conversion.
|
||||
|
||||
Filter, sharpness and frame-generation values are labeled `*-request`: configuration
|
||||
does not establish that a feature ran successfully. The route is a CPU command-recording
|
||||
decision, not a successful GPU present result. The host-aspect field reuses the exact
|
||||
aspect passed to the destination fit, without a second callback read. No-source frames
|
||||
report it unavailable, since fitting deliberately does not query the callback then.
|
||||
This diagnostic does not alter aspect policy.
|
||||
An unavailable/invalid aspect is explicit; invalid values are normalized for stable deduplication.
|
||||
Requests accompany geometry/route changes but do not themselves trigger a line, so dragging
|
||||
a sharpness slider cannot consume the geometry-change budget while the effective route stays the same.
|
||||
|
||||
Repeated unchanged samples are suppressed independently for game, interpolated (`made`)
|
||||
and startup/clear frames, so alternating ordinary/interpolated sources does not spam a
|
||||
line per frame. Up to 256 geometry changes are logged per process, followed by one
|
||||
suppression notice. Restart to reset the limit. Continuously changing dynamic resolution
|
||||
can reach that limit; this is a bounded diagnostic, not a full timeline capture.
|
||||
|
||||
`tests/test_present_geometry_trace.py` executes the production presentation-route and
|
||||
trace blocks with CPU command spies. It checks effective crop/rectangle/output/options,
|
||||
resize, direct fallback/scaled/clear routes, independent frame kinds, disabled/zero
|
||||
configuration, invalid aspects and the output bound. No GPU or game data is required.
|
||||
55
docs/presentation-geometry.md
Normal file
55
docs/presentation-geometry.md
Normal file
@@ -0,0 +1,55 @@
|
||||
# Presentation geometry
|
||||
|
||||
The texture allocation, its active crop, the display aspect of that picture and
|
||||
the swapchain size are separate inputs. `gpu/vk/vk_present_rect.h` is the pure
|
||||
geometry contract used by the Vulkan presentation path.
|
||||
|
||||
- Without a valid `PresentAspect`, fit the clamped crop's pixel aspect, preserving
|
||||
the existing integer rounding and invalid-source fallback.
|
||||
- A positive, finite `PresentAspect` describes the **presented picture after
|
||||
crop**, not the texture allocation. It can differ from the allocation's aspect:
|
||||
a wide camera picture in a 16:9 texture must still fill its intended wide window.
|
||||
- Fit that display aspect into the current destination and center any bars. Round
|
||||
the logical fit to the nearest pixel; a float camera ratio must not introduce a
|
||||
one-pixel bar when the destination already has its intended shape. A nonempty
|
||||
destination retains at least one pixel per axis at extreme ratios.
|
||||
- Zero, negative, NaN and infinite overrides fall back to the physical content
|
||||
aspect. The callback is read once per present; no host/backend ABI field is added.
|
||||
|
||||
For example, a 43:18 picture fills 1720x720. In a 1920x1009 destination it occupies
|
||||
1920x804, with 102 pixels above and 103 below, even if the physical source texture
|
||||
is 1920x1080. Changing the destination does not change the camera or render targets.
|
||||
|
||||
Normal and interpolated frames share `present_image`. Interpolation materializes
|
||||
the crop into a separate image, but keeps the same display aspect. The scaling
|
||||
filter receives the content rectangle's dimensions; the blit uses that same
|
||||
rectangle and the existing clear path covers its bars. The Insert overlay remains
|
||||
in swapchain coordinates. Internal screenshots retain their separate capture
|
||||
dimensions and are not post-presentation evidence of letterboxing.
|
||||
|
||||
Game-free checks:
|
||||
|
||||
```sh
|
||||
python -I -B tests/run_all.py present_rect test_present_aspect.py output_resolution --strict
|
||||
```
|
||||
|
||||
`present_rect_test.cpp` covers fit, crop, invalid values, float rounding and bounded
|
||||
centered rectangles across extreme sizes. `test_present_aspect.py` executes the
|
||||
production geometry/blit-selection block with recording command stubs: host aspect,
|
||||
normal/generated images, filtered/direct blits, bar clears and restore. It does not
|
||||
execute Vulkan; backend compilation and a synthetic GPU/window integration run are
|
||||
separate validation layers.
|
||||
|
||||
An optional GPU check (Vulkan 1.3 loader/device and `HOOKSHOT_VULKAN_HEADERS` needed):
|
||||
|
||||
```sh
|
||||
python -I -B tests/run_all.py present_aspect_vulkan --strict
|
||||
```
|
||||
|
||||
It executes the production transfer/present-selection block offscreen, reads the
|
||||
pixels back and checks content and black bars against fixed expected rectangles.
|
||||
Cases include matching wide output, maximize/restore geometry, a padded active crop
|
||||
and invalid aspect fallback. The padded-crop check excludes only the scaled
|
||||
half-source-texel footprint of linear filtering at content edges; every bar pixel
|
||||
is checked. It does not validate swapchain acquisition, a window/DPI transition,
|
||||
the interpolation shader or the upscaler. Ordinary CI skips this hardware check.
|
||||
25
docs/presentation-source-crop.md
Normal file
25
docs/presentation-source-crop.md
Normal file
@@ -0,0 +1,25 @@
|
||||
# Presentation source crops
|
||||
|
||||
Direct presentation, capture, scaling and interpolation must sample the same valid
|
||||
source region. `present_source_rect` is the common contract, independent of Vulkan:
|
||||
|
||||
- A null crop, or either nonpositive crop extent, selects the entire image, preserving
|
||||
the existing convention even when the unused origin fields are negative.
|
||||
- An active crop with a negative or out-of-image origin is empty.
|
||||
- Positive extents are clipped to the remaining source area by subtraction before
|
||||
addition. `x + width` in guest input must not overflow a signed integer.
|
||||
- Empty sources and dimensions beyond signed 32-bit image offsets are rejected.
|
||||
|
||||
The content-aspect calculation uses the same clipped region. Direct/capture blits
|
||||
clear their destination to opaque black for an empty source region instead of issuing
|
||||
an invalid Vulkan blit or copying undefined pixels. Zero destination extent records
|
||||
nothing. Scaling/interpolation decline an invalid crop before setup or allocation;
|
||||
ordinary presentation supplies the black fallback. Valid crops retain their offsets,
|
||||
filter and destination rectangle.
|
||||
|
||||
`tests/test_present_source_crop.py` executes the production blit, capture wrapper and
|
||||
complete interpolation recorder with CPU API spies. It checks the command offsets,
|
||||
interpolation copy push constants, clipped allocation size, invalid-input clear/skip,
|
||||
full-image convention and thousands of deterministic edge-crossing crops. It needs
|
||||
Vulkan headers and a C++20 compiler, but no GPU or game data. The independent scaler
|
||||
fixture separately executes the production scaling function.
|
||||
45
docs/recording-admission.md
Normal file
45
docs/recording-admission.md
Normal file
@@ -0,0 +1,45 @@
|
||||
# Per-recording allocation admission and retry
|
||||
|
||||
Recording streams carry CPU-authored shader banks, copied UBOs and generated
|
||||
indices. Their buffer, VMA allocation, mapping and ownership entry must all exist
|
||||
before a reservation can authorize a CPU write or return a binding.
|
||||
|
||||
`stream_insert` commits a complete mapped stream to its recording's container.
|
||||
Failed creation, missing mapping or container allocation releases only the new
|
||||
resource. Initial stream creation is independent of descriptor-pool retry, so a
|
||||
failed pool cannot append another primary stream on every recording. A failed
|
||||
primary stream remains absent and can be retried on the slot's next recording.
|
||||
The active stream is explicitly empty until that succeeds; reservations reject
|
||||
it rather than treating its requested capacity as allocated memory.
|
||||
|
||||
Growth preserves earlier streams because already-recorded bindings reference
|
||||
them. Its cursor, ownership and counters change only after the new entry commits.
|
||||
Reservation and MiB rounding check representability before arithmetic can wrap.
|
||||
Successful growth sizes, cached-stream reuse, initial sizes, retention and the
|
||||
opt-out policy are preserved.
|
||||
|
||||
Driver-bank cache generations and grid metadata commit after a successful
|
||||
upload, or after a confirmed identical-byte cache hit. Failed reservation keeps
|
||||
the previous generation attached to its previous bytes, so retry rebuilds the
|
||||
changed bank instead of incorrectly treating stale bytes as current.
|
||||
|
||||
The descriptor allocation fallback rejects a missing pool. A new extra pool is
|
||||
owned only after its first set allocation and the ownership entry both succeed.
|
||||
Failure destroys that new pool, including any set allocated from it, before any
|
||||
descriptor update/bind occurs. Existing pools remain valid for retry; a missing
|
||||
primary pool can be retried by the next recording. Push-descriptor behavior and
|
||||
the renderer interface are unchanged.
|
||||
|
||||
`tests/test_recording_admission.py` executes actual allocation/reservation/reset
|
||||
functions and the descriptor allocation tail against authored VMA/Vulkan calls.
|
||||
Only the container boundary is adapted to inject allocation exceptions. Cases
|
||||
cover healthy byte copies, initial/growth failures, absent mappings, commit
|
||||
failure, pool/set failure, retry, old binding/pool preservation, parallel slots,
|
||||
aligned bounds, overflow, disabled growth and retention. Global defaults are
|
||||
seeded outside this per-recording contract. The tests model resource failures;
|
||||
they do not prove native driver OOM recovery, campaign behavior or performance.
|
||||
Global default images/buffers, device loss and other resource families retain
|
||||
their existing behavior. No cache visibility policy is changed here.
|
||||
|
||||
References: [VMA allocation ownership](https://gpuopen-librariesandsdks.github.io/VulkanMemoryAllocator/html/group__group__alloc.html)
|
||||
and [Vulkan descriptor allocation](https://docs.vulkan.org/refpages/latest/refpages/source/vkAllocateDescriptorSets.html).
|
||||
49
docs/sampler-admission.md
Normal file
49
docs/sampler-admission.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# Sampler cache admission and retry
|
||||
|
||||
NVN samplers pass through an ID/generation table, a thread-local parameter/handle
|
||||
cache and a global owning cache. Ignoring `vkCreateSampler` failure could admit
|
||||
an undefined error-output handle, or retain null in those caches indefinitely.
|
||||
Checking only the factory result would still let the outer ID table retain null.
|
||||
|
||||
Only successful handles now enter the owning or acceleration caches. The ID
|
||||
table does not mark a failed creation resolved for that registration generation.
|
||||
Failure of one key is bounded to one attempt per present; later presents can
|
||||
retry the same ID, including on a fresh thread. Successful hits and unrelated
|
||||
keys remain usable. Failure keys are cleared at a later present. If failure
|
||||
tracking itself cannot allocate, new keys wait until the next present, while
|
||||
already-owned successful handles still work.
|
||||
|
||||
A newly created sampler whose global cache insertion throws is destroyed before
|
||||
returning null. An error-output handle is never returned or destroyed. The
|
||||
parameter and thread-local handle tables are acceleration caches: allocation
|
||||
failure there does not discard registered parameters or destroy a valid global
|
||||
handle. Existing registration/version checks, immutable successful sampler
|
||||
lifetimes, filter/LOD/compare/anisotropy/border/reduction policies and the
|
||||
renderer API are unchanged. Existing replay locking serializes the shared cache
|
||||
and failure tracking; no new worker or GPU wait is added.
|
||||
|
||||
Before either pushing descriptors or allocating/updating a descriptor set,
|
||||
`bind_descriptors` rejects a combined-image-sampler array containing a failed
|
||||
sampler. It leaves the pushed-state cache untouched and returns failure to
|
||||
the existing draw, compute and mesh-fallback callers. A later rebuild can retry;
|
||||
no replacement sampler changes the requested filtering policy. Storage images
|
||||
continue to accept their unused null sampler.
|
||||
|
||||
`tests/test_sampler_admission.py` executes the complete actual sampler factory,
|
||||
ID-cache consumer and parameter policy helpers, using the public NVN parameter
|
||||
layout. Vulkan, registration/environment/hash inputs and the replay-lock boundary
|
||||
are modeled. Standard containers use tagged throwing allocators to target each
|
||||
cache operation. The owning-cache fault is armed after successful resource
|
||||
creation, independent of an STL's sentinel/bucket allocation choices. The 108
|
||||
cases cover failure/retry/cache chains, ownership and allocation rollback,
|
||||
independent keys, registration changes, threads and unchanged sampler policies.
|
||||
The actual complete descriptor-binding function and fingerprint helpers run
|
||||
against modeled Vulkan calls too: cold and warm failures in an array's second
|
||||
element, graphics/compute, pooled sets, both pushed-state layouts, full pushes,
|
||||
next-present recovery and storage images. Rejected writes must produce no
|
||||
allocation, update, bind or push and must preserve the prior fingerprint.
|
||||
|
||||
Run `python -I -B tests/run_all.py sampler_admission --strict`, or invoke the
|
||||
test with `--headers <Vulkan include folder> --cxx <compiler>`; `--sanitize` is
|
||||
supported on Linux. Game-free failure models do not prove native OOM/device-loss
|
||||
recovery, image-quality changes, a full campaign or performance improvements.
|
||||
120
docs/saves.md
Normal file
120
docs/saves.md
Normal file
@@ -0,0 +1,120 @@
|
||||
# Using a save from an emulator
|
||||
|
||||
Hookshot and the emulators and ports that run the Switch 2 Ocarina of Time on a PC all keep the game's saves as the
|
||||
**same files**. You can carry on in Hookshot from a save made in Lichii (or Eglantine, Orchid, NX2navi, ...), or take
|
||||
your Hookshot save to one of them, at the same spot: nothing is converted, the files are copied across.
|
||||
`scripts/oot_saves.py` finds the folders and does the copy safely.
|
||||
|
||||
You need your own copy of the game. Nothing here
|
||||
comes with game files.
|
||||
|
||||
## The files
|
||||
|
||||
Every one of them keeps one folder per user profile, holding:
|
||||
|
||||
```
|
||||
account.sav
|
||||
option.sav (the game's own settings)
|
||||
slot_00/ ... slot_NN/ (one per save slot)
|
||||
progress.sav
|
||||
caption.sav
|
||||
```
|
||||
|
||||
Only the folder they are in differs.
|
||||
|
||||
## Where the saves are
|
||||
|
||||
| Program | Linux | Windows | How sure |
|
||||
|---|---|---|---|
|
||||
| Hookshot | `~/.local/share/hookshot/save/<user id>/` | `<game folder>\user\save\<user id>\` | Hookshot's source; Linux tested |
|
||||
| [Lichii](https://github.com/zortos293/Lichii) | `~/.local/share/eden/` + Eden's save path below | `%APPDATA%\eden\` + Eden's save path | Lichii's source; Linux tested, Windows not |
|
||||
| [Frizti (Fritchii)](https://github.com/Frizti/Lichii), a Lichii fork | (Windows only) | `%APPDATA%\eden\` + Eden's save path | its source (`fritchii-v5.5`); not tried |
|
||||
| [Orchid](https://github.com/MythosOfs/Orchid) | `~/.local/share/eden/` + Eden's save path | `%APPDATA%\eden\` + Eden's save path | its source; not tried |
|
||||
| [PokeDinner](https://git.eden-emu.dev/Franseven/PokeDinner-a-Eglantine-Fork) | `~/.local/share/eden/` + Eden's save path | `%APPDATA%\eden\` + Eden's save path | its source; not tried |
|
||||
| [Eglantine](https://github.com/Djihads80/Eglantine) | `~/.local/share/eglantine/` + Eden's save path | `%APPDATA%\eglantine\` + Eden's save path | its source; not tried |
|
||||
| [NX2navi](https://github.com/tangerineturtle1/NX2navi) | `<game folder>/profile/save/<account>/` (its AppImage builds into `~/Games/Ocarina`) | `<game folder>\profile\save\<account>\` (the launcher's game folder is `%LOCALAPPDATA%\Programs\NX2navi\game` unless you chose another) | its documentation (`docs/launcher.md`, `docs/installer.md`, `docs/README-BuildKit.md`); not tried |
|
||||
|
||||
Eden's save path, under that data folder: `nand/user/save/0000000000000000/<profile id>/0400486024DEC000/`.
|
||||
|
||||
- `0400486024DEC000` is the game's title id. Eden's `<profile id>` is the Switch user profile you play with;
|
||||
Hookshot's `<user id>` and NX2navi's `<account>` are one folder with a 32-character hex name.
|
||||
- **Lichii, Frizti, Orchid and PokeDinner share one folder**, Eden's (`eden`): a save there is the same save in all of
|
||||
them. Eglantine has its own (`eglantine`).
|
||||
- The emulators use a `user` folder beside the program (Linux: in the folder you start it from) instead of the paths
|
||||
above when that folder exists (their "portable" mode). Point the script at it with `--from`/`--to`.
|
||||
- Newer Eden code also reads a save from `nand/user/save/account/<profile>/0400486024DEC000/0/` when that folder
|
||||
exists, and then uses it first. The script looks there too.
|
||||
- NX2navi writes a save to `profile/save-tx` first and moves it into place after. While something is waiting there,
|
||||
that is the save the game loads next, so the script refuses to copy until NX2navi has been started and quit (or
|
||||
`--force`). NX2navi's launcher also has its own **Import Saves**, which takes an emulator's save folder.
|
||||
- `$XDG_DATA_HOME` replaces `~/.local/share` when it is set. Hookshot's `--user <folder>` puts its saves in
|
||||
`<folder>/save/<user id>/` ([command line arguments](../readme.md#running-the-game-from-the-command-line)).
|
||||
- The emulator paths come from each one's `src/common/fs/fs_paths.h`, `path_util.cpp` and
|
||||
`src/core/file_sys/savedata_factory.cpp`. Official Eden does not run Switch 2 games; its name is accepted because
|
||||
the forks above use its folder.
|
||||
- Not listed: Ryujinx and its forks (Ryubing, ...) do not run Switch 2 games. Any other program that keeps these files
|
||||
works through `--from`/`--to` with the folder that holds them.
|
||||
|
||||
## Copy with the script
|
||||
|
||||
`scripts/oot_saves.py` (Python 3, nothing to install) copies the slots, `account.sav` and `option.sav` one way, after
|
||||
backing up the side it overwrites. From the Hookshot folder (or a clone):
|
||||
|
||||
```sh
|
||||
python -I scripts/oot_saves.py list # every save it finds, its slots and when each was saved
|
||||
python -I scripts/oot_saves.py import lichii # Lichii -> Hookshot
|
||||
python -I scripts/oot_saves.py export lichii # Hookshot -> Lichii
|
||||
python -I scripts/oot_saves.py import --from ~/Downloads/my-save # any folder holding a save -> Hookshot
|
||||
python -I scripts/oot_saves.py export --to ~/Lichii/user # Hookshot -> a portable emulator
|
||||
```
|
||||
|
||||
The names are `lichii`, `frizti` (or `fritchii`), `orchid`, `pokedinner`, `eden`, `eglantine` and `nx2navi`.
|
||||
`to-lichii` and `to-hookshot` still work: they are `export lichii` and `import lichii`.
|
||||
|
||||
On Windows Hookshot's folder is your game folder, which the script cannot guess, so name it:
|
||||
|
||||
```powershell
|
||||
python -I scripts\oot_saves.py import lichii --hookshot "C:\Games\Hookshot"
|
||||
python -I scripts\oot_saves.py export nx2navi --hookshot "C:\Games\Hookshot"
|
||||
```
|
||||
|
||||
| Option | What it does |
|
||||
|---|---|
|
||||
| `--dry-run` | Says what it would copy and where, and changes nothing |
|
||||
| `--keep-settings` | Leaves the target's `option.sav` alone, so each program keeps its own settings |
|
||||
| `--profile <id>` | The emulator's profile to use. Needed when more than one profile has saves: the script lists them and stops rather than pick one |
|
||||
| `--from <folder>`, `--to <folder>` | The emulator's side: the save folder itself, or a folder above it to search (a data folder, a portable `user` folder, NX2navi's game folder). With a name, it replaces where that emulator is looked for; without one, any folder works |
|
||||
| `--hookshot <folder>` | Hookshot's data folder (Windows: the game folder), or its `save\<user id>` folder |
|
||||
| `--backup-dir <folder>` | Where the backup goes, outside the target save folder |
|
||||
| `--force` | Copies even if a game seems to be running, or NX2navi has a save waiting |
|
||||
|
||||
The script refuses to run while it sees Hookshot or any of the programs above running. The side being overwritten is
|
||||
copied to `oot-save-backups/<import or export>-<name>-<date>-<time>/` first: in `~/.local/share` on Linux, in
|
||||
`%LOCALAPPDATA%\Hookshot` on Windows. To undo a copy, put the backup's contents back.
|
||||
|
||||
## Copy by hand
|
||||
|
||||
Close both programs, then copy the **contents** of the folder you are leaving into the folder you are going to (see
|
||||
the table above). Keep a copy of the target first. Delete the target's old `slot_*` folders before copying, so a slot
|
||||
that only exists on that side does not stay mixed in with the new save.
|
||||
|
||||
## Before you copy
|
||||
|
||||
- **Close both programs first.** A game that is running can write its save again on exit and undo the copy.
|
||||
- **Never run two of them on the same save at the same time.** Pointing two at one folder with a symlink may work,
|
||||
since the files are the same, but it has not been tried.
|
||||
- **Settings follow the save.** `option.sav` holds the game's own settings (camera, text speed, ...) and is copied too.
|
||||
Use `--keep-settings` (or skip that file by hand) if you want different settings in each.
|
||||
- **Profiles.** If your emulator has several user profiles, make sure you copy to and from the one you play on.
|
||||
- **Backups.** The script keeps one for every copy. Delete old ones when you no longer need them.
|
||||
- Start each program once and save in it before the first copy, so that its save folder exists.
|
||||
|
||||
## What was tested
|
||||
|
||||
- Linux (Arch-based), Hookshot v2.0.5 and Lichii built from source on the same day.
|
||||
- A Hookshot save copied to Lichii loads in Lichii at the same spot (Kokiri Forest, the spawn by the flower patch).
|
||||
- A Lichii save copied to Hookshot arrives identical, but has not been played through on the Hookshot side.
|
||||
- The script's discovery for every program above, its copies and backups are covered by `tests/test_oot_saves.py`, on
|
||||
temporary folders.
|
||||
- **Not tested:** Windows, and every program but Lichii. Their folders come from their source or, for NX2navi, its
|
||||
documentation, as the table says.
|
||||
55
docs/swapchain-recovery.md
Normal file
55
docs/swapchain-recovery.md
Normal file
@@ -0,0 +1,55 @@
|
||||
# Failed swapchain replacement
|
||||
|
||||
Passing a non-null `oldSwapchain` to `vkCreateSwapchainKHR` retires it even if
|
||||
creation fails. It cannot be acquired again or passed as a non-retired old chain
|
||||
on the next attempt. See the [Vulkan swapchain creation contract](https://docs.vulkan.org/refpages/latest/refpages/source/VkSwapchainCreateInfoKHR.html).
|
||||
|
||||
The same restriction applies when creation succeeds but enumerating the new
|
||||
images or creating their presentation semaphores fails. Leaving the retired
|
||||
handle in `dev.swapchain` makes the following present use an invalid chain.
|
||||
|
||||
Replacement now drains outstanding work with the existing device/present-queue
|
||||
idle helper, invalidates Reflex's cached handle, and releases the retired chain
|
||||
and its per-image semaphores after the creation attempt. New images and
|
||||
semaphores are prepared before publishing the replacement. An unsuccessful
|
||||
image-data query, including `VK_INCOMPLETE`, discards the incomplete replacement.
|
||||
|
||||
At the start of presentation, failed recreation returns before advancing the
|
||||
frame slot, waiting/resetting its fence, or acquiring an image. The requested
|
||||
size is retained for another attempt unless the window has already published a
|
||||
newer size. A missing swapchain also triggers recreation without requiring a
|
||||
new window event. Failures before the creation attempt leave the existing chain
|
||||
owned but skip that presentation and retry the request.
|
||||
|
||||
## Regression coverage
|
||||
|
||||
`python -I -B tests/test_present_semaphores.py --cxx clang++` compiles extracted
|
||||
creation, presentation-prefix, semaphore-selection and Reflex-invalidation code
|
||||
with CPU Vulkan stubs. It covers:
|
||||
|
||||
- Four swapchain images across three frame slots, followed by two/five-image replacements.
|
||||
- Retirement even when `vkCreateSwapchainKHR` fails.
|
||||
- Semaphore allocation failure, image-count error/zero, and incomplete image data.
|
||||
- Release of partial resources and recovery on a later attempt.
|
||||
- A failed preflight retaining the current chain while skipping acquisition.
|
||||
- Retrying the requested size while preserving a newer resize during recreation.
|
||||
- Recreation requested by Reflex and recovery without a resize notification.
|
||||
- Invalidation of Reflex's cached handle before its swapchain is destroyed.
|
||||
|
||||
The test requires a C++20 compiler, with no game, SDK runtime, GPU or display.
|
||||
It is discovered by the existing source-regression runner. CPU stubs do not
|
||||
establish driver behavior, Vulkan validation-layer results, or timing under
|
||||
concurrent game rendering. Native WSI and game integration remain separate checks.
|
||||
|
||||
Reflex consumers check eligibility while holding their state mutex. A latency
|
||||
sleep captures its swapchain generation and timeline target before releasing
|
||||
that mutex for the CPU wait. After the wait, it checks the same generation,
|
||||
latest target and eligibility before marking a new frame. Retired or replaced
|
||||
chains, including a replacement reusing the same handle, cannot receive stale
|
||||
markers. `tests/test_reflex_swapchain_lifetime.py` compiles the actual consumers
|
||||
and exercises retirement before locking and during the wait, handle reuse,
|
||||
overlapping sleep targets, disabling Reflex/turning on FG, and later recovery.
|
||||
|
||||
This handles replacement failure rather than device-loss or surface-loss
|
||||
recovery. It preserves the existing sizing policy, per-image presentation
|
||||
semaphores, queue locking, and frame-generation presentation path.
|
||||
30
docs/tested-configurations.md
Normal file
30
docs/tested-configurations.md
Normal file
@@ -0,0 +1,30 @@
|
||||
# Tested configurations
|
||||
|
||||
What has actually been run, where, and how it went: one row per test run, newest first. The README's
|
||||
[status table](../readme.md#project-status) sums this up by vendor; this page keeps the runs themselves, so a
|
||||
regression can be traced to a build and a driver, and so the next tester knows what is already covered.
|
||||
|
||||
Add a row for your own runs (a PR, or paste it in an issue and a maintainer adds it). A run that worked is as useful as
|
||||
one that did not. Keep it to facts you saw: the build (`hookshot vX.Y.Z-N-gHASH` from the first line of
|
||||
`hookshot_log.txt`), the hardware and driver, the settings that matter, where in the game, and what you measured.
|
||||
Frame rates come from `hookshot_log.txt` ("N presents, X fps") at a named spot, standing still unless the row says
|
||||
otherwise.
|
||||
|
||||
## Runs
|
||||
|
||||
| Date | Build | OS / desktop | GPU / driver | CPU / RAM | Settings | Where | Result |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| 2026-11-10 | host `v2.0.11-438-gcd544a7` (source `8bdd7a54`); renderer main `8bac8aa` + #1502 + the host-clock capture change | Windows 11 Home N, 26300 | RX 9070 XT 16 GB, 32.0.32015.12012 | Ryzen 7 5700X3D, 64 GB | 2560x720 window, 32:9 camera, serial, FG off, dense capture, core validation; fresh prepared shader bank | Boot, title, save load; gameplay not reached | 96 s: 1,476 consecutive original capture records; all 1,461 records covered by the flushed native trace align one-to-one with native returns. Movement window rejected: no filled gameplay interval. No new validation IDs; existing `02251`, and two `08737` errors during full preparation. No gameplay speed or performance claim |
|
||||
| 2026-11-10 | v2.0.11-447-gcbe49eb, sampler admission branch (main 8bac8aa + sampler checks + staging fixture repair) | Windows 11, windowed | RTX 5070 Ti, 621.01 | Ryzen 7 9800X3D, 32 GB | 3440x1440 picture, native upscaling, fixed FG 2x, 1 MB recording streams; serial and parallel replay (2 workers, batch 1) | Kokiri Forest, inside Link's house; copied checkpoint | Both runs rendered the scene, maximized/restored (client 1444x720 to 1440x829 and back), and exited normally. Six representative PNGs decoded at 3440x1440; original checkpoint hashes unchanged. Healthy-path validation only; no FPS comparison or native OOM recovery claim |
|
||||
| 2026-11-10 | main b2bb214 (+ #1447 #1451 #1520 #1660 #1619 #1448 #1654 #1612 #1600, unmerged then) | Ubuntu 26.04.1, KDE Plasma 6.6.6 Wayland | RTX 4070 Ti 12 GB, 595.99.02 | Ryzen 7 9800X3D, 32 GB | 3840x2160 output, DLSS Quality, no mods | Hyrule Castle road (child) | 48-50 fps. Same spot at 2560x1440 output: 56.3 fps. No device lost, saves written |
|
||||
| 2026-11-10 | main b2bb214 + #1459 (Dear ImGui menu) | as above | as above | as above | as above, windowed | Title, Hyrule Castle road | F1 menu, help box, prompts, the game's font and the FPS badge all correct; 60 fps |
|
||||
| 2026-11-10 | main b2bb214 | as above | as above | as above | HDR on | start-up | KDE offers an HDR10 surface (`vk: HDR: an HDR10 surface (format 58)`) |
|
||||
| 2026-11-09 | main 7eac5bf | as above | as above | as above | 3840x2160, DLSS Quality, no mods | Hyrule Castle road | 52.8 fps (replay submit 17.5 ms, memory sync 4.8 ms, GPU 12.8 ms) |
|
||||
| 2026-11-09 | main 7eac5bf | as above | as above | as above | fullscreen (`HOS_FULLSCREEN=1`) | Hyrule Castle road | fullscreen works, no device lost |
|
||||
| 2026-11-09 | v2.0.11-142 (7dbdcdc) | as above | as above | as above | 3840x2160, DLSS Quality | Water Temple 1F, Hyrule Castle road, Deku Tree 2F | Water Temple walls missing with the 23.3 GB dump, drawn with stand-in textures (#1308) or a complete dump. Slingshot and bow aiming: no fps change (51-52 fps, #1305). Pause / unpause: no freeze |
|
||||
|
||||
## Limits of the original RTX 4070 Ti runs
|
||||
|
||||
- Real mouse aiming with Mouse camera = Native after #1462 (the camera states were traced, but not tested with a real mouse).
|
||||
- Frame generation at 120 Hz or above (the displays here top out at 75 Hz).
|
||||
- HDR output on an HDR monitor (the surface is offered, the picture was not judged on one).
|
||||
31
docs/texture-staging-publication.md
Normal file
31
docs/texture-staging-publication.md
Normal file
@@ -0,0 +1,31 @@
|
||||
# Texture staging cache publication
|
||||
|
||||
Texture staging arenas use mapped sequential-write host memory. That memory is
|
||||
not required to be coherent: writing the pointer alone does not publish encoded
|
||||
ASTC blocks or CPU-converted texels to the GPU.
|
||||
|
||||
Both staging consumers now flush the completed allocation-relative slice before
|
||||
its GPU use. Staging carries its VMA allocation alongside the buffer/offset, and
|
||||
slice starts respect `nonCoherentAtomSize` as well as the consumer alignment.
|
||||
This keeps a rounded cache flush from sharing an atom with another slice already
|
||||
being read by a submitted parallel segment. VMA handles allocation offsets, cache
|
||||
rounding and coherent-memory no-ops. No additional queue wait is introduced.
|
||||
|
||||
CPU uploads publish after conversion workers join, before the slice is added to
|
||||
the pending image copy. ASTC uploads publish after gathering their layers, before
|
||||
the GPU decoder is recorded. This also covers uploads redirected into an earlier
|
||||
parallel init command buffer, without depending on a later worker's submit hook.
|
||||
Arena sizes, allocation fallback, lifetime/reuse, format conversion, shader paths,
|
||||
descriptor policy and the renderer ABI remain unchanged.
|
||||
|
||||
The game-free fixture executes actual staging allocation and the actual ASTC
|
||||
gather/publication prefix and CPU publication/copy-description region. Authored
|
||||
CPU/device caches distinguish coherent and noncoherent memory; it does not execute
|
||||
the full CPU converters or ASTC shader. Cases cover nonzero offsets, multiple
|
||||
slices/arenas, completion-based reuse, 1024-byte atoms, failed allocation and empty
|
||||
flushes. A source-order guard keeps CPU publication after worker joins. The range
|
||||
helper itself is extracted from production. These checks do not establish a
|
||||
physical noncoherent GPU reproduction or campaign/performance coverage.
|
||||
|
||||
Run `tests/test_texture_staging_publication.py --headers <Vulkan include> --cxx <compiler>`.
|
||||
Linux additionally supports `--sanitize` for AddressSanitizer/UndefinedBehaviorSanitizer.
|
||||
19
docs/ui-texture-upload.md
Normal file
19
docs/ui-texture-upload.md
Normal file
@@ -0,0 +1,19 @@
|
||||
# UI texture staging visibility
|
||||
|
||||
The custom ImGui texture uploader packs the requested rectangles into mapped
|
||||
staging memory and flushes that allocation after its final pixel write, before GPU
|
||||
use. Sequential-write host-visible memory need not be coherent. VMA's flush helper
|
||||
handles cache-line alignment and treats coherent allocations appropriately.
|
||||
|
||||
This covers new font/logo textures, individual dirty rectangles and the fallback
|
||||
update rectangle, for RGBA and Alpha8 expansion. The existing image layouts,
|
||||
descriptor publication, retry behavior and staging retirement remain unchanged.
|
||||
The bundled ImGui backend maintains its own vertex/index upload caches; this
|
||||
corrects Hookshot's separate texture-upload path.
|
||||
|
||||
`tests/test_ui_texture_upload.py` executes production pixels/upload/retirement code
|
||||
with the pinned ImGui/Vulkan types. A CPU/GPU cache model covers coherent and
|
||||
noncoherent memory, full and partial uploads, untouched pixels, failure/retry and
|
||||
deferred destruction. GPU copies execute after recording returns, so flushing
|
||||
after recording but before submission remains a valid control. HDR color
|
||||
conversion and physical noncoherent hardware behavior are not modeled.
|
||||
191
docs/versioning.md
Normal file
191
docs/versioning.md
Normal file
@@ -0,0 +1,191 @@
|
||||
# Versioning and releases
|
||||
|
||||
Hookshot follows [Semantic Versioning 2.0.0](https://semver.org/) and the conventions of
|
||||
[semantic-release](https://semantic-release.org/). The current version is **0.1.0**.
|
||||
|
||||
The git tag is the only source of truth. No file carries a version that has to be edited for a release: the game reads
|
||||
the tag through `nx2/version.py`, and the builder and the AppImage are stamped from the same tag when they are
|
||||
packaged.
|
||||
|
||||
## The version number
|
||||
|
||||
`MAJOR.MINOR.PATCH`, with an optional prerelease suffix: `0.1.0`, `0.2.0`, `0.2.0-rc.1`. A release is tagged
|
||||
`v0.1.0`: the `v` belongs to the tag, not to the version. Build metadata is not used in tags.
|
||||
|
||||
One version covers the whole project: the game (`hookshot.exe` / `hookshot`), the Vulkan back end, the lift pipeline
|
||||
and its scripts, Hookshot Builder and the Linux AppImage. They are built from the same commit and released together,
|
||||
so they never carry different versions.
|
||||
|
||||
The "public" surface the rules protect is the command line of `make_game.py` and the builder, the `RX_` and `HOS_`
|
||||
switches, the files a build writes into a game folder (`hookshot_options.ini`, `hookshot_env.txt`, `language.txt`,
|
||||
`hookshot_version.txt`), saves, and the format of `hookshot_log.txt`. The C sources and internal headers are not an
|
||||
API and may change in any release.
|
||||
|
||||
## Commits decide the version
|
||||
|
||||
Releases are computed from the commits on the release branch since the last tag, so commit subjects must be
|
||||
[Conventional Commits](https://www.conventionalcommits.org/):
|
||||
|
||||
```
|
||||
type(scope): description
|
||||
type(scope)!: description (a breaking change)
|
||||
```
|
||||
|
||||
`scope` is optional. A `!` before the colon, or `BREAKING CHANGE:` / `BREAKING-CHANGE:` in the commit body, marks a
|
||||
breaking change. The description is written as an imperative ("Stop scanning...", "Fix the..."), because it becomes
|
||||
the changelog line.
|
||||
|
||||
| Commit | Next version |
|
||||
|---|---|
|
||||
| any commit with `!` or `BREAKING CHANGE:` | major (`0.1.0` -> `1.0.0`) |
|
||||
| `feat` | minor (`0.1.0` -> `0.2.0`) |
|
||||
| `fix` | patch (`0.1.0` -> `0.1.1`) |
|
||||
| `perf` | patch |
|
||||
| `revert` | patch |
|
||||
| `docs`, `refactor`, `style`, `test`, `build`, `ci`, `chore` | no release |
|
||||
|
||||
`git revert` writes `Revert "..."`, which is not a Conventional Commit: use `git revert --edit` and write the subject
|
||||
as `revert: ...`, or the revert makes no release.
|
||||
|
||||
A release takes the highest bump among its commits. Merge commits are ignored, and a commit with `[skip release]` or
|
||||
`[release skip]` in its message is left out of the analysis. `chore(release): 0.2.0 [skip ci] [skip release]`, the
|
||||
commit the release itself makes, is not a release (the notes in its body can mention a breaking change).
|
||||
|
||||
There is no manual version bump. If a change should not make a release, use one of the types in the last row; if it
|
||||
should, use `feat` or `fix`.
|
||||
|
||||
## 0.x and 1.0.0
|
||||
|
||||
semantic-release does not treat major version zero specially: a breaking change is a major bump even from `0.1.0`.
|
||||
This is deliberate. Its documentation says that SemVer's different rules for `0.x` are out of scope, and it recommends
|
||||
`1.0.0` as the first release.
|
||||
|
||||
Hookshot starts at `0.1.0` anyway, because the first versioned release is not stable. That tag is made once by hand
|
||||
(see below); after it, the release workflow is the only thing that creates tags. Consequence: the first breaking change
|
||||
merged to `main` releases `1.0.0`, not `0.2.0`.
|
||||
|
||||
`1.0.0` means the public surface above is considered stable. Until then, keep breaking changes on the `next` branch
|
||||
and release them there as `1.0.0-rc.N`, or accept that the release that carries one becomes `1.0.0`.
|
||||
|
||||
## Branches and the release workflow
|
||||
|
||||
| Branch | Purpose | Version |
|
||||
|---|---|---|
|
||||
| `main` | stable releases | `vX.Y.Z` |
|
||||
| `next` | release candidates | `vX.Y.Z-rc.N` |
|
||||
|
||||
Every push to `main` or `next` runs the [Release workflow](../.github/workflows/release.yml). It:
|
||||
|
||||
1. finds the last release tag and the commits since it;
|
||||
2. takes the highest bump in the table above;
|
||||
3. runs `scripts/changelog.py` to put the new entry at the top of `changelog.txt`;
|
||||
4. commits `changelog.txt` as `chore(release): <version> [skip ci] [skip release]`;
|
||||
5. tags the release commit. The tag starts the Linux AppImage and Windows builder workflows: the first to finish makes
|
||||
the GitHub release, and each attaches its files (the AppImage; `HookshotBuilder.exe`, the setup, and
|
||||
`hookshot-setup.txt`, with the new parts in the `builder-parts` release). `scripts/release_upload.sh` is shared by both.
|
||||
|
||||
A merge with no releasable commit (only docs, chores, and so on) makes no release. A release is never edited by hand,
|
||||
never re-tagged and never deleted. Require the *Game-free source regressions* workflow on `main` and `next` (branch
|
||||
protection): the release then runs for a commit whose tests have passed.
|
||||
|
||||
The workflow pushes back to the repository, so it needs a token with `contents: write` that may bypass `main`'s branch
|
||||
protection. Use a `RELEASE_TOKEN` secret (a fine-grained personal access token) rather than the built-in
|
||||
`GITHUB_TOKEN`: a tag pushed with `GITHUB_TOKEN` does not start another workflow, and the Linux AppImage and Windows
|
||||
builder workflows publish the release from the tag. With only `GITHUB_TOKEN`, build both with `workflow_dispatch` and
|
||||
create the release by hand.
|
||||
|
||||
## The first tag: 0.1.0
|
||||
|
||||
semantic-release's first release is `1.0.0` when it finds no tag. To make `0.1.0` the first version, tag the merge
|
||||
commit that introduces this scheme once:
|
||||
|
||||
```sh
|
||||
git tag -a v0.1.0 -m "Hookshot 0.1.0"
|
||||
git push origin v0.1.0
|
||||
```
|
||||
|
||||
The Release workflow does nothing while no version tag exists, so the merge cannot release `1.0.0` first. The pushed
|
||||
tag starts the Linux AppImage workflow, which creates the `0.1.0` GitHub release with the AppImage attached; create the
|
||||
release by hand if you do not want to wait for it. The `0.1.0` entry in `changelog.txt` is written for that tag. After
|
||||
it, do not tag by hand.
|
||||
|
||||
## Pre-releases
|
||||
|
||||
A release candidate is released from `next` with the same commands. semantic-release numbers it `v0.2.0-rc.1`,
|
||||
`v0.2.0-rc.2`, and so on; the GitHub release is marked as a prerelease and never becomes "latest". A prerelease is not
|
||||
written to `changelog.txt`: Factorio's parser accepts only `major.minor.patch`, and two release candidates share a base
|
||||
version. Its notes are in the GitHub prerelease. Merge `next` into `main` to release the final `0.2.0`, whose entry
|
||||
lists every commit since the last stable tag.
|
||||
|
||||
## Hotfixes
|
||||
|
||||
Branch from the release tag, not from `main`, when a fix must not include `main`'s unreleased features:
|
||||
|
||||
```sh
|
||||
git switch -c fix/crash v0.1.0
|
||||
```
|
||||
|
||||
Commit with `fix: ...` and open a pull request to `main`. If `main` has no unreleased feature commits, the release is a
|
||||
patch. The project does not keep maintenance branches for older minor versions: only the newest release is supported.
|
||||
|
||||
## Where the version appears
|
||||
|
||||
| Place | Where it comes from |
|
||||
|---|---|
|
||||
| git tag | semantic-release; the source of truth |
|
||||
| `changelog.txt` | `scripts/changelog.py` during the release |
|
||||
| `hookshot_log.txt` and the Insert menu | `nx2/version.py`: `HOOKSHOT_VERSION`, then `VERSION`/`version.txt`, then `git describe --tags --match "v[0-9]*"` |
|
||||
| `hookshot_version.txt` in a build | the `VERSION` file written when the tools are packaged (`nx2/make_game.py`) |
|
||||
| AppImage file name | `git describe --tags --match "v[0-9]*"` (`scripts/linux/appimage.sh`) |
|
||||
| `HookshotBuilder.exe` | the MSBuild `Version` passed at publish time (`dotnet publish -p:Version=<version>`, the Windows builder workflow); the `csproj`'s `0.1.0` is only the default for a local build |
|
||||
|
||||
`nx2/version.py` also appends a fingerprint of the source files, `(source 1a2b3c4d)`. It is not a version: it tells
|
||||
two builds of the same release apart, and it is the only identifier a build from a modified checkout has.
|
||||
|
||||
`package.json` and `package-lock.json` (if present) are the release tooling's, not the project's: `package.json` keeps
|
||||
the placeholder `0.0.0-development` on purpose, and the tag remains the version. The game never reads them.
|
||||
|
||||
## The changelog
|
||||
|
||||
`changelog.txt` follows [Factorio's changelog format](https://lua-api.factorio.com/latest/auxiliary/changelog-format.html).
|
||||
Newest first, with a 99-dash separator before each version:
|
||||
|
||||
```
|
||||
---------------------------------------------------------------------------------------------------
|
||||
Version: 0.1.0
|
||||
Date: 07. 11. 2026
|
||||
Features:
|
||||
- Something new.
|
||||
Bugfixes:
|
||||
- Something fixed.
|
||||
```
|
||||
|
||||
The date is day. month. year (`07. 11. 2026`), as in Factorio's own changelog. Section names are indented two
|
||||
spaces, bullets four, there are no tabs or trailing spaces, and a section is omitted when it is empty. The version is
|
||||
always three numbers: a prerelease has no entry (see above).
|
||||
|
||||
| Commit | Section |
|
||||
|---|---|
|
||||
| `feat` | Features |
|
||||
| `fix`, `revert` | Bugfixes |
|
||||
| `perf` | Optimizations |
|
||||
| `refactor` | Changes |
|
||||
| breaking change | Changes, as `Breaking: ...` |
|
||||
| `docs`, `style`, `test`, `build`, `ci`, `chore` | not listed |
|
||||
|
||||
A stable release writes the entry. Do not renumber, reorder or rewrite a released entry; a correction is a new commit
|
||||
and a new release. The `0.1.0` entry was written by hand because the history before this scheme does not use
|
||||
Conventional Commits.
|
||||
|
||||
## History
|
||||
|
||||
Tags from before this scheme (`Release`, `amd`, `text-hotfix`, and so on) are snapshots of the old workflow. They stay
|
||||
in the repository for history but are not releases: the tooling matches only `v[0-9]*` tags. `v0.1.0` is the first
|
||||
versioned release.
|
||||
|
||||
## Rules
|
||||
|
||||
- Do not edit a version number in a file, move a tag or reuse a version.
|
||||
- Do not tag a branch other than `main` (or `next` for a release candidate).
|
||||
- Do not release from a dirty working tree; the release commit contains `changelog.txt` and nothing else.
|
||||
- Do not put game files, keys, builds or anything made from a dump in a release.
|
||||
118
docs/workflow.md
Normal file
118
docs/workflow.md
Normal file
@@ -0,0 +1,118 @@
|
||||
# Workflow: from report to release
|
||||
|
||||
How work moves through Hookshot, and how to decide what to work on next. It is written for people and for AI agents
|
||||
alike (agents: [AGENTS.md](../AGENTS.md) sends you here). [CONTRIBUTING.md](../CONTRIBUTING.md) still holds the rules
|
||||
that never bend, above all that no game files or anything made from them leave your PC.
|
||||
|
||||
The goal of this page is the order of work: with dozens of contributors and agents working at once, the project moves
|
||||
fastest when everyone picks from the same priorities, claims before starting, and leaves a trail the next person can
|
||||
follow.
|
||||
|
||||
## 1. Where work comes from
|
||||
|
||||
| Source | What lands there |
|
||||
|---|---|
|
||||
| GitHub issues (game problem, build problem, idea) | the record: version, hardware, place in the game, `hookshot_log.txt` |
|
||||
| Tracking issues (`[Tracking] ...`) | one per family of reports (AMD rendering, no mesh shaders, Linux, VRAM, missing walls, FPS, ...) |
|
||||
| Discord #bug-reporting (forum) | first reports, often without a log; turn the reproducible ones into issues |
|
||||
| Discord #dev-work | claims, progress, review requests |
|
||||
| Discord #dev-stuff | security scan results (Syntak): handled before anything else |
|
||||
| CI | a red check on `main` is everyone's first problem |
|
||||
|
||||
## 2. Priority: what to work on first
|
||||
|
||||
Score a problem as **severity x reach x confidence** and work from the top. Write the three parts in the issue or the
|
||||
claim, so others can see why it was picked. The scored, sorted list of open work is **[docs/worklist.md](worklist.md)**:
|
||||
pick from its top, and add new problems there with their three parts.
|
||||
|
||||
### Severity
|
||||
|
||||
| Level | Weight | What it means |
|
||||
|---|---|---|
|
||||
| S0 | 8 | Security issue; crash, hang, device lost or a PC freeze; lost or corrupted saves; the builder cannot build |
|
||||
| S1 | 5 | Cannot progress or cannot start: black screen, missing walls that block a dungeon, softlock, menus unusable |
|
||||
| S2 | 3 | Wrong but playable: visual corruption, missing objects, audio out of sync, a large frame-rate loss (over 20%) |
|
||||
| S3 | 1 | Polish: small visual issues, small frame-rate gains, settings, features |
|
||||
|
||||
### Reach: the systems people run
|
||||
|
||||
Reach is the share of players on the hardware the problem affects (1.0 when it affects everyone). Two numbers feed it,
|
||||
and both change, so read them again rather than trusting old ones:
|
||||
|
||||
- **Who reports problems:** `python -I scripts/hardware_report.py` counts the GPU vendor, card class and OS in the
|
||||
issue forms (it needs the `gh` CLI). As of 2026-11-10, of 399 issues that named their hardware: AMD 51%, NVIDIA 35%,
|
||||
Intel 3.5%; by class RX 6000 16%, AMD APUs and handhelds 12%, RX 9000 10%, RTX 40 9%, RTX 50 8.5%, RTX 30 7%, cards
|
||||
without mesh shaders 14.5%; Windows 57%, Linux and SteamOS 17%. This over-counts hardware where things break.
|
||||
- **Who plays:** the [Steam Hardware Survey](https://store.steampowered.com/hwsurvey/) is the best public picture of
|
||||
PC gaming hardware (vendor, card, OS). Use its current numbers for the installed base.
|
||||
|
||||
A problem that hits a large installed base, or that dominates the reports, ranks high either way.
|
||||
|
||||
### Confidence
|
||||
|
||||
| | Weight |
|
||||
|---|---|
|
||||
| Reproduced by you, or by a maintainer, with a log | 1.0 |
|
||||
| Two or more reporters with logs | 0.7 |
|
||||
| One report, or no log | 0.4 (ask for the log first) |
|
||||
|
||||
### Hardware tiers: stability across machines
|
||||
|
||||
A change must not break a higher tier to help a lower one.
|
||||
|
||||
| Tier | Hardware | What it means |
|
||||
|---|---|---|
|
||||
| 1 | NVIDIA RTX 20-50 and AMD RX 6000-9000, on Windows; the same on Linux (NVIDIA driver, Mesa RADV); Steam Deck | Must keep working: test here before merge, regressions are S0/S1 |
|
||||
| 2 | AMD APUs and handhelds (ROG Ally, Legion Go, 780M/890M/8060S), Intel Arc | Fixed when reported, tested when someone with the hardware can |
|
||||
| 3 | Cards without mesh shaders (GTX 10/16, RX 5000 and older) | Best effort through the fallback; never at a Tier 1 cost |
|
||||
|
||||
Today Tier 1 testing is mostly NVIDIA; AMD files half the reports. AMD stability is the largest gap, so an AMD Tier 1
|
||||
problem with a log outranks an NVIDIA one of the same severity.
|
||||
|
||||
## 3. The lifecycle
|
||||
|
||||
1. **Triage.** Reproduce if you can. Find the tracking issue it belongs to, or open the issue (with the log). Score it
|
||||
(section 2).
|
||||
2. **Claim.** Before starting, say so in Discord #dev-work and on the issue: what, why (the score), and on what
|
||||
hardware you can test. Search open PRs first: two people fixing the same thing is the most common waste here. Drop
|
||||
a claim out loud when you stop.
|
||||
3. **Fix.** A branch from current `main`, one topic. Find the cause before the fix: a log, a trace, a measurement.
|
||||
4. **Test.**
|
||||
- A game-free test that fails without the change (see AGENTS.md: tests read the source).
|
||||
- `python -I -B tests/run_all.py`, all of it.
|
||||
- In the game, on your hardware: what you did and what you saw, with numbers from `hookshot_log.txt` for performance.
|
||||
- Add a row to [tested configurations](tested-configurations.md) for each run you measured.
|
||||
- Say what you could not test (another OS, another vendor, real input devices).
|
||||
5. **Pull request.** Open the description with why the PR is needed; then what changed, how it was tested, what was
|
||||
not tested, and screenshots when it shows on screen. Link the issue and the tracking issue. Post the link in
|
||||
#dev-work.
|
||||
6. **Review.** Answer every comment in the PR. Rebase onto `main` before asking for review again: a PR that does not
|
||||
merge cleanly waits.
|
||||
7. **Merge and release.** Release branches are merged back into `main` right after a release, so that `main`'s builds
|
||||
carry the release's fixes and the right version string (`git describe` only sees tags on `main`'s history).
|
||||
8. **Verify.** Ask the reporter to retest on the release, then close the issue, or keep it open on the tracking issue
|
||||
with what is left.
|
||||
|
||||
## 4. The security lane
|
||||
|
||||
Security comes first: a scan result in #dev-stuff is triaged before other work.
|
||||
|
||||
- Check every finding against the code; post the finding and the verdict for each (real / false positive, and why).
|
||||
- Fix the real ones in a PR, marked as security in the title.
|
||||
- Record false positives in the scanner as not exploitable, with the reason, so the next scan shows only new ones.
|
||||
- Never paste secrets, tokens or the scanner's private links in public channels.
|
||||
|
||||
## 5. Keeping the board readable
|
||||
|
||||
Suggested labels (a maintainer creates them): `priority:P0` to `priority:P3` (from the score), `hw:nvidia`, `hw:amd`,
|
||||
`hw:intel`, `hw:no-mesh-shaders`, `os:linux`, `os:steamdeck`, `needs-log`, `reproduced`, `claimed`. With them, the
|
||||
next thing to work on is the top of `is:open label:reproduced -label:claimed sort:priority`.
|
||||
|
||||
## For agents
|
||||
|
||||
- Start here, then AGENTS.md, then CONTRIBUTING.md.
|
||||
- Pick from the top of [docs/worklist.md](worklist.md); add a scored row there for a new problem you find.
|
||||
- Prefer problems you can reproduce on the hardware you run on.
|
||||
- Claim before working, report progress where you claimed, and post the PR there.
|
||||
- Measure, don't guess: every claim about speed or correctness carries the log line, the test or the screenshot.
|
||||
- Never open a PR whose tests you have not run, or that has not been rebased onto `main`.
|
||||
57
docs/worklist.md
Normal file
57
docs/worklist.md
Normal file
@@ -0,0 +1,57 @@
|
||||
# Work list
|
||||
|
||||
The prioritized list of open work, scored as [docs/workflow.md](workflow.md) describes: **score = severity weight x
|
||||
reach x confidence** (S0 8, S1 5, S2 3, S3 1; reach 0-1 from `scripts/hardware_report.py` and the Steam survey;
|
||||
confidence 1.0 reproduced with a log, 0.7 two or more reporters with logs, 0.4 one report or no log). Work from the
|
||||
top of **Open**. A GitHub issue stays the full record; this list is the order.
|
||||
|
||||
How to use it:
|
||||
|
||||
- **Pick:** take the highest-scored row with no owner that you can reproduce on your hardware, claim it in Discord
|
||||
#dev-work, and put your name in *Owner* in the same PR that starts the work (or a one-line PR).
|
||||
- **Add:** a new report becomes a row in **Open** with its three parts filled in, so the score can be checked. Link the
|
||||
issue or the Discord thread. One row per problem: if a row already covers it, add the link to that row and raise its
|
||||
confidence instead of adding a second row.
|
||||
- **Move:** when a PR merges, move the row to **Fixed, waiting for a release** with the PR; when a release ships it,
|
||||
delete the row (the release notes are the record). A row nobody can reproduce for two weeks, with logs asked for,
|
||||
goes to **Parked**.
|
||||
- Keep each table sorted by score, highest first. Re-score when new logs or hardware numbers arrive.
|
||||
|
||||
Agents may add and re-score rows; a person reviews the change like any other PR.
|
||||
|
||||
## Open
|
||||
|
||||
| Score | Problem | S | Reach | Conf. | Owner | Links |
|
||||
|---:|---|---|---:|---:|---|---|
|
||||
| 3.0 | Shader programs that still do not translate (TLD multisample bit 78, LDG descriptor without `.E`, programs using the lowering scratch registers, uniform-addressed global loads) | S2 | 1.0 | 1.0 | | `--prepare-shaders` log; Legion Go 2 log in #bug-reporting |
|
||||
| 2.1 | Ground and outdoor textures garble after a while of play and come back after a restart or going to the title; not the 4-8 GB over-budget case: on a 9070 XT 16 GB the textures sit at 5.3 of their 6.1 GB budget (VRAM 9.6 of 15.4 GB) with 287 texture retirements in the log; `RX_VK_ASTC_RGBA8=1` took effect (texture arrays 4x larger) and did not help. 9070 XT x2 (logs, v2.0.13), RTX 4080 (no log), RTX 5060 Ti (v2.0.5 log). Next: test whether retiring and re-making a texture brings back stale or wrong contents (force retirements with a small `RX_VK_VRAM_MB`) | S2 | 1.0 | 0.7 | | #993; #bug-reporting "texture glitch on 9070 XT", "Ground design is not loading properly", "Dodongo Cavern, floor texture glitch" |
|
||||
| 2.1 | Camera turns in Kokiri Forest stutter: 46 fps minimum, dips to 38 (14700KF); goal is a 60 fps minimum | S2 | 1.0 | 0.7 | Joey-Magnus | Reddit report; #1762 (+13.5% in the village view; 57 of 60); next: the capture wait on the main thread |
|
||||
| 2.0 | Antivirus flags HookshotBuilder.exe (Defender `Wacatac.H!ml` / `Wacatac.B!ml`, Chrome on v2.0.12): new unsigned exe, no reputation. Waiting on the SignPath open-source signing application; then sign in `windows-builder.yml` and publish the signed files | S1 | 0.57 | 0.7 | NotRexed (SignPath) | #1003, #525, #1620; checksums: #1307 |
|
||||
| 1.6 | Stops after the first cutscene (Navi, Deku Tree): black screen, or the PC freezes or crashes, with no error in the logs; AMD (RX 9060 XT, RX 6700 XT, RX 6600, 5700G, Z1 Extreme) and one NVIDIA card (1080 Ti) reported; Linux too | S0 | 0.51 | 0.4 | | #bug-reporting "black screen after a first cutscene", "Crashes entire computer" (one log is from v2.0.7, the other not read yet: confidence goes up with a current log) |
|
||||
| 1.2 | Link keeps playing a jump animation in dialogue after the Gohma fight (gameplay fine): cutscenes and dialogue run at the frame rate above a 60 cap since #1612 (v2.0.13); work-around: cap at 60 | S2 | 1.0 | 0.4 | | #1742 (#1726 closed into it); #bug-reporting "My link is jumping forever" |
|
||||
| 1.2 | Ground looks smeared, without detail: Kokiri Forest / Lost Woods, and also Dodongo's Cave and Hyrule Field on RX 9070 XT (driver 26.10.2, 1440p, no mods; log received, clears after a restart) | S2 | 1.0 | 0.4 | | #bug-reporting "Ground design is not loading properly" |
|
||||
| 1.2 | Trees float above the ground (Kakariko, Hyrule Field): reported as already fixed, check which change and that it shipped | S2 | 1.0 | 0.4 | | tracker #994 |
|
||||
| 1.0 | Source checks fail after stream/descriptor admission merges: missing production stream_insert dependency and descriptor-pool fixture environment. CI-first; reproduced on main | S3 | 1.0 | 1.0 | Arkex-ai | [#1813](https://github.com/NotRexed/Hookshot/issues/1813), [claim](https://discord.com/channels/1568810484818378804/1569019786766188665/1569763504758464609) |
|
||||
| 0.6 | Zora's Domain textures missing in the adult timeline (points to VRAM pressure, not the dump) | S2 | 0.3 | 0.7 | | #bug-reporting "Zora Domain textures missing?" |
|
||||
| 0.5 | Sacred Forest Meadow fairy fountain grotto: floor and walls invisible (RTX 2070 Super) | S2 | 0.4 | 0.4 | | #bug-reporting thread |
|
||||
| 0.4 | Voices in the wrong language | S3 | 1.0 | 0.4 | | #1460 (needs a save past Gohma) |
|
||||
| 0.29 | Amd RX 580 (Polaris, no mesh shaders): hair renders white and bright, and the game crashes when Link moves; a second RX 580 8 GB user reports the same | S1 | 0.145 | 0.4 | | #bug-reporting "Amd rx 580 crashes" (no log yet) |
|
||||
| 0.24 | Handheld with no keyboard (Asus ROG Xbox Ally X, controller only): fullscreen and the F1 options menu cannot be reached; no log yet | S1 | 0.12 | 0.4 | | #bug-reporting "Xinput devices cannot enter fullscreen or menu" |
|
||||
| 0.12 | AMD RX 9070 XT: texture glitch that appears after some play (since v2.0.5); same as the tracked AMD texture artifact | S2 | 0.10 | 0.4 | | #bug-reporting "texture glitch on 9070 XT aris"; #57, #1006 |
|
||||
| 0.11 | Intel GPU: black screen after loading a save (or on the Deku Tree), sound only, very low fps; three reports, no logs yet; tracked in the GitHub issue linked in #bug-reporting replies (#999) | S0 | 0.035 | 0.4 | | #bug-reporting "Intel G3E. Title and intro loads...", "intel arc b580 problems", "game gets stuck on a black screen when loading a save (intel gpu... gulp...)" |
|
||||
| 0.11 | Crash at the door after the owl intro (Navi wakes Link, 2.0.13): access violation in `igvk64.dll` at start-up; GPU not given yet, reach uses Intel's 3.5% until it is | S0 | 0.035 | 0.4 | | #bug-reporting "2.0.13 Crash after you see the owl during intro cutscene" (5 logs) |
|
||||
|
||||
|
||||
## Fixed, waiting for a release
|
||||
|
||||
| Problem | Fixed by | Notes |
|
||||
|---|---|---|
|
||||
| 1D surface loads (SULD.1D) stop 16-17 compute programs from translating; fits the green glow on a Legion Go 2 (890M) | #1447 | in `main`, not in v2.0.13 (log: #bug-reporting "Bugs 2.0.13 Dodaza") |
|
||||
| Integer min/max with a wide immediate (IMNMX) does not translate | #1451 | in `main`, not in v2.0.13 (log: #bug-reporting "Bugs 2.0.13 Dodaza") |
|
||||
| Insert menu purple with HDR10 on (v2.0.13 GDI overlay drawn into the HDR swapchain unconverted) | `main`'s ImGui overlay (`hdr10()`) | needs the release branch merged back into `main` |
|
||||
|
||||
## Parked
|
||||
|
||||
| Problem | Why parked | Links |
|
||||
|---|---|---|
|
||||
| | | |
|
||||
Reference in New Issue
Block a user