Describe what the code does in comments and docs.

Comments say what the code does and why, docs/maxwell.md keeps the Maxwell facts, and fetch_refs.sh clones only the checkout the build reads. Code is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
chrg6
2026-11-07 03:20:42 -05:00
parent a1871280a3
commit 1b39aabd37
128 changed files with 481 additions and 577 deletions

View File

@@ -1,6 +1,6 @@
"""Shared parts of the differential tests: Unicorn reference, operand generators, build and run of harness.c.
Adapted from LAS Recomp tools/test_lift.py and tools/test_simd.py @ da35b83 (MIT, Copyright (c) 2026 Peiikai).
Copyright (c) 2026 Peiikai (MIT; see THIRD_PARTY_NOTICES.md).
"""
import os
import shutil

View File

@@ -8,7 +8,7 @@ Skipped: system instructions and exceptions, whose single step cannot be compare
atomics, which atomic.py covers. Encodings the lifter rejects are counted per module and mnemonic; --coverage lifts
every distinct word instead of testing.
Adapted from LAS Recomp tools/test_lift.py @ da35b83 (MIT, Copyright (c) 2026 Peiikai).
Copyright (c) 2026 Peiikai (MIT; see THIRD_PARTY_NOTICES.md).
Usage: lift.py [--exefs DIR] [--modules main,sdk,subsdk0] [--encodings N] [--cases K] [--seed S] [--jobs J]
lift.py --coverage

View File

@@ -7,7 +7,7 @@ operands (NaNs, infinities, denormals, +-0, saturation edges, random FPCR.DN/FZ/
and, for stores, memory must match. FPSR is not compared; the lowering does not model the cumulative exception bits
or QC.
Adapted from LAS Recomp tools/test_simd.py @ da35b83 (MIT, Copyright (c) 2026 Peiikai).
Copyright (c) 2026 Peiikai (MIT; see THIRD_PARTY_NOTICES.md).
Usage: simd.py [--encodings N] [--cases K] [--seed S] [--jobs J] [--no-validity] [--filter HEX/MASK]
"""

View File

@@ -421,7 +421,7 @@ class CleanupTest(unittest.TestCase):
class BaseGameCheckTest(unittest.TestCase):
"""Issue #261: an update NSP chosen as the base game was reported as "not Ocarina of Time"."""
"""Issue #261: an update NSP chosen as the base game was reported as not being the game."""
BASE, UPDATE = '0100F2C0115B6000', '0100F2C0115B6800'
@staticmethod

View File

@@ -1,6 +1,6 @@
"""Check of recomp/jumptables.py on every `br` in the game's modules against an emulation oracle.
The oracle is LAS Recomp's jump-table method: find `cmp #N; b.hi/b.hs` before the br (or a `b.ls/b.lo` to the
The oracle is an emulated jump-table resolver: find `cmp #N; b.hi/b.hs` before the br (or a `b.ls/b.lo` to the
dispatch block), replay ADRP/ADR/ADD-imm from the function entry, then single-step Unicorn from the bounds check to
the br once per index. Functions are [seed, next seed) over the module's seeds plus BL targets.
@@ -11,7 +11,7 @@ as 'oracle wrong' only with independent evidence (oracle_wrong). Samples of ever
unresolved computed site are printed with disassembly. Output also goes to build/logs/jumptables-check.log, and one
row per table-like br to build/logs/jumptables-<module>.tsv.
The oracle is adapted from LAS Recomp tools/gen.py @ da35b83 (MIT, Copyright (c) 2026 Peiikai).
The oracle: Copyright (c) 2026 Peiikai (MIT; see THIRD_PARTY_NOTICES.md).
Exits with 1 if any disagreement is unexplained, a hoisted base is not verified, or a PLT stub counts as computed.
@@ -44,7 +44,7 @@ _md = capstone.Cs(capstone.CS_ARCH_ARM64, capstone.CS_MODE_ARM)
class Oracle:
"""LAS Recomp's Unicorn jump-table resolver (`Module.jump_table` and helpers in tools/gen.py)."""
"""The Unicorn jump-table resolver: bounds check, table base replay and one single-step run per index."""
def __init__(self, mod):
self.m = mod

View File

@@ -1,10 +1,10 @@
"""A 32-lane Maxwell warp emulator for checking the SM86 tensor lowering on the CPU.
tools/xlate/AmpereLowering.cs rewrites SM86 instructions as Maxwell (64-bit) instructions, and the patched Ryujinx shader
tools/xlate/AmpereLowering.cs rewrites SM86 instructions as Maxwell (64-bit) instructions, and the patched shader
translator (build/xlate/ryujinx/src/Ryujinx.Graphics.Shader) turns those into SPIR-V. This file executes the Maxwell
words the way that translator does: fields are decoded exactly as Ryujinx's Decoders/InstDecoders.cs, opcodes are
matched with the bit patterns of Decoders/InstTable.cs, and each instruction does what Ryujinx's Instructions/*.cs
emit for it (not what a real Maxwell GPU does where the two differ; see "Ryujinx semantics" below). With it, the result
words the way that translator does: fields are decoded exactly as its Decoders/InstDecoders.cs, opcodes are
matched with the bit patterns of Decoders/InstTable.cs, and each instruction does what its Instructions/*.cs
emit for it (not what a real Maxwell GPU does where the two differ; see "Translator semantics" below). With it, the result
of one lowered SM86 instruction (HMMA, LDSM, MOVM, HFMA2.RELU, ...) can be compared with vectors captured on real
hardware without running the game.
@@ -12,21 +12,21 @@ State per lane: R0-R254 (RZ = R255 reads 0, writes are dropped), P0-P6 (PT = P7
local memory; per warp: shared memory. Every lane is active at the start; a guard (bits 16-19: predicate 16-18, inverted
19) masks the writes of one instruction, EXIT retires lanes. Branches must be uniform across the active lanes.
Ryujinx semantics this follows (and real hardware may not):
Translator semantics this follows (and real hardware may not):
* Half-precision arithmetic (HADD2, HMUL2, HFMA2) works in F32 on the unpacked halves and rounds to F16 once at the
end (Ryujinx InstEmitFloatArithmetic EmitHadd2Hmul2 / EmitHfma2, InstEmitHelper GetHalfPacked: PackHalf2x16).
end (InstEmitFloatArithmetic EmitHadd2Hmul2 / EmitHfma2, InstEmitHelper GetHalfPacked: PackHalf2x16).
HFMA2 is therefore an F32 fused multiply-add rounded to F32, then to F16.
* That rounding is skipped when the packed value goes straight into another unpack: Ryujinx Optimizer.PropagatePack
* That rounding is skipped when the packed value goes straight into another unpack: Optimizer.PropagatePack
turns UnpackHalf2x16(PackHalf2x16(a, b)) into a (or b). So F2F.F16.F32 then F2F.F32.F16 of the same register, or
HFMA2 feeding HFMA2, passes the F32 value on unrounded; an integer instruction (SHFL, LOP, ISCADD, ...) in between
keeps the rounding, a MOV does not. Modelled per register (Warp.halves); a guarded write and a branch end it.
* FTZ, FMZ and rounding-mode fields of FFMA / FMUL / FADD / HFMA2 / HMUL2 are ignored, as Ryujinx ignores them.
* FTZ, FMZ and rounding-mode fields of FFMA / FMUL / FADD / HFMA2 / HMUL2 are ignored, as the translator ignores them.
Denormals are kept (IEEE), saturation is clamp(x, 0, 1) with NaN giving 0 (GLSL FClamp as min(max(x, 0), 1) on
a GPU whose min/max return the non-NaN operand).
* PRMT is not implemented in Ryujinx 1.3.2: InstEmit.PrmtR/PrmtI/PrmtC/PrmtRc only log "not implemented", so the
* PRMT is not implemented in the translator (1.3.2): InstEmit.PrmtR/PrmtI/PrmtC/PrmtRc only log "not implemented", so the
destination keeps its old value. That is the default here (prmt="ryujinx"); prmt="maxwell" gives the hardware
byte permute instead, to see what the lowering intended.
* SHFL follows Ryujinx's HelperFunctionManager GenerateShuffle* helpers (clamp and segment mask from operand C, the
* SHFL follows the translator's HelperFunctionManager GenerateShuffle* helpers (clamp and segment mask from operand C, the
lane's own value where the source lane is out of range).
* LDS / LDL read the 32-bit word at (address >> 2) and extract a sub-word at (address & 3) * 8 (InstEmitMemory
EmitLoad); STS of 8 and 16 bits stores those bytes at the address.
@@ -218,10 +218,10 @@ class Warp:
self.prmt = prmt
self.trace = trace
self.warnings: list[str] = []
# Ryujinx Optimizer.PropagatePack turns an UnpackHalf2x16 of a PackHalf2x16 result into the pack's F32
# The translator's Optimizer.PropagatePack turns an UnpackHalf2x16 of a PackHalf2x16 result into the pack's F32
# operand, so that half is never rounded to F16. Per lane and register: the F32 halves of the pack that wrote
# the register (sh_ok), kept until the register is written otherwise. A write under a guard ends it in every
# lane: Ryujinx puts the instruction in an if-block and merges old and new value in a phi, which is no pack.
# lane: the translator puts the instruction in an if-block and merges old and new value in a phi, which is no pack.
self.sh_lo = np.zeros((LANES, 256), dtype=F32)
self.sh_hi = np.zeros((LANES, 256), dtype=F32)
self.sh_ok = np.zeros((LANES, 256), dtype=bool)
@@ -479,7 +479,7 @@ def op_lop3_i(w, i, m):
def _prmt(w, i, m, a, sel, c):
if w.prmt == "ryujinx":
# Ryujinx InstEmit.PrmtR / PrmtI: "not implemented", nothing is emitted, the destination keeps its value.
# The translator's InstEmit.PrmtR / PrmtI: "not implemented", nothing is emitted, the destination keeps its value.
w.warnings.append(f"PRMT at {i} is a no-op in Ryujinx 1.3.2 (destination R{i.rd} unchanged)")
return
if i.bits(48, 4):
@@ -882,7 +882,7 @@ def op_exit(w, i, m):
OP_BRA = "bra"
# The decoder: patterns as Ryujinx Decoders/InstTable.cs writes them (bit 63 first, x = any). The most specific
# The decoder: patterns as the translator's Decoders/InstTable.cs writes them (bit 63 first, x = any). The most specific
# matching pattern wins, as InstTable's XBits ordering does.
TABLE = [
("0101000010110xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", "NOP", op_nop),
@@ -993,7 +993,7 @@ def run_warp(words, regs_in=None, smem=None, max_steps: int = 200000, **opts) ->
elif kind == "P" and n != PT:
w.p[lane, n] = bool(val)
addr_to_index = {slot_address(k): k for k in range(len(words) + 1)}
# Where control flow joins (branch targets) Ryujinx merges registers in phis, and an UnpackHalf2x16 of a phi is
# Where control flow joins (branch targets) the translator merges registers in phis, and an UnpackHalf2x16 of a phi is
# not folded: the PropagatePack state (Warp.sh_ok) is dropped there and at every BRA (conservative: a register no
# path writes needs no phi).
joins = set()

View File

@@ -1,11 +1,11 @@
"""Self-test of tests/maxwell_warp_emu.py: hand-encoded Maxwell sequences against hand-computed results.
Part 1 runs Maxwell words encoded here from the field layouts of Ryujinx Decoders/InstDecoders.cs (integer and logic
Part 1 runs Maxwell words encoded here from the field layouts of the translator's Decoders/InstDecoders.cs (integer and logic
ops, SHFL.IDX with clamp and segment masks, LDS/STS sizes, F2F, HFMA2, FFMA rounding, PRMT, predicates, EXIT, BRA).
Part 2, when build/xlate/nx2navi_xlate.exe exists, lowers SM86 HMMA.1688 / HMMA.16816, LDSM and MOVM with
nx2navi_xlate --lower and checks the emulated result against exactly representable reference values (small
integers, so every F16 rounding is exact and only the fragment layout can be wrong), and HFMA2.RELU as the translator
runs it (Ryujinx 1.3.2 drops PRMT, so AmpereLowering.cs Half must not use it). tests/tensor_lowering_check.py checks
runs it (the translator drops PRMT, so AmpereLowering.cs Half must not use it). tests/tensor_lowering_check.py checks
the same lowerings against hardware results.
python tests/maxwell_warp_emu_selftest.py
@@ -195,7 +195,7 @@ def part1():
# (1 + 2^-10)^2 = 1 + 2^-9 + 2^-20 -> F16 1 + 2^-9; 1 + 2^-11 is an F16 tie -> 1.0.
check("HFMA2 rounding", r[:, 10], hpair(1 + 2 ** -9, 1.0))
# Ryujinx Optimizer.PropagatePack: an F16 result unpacked again without an integer operation in between is the
# The translator's Optimizer.PropagatePack: an F16 result unpacked again without an integer operation in between is the
# unrounded F32 value (directly or through a MOV); LOP in between, or a guarded pack, keeps the F16 rounding.
third, third16 = f32bits(1 / 3), f32bits(float(np.float16(1 / 3)))
seq = [mov32i(1, third), f2f(2, 1, 1, 2), f2f(3, 2, 2, 1), # pack, unpack: folded
@@ -222,7 +222,7 @@ def part1():
check("FFMA single rounding at a tie", r[:, 4], 0x3F800001)
check("FFMA", r[:, 7], f32bits(4.5))
# PRMT: a no-op as Ryujinx translates it, the Maxwell byte permute otherwise (0xbb99: sign of bytes 1 and 3).
# PRMT: a no-op as the translator translates it, the Maxwell byte permute otherwise (0xbb99: sign of bytes 1 and 3).
seq = [mov32i(1, 0x7F80C000), mov32i(2, 0xBB99), mov32i(3, 0x11111111), prmt(3, 1, 2, 255),
mov32i(4, 0x3210), mov32i(5, 0x22222222), prmt(5, 1, 4, 255), EXIT]
r = emu.run(seq, {}, prmt="ryujinx")

View File

@@ -228,11 +228,11 @@ def restbl_sizes(table):
return dict(struct.unpack_from('<II', table, 22 + 8 * i) for i in range(count))
MAGIC = 0x18029a7f # MacNativeGames make_button_icons.py: a .bfotf's first word is its key XOR this
MAGIC = 0x18029a7f # a .bfotf's first word is its key XOR this
def bfotf(otf, key):
"""A .bfotf as MacNativeGames make_button_icons.py bfotf_encode writes it: 32-bit words XORed with the key."""
"""A .bfotf as tools/button_icons.py bfotf_encode writes it: 32-bit words XORed with the key."""
size = len(otf)
out = struct.pack('<II', MAGIC ^ key, struct.unpack('<I', size.to_bytes(4, 'big'))[0] ^ key)
out += b''.join(struct.pack('<I', struct.unpack_from('<I', otf, i)[0] ^ key) for i in range(0, size - size % 4, 4))

View File

@@ -1,4 +1,4 @@
# Adapted from nxnative dm/tools/tests/test_spin_waits.py @ 6d34984 (MIT, Copyright (c) 2026 seedds).
# Copyright (c) 2026 seedds (MIT; see THIRD_PARTY_NOTICES.md).
"""tools/spin_waits.py on hand-assembled AArch64 loops, without game files.
Run: .venv/Scripts/python tests/spin_waits_test.py
@@ -12,7 +12,7 @@ sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
from spin_waits import Search # noqa: E402
# A job queue poll (the shape of Breath of the Wild's main+0xef6354): take the queue's spin lock, look whether the wait
# A job queue poll: take the queue's spin lock, look whether the wait
# is over, release the lock, again.
# 0x00 T: cbz x20, X 0x2c str w12, [x20, #0x100] 0x50 ldr w13, [x21, #0x90]
# 0x04 str w11, [x20, #0x100] 0x30 ldrb w14, [x21, #0x98] 0x54 sub w8, w13, w25

View File

@@ -3,7 +3,7 @@
tools/xlate/AmpereLowering.cs rewrites the SM86 instructions of the game's neural-network passes as Maxwell code:
Lowerer.Hmma (HMMA.1688 / HMMA.16816, .F16 and .F32), Lowerer.Ldsm (LDSM.16.M88 / MT88 .1/.2/.4), Lowerer.Movm
(MOVM.16.MT88) and Lowerer.Half (HFMA2, HFMA2.RELU). This script lowers every case of the vector file with
`nx2navi_xlate --lower`, runs the Maxwell words on tests/maxwell_warp_emu.py (which executes them as the Ryujinx
`nx2navi_xlate --lower`, runs the Maxwell words on tests/maxwell_warp_emu.py (which executes them as the shader
translator does) and compares with the registers the GPU wrote. Exit status 1 on any failure. The vector file names
the GPU that produced it ("device"). So far that is an RTX 5090 running the compute_86 PTX through the driver's JIT,
as no SM86 part was at hand: the results follow the PTX semantics but need not show SM86's internal precision.