MacKwnoX/NX2navi
1
0
Fork 0
mirror of https://github.com/tangerineturtle1/NX2navi.git synced 2026-11-15 19:19:58 -04:00
Code Issues Packages Projects Releases Wiki Activity

Add an opt-in paired projective coordinate correction for water

Browse Source 
Carry clip coordinates through perspective interpolation for the identified water shader pair. Keep generated modules private and document the observed results and remaining portability limits.
This commit is contained in:
ChurchOfIlias
2026-11-11 21:19:29 -06:00
committed by chrg6
parent 2f39b27fb6
commit d0d0112c3a
3 changed files with 365 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

69
docs/water-projective.md Normal file
View File

@@ -0,0 +1,69 @@
# Experimental paired water coordinates
Some views of the Hyrule Castle moat and Castle Town canal produce a rectangle
over solid geometry, sometimes with extreme red output. A paired coordinate
rewrite removed both artifacts in the tested views. This is an opt-in reference
implementation, not a demonstrated correction to the translator or GPU driver.
Related investigation: https://github.com/tangerineturtle1/NX2navi/issues/398.
That earlier report used a different, fragment-coordinate workaround.
## What changes
The vertex output carries undivided clip X, Y and the actual output W. The
fragment input uses perspective interpolation instead of NoPerspective and
reconstructs X/W and Y/W before the original half-scale and Y inversion. The
fragment's material and sampling calculations remain intact. There are no
empirical offsets, color clamps, or object/scene coordinates in this recipe.
The runtime requires independent stage layouts and the exact original pair:
vertex FNV64 `f6d753852c47b0de`, fragment `a39c3d09241441fa` (descriptor-set-1
form `3d3a316ab4da3bba`). Both stages change together. Separate pre-raster and
fragment library variants prevent reuse of the replacement with another pair.
Other shader pairs and the default runtime behavior remain unchanged.
## Private local preparation
Use privately generated modules from the matching translator configuration:
```sh
python tools/projective_recipe.py --vertex <local-vertex.spv> --fragment <local-fragment.spv> --output build/water-projective
spirv-val --target-env vulkan1.2 build/water-projective/f6d753852c47b0de.spv
spirv-val --target-env vulkan1.2 build/water-projective/3d3a316ab4da3bba.spv
```
Unexpected input hashes and existing output files are rejected. Rebuild the
runtime using the documented UCRT64 workflow. Set `NX2NAVI_SPV_OVERRIDE` to the
absolute output directory in the test process only, then restart the game.
Keep the original installation and use separate pipeline caches for comparisons;
disable pipeline-binary reuse and boot prebuilding during this experiment.
Unset the override and restart to disable it. The directory is checked once at
startup/first pipeline use; do not change its contents during a run.
The exact source integration is compilation-checked, but has not itself had a
gameplay run. The equivalent embedded-module diagnostic correction was tested
on versions 0.7.1 and 0.8.0, game 1.1.1, RX 9070 XT, driver 26.10.1, in repeated
short visits to the moat and market canal. No timed endurance run was recorded.
Other vendors, drivers, resolutions, water scenes and translator versions need
validation before enabling this generally. A changed module deliberately fails
the narrow identity check rather than guessing a match.
## Evidence and limits
The recorded fragment invocations with UV error above 0.001 changed as follows:
| View | Original | Paired correction |
| --- | ---: | ---: |
| Castle moat | 4,999 / 18,065 | 0 / 20,400 |
| Market canal | 7,346 / 72,720 | 0 / 72,033 |
These are instrumented fragment invocations, not unique visible pixels. Views
were revisited manually and are not pixel-identical. Visual checks found no
rectangle or red artifact in those reproductions after correction. This supports
the coordinate representation as a successful workaround; it does not establish
which translation, interpolation, clipping or driver operation first diverged.
The dramatic red output is unintended in those scenes, but its complete
arithmetic origin and portability of this workaround remain unproven.
This source contains only authored patch logic and diagnostic identifiers.
Generated modules, disassembly, caches, game executables and raw GPU captures
must stay private under ignored build directories and must not be uploaded.

83
runtime/vk_pipeline.c
View File

@@ -3000,6 +3000,76 @@ static uint32_t vs_union_missing(const VkPipeState *s, GfxShaderIo fin[32]) {
return missing;
}
// Optional local recipe: both modules are generated privately by projective_recipe.py.
// The switch reuses the existing local override directory and is inactive unless both files exist.
static const char *water_projective_dir(void) {
static PlatMutex lock = PLAT_MUTEX_INIT;
static int checked;
static const char *dir;
plat_mutex_lock(&lock);
if (!checked) {
checked = 1;
const char *candidate = cfg_get("NX2NAVI_SPV_OVERRIDE");
char v[700], f[700];
if (candidate && *candidate) {
snprintf(v, sizeof v, "%s/f6d753852c47b0de.spv", candidate);
snprintf(f, sizeof f, "%s/3d3a316ab4da3bba.spv", candidate);
if (plat_file_exists(v) && plat_file_exists(f)) {
dir = candidate;
rt_log("water projective: local paired recipe enabled; other shader pairs retain their original calculation");
}
}
}
const char *result = dir;
plat_mutex_unlock(&lock);
return result;
}
static int water_projective_pair(const VkPipeState *s) {
if (!P.indep || !s->vs || !s->fs || !water_projective_dir()) return 0;
uint64_t vn = 0, fn = 0;
const uint32_t *v = gfx_shader_spirv(s->vs, &vn), *f = gfx_shader_spirv(s->fs, &fn);
if (!v || !f || fnv(FNV0, v, (size_t)vn * 4) != 0xf6d753852c47b0deull) return 0;
uint64_t h = fnv(FNV0, f, (size_t)fn * 4);
return h == 0xa39c3d09241441faull;
}
static VkShaderModule water_pair_module(const uint32_t *w, uint64_t n, int early, int projective) {
if (!projective) return make_module(w, n, early);
uint64_t h = fnv(FNV0, w, (size_t)n * 4);
int stage = h == 0xf6d753852c47b0deull ? 0 : h == 0x3d3a316ab4da3bbaull ? 4 : -1;
if (stage < 0) rt_fatal("water projective: paired module changed before the override; refusing a partial recipe");
const char *dir = water_projective_dir();
if (!dir) rt_fatal("water projective: recipe directory unavailable");
char path[700];
snprintf(path, sizeof path, "%s/%016llx.spv", dir, (unsigned long long)h);
FILE *file = fopen(path, "rb");
if (!file) rt_fatal("water projective: local module missing: %s", path);
fseek(file, 0, SEEK_END);
long bytes = ftell(file);
rewind(file);
uint32_t *code = bytes >= 20 && bytes <= 1048576 && !(bytes & 3) ? malloc((size_t)bytes) : NULL;
if (!code || fread(code, 1, (size_t)bytes, file) != (size_t)bytes)
rt_fatal("water projective: invalid local module: %s", path);
fclose(file);
if (code[0] != 0x07230203u) rt_fatal("water projective: invalid SPIR-V header");
int entries = 0;
for (size_t at = 5; at < (size_t)bytes / 4;) {
uint32_t len = code[at] >> 16, op = code[at] & 65535;
if (!len || at + len > (size_t)bytes / 4) rt_fatal("water projective: malformed instruction");
if (op == 15) {
if (len < 4 || code[at + 1] != (uint32_t)stage) rt_fatal("water projective: wrong module stage");
entries++;
}
at += len;
}
if (entries != 1) rt_fatal("water projective: unexpected entry-point count");
VkShaderModule result = make_module(code, (uint64_t)bytes / 4, early);
free(code);
return result;
}
static VkPipeline create_graphics(const VkPipeState *s, const GKey *k, const Layout *l, uint32_t fs_off) {
VkShaderModule vm = VK_NULL_HANDLE, fm = VK_NULL_HANDLE;
uint64_t vw, fw, vm_hash = 0, fm_hash = 0; // the hashes: of the final modules, for the PIPE_FEEDBACK 2 line
@@ -3062,9 +3132,9 @@ static VkPipeline create_graphics(const VkPipeState *s, const GKey *k, const Lay
fcode = fpatched = r;
}
// A bound colour target outside the shader's output map gets write mask 0 (target_wmask)
if (fcode) fm = make_module(fcode, fw, !k->a2c), fm_hash = mm_hash;
if (fcode) fm = water_pair_module(fcode, fw, !k->a2c, water_projective_pair(s)), fm_hash = mm_hash;
}
vm = make_module(vcode, vw, 0), vm_hash = mm_hash;
vm = water_pair_module(vcode, vw, 0, water_projective_pair(s)), vm_hash = mm_hash;
free(vpatched), free(fpatched);
VkPipeline pipe = VK_NULL_HANDLE;
if (!vm || (s->fs && !fm) || (mesh && s->ts && !tm)) goto out;
@@ -3221,8 +3291,8 @@ typedef struct Lib { struct Lib *next; uint64_t hash; VkPipeline pipe; uint32_t
static Lib *libs[LIB_BUCKETS];
typedef struct { uint32_t kind, topo_class, attr_fmt[32], attr_off[32]; uint8_t attr_bind[32], rate[17], pad[3]; } ViKey;
typedef struct { uint32_t kind, missing; const GfxShader *vs; uint8_t clamp, poly_mode, minus_w, pad; GfxShaderIo fin[32]; } PrKey;
typedef struct { uint32_t kind; uint8_t dsb, samples, a2c, pad; const GfxShader *fs; } FsKey;
typedef struct { uint32_t kind, missing; const GfxShader *vs; uint8_t clamp, poly_mode, minus_w, water_projective; GfxShaderIo fin[32]; } PrKey;
typedef struct { uint32_t kind; uint8_t dsb, samples, a2c, water_projective; const GfxShader *fs; } FsKey;
typedef struct { uint32_t kind; uint8_t n_color, samples, a2c, logic_op, blend_en, fbl, pad[2]; uint32_t color[8], depth, stencil, wmask;
uint8_t blend[8][6]; } FoKey; // fbl: GKey.fbl, the feedback loop flags of the library (gpl_feedback)
@@ -3297,7 +3367,7 @@ static VkPipeline lib_create(int kind, const VkPipeState *s, const GKey *k, VkPi
sw = vw;
if (pk->missing && (patched = spirv_add_outputs(vcode, vw, pk->missing, pk->fin, &n2))) vcode = patched, vw = n2;
else if (pk->missing) LOG_ONCE("vk_pipeline: cannot add missing vertex outputs (mask %#x)", pk->missing);
if (!(m = make_module(vcode, vw, 0))) goto out;
if (!(m = water_pair_module(vcode, vw, 0, pk->water_projective))) goto out;
st.stage = VK_SHADER_STAGE_VERTEX_BIT, st.module = m;
pin_subgroup32(&st, &rq, scode, sw);
pi.stageCount = 1, pi.pStages = &st, pi.pViewportState = &vp, pi.pRasterizationState = &rs;
@@ -3314,7 +3384,7 @@ static VkPipeline lib_create(int kind, const VkPipeState *s, const GKey *k, VkPi
}
patched = fcode ? spirv_set(fcode, fw, 1) : NULL;
free(d);
if (!patched || !(m = make_module(patched, fw, !k->a2c))) goto out;
if (!patched || !(m = water_pair_module(patched, fw, !k->a2c, ((const FsKey *)key)->water_projective))) goto out;
st.stage = VK_SHADER_STAGE_FRAGMENT_BIT, st.module = m;
pin_subgroup32(&st, &rq, scode, sw);
pi.stageCount = 1, pi.pStages = &st;
@@ -3519,6 +3589,7 @@ static void lib_keys(const VkPipeState *s, const GKey *k, ViKey *v, PrKey *pk, F
if (pk->missing >> i & 1) pk->fin[i] = fin[i];
memset(fk, 0, sizeof *fk);
fk->kind = LIB_FS, fk->fs = s->fs, fk->dsb = k->dsb, fk->samples = k->samples, fk->a2c = k->a2c;
pk->water_projective = fk->water_projective = (uint8_t)water_projective_pair(s);
memset(ok, 0, sizeof *ok);
ok->kind = LIB_FO, ok->n_color = k->n_color, ok->samples = k->samples, ok->a2c = k->a2c, ok->logic_op = k->logic_op;
ok->blend_en = k->blend_en, ok->fbl = k->fbl, ok->depth = k->depth, ok->stencil = k->stencil, ok->wmask = k->wmask;

219
tools/projective_recipe.py Normal file
View File

@@ -0,0 +1,219 @@
"""Opt-in source recipe for locally held modules; this file contains no game shader.
Only the identified input hashes are accepted. Generated SPIR-V is private build
output and must not be attached to an issue or committed to the repository.
The recipe implements the tested coordinate representation, not its recorder.
"""
from pathlib import Path
import argparse
import json
import struct
VS_HASH = 0xF6D753852C47B0DE
FS_HASH = 0xA39C3D09241441FA
FS_SET1_HASH = 0x3D3A316AB4DA3BBA
def fnv(data):
value = 0xCBF29CE484222325
for byte in data:
value = ((value ^ byte) * 0x100000001B3) & 0xFFFFFFFFFFFFFFFF
return value
def string(words):
return struct.pack('<%dI' % len(words), *words).split(b'\0', 1)[0].decode()
class Module:
def __init__(self, data):
if len(data) < 20 or len(data) % 4:
raise ValueError('Not a complete SPIR-V module')
words = list(struct.unpack('<%dI' % (len(data) // 4), data))
if words[0] != 0x07230203:
raise ValueError('Expected little-endian SPIR-V')
self.header, self.ops = words[:5], []
at = 5
while at < len(words):
size, op = words[at] >> 16, words[at] & 65535
if not size or at + size > len(words):
raise ValueError('Malformed instruction')
self.ops.append((op, words[at + 1:at + size]))
at += size
entries = [args for op, args in self.ops if op == 15]
if len(entries) != 1 or sum(op == 54 for op, _ in self.ops) != 1:
raise ValueError('This narrow recipe expects one entry point and function')
if sum(op == 253 for op, _ in self.ops) != 1:
raise ValueError('This narrow recipe expects one return')
self.model = entries[0][0]
self.types = {a[0]: (op, a[1:]) for op, a in self.ops if 19 <= op <= 33}
self.variables = {a[1]: (a[0], a[2]) for op, a in self.ops if op == 59}
self.extra = []
def new(self):
result = self.header[3]
self.header[3] += 1
return result
def type(self, op, *args):
for ident, definition in self.types.items():
if definition == (op, list(args)):
return ident
ident = self.new()
self.types[ident] = (op, list(args))
self.extra.append((op, [ident, *args]))
return ident
def constant(self, ty, bits):
for op, args in self.ops + self.extra:
if op == 43 and args[0] == ty and args[2:] == [bits]:
return args[1]
ident = self.new()
self.extra.append((43, [ty, ident, bits]))
return ident
def location0(self, storage):
ids = [a[0] for op, a in self.ops if op == 71 and a[1:] == [30, 0]
and a[0] in self.variables and self.variables[a[0]][1] == storage]
if len(ids) != 1:
raise ValueError('Expected one location-zero interface variable')
var = ids[0]
pointer = self.variables[var][0]
op, args = self.types[pointer]
if op != 32 or args[0] != storage:
raise ValueError('Unexpected interface pointer')
vec = args[1]
op, args = self.types[vec]
if op != 23 or args[1] != 4 or self.types[args[0]] != (22, [32]):
raise ValueError('Expected a four-component float interface')
return var, vec, args[0]
def install_declarations(self):
at = next(i for i, (op, _) in enumerate(self.ops) if op == 54)
self.ops[at:at] = self.extra
self.extra = []
def bytes(self):
words = self.header[:]
for op, args in self.ops:
words.extend([((len(args) + 1) << 16) | op, *args])
return struct.pack('<%dI' % len(words), *words)
def vertex(data):
mod = Module(data)
if mod.model != 0:
raise ValueError('Expected vertex stage')
varying, vec, scalar = mod.location0(3)
positions = [(a[0], a[1]) for op, a in mod.ops
if op == 72 and a[2:] == [11, 0]]
if len(positions) != 1:
raise ValueError('Expected one Position structure member')
structure, member = positions[0]
candidates = [var for var, (ptr, storage) in mod.variables.items()
if storage == 3 and mod.types[ptr] == (32, [3, structure])]
if len(candidates) != 1 or mod.types[structure][1][member] != vec:
raise ValueError('Unexpected Position block')
pointer = mod.type(32, 3, vec)
integer = mod.type(21, 32, 1)
member_id = mod.constant(integer, member)
one = mod.constant(scalar, 0x3F800000)
ptr, pos, x, y, w, encoded = [mod.new() for _ in range(6)]
code = [(65, [pointer, ptr, candidates[0], member_id]),
(61, [vec, pos, ptr]),
(81, [scalar, x, pos, 0]), (81, [scalar, y, pos, 1]),
(81, [scalar, w, pos, 3]),
(80, [vec, encoded, x, y, w, one]), (62, [varying, encoded])]
at = next(i for i, (op, _) in enumerate(mod.ops) if op == 253)
mod.ops[at:at] = code
mod.install_declarations()
return mod.bytes()
def fragment(data):
mod = Module(data)
if mod.model != 4:
raise ValueError('Expected fragment stage')
varying, vec, scalar = mod.location0(1)
if sum(op == 71 and a == [varying, 13] for op, a in mod.ops) != 1:
raise ValueError('Expected NoPerspective on the original input')
mod.ops = [(op, a) for op, a in mod.ops if not (op == 71 and a == [varying, 13])]
imports = [a[0] for op, a in mod.ops if op == 11 and string(a[1:]) == 'GLSL.std.450']
if len(imports) != 1:
raise ValueError('Expected GLSL extended instructions')
local_pointer = mod.type(32, 7, vec)
local = mod.new()
# Redirect only original accesses. The raw input load added below must remain an input load.
count = 0
for op, args in mod.ops:
if op in (65, 66) and args[2] == varying:
old_op, old_args = mod.types[args[0]]
if old_op != 32 or old_args[0] != 1:
raise ValueError('Unexpected access pointer')
args[0] = mod.type(32, 7, old_args[1])
args[2] = local
count += 1
elif op == 61 and args[2] == varying:
args[2] = local
count += 1
if count != 2:
raise ValueError('Unexpected original varying-access count')
zero = mod.constant(scalar, 0)
one = mod.constant(scalar, 0x3F800000)
half = mod.constant(scalar, 0x3F000000)
negative_half = mod.constant(scalar, 0xBF000000)
raw, x, y, w, nx, ny, u, v, decoded = [mod.new() for _ in range(9)]
code = [(61, [vec, raw, varying]),
(81, [scalar, x, raw, 0]), (81, [scalar, y, raw, 1]),
(81, [scalar, w, raw, 2]),
(136, [scalar, nx, x, w]), (136, [scalar, ny, y, w]),
(12, [scalar, u, imports[0], 50, nx, half, half]),
(12, [scalar, v, imports[0], 50, ny, negative_half, half]),
(80, [vec, decoded, u, v, zero, one]), (62, [local, decoded])]
at = next(i for i, (op, _) in enumerate(mod.ops) if op == 248) + 1
while mod.ops[at][0] == 59:
at += 1
mod.ops[at:at] = [(59, [local_pointer, local, 7]), *code]
mod.install_declarations()
return mod.bytes()
def prepare(vs, fs):
if fnv(vs) != VS_HASH or fnv(fs) not in (FS_HASH, FS_SET1_HASH):
raise ValueError('Input hash differs from the narrowly tested shader pair')
if fnv(fs) == FS_HASH:
mod = Module(fs)
for op, args in mod.ops:
if op == 71 and args[1] == 34:
if args[2] != 0:
raise ValueError('Unexpected descriptor set')
args[2] = 1
fs = mod.bytes()
if fnv(fs) != FS_SET1_HASH:
raise ValueError('Descriptor remapping did not match the recorded module')
return vertex(vs), fragment(fs)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--vertex', required=True, type=Path)
parser.add_argument('--fragment', required=True, type=Path)
parser.add_argument('--output', required=True, type=Path)
args = parser.parse_args()
vs, fs = prepare(args.vertex.read_bytes(), args.fragment.read_bytes())
args.output.mkdir(parents=True, exist_ok=True)
files = {f'{VS_HASH:016x}.spv': vs, f'{FS_SET1_HASH:016x}.spv': fs}
if any((args.output / name).exists() for name in [*files, 'recipe-manifest.json']):
raise ValueError('Output exists; choose a new ignored build directory')
for name, data in files.items():
(args.output / name).write_bytes(data)
manifest = {'scope': 'Only the identified vertex/fragment pair, changed together',
'outputs': {name: {'fnv': f'{fnv(data):016x}', 'bytes': len(data)}
for name, data in files.items()},
'warning': 'Generated modules are game-derived private build output; do not upload them'}
(args.output / 'recipe-manifest.json').write_text(json.dumps(manifest, indent=2))
print('Created private local modules. Validate them before integration; do not upload them.')
if __name__ == '__main__':
main()