Guard water absorption distance alongside paired coordinates

Add the exact-material nonnegative distance guard and the recorded NVIDIA translation profiles. Keep generated shaders and the private native patcher out of the repository. Document reported cross-vendor testing and the unresolved origin of invalid absorption distance.
This commit is contained in:
ChurchOfIlias
2026-11-12 03:15:45 -06:00
committed by chrg6
parent d0d0112c3a
commit 37e8e6dbda
3 changed files with 205 additions and 84 deletions

View File

@@ -1,55 +1,88 @@
# Experimental paired water coordinates
# Paired water coordinates and absorption-distance guard
Some views of the Hyrule Castle moat and Castle Town canal produce a rectangle
over solid geometry, sometimes with extreme red output. A paired coordinate
rewrite removed both artifacts in the tested views. This is an opt-in reference
implementation, not a demonstrated correction to the translator or GPU driver.
Some views of the Hyrule Castle moat and Castle Town canal place water rendering
over solid geometry as a rectangle. A second defect can amplify that rendering
into an extreme red beam. These corrections address two separate calculations;
they do not establish the original translator/compiler/driver fault.
Related investigation: https://github.com/tangerineturtle1/NX2navi/issues/398.
That earlier report used a different, fragment-coordinate workaround.
The earlier fragment-coordinate/offset workaround is superseded by the paired
coordinate representation below.
## What changes
## Coordinate correction
The vertex output carries undivided clip X, Y and the actual output W. The
fragment input uses perspective interpolation instead of NoPerspective and
reconstructs X/W and Y/W before the original half-scale and Y inversion. The
fragment's material and sampling calculations remain intact. There are no
empirical offsets, color clamps, or object/scene coordinates in this recipe.
fragment's subsequent material and sampling calculations remain intact. There
are no empirical offsets or object/scene coordinates.
The runtime requires independent stage layouts and the exact original pair:
vertex FNV64 `f6d753852c47b0de`, fragment `a39c3d09241441fa` (descriptor-set-1
form `3d3a316ab4da3bba`). Both stages change together. Separate pre-raster and
fragment library variants prevent reuse of the replacement with another pair.
Other shader pairs and the default runtime behavior remain unchanged.
Both stages change together, only for an exact supported pair and when both
private replacement files exist. Separate pre-raster and fragment library
variants keep the correction from being reused with other shader pairs.
## Independent red-amplification guard
The identified absorption material stores a scalar distance used in attenuation.
A negative distance can turn attenuation into amplification. The recipe inserts
`NMax(distance, 0)` immediately before that exact store. It does not inspect or
suppress red color, alter valid positive distances, or change unrelated effects.
`NMax` also selects zero when only the distance operand is NaN.
This is a defensive bound on an invalid input. The reason the original distance
becomes invalid remains unresolved; this must not be described as a proven
translator or driver root-cause fix. It is independent of the UV representation
and can activate when the paired coordinate correction is not selected.
## Exact translation profiles
| Profile | Vertex | Fragment | Fragment set 1 | Material | Material set 1 |
| --- | --- | --- | --- | --- | --- |
| AMD | f6d753852c47b0de | a39c3d09241441fa | 3d3a316ab4da3bba | e73dfc0f4c556df8 | abfccea256824a78 |
| NVIDIA | 8d83e3705d790fed | 3c20cb98dbaeb257 | 216bd323c6b48ee2 | 061318799d885bcf | b3cec6d0d78ac25a |
These are FNV64 identities of translated modules, not hashes of a game EXE.
Unknown modules retain their original calculation. The runtime requires
independent stage layouts. Its graphics-library variant byte distinguishes
original, paired-coordinate and absorption-guard pipelines.
## Private local preparation
Use privately generated modules from the matching translator configuration:
Use locally held inputs from the same supported translator profile:
```sh
python tools/projective_recipe.py --vertex <local-vertex.spv> --fragment <local-fragment.spv> --output build/water-projective
spirv-val --target-env vulkan1.2 build/water-projective/f6d753852c47b0de.spv
spirv-val --target-env vulkan1.2 build/water-projective/3d3a316ab4da3bba.spv
python tools/projective_recipe.py --vertex <local-vertex.spv> --fragment <local-fragment.spv> --material <local-material.spv> --output build/water-projective
```
Unexpected input hashes and existing output files are rejected. Rebuild the
runtime using the documented UCRT64 workflow. Set `NX2NAVI_SPV_OVERRIDE` to the
absolute output directory in the test process only, then restart the game.
Keep the original installation and use separate pipeline caches for comparisons;
disable pipeline-binary reuse and boot prebuilding during this experiment.
Unset the override and restart to disable it. The directory is checked once at
startup/first pipeline use; do not change its contents during a run.
The material argument is optional for a coordinate-only comparison. The recipe
rejects unknown inputs, mixed profiles and existing output files. It writes
replacement modules named by the original vertex or set-1 fragment identities,
plus a manifest. Validate every generated module with
`spirv-val --target-env vulkan1.2 <module.spv>` before integration.
The exact source integration is compilation-checked, but has not itself had a
gameplay run. The equivalent embedded-module diagnostic correction was tested
on versions 0.7.1 and 0.8.0, game 1.1.1, RX 9070 XT, driver 26.10.1, in repeated
short visits to the moat and market canal. No timed endurance run was recorded.
Other vendors, drivers, resolutions, water scenes and translator versions need
validation before enabling this generally. A changed module deliberately fails
the narrow identity check rather than guessing a match.
Rebuild using the documented UCRT64 workflow. Set `NX2NAVI_SPV_OVERRIDE` to the
absolute output directory in the test process, then restart. Default runtime
behavior is unchanged. Use separate pipeline caches for comparisons and disable
pipeline-binary reuse and boot prebuilding during the experiment. Unset the
override and restart to disable it. Do not change the directory during a run.
## Evidence and limits
## Validation and limits
The recorded fragment invocations with UV error above 0.001 changed as follows:
The equivalent embedded corrections were visually tested on 0.7.1 and 0.8.0,
game 1.1.1, RX 9070 XT / driver 26.10.1, in repeated short visits to the moat
and market canal. No timed endurance run was recorded. The reporter also
reports successful AMD testing beyond the recorded system.
A separate tester has now reported the combined native patch runs cleanly on
NVIDIA without observed breakage. The exact NVIDIA model, driver, duration and
complete scene coverage were not recorded in that report. This is reported
cross-vendor validation of the equivalent embedded correction, not proof that
every GPU, scene, or the source file-loading integration has been gameplay-tested.
The source integration is compilation-checked. Its generated corrections match
the six AMD/NVIDIA modules used by the native test patch byte for byte and pass
SPIR-V validation. The private native patcher is a distribution/test aid and is
not part of this PR.
Recorded fragment invocations with UV error above 0.001:
| View | Original | Paired correction |
| --- | ---: | ---: |
@@ -57,13 +90,15 @@ The recorded fragment invocations with UV error above 0.001 changed as follows:
| Market canal | 7,346 / 72,720 | 0 / 72,033 |
These are instrumented fragment invocations, not unique visible pixels. Views
were revisited manually and are not pixel-identical. Visual checks found no
rectangle or red artifact in those reproductions after correction. This supports
the coordinate representation as a successful workaround; it does not establish
which translation, interpolation, clipping or driver operation first diverged.
The dramatic red output is unintended in those scenes, but its complete
arithmetic origin and portability of this workaround remain unproven.
were revisited manually and are not pixel-identical. The coordinate error
measurements concern the paired correction, not the absorption guard.
This source contains only authored patch logic and diagnostic identifiers.
Generated modules, disassembly, caches, game executables and raw GPU captures
must stay private under ignored build directories and must not be uploaded.
Sewer coverage, other translator outputs, drivers, resolutions and water scenes
remain open validation work. A brief sewer red flash was reported but was not
captured or conclusively attributed to these exact shaders. Do not infer that
all red flashes, the separate full-screen flash bug, or unrelated shadow defects
are fixed by this change. The texture-owner correction is separate (merged #535).
Only authored patch logic, diagnostic identities and documentation are committed.
Generated modules, disassembly, caches, executables and GPU captures stay private
under ignored build directories and must not be uploaded.

View File

@@ -3001,8 +3001,8 @@ static uint32_t vs_union_missing(const VkPipeState *s, GfxShaderIo fin[32]) {
}
// Optional local recipe: both modules are generated privately by projective_recipe.py.
// The switch reuses the existing local override directory and is inactive unless both files exist.
// Optional private modules generated from exact local inputs by projective_recipe.py.
// Mode 1 changes a complete coordinate pair; mode 2 guards one absorption material.
static const char *water_projective_dir(void) {
static PlatMutex lock = PLAT_MUTEX_INIT;
static int checked;
@@ -3011,39 +3011,60 @@ static const char *water_projective_dir(void) {
if (!checked) {
checked = 1;
const char *candidate = cfg_get("NX2NAVI_SPV_OVERRIDE");
char v[700], f[700];
if (candidate && *candidate) {
snprintf(v, sizeof v, "%s/f6d753852c47b0de.spv", candidate);
snprintf(f, sizeof f, "%s/3d3a316ab4da3bba.spv", candidate);
if (plat_file_exists(v) && plat_file_exists(f)) {
dir = candidate;
rt_log("water projective: local paired recipe enabled; other shader pairs retain their original calculation");
}
}
if (candidate && *candidate) dir = candidate;
}
const char *result = dir;
plat_mutex_unlock(&lock);
return result;
}
static int water_projective_pair(const VkPipeState *s) {
if (!P.indep || !s->vs || !s->fs || !water_projective_dir()) return 0;
uint64_t vn = 0, fn = 0;
const uint32_t *v = gfx_shader_spirv(s->vs, &vn), *f = gfx_shader_spirv(s->fs, &fn);
if (!v || !f || fnv(FNV0, v, (size_t)vn * 4) != 0xf6d753852c47b0deull) return 0;
uint64_t h = fnv(FNV0, f, (size_t)fn * 4);
return h == 0xa39c3d09241441faull;
static int water_local_module_exists(uint64_t hash) {
const char *dir = water_projective_dir();
if (!dir) return 0;
char path[700];
int n = snprintf(path, sizeof path, "%s/%016llx.spv", dir, (unsigned long long)hash);
return n > 0 && n < (int)sizeof path && plat_file_exists(path);
}
static VkShaderModule water_pair_module(const uint32_t *w, uint64_t n, int early, int projective) {
if (!projective) return make_module(w, n, early);
static int water_projective_pair(const VkPipeState *s) {
if (!P.indep || !s->fs || !water_projective_dir()) return 0;
uint64_t vn = 0, fn = 0;
const uint32_t *f = gfx_shader_spirv(s->fs, &fn);
if (!f) return 0;
uint64_t fh = fnv(FNV0, f, (size_t)fn * 4);
// The absorption correction is independent of the coordinate pair.
if ((fh == 0xe73dfc0f4c556df8ull && water_local_module_exists(0xabfccea256824a78ull)) ||
(fh == 0x061318799d885bcfull && water_local_module_exists(0xb3cec6d0d78ac25aull))) return 2;
if (!s->vs) return 0;
const uint32_t *v = gfx_shader_spirv(s->vs, &vn);
if (!v) return 0;
uint64_t vh = fnv(FNV0, v, (size_t)vn * 4);
if (vh == 0xf6d753852c47b0deull && fh == 0xa39c3d09241441faull)
return water_local_module_exists(vh) && water_local_module_exists(0x3d3a316ab4da3bbaull);
if (vh == 0x8d83e3705d790fedull && fh == 0x3c20cb98dbaeb257ull)
return water_local_module_exists(vh) && water_local_module_exists(0x216bd323c6b48ee2ull);
return 0;
}
static VkShaderModule water_pair_module(const uint32_t *w, uint64_t n, int early, int mode) {
if (!mode) return make_module(w, n, early);
uint64_t h = fnv(FNV0, w, (size_t)n * 4);
int stage = h == 0xf6d753852c47b0deull ? 0 : h == 0x3d3a316ab4da3bbaull ? 4 : -1;
if (stage < 0) rt_fatal("water projective: paired module changed before the override; refusing a partial recipe");
int stage = -1;
if (mode == 2) {
// Vertex modules and unrelated fragment translations remain original.
if (h != 0xabfccea256824a78ull && h != 0xb3cec6d0d78ac25aull)
return make_module(w, n, early);
stage = 4;
} else {
if (h == 0xf6d753852c47b0deull || h == 0x8d83e3705d790fedull) stage = 0;
if (h == 0x3d3a316ab4da3bbaull || h == 0x216bd323c6b48ee2ull) stage = 4;
if (stage < 0) rt_fatal("water projective: paired module changed; refusing a partial recipe");
}
const char *dir = water_projective_dir();
if (!dir) rt_fatal("water projective: recipe directory unavailable");
char path[700];
snprintf(path, sizeof path, "%s/%016llx.spv", dir, (unsigned long long)h);
int path_len = snprintf(path, sizeof path, "%s/%016llx.spv", dir, (unsigned long long)h);
if (path_len < 0 || path_len >= (int)sizeof path) rt_fatal("water projective: module path too long");
FILE *file = fopen(path, "rb");
if (!file) rt_fatal("water projective: local module missing: %s", path);
fseek(file, 0, SEEK_END);

View File

@@ -9,9 +9,16 @@ import argparse
import json
import struct
VS_HASH = 0xF6D753852C47B0DE
FS_HASH = 0xA39C3D09241441FA
FS_SET1_HASH = 0x3D3A316AB4DA3BBA
# Known translator outputs: (vertex, fragment, descriptor-set-1 fragment).
PAIRS = {
'AMD': (0xF6D753852C47B0DE, 0xA39C3D09241441FA, 0x3D3A316AB4DA3BBA),
'NVIDIA': (0x8D83E3705D790FED, 0x3C20CB98DBAEB257, 0x216BD323C6B48EE2),
}
# Exact material profiles: (original, descriptor-set-1 form, distance pointer/value).
ABSORPTION = {
'AMD': (0xE73DFC0F4C556DF8, 0xABFCCEA256824A78, 1965, 8490),
'NVIDIA': (0x061318799D885BCF, 0xB3CEC6D0D78AC25A, 1969, 8826),
}
def fnv(data):
@@ -178,36 +185,94 @@ def fragment(data):
return mod.bytes()
def set1(data):
mod = Module(data)
for op, args in mod.ops:
if op == 71 and args[1] == 34:
if args[2] not in (0, 1):
raise ValueError('Unexpected descriptor set')
args[2] = 1
return mod.bytes()
def pair_profile(vs, fs):
hashes = fnv(vs), fnv(fs)
matches = [(name, profile) for name, profile in PAIRS.items()
if hashes[0] == profile[0] and hashes[1] in profile[1:]]
if len(matches) != 1:
raise ValueError('Input differs from the narrowly tested shader pairs')
return matches[0]
def prepare(vs, fs):
if fnv(vs) != VS_HASH or fnv(fs) not in (FS_HASH, FS_SET1_HASH):
raise ValueError('Input hash differs from the narrowly tested shader pair')
if fnv(fs) == FS_HASH:
mod = Module(fs)
for op, args in mod.ops:
if op == 71 and args[1] == 34:
if args[2] != 0:
raise ValueError('Unexpected descriptor set')
args[2] = 1
fs = mod.bytes()
if fnv(fs) != FS_SET1_HASH:
raise ValueError('Descriptor remapping did not match the recorded module')
_, profile = pair_profile(vs, fs)
fs = set1(fs)
if fnv(fs) != profile[2]:
raise ValueError('Descriptor remapping did not match the recorded module')
return vertex(vs), fragment(fs)
def absorption(data):
matches = [(name, profile) for name, profile in ABSORPTION.items()
if fnv(data) in profile[:2]]
if len(matches) != 1:
raise ValueError('Input differs from the narrowly tested absorption materials')
name, profile = matches[0]
data = set1(data)
if fnv(data) != profile[1]:
raise ValueError('Material descriptor remapping differs')
mod = Module(data)
if mod.model != 4:
raise ValueError('Expected fragment material')
pointer, distance = profile[2:]
stores = [i for i, (op, args) in enumerate(mod.ops)
if op == 62 and args == [pointer, distance]]
if len(stores) != 1:
raise ValueError('Expected one exact absorption-distance store')
definitions = [args for op, args in mod.ops
if op == 133 and len(args) == 4 and args[1] == distance]
if len(definitions) != 1 or mod.types[definitions[0][0]] != (22, [32]):
raise ValueError('Expected the recorded scalar distance product')
scalar = definitions[0][0]
imports = [args[0] for op, args in mod.ops
if op == 11 and string(args[1:]) == 'GLSL.std.450']
if len(imports) != 1:
raise ValueError('Expected one GLSL extended-instruction import')
zero = mod.constant(scalar, 0)
guarded = mod.new()
# NMax(distance, 0): negative optical distance must not amplify attenuation.
# This targets the distance expression, never a red color or other material.
mod.ops[stores[0]:stores[0] + 1] = [
(12, [scalar, guarded, imports[0], 80, distance, zero]),
(62, [pointer, guarded]),
]
mod.install_declarations()
return name, profile[1], mod.bytes()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--vertex', required=True, type=Path)
parser.add_argument('--fragment', required=True, type=Path)
parser.add_argument('--material', type=Path, help='Optional exact absorption material')
parser.add_argument('--output', required=True, type=Path)
args = parser.parse_args()
vs, fs = prepare(args.vertex.read_bytes(), args.fragment.read_bytes())
vertex_input, fragment_input = args.vertex.read_bytes(), args.fragment.read_bytes()
family, profile = pair_profile(vertex_input, fragment_input)
vs, fs = prepare(vertex_input, fragment_input)
args.output.mkdir(parents=True, exist_ok=True)
files = {f'{VS_HASH:016x}.spv': vs, f'{FS_SET1_HASH:016x}.spv': fs}
files = {f'{profile[0]:016x}.spv': vs, f'{profile[2]:016x}.spv': fs}
if args.material:
material_family, material_hash, material = absorption(args.material.read_bytes())
if material_family != family:
raise ValueError('Pair and material must use the same translator profile')
files[f'{material_hash:016x}.spv'] = material
if any((args.output / name).exists() for name in [*files, 'recipe-manifest.json']):
raise ValueError('Output exists; choose a new ignored build directory')
for name, data in files.items():
(args.output / name).write_bytes(data)
manifest = {'scope': 'Only the identified vertex/fragment pair, changed together',
manifest = {'scope': 'Exact paired coordinates and optional exact-material distance guard',
'family': family,
'outputs': {name: {'fnv': f'{fnv(data):016x}', 'bytes': len(data)}
for name, data in files.items()},
'warning': 'Generated modules are game-derived private build output; do not upload them'}