mirror of
https://github.com/tangerineturtle1/NX2navi.git
synced 2026-11-17 00:39:57 -04:00
Guard water absorption distance alongside paired coordinates
Add the exact-material nonnegative distance guard and the recorded NVIDIA translation profiles. Keep generated shaders and the private native patcher out of the repository. Document reported cross-vendor testing and the unresolved origin of invalid absorption distance.
This commit is contained in:
@@ -1,55 +1,88 @@
|
||||
# Experimental paired water coordinates
|
||||
# Paired water coordinates and absorption-distance guard
|
||||
|
||||
Some views of the Hyrule Castle moat and Castle Town canal produce a rectangle
|
||||
over solid geometry, sometimes with extreme red output. A paired coordinate
|
||||
rewrite removed both artifacts in the tested views. This is an opt-in reference
|
||||
implementation, not a demonstrated correction to the translator or GPU driver.
|
||||
Some views of the Hyrule Castle moat and Castle Town canal place water rendering
|
||||
over solid geometry as a rectangle. A second defect can amplify that rendering
|
||||
into an extreme red beam. These corrections address two separate calculations;
|
||||
they do not establish the original translator/compiler/driver fault.
|
||||
Related investigation: https://github.com/tangerineturtle1/NX2navi/issues/398.
|
||||
That earlier report used a different, fragment-coordinate workaround.
|
||||
The earlier fragment-coordinate/offset workaround is superseded by the paired
|
||||
coordinate representation below.
|
||||
|
||||
## What changes
|
||||
## Coordinate correction
|
||||
|
||||
The vertex output carries undivided clip X, Y and the actual output W. The
|
||||
fragment input uses perspective interpolation instead of NoPerspective and
|
||||
reconstructs X/W and Y/W before the original half-scale and Y inversion. The
|
||||
fragment's material and sampling calculations remain intact. There are no
|
||||
empirical offsets, color clamps, or object/scene coordinates in this recipe.
|
||||
fragment's subsequent material and sampling calculations remain intact. There
|
||||
are no empirical offsets or object/scene coordinates.
|
||||
|
||||
The runtime requires independent stage layouts and the exact original pair:
|
||||
vertex FNV64 `f6d753852c47b0de`, fragment `a39c3d09241441fa` (descriptor-set-1
|
||||
form `3d3a316ab4da3bba`). Both stages change together. Separate pre-raster and
|
||||
fragment library variants prevent reuse of the replacement with another pair.
|
||||
Other shader pairs and the default runtime behavior remain unchanged.
|
||||
Both stages change together, only for an exact supported pair and when both
|
||||
private replacement files exist. Separate pre-raster and fragment library
|
||||
variants keep the correction from being reused with other shader pairs.
|
||||
|
||||
## Independent red-amplification guard
|
||||
|
||||
The identified absorption material stores a scalar distance used in attenuation.
|
||||
A negative distance can turn attenuation into amplification. The recipe inserts
|
||||
`NMax(distance, 0)` immediately before that exact store. It does not inspect or
|
||||
suppress red color, alter valid positive distances, or change unrelated effects.
|
||||
`NMax` also selects zero when only the distance operand is NaN.
|
||||
|
||||
This is a defensive bound on an invalid input. The reason the original distance
|
||||
becomes invalid remains unresolved; this must not be described as a proven
|
||||
translator or driver root-cause fix. It is independent of the UV representation
|
||||
and can activate when the paired coordinate correction is not selected.
|
||||
|
||||
## Exact translation profiles
|
||||
|
||||
| Profile | Vertex | Fragment | Fragment set 1 | Material | Material set 1 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| AMD | f6d753852c47b0de | a39c3d09241441fa | 3d3a316ab4da3bba | e73dfc0f4c556df8 | abfccea256824a78 |
|
||||
| NVIDIA | 8d83e3705d790fed | 3c20cb98dbaeb257 | 216bd323c6b48ee2 | 061318799d885bcf | b3cec6d0d78ac25a |
|
||||
|
||||
These are FNV64 identities of translated modules, not hashes of a game EXE.
|
||||
Unknown modules retain their original calculation. The runtime requires
|
||||
independent stage layouts. Its graphics-library variant byte distinguishes
|
||||
original, paired-coordinate and absorption-guard pipelines.
|
||||
|
||||
## Private local preparation
|
||||
|
||||
Use privately generated modules from the matching translator configuration:
|
||||
Use locally held inputs from the same supported translator profile:
|
||||
|
||||
```sh
|
||||
python tools/projective_recipe.py --vertex <local-vertex.spv> --fragment <local-fragment.spv> --output build/water-projective
|
||||
spirv-val --target-env vulkan1.2 build/water-projective/f6d753852c47b0de.spv
|
||||
spirv-val --target-env vulkan1.2 build/water-projective/3d3a316ab4da3bba.spv
|
||||
python tools/projective_recipe.py --vertex <local-vertex.spv> --fragment <local-fragment.spv> --material <local-material.spv> --output build/water-projective
|
||||
```
|
||||
|
||||
Unexpected input hashes and existing output files are rejected. Rebuild the
|
||||
runtime using the documented UCRT64 workflow. Set `NX2NAVI_SPV_OVERRIDE` to the
|
||||
absolute output directory in the test process only, then restart the game.
|
||||
Keep the original installation and use separate pipeline caches for comparisons;
|
||||
disable pipeline-binary reuse and boot prebuilding during this experiment.
|
||||
Unset the override and restart to disable it. The directory is checked once at
|
||||
startup/first pipeline use; do not change its contents during a run.
|
||||
The material argument is optional for a coordinate-only comparison. The recipe
|
||||
rejects unknown inputs, mixed profiles and existing output files. It writes
|
||||
replacement modules named by the original vertex or set-1 fragment identities,
|
||||
plus a manifest. Validate every generated module with
|
||||
`spirv-val --target-env vulkan1.2 <module.spv>` before integration.
|
||||
|
||||
The exact source integration is compilation-checked, but has not itself had a
|
||||
gameplay run. The equivalent embedded-module diagnostic correction was tested
|
||||
on versions 0.7.1 and 0.8.0, game 1.1.1, RX 9070 XT, driver 26.10.1, in repeated
|
||||
short visits to the moat and market canal. No timed endurance run was recorded.
|
||||
Other vendors, drivers, resolutions, water scenes and translator versions need
|
||||
validation before enabling this generally. A changed module deliberately fails
|
||||
the narrow identity check rather than guessing a match.
|
||||
Rebuild using the documented UCRT64 workflow. Set `NX2NAVI_SPV_OVERRIDE` to the
|
||||
absolute output directory in the test process, then restart. Default runtime
|
||||
behavior is unchanged. Use separate pipeline caches for comparisons and disable
|
||||
pipeline-binary reuse and boot prebuilding during the experiment. Unset the
|
||||
override and restart to disable it. Do not change the directory during a run.
|
||||
|
||||
## Evidence and limits
|
||||
## Validation and limits
|
||||
|
||||
The recorded fragment invocations with UV error above 0.001 changed as follows:
|
||||
The equivalent embedded corrections were visually tested on 0.7.1 and 0.8.0,
|
||||
game 1.1.1, RX 9070 XT / driver 26.10.1, in repeated short visits to the moat
|
||||
and market canal. No timed endurance run was recorded. The reporter also
|
||||
reports successful AMD testing beyond the recorded system.
|
||||
|
||||
A separate tester has now reported the combined native patch runs cleanly on
|
||||
NVIDIA without observed breakage. The exact NVIDIA model, driver, duration and
|
||||
complete scene coverage were not recorded in that report. This is reported
|
||||
cross-vendor validation of the equivalent embedded correction, not proof that
|
||||
every GPU, scene, or the source file-loading integration has been gameplay-tested.
|
||||
The source integration is compilation-checked. Its generated corrections match
|
||||
the six AMD/NVIDIA modules used by the native test patch byte for byte and pass
|
||||
SPIR-V validation. The private native patcher is a distribution/test aid and is
|
||||
not part of this PR.
|
||||
|
||||
Recorded fragment invocations with UV error above 0.001:
|
||||
|
||||
| View | Original | Paired correction |
|
||||
| --- | ---: | ---: |
|
||||
@@ -57,13 +90,15 @@ The recorded fragment invocations with UV error above 0.001 changed as follows:
|
||||
| Market canal | 7,346 / 72,720 | 0 / 72,033 |
|
||||
|
||||
These are instrumented fragment invocations, not unique visible pixels. Views
|
||||
were revisited manually and are not pixel-identical. Visual checks found no
|
||||
rectangle or red artifact in those reproductions after correction. This supports
|
||||
the coordinate representation as a successful workaround; it does not establish
|
||||
which translation, interpolation, clipping or driver operation first diverged.
|
||||
The dramatic red output is unintended in those scenes, but its complete
|
||||
arithmetic origin and portability of this workaround remain unproven.
|
||||
were revisited manually and are not pixel-identical. The coordinate error
|
||||
measurements concern the paired correction, not the absorption guard.
|
||||
|
||||
This source contains only authored patch logic and diagnostic identifiers.
|
||||
Generated modules, disassembly, caches, game executables and raw GPU captures
|
||||
must stay private under ignored build directories and must not be uploaded.
|
||||
Sewer coverage, other translator outputs, drivers, resolutions and water scenes
|
||||
remain open validation work. A brief sewer red flash was reported but was not
|
||||
captured or conclusively attributed to these exact shaders. Do not infer that
|
||||
all red flashes, the separate full-screen flash bug, or unrelated shadow defects
|
||||
are fixed by this change. The texture-owner correction is separate (merged #535).
|
||||
|
||||
Only authored patch logic, diagnostic identities and documentation are committed.
|
||||
Generated modules, disassembly, caches, executables and GPU captures stay private
|
||||
under ignored build directories and must not be uploaded.
|
||||
|
||||
@@ -3001,8 +3001,8 @@ static uint32_t vs_union_missing(const VkPipeState *s, GfxShaderIo fin[32]) {
|
||||
}
|
||||
|
||||
|
||||
// Optional local recipe: both modules are generated privately by projective_recipe.py.
|
||||
// The switch reuses the existing local override directory and is inactive unless both files exist.
|
||||
// Optional private modules generated from exact local inputs by projective_recipe.py.
|
||||
// Mode 1 changes a complete coordinate pair; mode 2 guards one absorption material.
|
||||
static const char *water_projective_dir(void) {
|
||||
static PlatMutex lock = PLAT_MUTEX_INIT;
|
||||
static int checked;
|
||||
@@ -3011,39 +3011,60 @@ static const char *water_projective_dir(void) {
|
||||
if (!checked) {
|
||||
checked = 1;
|
||||
const char *candidate = cfg_get("NX2NAVI_SPV_OVERRIDE");
|
||||
char v[700], f[700];
|
||||
if (candidate && *candidate) {
|
||||
snprintf(v, sizeof v, "%s/f6d753852c47b0de.spv", candidate);
|
||||
snprintf(f, sizeof f, "%s/3d3a316ab4da3bba.spv", candidate);
|
||||
if (plat_file_exists(v) && plat_file_exists(f)) {
|
||||
dir = candidate;
|
||||
rt_log("water projective: local paired recipe enabled; other shader pairs retain their original calculation");
|
||||
}
|
||||
}
|
||||
if (candidate && *candidate) dir = candidate;
|
||||
}
|
||||
const char *result = dir;
|
||||
plat_mutex_unlock(&lock);
|
||||
return result;
|
||||
}
|
||||
|
||||
static int water_projective_pair(const VkPipeState *s) {
|
||||
if (!P.indep || !s->vs || !s->fs || !water_projective_dir()) return 0;
|
||||
uint64_t vn = 0, fn = 0;
|
||||
const uint32_t *v = gfx_shader_spirv(s->vs, &vn), *f = gfx_shader_spirv(s->fs, &fn);
|
||||
if (!v || !f || fnv(FNV0, v, (size_t)vn * 4) != 0xf6d753852c47b0deull) return 0;
|
||||
uint64_t h = fnv(FNV0, f, (size_t)fn * 4);
|
||||
return h == 0xa39c3d09241441faull;
|
||||
static int water_local_module_exists(uint64_t hash) {
|
||||
const char *dir = water_projective_dir();
|
||||
if (!dir) return 0;
|
||||
char path[700];
|
||||
int n = snprintf(path, sizeof path, "%s/%016llx.spv", dir, (unsigned long long)hash);
|
||||
return n > 0 && n < (int)sizeof path && plat_file_exists(path);
|
||||
}
|
||||
|
||||
static VkShaderModule water_pair_module(const uint32_t *w, uint64_t n, int early, int projective) {
|
||||
if (!projective) return make_module(w, n, early);
|
||||
static int water_projective_pair(const VkPipeState *s) {
|
||||
if (!P.indep || !s->fs || !water_projective_dir()) return 0;
|
||||
uint64_t vn = 0, fn = 0;
|
||||
const uint32_t *f = gfx_shader_spirv(s->fs, &fn);
|
||||
if (!f) return 0;
|
||||
uint64_t fh = fnv(FNV0, f, (size_t)fn * 4);
|
||||
// The absorption correction is independent of the coordinate pair.
|
||||
if ((fh == 0xe73dfc0f4c556df8ull && water_local_module_exists(0xabfccea256824a78ull)) ||
|
||||
(fh == 0x061318799d885bcfull && water_local_module_exists(0xb3cec6d0d78ac25aull))) return 2;
|
||||
if (!s->vs) return 0;
|
||||
const uint32_t *v = gfx_shader_spirv(s->vs, &vn);
|
||||
if (!v) return 0;
|
||||
uint64_t vh = fnv(FNV0, v, (size_t)vn * 4);
|
||||
if (vh == 0xf6d753852c47b0deull && fh == 0xa39c3d09241441faull)
|
||||
return water_local_module_exists(vh) && water_local_module_exists(0x3d3a316ab4da3bbaull);
|
||||
if (vh == 0x8d83e3705d790fedull && fh == 0x3c20cb98dbaeb257ull)
|
||||
return water_local_module_exists(vh) && water_local_module_exists(0x216bd323c6b48ee2ull);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static VkShaderModule water_pair_module(const uint32_t *w, uint64_t n, int early, int mode) {
|
||||
if (!mode) return make_module(w, n, early);
|
||||
uint64_t h = fnv(FNV0, w, (size_t)n * 4);
|
||||
int stage = h == 0xf6d753852c47b0deull ? 0 : h == 0x3d3a316ab4da3bbaull ? 4 : -1;
|
||||
if (stage < 0) rt_fatal("water projective: paired module changed before the override; refusing a partial recipe");
|
||||
int stage = -1;
|
||||
if (mode == 2) {
|
||||
// Vertex modules and unrelated fragment translations remain original.
|
||||
if (h != 0xabfccea256824a78ull && h != 0xb3cec6d0d78ac25aull)
|
||||
return make_module(w, n, early);
|
||||
stage = 4;
|
||||
} else {
|
||||
if (h == 0xf6d753852c47b0deull || h == 0x8d83e3705d790fedull) stage = 0;
|
||||
if (h == 0x3d3a316ab4da3bbaull || h == 0x216bd323c6b48ee2ull) stage = 4;
|
||||
if (stage < 0) rt_fatal("water projective: paired module changed; refusing a partial recipe");
|
||||
}
|
||||
const char *dir = water_projective_dir();
|
||||
if (!dir) rt_fatal("water projective: recipe directory unavailable");
|
||||
char path[700];
|
||||
snprintf(path, sizeof path, "%s/%016llx.spv", dir, (unsigned long long)h);
|
||||
int path_len = snprintf(path, sizeof path, "%s/%016llx.spv", dir, (unsigned long long)h);
|
||||
if (path_len < 0 || path_len >= (int)sizeof path) rt_fatal("water projective: module path too long");
|
||||
FILE *file = fopen(path, "rb");
|
||||
if (!file) rt_fatal("water projective: local module missing: %s", path);
|
||||
fseek(file, 0, SEEK_END);
|
||||
|
||||
@@ -9,9 +9,16 @@ import argparse
|
||||
import json
|
||||
import struct
|
||||
|
||||
VS_HASH = 0xF6D753852C47B0DE
|
||||
FS_HASH = 0xA39C3D09241441FA
|
||||
FS_SET1_HASH = 0x3D3A316AB4DA3BBA
|
||||
# Known translator outputs: (vertex, fragment, descriptor-set-1 fragment).
|
||||
PAIRS = {
|
||||
'AMD': (0xF6D753852C47B0DE, 0xA39C3D09241441FA, 0x3D3A316AB4DA3BBA),
|
||||
'NVIDIA': (0x8D83E3705D790FED, 0x3C20CB98DBAEB257, 0x216BD323C6B48EE2),
|
||||
}
|
||||
# Exact material profiles: (original, descriptor-set-1 form, distance pointer/value).
|
||||
ABSORPTION = {
|
||||
'AMD': (0xE73DFC0F4C556DF8, 0xABFCCEA256824A78, 1965, 8490),
|
||||
'NVIDIA': (0x061318799D885BCF, 0xB3CEC6D0D78AC25A, 1969, 8826),
|
||||
}
|
||||
|
||||
|
||||
def fnv(data):
|
||||
@@ -178,36 +185,94 @@ def fragment(data):
|
||||
return mod.bytes()
|
||||
|
||||
|
||||
def set1(data):
|
||||
mod = Module(data)
|
||||
for op, args in mod.ops:
|
||||
if op == 71 and args[1] == 34:
|
||||
if args[2] not in (0, 1):
|
||||
raise ValueError('Unexpected descriptor set')
|
||||
args[2] = 1
|
||||
return mod.bytes()
|
||||
|
||||
|
||||
def pair_profile(vs, fs):
|
||||
hashes = fnv(vs), fnv(fs)
|
||||
matches = [(name, profile) for name, profile in PAIRS.items()
|
||||
if hashes[0] == profile[0] and hashes[1] in profile[1:]]
|
||||
if len(matches) != 1:
|
||||
raise ValueError('Input differs from the narrowly tested shader pairs')
|
||||
return matches[0]
|
||||
|
||||
|
||||
def prepare(vs, fs):
|
||||
if fnv(vs) != VS_HASH or fnv(fs) not in (FS_HASH, FS_SET1_HASH):
|
||||
raise ValueError('Input hash differs from the narrowly tested shader pair')
|
||||
if fnv(fs) == FS_HASH:
|
||||
mod = Module(fs)
|
||||
for op, args in mod.ops:
|
||||
if op == 71 and args[1] == 34:
|
||||
if args[2] != 0:
|
||||
raise ValueError('Unexpected descriptor set')
|
||||
args[2] = 1
|
||||
fs = mod.bytes()
|
||||
if fnv(fs) != FS_SET1_HASH:
|
||||
raise ValueError('Descriptor remapping did not match the recorded module')
|
||||
_, profile = pair_profile(vs, fs)
|
||||
fs = set1(fs)
|
||||
if fnv(fs) != profile[2]:
|
||||
raise ValueError('Descriptor remapping did not match the recorded module')
|
||||
return vertex(vs), fragment(fs)
|
||||
|
||||
|
||||
def absorption(data):
|
||||
matches = [(name, profile) for name, profile in ABSORPTION.items()
|
||||
if fnv(data) in profile[:2]]
|
||||
if len(matches) != 1:
|
||||
raise ValueError('Input differs from the narrowly tested absorption materials')
|
||||
name, profile = matches[0]
|
||||
data = set1(data)
|
||||
if fnv(data) != profile[1]:
|
||||
raise ValueError('Material descriptor remapping differs')
|
||||
mod = Module(data)
|
||||
if mod.model != 4:
|
||||
raise ValueError('Expected fragment material')
|
||||
pointer, distance = profile[2:]
|
||||
stores = [i for i, (op, args) in enumerate(mod.ops)
|
||||
if op == 62 and args == [pointer, distance]]
|
||||
if len(stores) != 1:
|
||||
raise ValueError('Expected one exact absorption-distance store')
|
||||
definitions = [args for op, args in mod.ops
|
||||
if op == 133 and len(args) == 4 and args[1] == distance]
|
||||
if len(definitions) != 1 or mod.types[definitions[0][0]] != (22, [32]):
|
||||
raise ValueError('Expected the recorded scalar distance product')
|
||||
scalar = definitions[0][0]
|
||||
imports = [args[0] for op, args in mod.ops
|
||||
if op == 11 and string(args[1:]) == 'GLSL.std.450']
|
||||
if len(imports) != 1:
|
||||
raise ValueError('Expected one GLSL extended-instruction import')
|
||||
zero = mod.constant(scalar, 0)
|
||||
guarded = mod.new()
|
||||
# NMax(distance, 0): negative optical distance must not amplify attenuation.
|
||||
# This targets the distance expression, never a red color or other material.
|
||||
mod.ops[stores[0]:stores[0] + 1] = [
|
||||
(12, [scalar, guarded, imports[0], 80, distance, zero]),
|
||||
(62, [pointer, guarded]),
|
||||
]
|
||||
mod.install_declarations()
|
||||
return name, profile[1], mod.bytes()
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--vertex', required=True, type=Path)
|
||||
parser.add_argument('--fragment', required=True, type=Path)
|
||||
parser.add_argument('--material', type=Path, help='Optional exact absorption material')
|
||||
parser.add_argument('--output', required=True, type=Path)
|
||||
args = parser.parse_args()
|
||||
vs, fs = prepare(args.vertex.read_bytes(), args.fragment.read_bytes())
|
||||
vertex_input, fragment_input = args.vertex.read_bytes(), args.fragment.read_bytes()
|
||||
family, profile = pair_profile(vertex_input, fragment_input)
|
||||
vs, fs = prepare(vertex_input, fragment_input)
|
||||
args.output.mkdir(parents=True, exist_ok=True)
|
||||
files = {f'{VS_HASH:016x}.spv': vs, f'{FS_SET1_HASH:016x}.spv': fs}
|
||||
files = {f'{profile[0]:016x}.spv': vs, f'{profile[2]:016x}.spv': fs}
|
||||
if args.material:
|
||||
material_family, material_hash, material = absorption(args.material.read_bytes())
|
||||
if material_family != family:
|
||||
raise ValueError('Pair and material must use the same translator profile')
|
||||
files[f'{material_hash:016x}.spv'] = material
|
||||
if any((args.output / name).exists() for name in [*files, 'recipe-manifest.json']):
|
||||
raise ValueError('Output exists; choose a new ignored build directory')
|
||||
for name, data in files.items():
|
||||
(args.output / name).write_bytes(data)
|
||||
manifest = {'scope': 'Only the identified vertex/fragment pair, changed together',
|
||||
manifest = {'scope': 'Exact paired coordinates and optional exact-material distance guard',
|
||||
'family': family,
|
||||
'outputs': {name: {'fnv': f'{fnv(data):016x}', 'bytes': len(data)}
|
||||
for name, data in files.items()},
|
||||
'warning': 'Generated modules are game-derived private build output; do not upload them'}
|
||||
|
||||
Reference in New Issue
Block a user