diff --git a/docs/releases/v0.9.1.md b/docs/releases/v0.9.1.md index bfe52ad0..74dcfd88 100644 --- a/docs/releases/v0.9.1.md +++ b/docs/releases/v0.9.1.md @@ -18,7 +18,7 @@ for the launcher; a GPU with Vulkan 1.3 and `VK_EXT_external_memory_host` (curre ## Faster - **A new install no longer builds its pipelines while you play.** A release now carries a list of the graphics - pipelines the game is known to draw, 8,605 of them in this one. The install builds all of them while it prepares the + pipelines the game is known to draw, 8,715 of them in this one. The install builds all of them while it prepares the shaders, and every start builds them before play, as it already did for the pipelines your own install had drawn. Before, a new install met each pipeline in play first: in Hyrule Field one ten-second stretch of a first visit had 1,769 draws wait for their pipelines, 461 ms in all, the longest frame 88 ms. In a first walk through Kakariko @@ -55,16 +55,17 @@ for the launcher; a GPU with Vulkan 1.3 and `VK_EXT_external_memory_host` (curre 5 to 180 m in one frame; such a frame now starts a new history, and frame generation is told of every new start, so it no longer makes frames between two unrelated pictures. `dlaa_cut = 0` turns the camera test off. By bct8925 (pull request #651). +- **Water drawn as a rectangle over solid ground at the Hyrule Castle moat and the Castle Town canal, and a red beam + there** (issue #398). The water's shader worked out each corner's place on the screen and blended that across the + surface, which goes wrong as soon as a corner of the surface is behind the camera; its place is now worked out per + pixel. The distance through the water that dims what lies under it can no longer be negative, which had turned the + dimming into a gain. Found and corrected by ChurchOfIlias (pull request #666); the game applies the correction to + those two shaders itself. `water_projective = 0` in nx2navi.ini leaves them as they are. The correction was compared + byte for byte on an NVIDIA card; on AMD it runs the same steps on AMD's translation of the shaders and was not run by + us. Why the distance comes out negative is not known yet. - **The launcher run from the source with an older Python** stopped at *Gather Build Tools* with nothing but "No module named 'zstandard'". It now says so on its first screen, with the command that installs what is missing. -## For testers - -- **Water drawn as a rectangle over solid ground at the Hyrule Castle moat and the Castle Town canal, and a red beam - there** (issue #398): the source has an opt-in correction by ChurchOfIlias (pull request #666). It takes replacement - shaders made on your own PC from your own shader cache (`tools/projective_recipe.py`, `docs/water-projective.md`) - and is off unless `spv_override` names their folder. The release does not turn it on. - ## Known issues - A rare one-frame flicker of shadows or lighting, on grass and objects, may remain; the fix for the black part of the diff --git a/docs/water-projective.md b/docs/water-projective.md index 13c5132f..69135283 100644 --- a/docs/water-projective.md +++ b/docs/water-projective.md @@ -48,14 +48,31 @@ original, paired-coordinate and absorption-guard pipelines. The vertex replacement stands for the translation itself: the outputs an install's vertex output union adds are added to the replacement as to any other module, so the pair holds whatever other fragment shaders that install has met with the vertex -shader. A pair blended with dual-source factors keeps its original modules. The log -names each replacement the first time it is built ("water projective: ... module"). +shader. A pair blended with dual-source factors keeps its original modules. The translations of game 1.1.1 are the same under the 0.9.0 translator as under 0.8.0's on NVIDIA (RTX 5090), so the profiles above still apply there. -## Private local preparation +## In the runtime, on by default -Use locally held inputs from the same supported translator profile: +The runtime makes both corrections itself (`runtime/spirv_water.c`, called from +`runtime/vk_pipeline.c` `water_projective_pair`): when a pipeline's shaders are one of +the translations above, it edits their SPIR-V in memory, with the steps of +`tools/projective_recipe.py` in the same order, and builds the pipeline from the +result. No file is read or written and nothing derived from the game is shipped. +`water_projective = 0` in `nx2navi.ini` builds the translations as they are. The log +names each corrected module the first time it is built ("vk_pipeline: water: ..."); +a module that is not of the shape an edit expects is built as it is, with a line +that says so. A coordinate pair is corrected in both stages or in neither. + +On the NVIDIA profile the runtime's three modules are byte for byte the ones the +recipe writes (RTX 5090, game 1.1.1, 0.9.0's translator). The AMD profile runs the +same code on AMD's translations, which were not at hand to compare. +`python tests/water_projective_check.py` compares the two on made-up modules. + +## The recipe as a tool + +`tools/projective_recipe.py` makes the same modules as files from locally held +inputs, for comparing or inspecting them: ```sh python tools/projective_recipe.py --vertex --fragment --material --output build/water-projective @@ -64,14 +81,8 @@ python tools/projective_recipe.py --vertex --fragment ` before integration. - -Rebuild using the documented UCRT64 workflow. Set `NX2NAVI_SPV_OVERRIDE` to the -absolute output directory in the test process, then restart. Default runtime -behavior is unchanged. Use separate pipeline caches for comparisons and disable -pipeline-binary reuse and boot prebuilding during the experiment. Unset the -override and restart to disable it. Do not change the directory during a run. +plus a manifest. Validate a generated module with +`spirv-val --target-env vulkan1.2 `. The runtime does not read them. ## Validation and limits diff --git a/nx2navi.ini.example b/nx2navi.ini.example index 9c414006..624d06db 100644 --- a/nx2navi.ini.example +++ b/nx2navi.ini.example @@ -305,6 +305,10 @@ # fix_c0 = 1 # New GPU memory pools start zero-filled, as Switch memory does. # fix_zero = 1 +# Build the water's screen coordinates per pixel and keep its absorption distance at 0 or more: without it water is +# drawn as a rectangle over solid ground at the Hyrule Castle moat and the Castle Town canal from some camera +# positions, and can turn into a red beam there. 0: the water's shaders as they are. +# water_projective = 1 # Optimisations, 0 turns each off: skip bindings that did not change, remember shader lookups, reuse the previous # draw's pipeline when its state repeats, remember texture descriptors between draws, update descriptors in bulk, # skip push constants that did not change. diff --git a/runtime/cfg.c b/runtime/cfg.c index 3867cff3..b3e062d0 100644 --- a/runtime/cfg.c +++ b/runtime/cfg.c @@ -71,7 +71,7 @@ static const char *const known[] = { "SHOTS_FULL", "SPEED_PROBE", "SPIN", "SPIN_HANDOFF_US", "SPLIT_DRAWS", "SPLIT_IDLE", "SPLIT_IN_PASS", "SPLIT_US", "SPV_MARK_TEST", "SPV_OVERRIDE", "STICK_DEADZONE", "STICK_SENS", "SWKBD_ANSWER", "SET_AT", "SWKBD_TEST", "TAA", "DBG_TEST", "DEBUG_MENU", "MENU_TEST_DEBUG", "FSR4", "FSR4_FG", "FG4_VERSION", "FG4_SWAPCHAIN", "TEST_PAD", "TEST_PAD_AT", "TEST_PREPARE_SHADERS", "TEX_ALIAS", "DEPTH_INIT_INLINE", "DEPTH_ALIAS_LOG", "TEX_DISUSE", "TEX_DISUSE_EVICT", "TEX_GEN_UNDO", "TEX_MEMO", "TEX_VRAM_PRESSURE", "TEX_RELEASED_MB", "TEX_RELEASED_MAX", "TEX_REUSE", "TEX_SPLIT_FILL", "TEX_COMMIT_FREE_MB", "TEX_PROC_COMMIT_MB", "TEX_EVICT", "TEX_COMMIT_EVICT", "MIRROR_EVICT", "VRAM_BUDGET_MB", "TIMER_1MS", "TRACE", "TXD_TRANSPOSE", "UBO_SNAPSHOT", "EVO_TRACE", "EVENT_AT_SYNC", "UPSCALER", "VK_DYN_A2C", "VK_DYN_VERTEX", "VK_NAN_PRESERVE", "VK_NOOPT", "VK_NO_FLOAT64", - "VK_PRESENT", "VK_RECONVERGE", "VK_NAN_BITS", "VK_RECORDER", "VK_SSBO_ALIGN", "VK_SUBGROUP", "VK_SUBGROUP_PIN", "VK_VALIDATION", "VOLUME", "VRAM_HEADROOM", "VS_OUTS", + "VK_PRESENT", "VK_RECONVERGE", "VK_NAN_BITS", "VK_RECORDER", "VK_SSBO_ALIGN", "VK_SUBGROUP", "VK_SUBGROUP_PIN", "VK_VALIDATION", "VOLUME", "VRAM_HEADROOM", "VS_OUTS", "WATER_PROJECTIVE", "WAIT_WATCH_S", "WALK_KEY", "WALK_SCALE", "WB_EXACT", "WB_FENCE_LIMIT", "WB_MERGE", "WB_NO_REUP", "WB_MERGE_MAX", "WB_PROBE", "WB_SPLIT", "WB_LAZY", "WB_LAZY_CHUNK", "WB_LAZY_TOUCH", "WINDOW_SIZE", "XLATE", "XLATE_ID", "XLATE_KEEP", "XLATE_PROCS", "ZSTD_CHECK", "ZSTD_NATIVE", "BIND_REUSE", "DYN_DELTA", "STG_ARENA", "SUBMIT_BATCH", "ASTC_GPU_DESWIZZLE", "ASTC_SCRATCH_KEEP", "ASTC_ARENA", "ASTC_BC1", @@ -135,7 +135,7 @@ static const struct { const char *key, *section; } sections[] = { {"LIB_PREBUILD", "performance"}, {"LIB_PREBUILD_THREADS", "performance"}, {"LIB_WARM", "performance"}, {"PIPE_LINK_MS", "performance"}, {"PIPE_LINK_DRAW_MS", "performance"}, {"PIPE_AHEAD", "performance"}, {"PIPE_FAST_WORKERS", "performance"}, {"GPL_FEEDBACK", "performance"}, {"FRAME_HOLD_MS", "performance"}, {"PACE_SMOOTH", "performance"}, {"OQ_BATCH", "performance"}, {"GATE_WAIT_MS", "performance"}, {"PIPE_PREBUILD", "performance"}, {"PIPE_SEED", "performance"}, {"COMPUTE_KEEP", "performance"}, {"PREWARM_XLATE", "performance"}, {"GPL", "performance"}, {"ASTC", "performance"}, {"MEMORY", "performance"}, - {"TEX_ALIAS", "performance"}, {"DEPTH_INIT_INLINE", "performance"}, {"DEPTH_ALIAS_LOG", "diagnostics"}, {"TEX_DISUSE", "performance"}, {"TEX_DISUSE_EVICT", "performance"}, {"TEX_GEN_UNDO", "performance"}, {"TEX_REUSE", "performance"}, {"TEX_SPLIT_FILL", "performance"}, {"TEX_VRAM_PRESSURE", "performance"}, {"TEX_PROC_COMMIT_MB", "performance"}, {"ASTC_BC", "performance"}, {"LOD_GUARD", "performance"}, {"FIX_C0", "performance"}, {"FIX_ZERO", "performance"}, + {"TEX_ALIAS", "performance"}, {"DEPTH_INIT_INLINE", "performance"}, {"DEPTH_ALIAS_LOG", "diagnostics"}, {"TEX_DISUSE", "performance"}, {"TEX_DISUSE_EVICT", "performance"}, {"TEX_GEN_UNDO", "performance"}, {"TEX_REUSE", "performance"}, {"TEX_SPLIT_FILL", "performance"}, {"TEX_VRAM_PRESSURE", "performance"}, {"TEX_PROC_COMMIT_MB", "performance"}, {"ASTC_BC", "performance"}, {"LOD_GUARD", "performance"}, {"FIX_C0", "performance"}, {"FIX_ZERO", "performance"}, {"WATER_PROJECTIVE", "performance"}, {"BIND_SKIP", "performance"}, {"SHADER_CACHE_FLUSH", "performance"}, {"SHADER_CARRY", "performance"}, {"PIPELINE_CACHE_PLAY", "performance"}, {"PIPELINE_CACHE_PLAY_MB", "performance"}, {"FFX_PIPE_CACHE", "performance"}, {"SHADER_MEMO", "performance"}, {"PIPE_MEMO", "performance"}, {"TEX_MEMO", "performance"}, {"DESC_TEMPLATE", "performance"}, {"PUSH_SKIP", "performance"}, {"SPLIT_DRAWS", "performance"}, {"SPLIT_IDLE", "performance"}, {"SPLIT_IN_PASS", "performance"}, {"SPLIT_US", "performance"}, {"VK_VALIDATION", "performance"}, {"VK_DYN_VERTEX", "performance"}, {"VK_DYN_A2C", "performance"}, {"VK_SUBGROUP_PIN", "performance"}, {"VK_RECONVERGE", "performance"}, @@ -299,6 +299,7 @@ static const CfgMeta metas[] = { {"ASTC_SCRATCH_KEEP", "1", "0|1", RESTART}, {"ASTC_ARENA", "1", "0|1", RESTART}, // vk_astc.c scratch_get, vk_texture.c astc_arena // render fixes {"LOD_GUARD", "1", "0|1", RESTART}, {"FIX_C0", "1", "0|1", RESTART}, {"FIX_ZERO", "1", "0|1", RESTART}, + {"WATER_PROJECTIVE", "1", "0|1", RESTART}, // vk_pipeline.c water_projective_pair {"BIND_SKIP", "1", "0|1", RESTART}, {"SHADER_CACHE_FLUSH", "0", "0|1", RESTART}, {"SHADER_MEMO", "1", "0|1", RESTART}, {"PIPE_MEMO", "1", "0|1", RESTART}, {"TEX_MEMO", "1", "0|1", RESTART}, {"BIND_REUSE", "1", "0|1", RESTART}, {"DYN_DELTA", "1", "0|1", RESTART}, {"DESC_TEMPLATE", "1", "0|1", RESTART}, diff --git a/runtime/spirv_water.c b/runtime/spirv_water.c new file mode 100644 index 00000000..ea03e23d --- /dev/null +++ b/runtime/spirv_water.c @@ -0,0 +1,297 @@ +// The water corrections of docs/water-projective.md, made on a SPIR-V module in memory. Each edit follows +// tools/projective_recipe.py step for step, new ids and declarations in the same order, so the module it returns is the +// one that tool writes for the same input (tests/water_projective_check.py compares them on a made-up module). +// +// A module is taken apart into its instructions, edited as a list, and put together again. New types and constants +// are collected and placed before the module's one function; an existing type or constant of the same definition is +// used instead of a new one. Every edit refuses a module that is not of the shape it was written for: one entry +// point, one function, one return, and the interface or the store it looks for found exactly once. +#include +#include + +#include "spirv_water.h" + +enum { OP_EXT_INST_IMPORT = 11, OP_EXT_INST = 12, OP_ENTRY_POINT = 15, OP_TYPE_INT = 21, OP_TYPE_FLOAT = 22, + OP_TYPE_VECTOR = 23, OP_TYPE_STRUCT = 30, OP_TYPE_POINTER = 32, OP_CONSTANT = 43, OP_FUNCTION = 54, + OP_VARIABLE = 59, OP_LOAD = 61, OP_STORE = 62, OP_ACCESS_CHAIN = 65, OP_IN_BOUNDS_ACCESS_CHAIN = 66, + OP_DECORATE = 71, OP_MEMBER_DECORATE = 72, OP_COMPOSITE_CONSTRUCT = 80, OP_COMPOSITE_EXTRACT = 81, + OP_FMUL = 133, OP_FDIV = 136, OP_LABEL = 248, OP_RETURN = 253 }; +enum { DEC_BUILTIN = 11, DEC_NOPERSPECTIVE = 13, DEC_LOCATION = 30, SC_INPUT = 1, SC_OUTPUT = 3, SC_FUNCTION = 7, + GLSL_FMA = 50, GLSL_NMAX = 80 }; + +typedef struct { uint32_t op, n; uint32_t *a; } Ins; // n: words after the first; a: those words +typedef struct { + uint32_t hdr[5]; + Ins *ops; + size_t nops, cap; + Ins extra[24]; // new types and constants, in the order made + size_t nextra; + uint32_t *src; // the module's words: the instructions' arguments point into this copy + uint32_t pool[256]; // the arguments of the instructions added + size_t np; + uint32_t model; + int bad; // an edit ran out of room: the module is refused +} Mod; + +static void mod_free(Mod *m) { free(m->ops), free(m->src); } + +static int mod_parse(Mod *m, const uint32_t *w, uint64_t n) { + memset(m, 0, sizeof *m); + if (!w || n < 5 || w[0] != 0x07230203u) return 0; + if (!(m->src = malloc((size_t)n * 4))) return 0; + memcpy(m->src, w, (size_t)n * 4); + memcpy(m->hdr, w, sizeof m->hdr); + m->cap = 1024; + if (!(m->ops = malloc(m->cap * sizeof *m->ops))) return mod_free(m), 0; + unsigned entries = 0, functions = 0, returns = 0; + for (uint64_t at = 5; at < n;) { + uint32_t len = m->src[at] >> 16, op = m->src[at] & 65535; + if (!len || at + len > n) return mod_free(m), 0; + if (m->nops + 64 > m->cap) { // 64: room for what the edits insert + Ins *t = realloc(m->ops, (m->cap *= 2) * sizeof *t); + if (!t) return mod_free(m), 0; + m->ops = t; + } + m->ops[m->nops++] = (Ins){op, len - 1, m->src + at + 1}; + if (op == OP_ENTRY_POINT && len >= 2) entries++, m->model = m->src[at + 1]; + functions += op == OP_FUNCTION, returns += op == OP_RETURN; + at += len; + } + if (entries != 1 || functions != 1 || returns != 1) return mod_free(m), 0; + return 1; +} + +static uint32_t new_id(Mod *m) { return m->hdr[3]++; } + +static uint32_t *words(Mod *m, uint32_t n, const uint32_t *v) { + if (m->np + n > sizeof m->pool / sizeof *m->pool) { m->bad = 1; return m->pool; } + uint32_t *p = m->pool + m->np; + memcpy(p, v, n * 4); + m->np += n; + return p; +} + +#define INS(m, op, ...) ((Ins){op, sizeof((uint32_t[]){__VA_ARGS__}) / 4, \ + words(m, sizeof((uint32_t[]){__VA_ARGS__}) / 4, (uint32_t[]){__VA_ARGS__})}) + +static void add_extra(Mod *m, Ins i) { + if (m->nextra == sizeof m->extra / sizeof *m->extra) { m->bad = 1; return; } + m->extra[m->nextra++] = i; +} + +static int is_type(uint32_t op) { return op >= 19 && op <= 33; } + +// The type or variable instruction that defines id, among the module's own and the new ones. +static const Ins *type_of(const Mod *m, uint32_t id) { + for (size_t i = 0; i < m->nops; i++) + if (is_type(m->ops[i].op) && m->ops[i].n >= 1 && m->ops[i].a[0] == id) return &m->ops[i]; + for (size_t i = 0; i < m->nextra; i++) + if (is_type(m->extra[i].op) && m->extra[i].a[0] == id) return &m->extra[i]; + return NULL; +} + +static const Ins *variable_of(const Mod *m, uint32_t id) { + for (size_t i = 0; i < m->nops; i++) + if (m->ops[i].op == OP_VARIABLE && m->ops[i].n >= 3 && m->ops[i].a[1] == id) return &m->ops[i]; + return NULL; +} + +// The id of the type (op, v[0..k)), an existing one or a new one. +static uint32_t type_id(Mod *m, uint32_t op, uint32_t k, const uint32_t *v) { + for (size_t i = 0; i < m->nops; i++) + if (m->ops[i].op == op && m->ops[i].n == k + 1 && !memcmp(m->ops[i].a + 1, v, k * 4)) return m->ops[i].a[0]; + for (size_t i = 0; i < m->nextra; i++) + if (m->extra[i].op == op && m->extra[i].n == k + 1 && !memcmp(m->extra[i].a + 1, v, k * 4)) return m->extra[i].a[0]; + uint32_t id = new_id(m), all[4] = {id}; + memcpy(all + 1, v, k * 4); + add_extra(m, (Ins){op, k + 1, words(m, k + 1, all)}); + return id; +} +#define TYPE(m, op, ...) type_id(m, op, sizeof((uint32_t[]){__VA_ARGS__}) / 4, (uint32_t[]){__VA_ARGS__}) + +// The id of the 32-bit constant `bits` of type ty, an existing one or a new one. +static uint32_t constant_id(Mod *m, uint32_t ty, uint32_t bits) { + for (size_t i = 0; i < m->nops; i++) + if (m->ops[i].op == OP_CONSTANT && m->ops[i].n == 3 && m->ops[i].a[0] == ty && m->ops[i].a[2] == bits) + return m->ops[i].a[1]; + for (size_t i = 0; i < m->nextra; i++) + if (m->extra[i].op == OP_CONSTANT && m->extra[i].a[0] == ty && m->extra[i].a[2] == bits) return m->extra[i].a[1]; + uint32_t id = new_id(m); + add_extra(m, INS(m, OP_CONSTANT, ty, id, bits)); + return id; +} + +// The one interface variable of the storage class at location 0, a four-component 32-bit float vector. +static int location0(const Mod *m, uint32_t storage, uint32_t *var, uint32_t *vec, uint32_t *scalar) { + unsigned found = 0; + for (size_t i = 0; i < m->nops; i++) { + const Ins *d = &m->ops[i]; + if (d->op != OP_DECORATE || d->n != 3 || d->a[1] != DEC_LOCATION || d->a[2] != 0) continue; + const Ins *v = variable_of(m, d->a[0]); + if (v && v->a[2] == storage) *var = d->a[0], found++; + } + if (found != 1) return 0; + const Ins *p = type_of(m, variable_of(m, *var)->a[0]); + if (!p || p->op != OP_TYPE_POINTER || p->n != 3 || p->a[1] != storage) return 0; + const Ins *v = type_of(m, p->a[2]); + if (!v || v->op != OP_TYPE_VECTOR || v->n != 3 || v->a[2] != 4) return 0; + const Ins *s = type_of(m, v->a[1]); + if (!s || s->op != OP_TYPE_FLOAT || s->n != 2 || s->a[1] != 32) return 0; + *vec = p->a[2], *scalar = v->a[1]; + return 1; +} + +// The id of the GLSL.std.450 import when the module has exactly one. +static int glsl_import(const Mod *m, uint32_t *id) { + unsigned found = 0; + for (size_t i = 0; i < m->nops; i++) { + const Ins *e = &m->ops[i]; + static const char name[] = "GLSL.std.450"; + if (e->op != OP_EXT_INST_IMPORT || (e->n - 1) * 4 < sizeof name) continue; + if (!memcmp(e->a + 1, name, sizeof name)) *id = e->a[0], found++; + } + return found == 1; +} + +static void insert(Mod *m, size_t at, const Ins *list, size_t k) { // mod_parse left room for the edits' instructions + if (m->nops + k > m->cap) { m->bad = 1; return; } + memmove(m->ops + at + k, m->ops + at, (m->nops - at) * sizeof *m->ops); + memcpy(m->ops + at, list, k * sizeof *list); + m->nops += k; +} + +static size_t first(const Mod *m, uint32_t op) { + for (size_t i = 0; i < m->nops; i++) + if (m->ops[i].op == op) return i; + return m->nops; +} + +// The edited module's words, the new declarations placed before the function; frees m. +static uint32_t *mod_finish(Mod *m, uint64_t *out_n) { + insert(m, first(m, OP_FUNCTION), m->extra, m->nextra); + uint64_t n = 5; + for (size_t i = 0; i < m->nops; i++) n += m->ops[i].n + 1; + uint32_t *out = m->bad ? NULL : malloc((size_t)n * 4); + if (out) { + memcpy(out, m->hdr, sizeof m->hdr); + uint64_t at = 5; + for (size_t i = 0; i < m->nops; i++) { + out[at++] = (m->ops[i].n + 1) << 16 | m->ops[i].op; + memcpy(out + at, m->ops[i].a, m->ops[i].n * 4); + at += m->ops[i].n; + } + *out_n = n; + } + mod_free(m); + return out; +} + +uint32_t *spirv_water_vertex(const uint32_t *w, uint64_t n, uint64_t *out_n) { + Mod m; + if (!mod_parse(&m, w, n)) return NULL; + uint32_t varying, vec, scalar, structure = 0, member = 0, block = 0; + unsigned positions = 0, blocks = 0; + if (m.model != 0 || !location0(&m, SC_OUTPUT, &varying, &vec, &scalar)) return mod_free(&m), NULL; + for (size_t i = 0; i < m.nops; i++) { + const Ins *d = &m.ops[i]; + if (d->op == OP_MEMBER_DECORATE && d->n == 4 && d->a[2] == DEC_BUILTIN && d->a[3] == 0) // BuiltIn Position + structure = d->a[0], member = d->a[1], positions++; + } + if (positions != 1) return mod_free(&m), NULL; + for (size_t i = 0; i < m.nops; i++) { + const Ins *v = &m.ops[i]; + if (v->op != OP_VARIABLE || v->n < 3 || v->a[2] != SC_OUTPUT) continue; + const Ins *p = type_of(&m, v->a[0]); + if (p && p->op == OP_TYPE_POINTER && p->n == 3 && p->a[1] == SC_OUTPUT && p->a[2] == structure) block = v->a[1], blocks++; + } + const Ins *st = type_of(&m, structure); + if (blocks != 1 || !st || st->op != OP_TYPE_STRUCT || member + 1 >= st->n || st->a[1 + member] != vec) + return mod_free(&m), NULL; + uint32_t pointer = TYPE(&m, OP_TYPE_POINTER, SC_OUTPUT, vec); + uint32_t integer = TYPE(&m, OP_TYPE_INT, 32, 1); + uint32_t member_id = constant_id(&m, integer, member); + uint32_t one = constant_id(&m, scalar, 0x3F800000); + uint32_t ptr = new_id(&m), pos = new_id(&m), x = new_id(&m), y = new_id(&m), cw = new_id(&m), enc = new_id(&m); + Ins code[] = {INS(&m, OP_ACCESS_CHAIN, pointer, ptr, block, member_id), + INS(&m, OP_LOAD, vec, pos, ptr), + INS(&m, OP_COMPOSITE_EXTRACT, scalar, x, pos, 0), + INS(&m, OP_COMPOSITE_EXTRACT, scalar, y, pos, 1), + INS(&m, OP_COMPOSITE_EXTRACT, scalar, cw, pos, 3), + INS(&m, OP_COMPOSITE_CONSTRUCT, vec, enc, x, y, cw, one), + INS(&m, OP_STORE, varying, enc)}; + insert(&m, first(&m, OP_RETURN), code, sizeof code / sizeof *code); + return mod_finish(&m, out_n); +} + +uint32_t *spirv_water_fragment(const uint32_t *w, uint64_t n, uint64_t *out_n) { + Mod m; + if (!mod_parse(&m, w, n)) return NULL; + uint32_t varying, vec, scalar, glsl; + if (m.model != 4 || !location0(&m, SC_INPUT, &varying, &vec, &scalar)) return mod_free(&m), NULL; + size_t flat = m.nops; + unsigned flats = 0; + for (size_t i = 0; i < m.nops; i++) + if (m.ops[i].op == OP_DECORATE && m.ops[i].n == 2 && m.ops[i].a[0] == varying && m.ops[i].a[1] == DEC_NOPERSPECTIVE) + flat = i, flats++; + if (flats != 1) return mod_free(&m), NULL; + memmove(m.ops + flat, m.ops + flat + 1, (m.nops - flat - 1) * sizeof *m.ops); // the NoPerspective goes + m.nops--; + if (!glsl_import(&m, &glsl)) return mod_free(&m), NULL; + uint32_t local_pointer = TYPE(&m, OP_TYPE_POINTER, SC_FUNCTION, vec), local = new_id(&m); + unsigned count = 0; // the stage's own reads of the input go to the local + for (size_t i = 0; i < m.nops; i++) { + Ins *o = &m.ops[i]; + if ((o->op == OP_ACCESS_CHAIN || o->op == OP_IN_BOUNDS_ACCESS_CHAIN) && o->n >= 3 && o->a[2] == varying) { + const Ins *p = type_of(&m, o->a[0]); + if (!p || p->op != OP_TYPE_POINTER || p->n != 3 || p->a[1] != SC_INPUT) return mod_free(&m), NULL; + o->a[0] = TYPE(&m, OP_TYPE_POINTER, SC_FUNCTION, p->a[2]); + o->a[2] = local, count++; + } else if (o->op == OP_LOAD && o->n >= 3 && o->a[2] == varying) { + o->a[2] = local, count++; + } + } + if (count != 2) return mod_free(&m), NULL; + uint32_t zero = constant_id(&m, scalar, 0), one = constant_id(&m, scalar, 0x3F800000); + uint32_t half = constant_id(&m, scalar, 0x3F000000), nhalf = constant_id(&m, scalar, 0xBF000000); + uint32_t raw = new_id(&m), x = new_id(&m), y = new_id(&m), cw = new_id(&m), nx = new_id(&m), ny = new_id(&m); + uint32_t u = new_id(&m), v = new_id(&m), dec = new_id(&m); + Ins code[] = {INS(&m, OP_VARIABLE, local_pointer, local, SC_FUNCTION), + INS(&m, OP_LOAD, vec, raw, varying), + INS(&m, OP_COMPOSITE_EXTRACT, scalar, x, raw, 0), + INS(&m, OP_COMPOSITE_EXTRACT, scalar, y, raw, 1), + INS(&m, OP_COMPOSITE_EXTRACT, scalar, cw, raw, 2), + INS(&m, OP_FDIV, scalar, nx, x, cw), + INS(&m, OP_FDIV, scalar, ny, y, cw), + INS(&m, OP_EXT_INST, scalar, u, glsl, GLSL_FMA, nx, half, half), + INS(&m, OP_EXT_INST, scalar, v, glsl, GLSL_FMA, ny, nhalf, half), + INS(&m, OP_COMPOSITE_CONSTRUCT, vec, dec, u, v, zero, one), + INS(&m, OP_STORE, local, dec)}; + size_t at = first(&m, OP_LABEL) + 1; // after the function's own variables + while (at < m.nops && m.ops[at].op == OP_VARIABLE) at++; + if (at > m.nops) return mod_free(&m), NULL; + insert(&m, at, code, sizeof code / sizeof *code); + return mod_finish(&m, out_n); +} + +uint32_t *spirv_water_absorption(const uint32_t *w, uint64_t n, uint32_t pointer, uint32_t distance, uint64_t *out_n) { + Mod m; + if (!mod_parse(&m, w, n)) return NULL; + size_t store = 0; + unsigned stores = 0, products = 0; + uint32_t scalar = 0, glsl; + if (m.model != 4) return mod_free(&m), NULL; + for (size_t i = 0; i < m.nops; i++) { + const Ins *o = &m.ops[i]; + if (o->op == OP_STORE && o->n == 2 && o->a[0] == pointer && o->a[1] == distance) store = i, stores++; + if (o->op == OP_FMUL && o->n == 4 && o->a[1] == distance) scalar = o->a[0], products++; + } + const Ins *s = products == 1 ? type_of(&m, scalar) : NULL; + if (stores != 1 || !s || s->op != OP_TYPE_FLOAT || s->n != 2 || s->a[1] != 32 || !glsl_import(&m, &glsl)) + return mod_free(&m), NULL; + uint32_t zero = constant_id(&m, scalar, 0), guarded = new_id(&m); + // NMax(distance, 0): a negative distance through the water must not turn the dimming into a gain + Ins guard = INS(&m, OP_EXT_INST, scalar, guarded, glsl, GLSL_NMAX, distance, zero); + m.ops[store] = INS(&m, OP_STORE, pointer, guarded); + insert(&m, store, &guard, 1); + return mod_finish(&m, out_n); +} diff --git a/runtime/spirv_water.h b/runtime/spirv_water.h new file mode 100644 index 00000000..85c1cf31 --- /dev/null +++ b/runtime/spirv_water.h @@ -0,0 +1,14 @@ +// The water corrections of docs/water-projective.md as edits of a SPIR-V module in memory (spirv_water.c); the same +// edits tools/projective_recipe.py makes on files. vk_pipeline.c applies them to the exact translations it knows. +#pragma once +#include + +// Each returns a new module (free it) with its word count in *out_n, or NULL when the module is not of the shape the +// edit expects; the caller then keeps the original. +// vertex: the location 0 output gets (clip x, clip y, clip w, 1) of the position the stage wrote, before its return. +uint32_t *spirv_water_vertex(const uint32_t *w, uint64_t n, uint64_t *out_n); +// fragment: the location 0 input loses NoPerspective and is read through a local holding (x / w * 0.5 + 0.5, +// y / w * -0.5 + 0.5, 0, 1), which is what the vertex stage wrote there before, now divided per fragment. +uint32_t *spirv_water_fragment(const uint32_t *w, uint64_t n, uint64_t *out_n); +// absorption: the store of id `distance` through id `pointer` stores NMax(distance, 0) instead. +uint32_t *spirv_water_absorption(const uint32_t *w, uint64_t n, uint32_t pointer, uint32_t distance, uint64_t *out_n); diff --git a/runtime/vk_pipeline.c b/runtime/vk_pipeline.c index d3c95bc3..15a41f5e 100644 --- a/runtime/vk_pipeline.c +++ b/runtime/vk_pipeline.c @@ -48,6 +48,7 @@ #include "nvn_hmap.h" #include "platform/platform.h" #include "prp_file.h" +#include "spirv_water.h" #include "tex.h" #include "vk_build_gate.h" #include "vk_cache_file.h" @@ -3001,127 +3002,105 @@ static uint32_t vs_union_missing(const VkPipeState *s, GfxShaderIo fin[32]) { } -// Optional private modules generated from exact local inputs by projective_recipe.py. -// Mode 1 changes a complete coordinate pair; mode 2 guards one absorption material. -static const char *water_projective_dir(void) { - static PlatMutex lock = PLAT_MUTEX_INIT; - static int checked; - static const char *dir; - plat_mutex_lock(&lock); - if (!checked) { - checked = 1; - const char *candidate = cfg_get("NX2NAVI_SPV_OVERRIDE"); - if (candidate && *candidate) dir = candidate; - } - const char *result = dir; - plat_mutex_unlock(&lock); - return result; -} +// ---- water corrections (docs/water-projective.md; NX2NAVI_WATER_PROJECTIVE, default 1; 0: the translations as built) +// Two of the game's water translations are built from an edited module (spirv_water.c): +// - the coordinate pair (mode 1). The vertex stage worked out each vertex's place on the screen (x / w and y / w, +// scaled to 0..1) and the fragment stage read it interpolated without perspective, to sample the scene behind the +// water. A triangle with a corner behind the camera has no such place for that corner, and the scene was sampled +// elsewhere: a rectangle of water over solid ground at the Hyrule Castle moat and the Castle Town canal. The pair +// now hands x, y and w down undivided and the fragment stage divides. Both stages change, or neither. +// - the absorption material (mode 2): its distance through the water could come out negative, which turned the +// dimming into a gain (a red beam). The distance stored is at least 0. +// The translations are known by the FNV-1a of their SPIR-V, one profile per translator output (the host options of +// AMD's and NVIDIA's drivers give different modules); every other module is built as it is. The edited modules are +// made at their first use and kept. Modes go into the stage libraries' keys (PrKey, FsKey), so a library of an edited +// module is never linked with one of the original. +typedef struct { + uint64_t vs, fs, fs_set1; // the coordinate pair: vertex, fragment, the fragment in its set 1 form + uint64_t mat, mat_set1; // the absorption material, and in its set 1 form + uint32_t mat_pointer, mat_distance; // the ids of the store its distance goes through +} WaterProfile; +static const WaterProfile water_profiles[] = { + {0xf6d753852c47b0deull, 0xa39c3d09241441faull, 0x3d3a316ab4da3bbaull, 0xe73dfc0f4c556df8ull, 0xabfccea256824a78ull, 1965, 8490}, + {0x8d83e3705d790fedull, 0x3c20cb98dbaeb257ull, 0x216bd323c6b48ee2ull, 0x061318799d885bcfull, 0xb3cec6d0d78ac25aull, 1969, 8826}, +}; +enum { WATER_VERTEX, WATER_FRAGMENT, WATER_ABSORPTION }; -static int water_local_module_exists(uint64_t hash) { - const char *dir = water_projective_dir(); - if (!dir) return 0; - char path[700]; - int n = snprintf(path, sizeof path, "%s/%016llx.spv", dir, (unsigned long long)hash); - return n > 0 && n < (int)sizeof path && plat_file_exists(path); -} +// The edited modules, by the hash of the module each stands for as it is built: a vertex translation, or a fragment +// module in its set 1 form. state: 0 empty, 1 made, -1 the edit refused the module (it is built as it is). +static struct { uint64_t key; uint32_t *code; uint64_t words; int state; } water_mods[8]; +static PlatMutex water_lock = PLAT_MUTEX_INIT; -static int water_projective_pair(const VkPipeState *s) { - if (!P.indep || !s->fs || !water_projective_dir()) return 0; - uint64_t vn = 0, fn = 0; - const uint32_t *f = gfx_shader_spirv(s->fs, &fn); - if (!f) return 0; - uint64_t fh = fnv(FNV0, f, (size_t)fn * 4); - // The absorption correction is independent of the coordinate pair. - if ((fh == 0xe73dfc0f4c556df8ull && water_local_module_exists(0xabfccea256824a78ull)) || - (fh == 0x061318799d885bcfull && water_local_module_exists(0xb3cec6d0d78ac25aull))) return 2; - if (!s->vs) return 0; - const uint32_t *v = gfx_shader_spirv(s->vs, &vn); - if (!v) return 0; - uint64_t vh = fnv(FNV0, v, (size_t)vn * 4); - if (vh == 0xf6d753852c47b0deull && fh == 0xa39c3d09241441faull) - return water_local_module_exists(vh) && water_local_module_exists(0x3d3a316ab4da3bbaull); - if (vh == 0x8d83e3705d790fedull && fh == 0x3c20cb98dbaeb257ull) - return water_local_module_exists(vh) && water_local_module_exists(0x216bd323c6b48ee2ull); - return 0; -} - -// The local module that stands for the module of hash h, checked: one entry point, of the stage given (0 vertex, -// 4 fragment). The caller frees it. A line in the log names each module the first time it is taken. -static uint32_t *water_local_module(uint64_t h, int stage, uint64_t *words) { - const char *dir = water_projective_dir(); - if (!dir) rt_fatal("water projective: recipe directory unavailable"); - char path[700]; - int path_len = snprintf(path, sizeof path, "%s/%016llx.spv", dir, (unsigned long long)h); - if (path_len < 0 || path_len >= (int)sizeof path) rt_fatal("water projective: module path too long"); - FILE *file = fopen(path, "rb"); - if (!file) rt_fatal("water projective: local module missing: %s", path); - fseek(file, 0, SEEK_END); - long bytes = ftell(file); - rewind(file); - uint32_t *code = bytes >= 20 && bytes <= 1048576 && !(bytes & 3) ? malloc((size_t)bytes) : NULL; - if (!code || fread(code, 1, (size_t)bytes, file) != (size_t)bytes) - rt_fatal("water projective: invalid local module: %s", path); - fclose(file); - if (code[0] != 0x07230203u) rt_fatal("water projective: invalid SPIR-V header"); - int entries = 0; - for (size_t at = 5; at < (size_t)bytes / 4;) { - uint32_t len = code[at] >> 16, op = code[at] & 65535; - if (!len || at + len > (size_t)bytes / 4) rt_fatal("water projective: malformed instruction"); - if (op == 15) { - if (len < 4 || code[at + 1] != (uint32_t)stage) rt_fatal("water projective: wrong module stage"); - entries++; - } - at += len; +// The edited module that stands for the module of hash `key`, or NULL. w: the translation it is made from at the +// first call (NULL: only look); a fragment translation is first put into its set 1 form, which must be the module +// `key` names, as it is the form the pipeline builds. +static const uint32_t *water_module(uint64_t key, int kind, const WaterProfile *p, const uint32_t *w, uint64_t n, + uint64_t *words) { + const uint32_t *code = NULL; + plat_mutex_lock(&water_lock); + int i = 0; + while (i < 8 && water_mods[i].state && water_mods[i].key != key) i++; + if (i < 8 && !water_mods[i].state && w) { + uint64_t wn = 0; + uint32_t *c = NULL, *set1 = kind == WATER_VERTEX ? NULL : spirv_set(w, n, 1); + if (kind == WATER_VERTEX) c = spirv_water_vertex(w, n, &wn); + else if (set1 && fnv(FNV0, set1, (size_t)n * 4) == key) + c = kind == WATER_FRAGMENT ? spirv_water_fragment(set1, n, &wn) + : spirv_water_absorption(set1, n, p->mat_pointer, p->mat_distance, &wn); + free(set1); + water_mods[i].key = key, water_mods[i].code = c, water_mods[i].words = wn, water_mods[i].state = c ? 1 : -1; + static const char *const names[] = {"vertex stage of the coordinate pair", "fragment stage of the coordinate pair", + "absorption material"}; + if (c) rt_log("vk_pipeline: water: the %s (module %016llx) is built from its corrected form " + "(NX2NAVI_WATER_PROJECTIVE)", names[kind], (unsigned long long)key); + else rt_log("vk_pipeline: water: the %s (module %016llx) is not of the shape its correction expects; built as " + "it is", names[kind], (unsigned long long)key); } - if (entries != 1) rt_fatal("water projective: unexpected entry-point count"); - static volatile uint64_t told[8]; - for (int i = 0; i < 8; i++) { - uint64_t was = __atomic_load_n(&told[i], __ATOMIC_RELAXED); - if (was == h) break; - if (!was && __atomic_compare_exchange_n(&told[i], &was, h, 0, __ATOMIC_RELAXED, __ATOMIC_RELAXED)) { - rt_log("water projective: %s module %016llx is built from %s", stage ? "fragment" : "vertex", - (unsigned long long)h, path); - break; - } - if (was == h) break; // another thread took the slot for the same module - } - *words = (uint64_t)bytes / 4; + if (i < 8 && water_mods[i].state == 1) code = water_mods[i].code, *words = water_mods[i].words; + plat_mutex_unlock(&water_lock); return code; } -// The fragment module of a pipeline in water mode `mode` (water_mode): the local one for the module as it is built (in -// its descriptor set 1 form), else the module itself. -static VkShaderModule water_pair_module(const uint32_t *w, uint64_t n, int early, int mode) { - if (!mode) return make_module(w, n, early); - uint64_t h = fnv(FNV0, w, (size_t)n * 4); - if (mode == 2) { - // Unrelated fragment translations remain original. - if (h != 0xabfccea256824a78ull && h != 0xb3cec6d0d78ac25aull) - return make_module(w, n, early); - } else if (h != 0x3d3a316ab4da3bbaull && h != 0x216bd323c6b48ee2ull) { - rt_fatal("water projective: paired module changed; refusing a partial recipe"); +// The water mode of a pipeline's shaders: 1 a coordinate pair whose two edited modules exist, 2 the absorption +// material whose edited module exists, else 0. +static int water_projective_pair(const VkPipeState *s) { + static int on = -1; + if (on < 0) on = cfg_bool("NX2NAVI_WATER_PROJECTIVE"); + if (!on || !P.indep || !s->fs) return 0; + uint64_t vn = 0, fn = 0, wn; + const uint32_t *f = gfx_shader_spirv(s->fs, &fn); + if (!f) return 0; + uint64_t fh = fnv(FNV0, f, (size_t)fn * 4); + for (size_t i = 0; i < sizeof water_profiles / sizeof *water_profiles; i++) { + const WaterProfile *p = &water_profiles[i]; + // The absorption correction is independent of the coordinate pair. + if (fh == p->mat) return water_module(p->mat_set1, WATER_ABSORPTION, p, f, fn, &wn) ? 2 : 0; + if (fh != p->fs) continue; + const uint32_t *v = s->vs ? gfx_shader_spirv(s->vs, &vn) : NULL; + if (!v || fnv(FNV0, v, (size_t)vn * 4) != p->vs) return 0; + return water_module(p->vs, WATER_VERTEX, p, v, vn, &wn) && water_module(p->fs_set1, WATER_FRAGMENT, p, f, fn, &wn); } - uint64_t words = 0; - uint32_t *code = water_local_module(h, 4, &words); - VkShaderModule result = make_module(code, words, early); - free(code); - return result; + return 0; } -// The vertex stage of a coordinate pair (mode 1): the local module for the translation itself, taken before the +// The fragment module of a pipeline in water mode `mode` (water_mode): the edited one for the module as it is built +// (in its set 1 form), else the module itself. +static VkShaderModule water_pair_module(const uint32_t *w, uint64_t n, int early, int mode) { + uint64_t words = 0; + const uint32_t *code = mode ? water_module(fnv(FNV0, w, (size_t)n * 4), 0, NULL, NULL, 0, &words) : NULL; + if (mode == 1 && !code) LOG_ONCE("vk_pipeline: water: a coordinate pair's fragment module is not in its known form; built as it is"); + return code ? make_module(code, words, early) : make_module(w, n, early); +} + +// The vertex stage of a coordinate pair (mode 1): the edited module for the translation itself, taken before the // outputs the vertex shader's union adds (spirv_add_outputs), which differ from one install to the next and are added -// to the local module as to any other. NULL in every other case; the caller frees it. -static uint32_t *water_pair_vertex(const uint32_t *v, uint64_t n, int mode, uint64_t *words) { - if (mode != 1 || !v) return NULL; - uint64_t h = fnv(FNV0, v, (size_t)n * 4); - if (h != 0xf6d753852c47b0deull && h != 0x8d83e3705d790fedull) - rt_fatal("water projective: paired module changed; refusing a partial recipe"); - return water_local_module(h, 0, words); +// to the edited module as to any other. NULL in every other case. The module stays the table's. +static const uint32_t *water_pair_vertex(const uint32_t *v, uint64_t n, int mode, uint64_t *words) { + return mode == 1 && v ? water_module(fnv(FNV0, v, (size_t)n * 4), 0, NULL, NULL, 0, words) : NULL; } // water_projective_pair for a pipeline of key k. A coordinate pair blended with dual-source factors keeps its original -// modules: spirv_dual_source changes the fragment module before its set 1 form, which is then none the recipe knows. +// modules: spirv_dual_source changes the fragment module before its set 1 form, which is then none the table holds. static int water_mode(const VkPipeState *s, const GKey *k) { int mode = water_projective_pair(s); return mode == 1 && k->dsb ? 0 : mode; @@ -3133,9 +3112,9 @@ static VkPipeline create_graphics(const VkPipeState *s, const GKey *k, const Lay const uint32_t *vcode = gfx_shader_spirv(s->vs, &vw); uint32_t *vpatched = NULL, *fpatched = NULL; int mesh = is_mesh(s); - int wmode = mesh ? 0 : water_mode(s, k); + int wmode = mesh || !l->set1 ? 0 : water_mode(s, k); // the edited fragment modules are in their set 1 form uint64_t wn = 0; - uint32_t *vwater = water_pair_vertex(vcode, vw, wmode, &wn); + const uint32_t *vwater = water_pair_vertex(vcode, vw, wmode, &wn); if (vwater) vcode = vwater, vw = wn; VkShaderModule tm = VK_NULL_HANDLE; if (mesh && s->ts) { // the task stage's bindings follow the fragment stage's (pipe_miss) @@ -3196,7 +3175,7 @@ static VkPipeline create_graphics(const VkPipeState *s, const GKey *k, const Lay if (fcode) fm = water_pair_module(fcode, fw, !k->a2c, wmode), fm_hash = mm_hash; } vm = make_module(vcode, vw, 0), vm_hash = mm_hash; - free(vpatched), free(fpatched), free(vwater); + free(vpatched), free(fpatched); VkPipeline pipe = VK_NULL_HANDLE; if (!vm || (s->fs && !fm) || (mesh && s->ts && !tm)) goto out; @@ -3427,13 +3406,11 @@ static VkPipeline lib_create(int kind, const VkPipeState *s, const GKey *k, VkPi const uint32_t *vcode = gfx_shader_spirv(s->vs, &vw), *scode = vcode; sw = vw; uint64_t wn = 0; - uint32_t *vwater = water_pair_vertex(vcode, vw, pk->water_projective, &wn); + const uint32_t *vwater = water_pair_vertex(vcode, vw, pk->water_projective, &wn); if (vwater) vcode = vwater, vw = wn; if (pk->missing && (patched = spirv_add_outputs(vcode, vw, pk->missing, pk->fin, &n2))) vcode = patched, vw = n2; else if (pk->missing) LOG_ONCE("vk_pipeline: cannot add missing vertex outputs (mask %#x)", pk->missing); - m = make_module(vcode, vw, 0); - free(vwater); - if (!m) goto out; + if (!(m = make_module(vcode, vw, 0))) goto out; st.stage = VK_SHADER_STAGE_VERTEX_BIT, st.module = m; pin_subgroup32(&st, &rq, scode, sw); pi.stageCount = 1, pi.pStages = &st, pi.pViewportState = &vp, pi.pRasterizationState = &rs; diff --git a/tests/water_projective_check.py b/tests/water_projective_check.py new file mode 100644 index 00000000..d1118798 --- /dev/null +++ b/tests/water_projective_check.py @@ -0,0 +1,133 @@ +"""The water corrections the runtime makes in memory (runtime/spirv_water.c) against the same steps of +tools/projective_recipe.py, on two made-up modules: a vertex stage that writes a position and a location 0 output, and +a fragment stage that reads a location 0 input without perspective. Each side's result must be the same words, and +valid SPIR-V. A module of another shape must be refused. Needs spirv-as and spirv-val (MSYS2 UCRT64 spirv-tools). +Usage: python tests/water_projective_check.py [--cc CLANG]""" +import argparse +import shutil +import subprocess +import sys +from pathlib import Path + +root = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(root / 'tools')) +import projective_recipe as recipe # noqa: E402 + +VERTEX = ''' + OpCapability Shader + %1 = OpExtInstImport "GLSL.std.450" + OpMemoryModel Logical GLSL450 + OpEntryPoint Vertex %main "main" %out0 %pv %in0 + OpDecorate %out0 Location 0 + OpDecorate %in0 Location 0 + OpMemberDecorate %PerVertex 0 BuiltIn Position + OpDecorate %PerVertex Block + %void = OpTypeVoid + %fn = OpTypeFunction %void + %float = OpTypeFloat 32 + %v4 = OpTypeVector %float 4 + %PerVertex = OpTypeStruct %v4 + %p_out_pv = OpTypePointer Output %PerVertex + %pv = OpVariable %p_out_pv Output + %p_out_v4 = OpTypePointer Output %v4 + %out0 = OpVariable %p_out_v4 Output + %p_in_v4 = OpTypePointer Input %v4 + %in0 = OpVariable %p_in_v4 Input + %int = OpTypeInt 32 1 + %int_0 = OpConstant %int 0 + %main = OpFunction %void None %fn + %entry = OpLabel + %a = OpLoad %v4 %in0 + %p = OpAccessChain %p_out_v4 %pv %int_0 + OpStore %p %a + OpStore %out0 %a + OpReturn + OpFunctionEnd +''' +FRAGMENT = ''' + OpCapability Shader + %1 = OpExtInstImport "GLSL.std.450" + OpMemoryModel Logical GLSL450 + OpEntryPoint Fragment %main "main" %in0 %col + OpExecutionMode %main OriginUpperLeft + OpDecorate %in0 Location 0 + OpDecorate %in0 NoPerspective + OpDecorate %col Location 0 + %void = OpTypeVoid + %fn = OpTypeFunction %void + %float = OpTypeFloat 32 + %v4 = OpTypeVector %float 4 + %p_in_v4 = OpTypePointer Input %v4 + %p_in_f = OpTypePointer Input %float + %in0 = OpVariable %p_in_v4 Input + %p_out_v4 = OpTypePointer Output %v4 + %col = OpVariable %p_out_v4 Output + %int = OpTypeInt 32 1 + %int_1 = OpConstant %int 1 + %p_fn_f = OpTypePointer Function %float + %main = OpFunction %void None %fn + %entry = OpLabel + %tmp = OpVariable %p_fn_f Function + %a = OpLoad %v4 %in0 + %py = OpAccessChain %p_in_f %in0 %int_1 + %y = OpLoad %float %py + OpStore %tmp %y + OpStore %col %a + OpReturn + OpFunctionEnd +''' +HARNESS = r''' +#include +#include +#include +#include "spirv_water.h" +int main(int argc, char **argv) { // check : 0 written, 3 refused + (void)argc; + FILE *f = fopen(argv[2], "rb"); + fseek(f, 0, SEEK_END); + long bytes = ftell(f); + rewind(f); + uint32_t *w = malloc((size_t)bytes); + if (fread(w, 1, (size_t)bytes, f) != (size_t)bytes) return 2; + fclose(f); + uint64_t n = 0; + uint32_t *o = argv[1][0] == 'v' ? spirv_water_vertex(w, (uint64_t)bytes / 4, &n) + : spirv_water_fragment(w, (uint64_t)bytes / 4, &n); + if (!o) return 3; + f = fopen(argv[3], "wb"); + fwrite(o, 4, (size_t)n, f); + return fclose(f) != 0; +} +''' + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument('--cc', default='clang') + a = ap.parse_args() + tools = [shutil.which(t) for t in ('spirv-as', 'spirv-val')] + if not all(tools): + sys.exit('water projective: SKIP (no spirv-as and spirv-val on PATH)') + out = root / 'build/test/water-projective' + out.mkdir(parents=True, exist_ok=True) + (out / 'check.c').write_text(HARNESS) + exe = out / 'check.exe' + subprocess.run([a.cc, '-std=gnu11', '-O2', '-Wall', '-Wextra', '-Werror', '-I', str(root / 'runtime'), + str(out / 'check.c'), str(root / 'runtime/spirv_water.c'), '-o', str(exe)], check=True) + for name, text, edit in (('vertex', VERTEX, recipe.vertex), ('fragment', FRAGMENT, recipe.fragment)): + src, mine = out / f'{name}.spv', out / f'{name}-runtime.spv' + (out / f'{name}.spvasm').write_text(text) + subprocess.run([tools[0], '--target-env', 'vulkan1.2', str(out / f'{name}.spvasm'), '-o', str(src)], check=True) + subprocess.run([str(exe), name, str(src), str(mine)], check=True) + want = edit(src.read_bytes()) + assert mine.read_bytes() == want, f'{name}: the runtime\'s module differs from the recipe\'s' + subprocess.run([tools[1], '--target-env', 'vulkan1.2', str(mine)], check=True) + # each edit refuses the other stage's module + for name, other in (('vertex', 'fragment'), ('fragment', 'vertex')): + rc = subprocess.run([str(exe), name, str(out / f'{other}.spv'), str(out / 'refused.spv')]).returncode + assert rc == 3, f'the {name} edit took a {other} module (exit {rc})' + print('water projective: the runtime\'s vertex and fragment edits match the recipe\'s, valid, other shapes refused') + + +if __name__ == '__main__': + main()