From cc5c8f87ad9d10a406daa6413a1d847ff660afbb Mon Sep 17 00:00:00 2001 From: chrg6 Date: Fri, 13 Nov 2026 08:33:09 -0500 Subject: [PATCH] Add shadow_pcf, which makes every depth-compare sampler filter linearly so each shadow test blends the 2x2 shadow map texels around it, and log the filters of the compare samplers made. Through a nearest filter each of the game's shadow compares tests one texel, so shadow edges are stair steps of shadow map texels that crawl over the ground as the sun cascades follow the camera. The setting is off by default and read at start; with shadow_scale above 1 the samplers were already linear. Co-Authored-By: Claude Opus 5.5 --- docs/maxwell.md | 2 +- nx2navi.ini.example | 6 ++++++ runtime/cfg.c | 5 ++++- runtime/vk_texture.c | 36 ++++++++++++++++++++++++++++++++++-- 4 files changed, 45 insertions(+), 4 deletions(-) diff --git a/docs/maxwell.md b/docs/maxwell.md index 3984bdfa..edc25dfc 100644 --- a/docs/maxwell.md +++ b/docs/maxwell.md @@ -55,7 +55,7 @@ Source shorthands: | 2.10 | **Texture gather rounding**: Maxwell sub-texel precision differs from AMD/Intel; edge texels picked by TLD4 differ by one. | Implementation-defined `subTexelPrecisionBits`. | A sub-pixel offset on gathers on non-NVIDIA hosts fixes shadow artifacts at object edges. Ours: the translator biases gather coordinates by the host's `subTexelPrecisionBits` on every non-NVIDIA GPU (`vk_caps.c` → job `gather_bias` → XL `TexturePass.InsertCoordGatherBias`); without it RADV drew dashed dark lines along wall and floor edges that changed with the camera angle. `NX2NAVI_GATHER_BIAS=0` turns it off. | — | | 2.11 | **Gather offsets**: `TLD4.PTP` takes four per-texel offsets, possibly from registers. | `ConstOffsets` must be constant. | Ours: `QueryHostSupportsNonConstantTextureOffset() => false` makes the translator emulate register offsets (`tools/xlate/Program.cs`). | XL `Translation/Optimizations` (TexturePass) | | 2.13 | **TXD gradient register order**: NX2navi's terrain shaders fill the gradient registers of a `TXD` per coordinate (du/dx, du/dy, dv/dx, dv/dy). | Vulkan's `Grad` takes dPdx (du/dx, dv/dx) and dPdy (du/dy, dv/dy). | The translator's `EmitTxd` takes the first two registers as dPdx and the last two as dPdy, so the mip level came from the wrong derivatives and was too sharp on terrain faces turned across the screen: patches that followed the faces crawled when the camera moved (found from an F9 frame: a gradient sample's coordinate and gradients traced back to `DPdx` and `DPdy` of the interpolants). Ours: `gfx_shader.c` `patch_txd_transpose` swaps the two inner components of every two-component gradient pair in the loaded SPIR-V (the shader cache is untouched). `NX2NAVI_TXD_TRANSPOSE=0` turns it off. Only 2D gradients are handled; 3D and cube ones are left as translated. A Switch 2 title (NVN 200 and up) is left out: `AmpereLowering.cs` `Txd` puts SM86's interleaved derivatives in the translator's order already, and transposing again swapped them back (`=1` forces it on). | — | -| 2.12 | **Depth compare** (`TSC` depth compare + `TIC_4_DEPTH_TEXTURE`), min/max reduction filtering. | `compareEnable`; reduction mode needs `samplerFilterMinmax` and is not allowed with compare. | Ours: reduction dropped when compare is on (`vk_texture.c` `make_sampler`). A depth-compare instruction through a sampler whose compare is off still compares (the game's shadow reads do this); in Vulkan that is undefined, NVIDIA's driver compares and AMD's returns the depth. Ours: such a read through a bound sampler gets the slot's sampler with the compare on (`vk_sampler_compare_by_id_locked`; the game's function, else LEQUAL; `NX2NAVI_SHADOW_COMPARE=0` off). Reads through the sampler pool (bindless) still take the game's sampler as it is. | — | +| 2.12 | **Depth compare** (`TSC` depth compare + `TIC_4_DEPTH_TEXTURE`), min/max reduction filtering. | `compareEnable`; reduction mode needs `samplerFilterMinmax` and is not allowed with compare. | Ours: reduction dropped when compare is on (`vk_texture.c` `make_sampler`). A depth-compare instruction through a sampler whose compare is off still compares (the game's shadow reads do this); in Vulkan that is undefined, NVIDIA's driver compares and AMD's returns the depth. Ours: such a read through a bound sampler gets the slot's sampler with the compare on (`vk_sampler_compare_by_id_locked`; the game's function, else LEQUAL; `NX2NAVI_SHADOW_COMPARE=0` off). Reads through the sampler pool (bindless) still take the game's sampler as it is. A compare through a linear filter tests the 2x2 texels around the coordinate and blends the four results (percentage-closer filtering), through a nearest one it tests one texel: shadow edges in steps of shadow map texels. Ours: the game's filter, linear with `NX2NAVI_SHADOW_SCALE` above 1 or `NX2NAVI_SHADOW_PCF=1` (`vk_texture.c` `make_sampler`, `shadow_pcf`); the log lists the filters of the compare samplers made ("depth-compare sampler:"). | — | | 2.13 | **ASTC / BC**: native ASTC; zero ASTC block = error colour on hosts. | AMD and NVIDIA desktop GPUs lack ASTC: decode to RGBA8. | Ours: native upload where the device samples ASTC, else a compute-shader decode (`vk_astc.c`, `NX2NAVI_ASTC`), CPU decode (`tex_astc.c`) as the fallback. | `runtime/vk_astc.c` | | 2.14 | **Linear CPU-written textures** (`FLAGS_LINEAR`, small ramps, glyph caches): CPU writes visible to the GPU directly. | Must re-upload or use linear images over host memory. | Without them a sky drawn from 64×1 RGBA16F ramps is black. | — | diff --git a/nx2navi.ini.example b/nx2navi.ini.example index 23cb4409..6c951b5c 100644 --- a/nx2navi.ini.example +++ b/nx2navi.ini.example @@ -627,6 +627,12 @@ # where an object meets the ground and softer the further its shadow falls. Done in the translated shaders (translator # patch 0010), so the shader cache is rebuilt once after an update. Read at start. # pcss = 0 +# Shadow edge filtering, 0 or 1. Default: off. 1 makes every shadow-map test blend the 2x2 shadow map texels around +# it (hardware percentage-closer filtering) instead of testing one texel where the game's sampler asks for nearest +# filtering: shadow edges turn from stair steps of shadow-map texels into soft ramps, and crawl less on the ground as the +# camera moves. Costs a little GPU time. With shadow_scale above 1 it is on anyway. The log lists the filters of the +# game's shadow samplers ("depth-compare sampler:"); where they are already linear this changes nothing. Read at start. +# shadow_pcf = 0 # NVIDIA DLSS / DLAA for the 3D scene, on NVIDIA RTX GPUs with the DLSS model from NVIDIA's DLSS SDK next to the # executable (Windows: nvngx_dlss.dll next to nx2navi.exe; Linux: libnvidia-ngx-dlss.so. next to nx2navi, with the diff --git a/runtime/cfg.c b/runtime/cfg.c index a67ee9ce..48873237 100644 --- a/runtime/cfg.c +++ b/runtime/cfg.c @@ -68,7 +68,7 @@ static const char *const known[] = { "PRESNAP_PLACE", "REPLAY_CHECK", "RESOLVE_CHECK", "RESTART_WAIT", "ROMFS", "ROMFS_INDEX", "ROMFS_MMAP", "ROMFS_MODS", "ROMFS_PATCHES", "ROMFS_PROOF_DIR", "ROMFS_VERIFY", "RSTICK_DEADZONE", "RUMBLE", "RUMBLE_CURVE", "RUMBLE_HD", "RUMBLE_HD_MS", "RUMBLE_STRENGTH", "RUMBLE_TRIGGERS", "SHADER_CACHE", - "SHADER_CACHE_FLUSH", "SHADER_CARRY", "SHADER_COMPRESS", "SHADER_COMPRESS_IDLE_S", "SHADER_MEMO", "SHADOW_BULK_WRITE", "SHADOW_CHECK", "SHADOW_COMPARE", "SHADOW_RANGE", "SHADOW_RES", "SHADOW_SCALE", "SHADOW_TABLE_STORES", "SHARED_FONTS", "SHARPNESS", "SHOTS", "SHOTS_DIR", + "SHADER_CACHE_FLUSH", "SHADER_CARRY", "SHADER_COMPRESS", "SHADER_COMPRESS_IDLE_S", "SHADER_MEMO", "SHADOW_BULK_WRITE", "SHADOW_CHECK", "SHADOW_COMPARE", "SHADOW_PCF", "SHADOW_RANGE", "SHADOW_RES", "SHADOW_SCALE", "SHADOW_TABLE_STORES", "SHARED_FONTS", "SHARPNESS", "SHOTS", "SHOTS_DIR", "SVC_SLOW_MS", "SHOTS_FULL", "SPEED_PROBE", "SPIN", "SPIN_HANDOFF_US", "SPLIT_DRAWS", "SPLIT_IDLE", "SPLIT_IN_PASS", "SPLIT_US", "SPV_MARK_TEST", "SPV_OVERRIDE", "STICK_DEADZONE", "STICK_SENS", "SWKBD_ANSWER", @@ -123,6 +123,7 @@ static const struct { const char *key, *section; } sections[] = { {"COOP_STEAM", "coop"}, {"COOP_SHARE", "coop"}, {"COOP_NAMES", "coop"}, {"COOP_CHAT", "coop"}, {"SHADOW_SCALE", "display"}, {"SHADOW_RANGE", "display"}, {"CONTACT_SHADOWS", "display"}, {"PCSS", "display"}, + {"SHADOW_PCF", "display"}, {"MODS_DIR", "mods"}, {"MODS_ORDER", "mods"}, {"MODS_DISABLED", "mods"}, {"MODS_REBUILD", "mods"}, {"MENU_LANG", "language"}, {"LANG", "language"}, {"OPERATION_MODE", "language"}, {"TEXT_SPEED", "language"}, {"VOLUME", "audio"}, {"AUDIO_CHANNELS", "audio"}, {"AUDIO_DEVICE", "audio"}, {"MICROPHONE", "audio"}, {"MENU_SOUND_VOLUME", "audio"}, @@ -236,6 +237,8 @@ static const CfgMeta metas[] = { {"SVC_SLOW_MS", "0", "0..60000", RESTART}, // kernel.c kern_svc: log services slower than this {"SHADOW_RANGE", "100", "50..400", RESTART}, {"CONTACT_SHADOWS", "0", "0..100", RESTART}, {"PCSS", "0", "0..200", RESTART}, + // vk_texture.c shadow_pcf: every depth-compare sampler filters linearly (2x2 percentage-closer filtering) + {"SHADOW_PCF", "0", "0|1", RESTART}, // Game {"FPS_LIMIT", "60", "30|40|45|60|90|120|144|165|240|display|unlimited", LIVE}, {"FPS60", "1", "0|1", LIVE}, {"RTSS", "1", "0|1", RESTART}, // rtss_limit.c: RivaTuner Statistics Server takes the limit where it runs diff --git a/runtime/vk_texture.c b/runtime/vk_texture.c index 5296a274..fb584aa9 100644 --- a/runtime/vk_texture.c +++ b/runtime/vk_texture.c @@ -993,6 +993,36 @@ static int shadow_scale_of(const nvn_texdesc *d, const TexVkFormat *f, uint32_t return d->width * (uint32_t)scale <= most ? scale : 1; } +// NX2NAVI_SHADOW_PCF (default 0): every depth-compare sampler filters linearly. Through a nearest filter each of the +// game's shadow compares tests one texel and returns 0 or 1, so a shadow edge is a staircase of shadow map texels, and +// the sun cascades follow the camera, so the stairs crawl along the ground as it moves (Hyrule Field's hills). Through a +// linear filter the GPU tests the 2x2 texels around the coordinate and blends the four results (percentage-closer +// filtering), which the game's own four compares then smooth further. With NX2NAVI_SHADOW_SCALE above 1 the samplers +// filter linearly anyway (make_sampler). Read once: samplers live until shutdown. +static int shadow_pcf(void) { + static int on = -1; + if (on < 0) { + on = cfg_bool("NX2NAVI_SHADOW_PCF"); + rt_log("vk: depth-compare samplers: %s (NX2NAVI_SHADOW_PCF=%d)", + on ? "linear, each compare blends the 2x2 texels around it" : "the game's filters", on); + } + return on; +} + +// The filters of the depth-compare samplers made, each kind once (8 at most): whether NX2NAVI_SHADOW_PCF changes +// anything shows in the log (with every one already linear it does not). Caller holds the lock. +static void compare_sampler_note(const samp_par *d) { + static uint32_t seen[8]; + static int n; + uint32_t kind = (uint32_t)d->minf | (uint32_t)d->magf << 8 | (uint32_t)d->cmp_func << 16 | 1u << 24; + for (int i = 0; i < n; i++) + if (seen[i] == kind) return; + if (n == 8) return; + seen[n++] = kind; + rt_log("vk: depth-compare sampler: min filter %u, mag filter %u (0 nearest, 1 linear, more: mipmapped), function %u", + (unsigned)d->minf, (unsigned)d->magf, (unsigned)d->cmp_func); +} + // ---- ASTC as BC1 (NX2NAVI_ASTC_BC1, vk_astc.c) // A texture the CPU provides whose ASTC blocks all decode opaque is held as BC1, not BC3: the encoder then writes only // the colour half of the BC3 blocks it would have written (bc3_encode.comp), which BC1 decodes to the same texels, in @@ -5698,8 +5728,10 @@ static VkSampler make_sampler(VkCtx *c, const samp_par *d) { si.compareOp = compare_op(d->cmp_func); // A depth-compare sampler set to nearest returns 0 or 1 per tap, and the game's shaders smooth shadow edges by // a few taps spaced for its own shadow map texels. With NX2NAVI_SHADOW_SCALE those taps cover a fraction of the - // area and the edges turn hard and stepped; linear filtering makes every tap a 2x2 PCF instead. - if (shadow_scale() > 1) si.magFilter = si.minFilter = VK_FILTER_LINEAR; + // area and the edges turn hard and stepped; linear filtering makes every tap a 2x2 PCF instead. NX2NAVI_SHADOW_PCF + // does the same at the game's own map size (shadow_pcf). + compare_sampler_note(d); + if (shadow_scale() > 1 || shadow_pcf()) si.magFilter = si.minFilter = VK_FILTER_LINEAR; } VkSamplerCustomBorderColorCreateInfoEXT cb = {.sType = VK_STRUCTURE_TYPE_SAMPLER_CUSTOM_BORDER_COLOR_CREATE_INFO_EXT}; VkSamplerReductionModeCreateInfo rm = {.sType = VK_STRUCTURE_TYPE_SAMPLER_REDUCTION_MODE_CREATE_INFO};