Triforce v0.5.0

In-game mods (GameBanana browser, search and installs, with the Mask Salesman), code-patch and cheat mods, the devs' test18 to test45 work (full performance overlay, frame pacing on, steady frame rate, save export and import, bug capture, new options rows, Setup's Mods page and Add to Steam), occlusion queries and new defaults, Auto resolution that scales, replay helper threads and robust buffers off by default, the options in sections.
This commit is contained in:
ruckus7382sfgsgdf
2026-11-17 22:40:50 -06:00
parent 93eeb714c2
commit 6f338c8f56
600 changed files with 116254 additions and 2522 deletions

View File

@@ -146,7 +146,9 @@ Designed after totk-pc-recomp's runtime/vk_*.c (MIT) and Eden's renderer_vulkan,
- **Mesh shading**: `NOP_DRAW_MESH` records NVN2's DrawMeshTasks*; programs whose code writes the internal stage
buffer are the task / mesh stages (marked in their headers as Eden's GraphicsEnvironment does). Unverified.
- **Frame interpolation** (`vk_framegen.cpp`, `RX_VK_FRAMEGEN=1`): the WebGPU renderer's, ported to GLSL
(`shaders/`, compiled with glslang): made frames between the game's, shown evenly by a display thread.
(`shaders/`, compiled with glslang): made frames between the game's, shown evenly by a display thread. (Triforce) Its
vectors are refined to one per 8x8 pixels, to a fraction of a pixel, and each made pixel takes one motion
(`fg_half`, `fg_fine_flow`, `fg_fine_pick`, `fg_fine_synth`; `RX_VK_FRAMEGEN_FINE=0`: the 16x16 blocks as ported).
- **Headless**: without a window presents render offscreen; `RX_VK_SHOTS=<dir>` writes every `RX_VK_SHOT_EVERY`-th
as a PNG. `work/run-vk.ps1` runs it (refuses while Siege runs, stops past `-MaxGB` of working set);
`work/vksum.py <run>` summarises the log.

141
gpu/vk/aa_rows.h Normal file
View File

@@ -0,0 +1,141 @@
/* (Triforce) The Insert menu's Anti-aliasing row (vk_menu.cpp), and the rule that finds the game's own anti-aliasing passes
among a frame's compute dispatches (vk_draw.cpp aa_stand_in, vk_aa.cpp), as plain functions. No Vulkan, no menu:
tests/test_aa_row.py compiles and runs them. After NX2navi's aa setting (its runtime/vk_aa.c and vk_exec_draw.c
aa_dispatch).
OoT anti-aliases every frame of the 3D with SMAA in three compute passes, right after the draw that adds the bloom and
tone maps: the edges (4 x 4 pixel tiles), the blending weights from them (an indirect dispatch over the tiles with edges)
and the composition, which samples the tone mapped picture and the weights and stores the smoothed picture into the
target the gameplay upscale then takes. It stores each channel's square root, the three packed as one RGB9E5 word, through
an integer view of that B10G11R11 target (shaders/aa_common.glsl). */
#pragma once
#include <cstdint>
#include "upscaler_rows.h"
namespace mkwvk::aa_rows {
/// The Anti-aliasing row (opts.aa, saved as anti-aliasing; RX_VK_AA over it): the game's own SMAA as it is, none (the tone
/// mapped picture stored as it is), FXAA over it (shaders/aa_fxaa.glsl), or SMAA at its Ultra preset in place of the game's
/// (shaders/smaa_*.comp). The saved values keep their meaning: a new choice comes after these
enum { GAME = 0, OFF = 1, FXAA = 2, SMAA = 3, kValues = 4 };
inline const char* name(int v) {
switch (v) {
case OFF: return "Off";
case FXAA: return "FXAA";
case SMAA: return "SMAA Ultra";
default: return "Game SMAA"; // (also for a value past the row; "Game", the owner's word, 11-16)
}
}
/// The row's value as one of its choices, RX_VK_AA=<0..3> over it (env: its number, -1 unset). Anything else is the game's own
inline int chosen(int row, int env) {
const int v = env >= 0 ? env : row;
return v >= GAME && v < kValues ? v : GAME;
}
/// DLSS, DLAA, FSR 3.1, FSR 4 and their native modes anti-alias the 3D themselves, from frames the game's own pass has not
/// smoothed any better: while one of them draws (upscaler: what runs, upscaler_rows.h) the row does not apply
inline bool upscaler_own(int upscaler) { return upscaler != upscaler_rows::GAME; }
/// What is in effect: the choice, or the game's own while such an upscaler draws (its frame is then drawn as before the row)
inline int effective(int choice, int upscaler) { return upscaler_own(upscaler) ? (int)GAME : choice; }
/// The row has nothing to choose then (shown without arrows; Left and Right leave the saved value alone), nor with
/// RX_VK_AA set for the run
inline bool fixed(int upscaler, bool env_set) { return upscaler_own(upscaler) || env_set; }
// ---- the game's passes ----
/// The game's three SMAA programs, by their keys (vk_shader.cpp code_key: a hash of the compute stage's control section and
/// code as the game hands them over, the same on every PC and with every setting). Game build e0f3ccfa. NX2navi knows the
/// same three by another hash of the same code (fcb21f2afa88378c, e44377cbfafc1101, 470d661afb1cb73a in its vk_aa.c): these
/// are the programs of a build's shader list (triforce_shaders.bin) with those hashes. Another game version has other keys:
/// nothing is then found, and the frame is drawn as before the row (vk_aa.cpp says so in the log)
enum { NONE = 0, EDGES = 1, WEIGHTS = 2, COMPOSE = 3 };
constexpr uint64_t kEdgesKey = 0x26fe0c70ab9fc547ull, kWeightsKey = 0x685adb41edc19de7ull, kComposeKey = 0x6637ac1d052d22d4ull;
constexpr int role(uint64_t key) {
return key == kEdgesKey ? EDGES : key == kWeightsKey ? WEIGHTS : key == kComposeKey ? COMPOSE : NONE;
}
/// What a dispatch has at one of its program's texture or image bindings, as far as the rule looks
enum { OTHER = 0, R11 = 1, E5 = 2 }; // (the image's own format: B10G11R11, the scene's and the picture's; E5B9G9R9; anything else)
struct Bound {
uint64_t image = 0; // (which image: any number that tells two apart; 0 nothing bound)
int format = OTHER;
uint32_t width = 0, height = 0;
bool flat = false; // (a plain 2D image, one sample, seen as 2D at its first level)
bool written = false; // (an image binding the program stores into)
bool integer = false; // (the program reads or stores it as integer words, not as colours: the variant of the program
// that runs is the one translated for the views bound, so this is the view's kind)
};
/// The composition's two ends among what its dispatch has bound (the program's own bindings, in its order): the target, the one
/// flat B10G11R11 (or E5B9G9R9: the same words) image it stores integer words into, and the tone mapped picture, the one flat
/// B10G11R11 texture of the target's size it samples as colours that is another image (the weights it samples too are read
/// as integer words). src and dst: their places in the lists. why: nullptr when both are found, else what was not as expected;
/// the game's own pass then runs, nothing is guessed
struct Ends {
int src = -1, dst = -1;
const char* why = nullptr;
};
inline Ends compose_ends(const Bound* textures, int ntex, const Bound* images, int nimg) {
Ends e;
int found = 0;
for (int i = 0; i < nimg; i++) {
const Bound& b = images[i];
if (!b.image || !b.written || !b.integer || !b.flat || (b.format != R11 && b.format != E5) || !b.width || !b.height) continue;
if (!found++) e.dst = i;
}
if (found != 1) return e.dst = -1, e.why = found ? "it stores into more than one such target" : "it stores into no B10G11R11 target as integer words", e;
const Bound& d = images[e.dst];
found = 0;
for (int i = 0; i < ntex; i++) {
const Bound& b = textures[i];
if (!b.image || b.image == d.image || b.integer || !b.flat || b.format != R11 || b.width != d.width || b.height != d.height) continue;
if (!found++) e.src = i;
}
if (found != 1) return e.src = e.dst = -1, e.why = found ? "it samples more than one picture of its target's size" : "it samples no B10G11R11 picture of its target's size", e;
return e;
}
/// Whether a dispatch of the game's edge or weight pass is left out: only the composition reads what they make, so they go
/// while another choice than the game's own is in effect and the composition before this one was replaced. One that could
/// not be replaced runs as the game's, and these two run again from the next frame on, until one is replaced again.
/// keep (RX_VK_AA_KEEP=1): never left out
constexpr bool leave_out(int role, int mode, bool last_replaced, bool keep) {
return (role == EDGES || role == WEIGHTS) && mode != GAME && last_replaced && !keep;
}
/// What the row says under its help about the passes (vk_aa.cpp aa_state): nothing asked or nothing seen yet, the game's
/// composition replaced, or none replaced for a while with another choice than the game's own in effect
enum { QUIET = 0, REPLACED = 1, MISSING = 2 };
/// The frames with a choice in effect and no composition replaced after which that is said (a minute at 60 a second: a
/// loading screen or a menu without the 3D that lasts longer says it too)
constexpr uint64_t kMissingFrames = 3600;
/// What vk_aa.cpp keeps about the game's passes from frame to frame (one for the run, under its lock)
struct Track {
int mode = GAME; // (the choice in effect in the frame before)
bool last_replaced = false; // (the last composition met was replaced: what leave_out asks)
int state = QUIET;
uint64_t since = 0; // (the frame the choice came into effect, or a composition was last replaced)
/// A frame begins with `want` in effect. true: the row's line became MISSING just now (said once in the log)
constexpr bool begin(uint64_t frame, int want) {
bool missing = false;
if (want == GAME) {
// (the game's own passes all run from here on: a composition replaced before says nothing about the next choice's
// first frame, whose edge and weight passes must run until one is replaced again)
last_replaced = false, state = QUIET;
} else if (mode != want) {
since = frame, state = QUIET; // (another choice: counted anew)
} else if (state != MISSING && frame - since >= kMissingFrames) {
state = MISSING, missing = true;
}
mode = want;
return missing;
}
/// A composition met with a choice in effect: replaced, or left to the game
constexpr void composed(uint64_t frame, bool replaced) {
last_replaced = replaced;
if (replaced) since = frame, state = REPLACED;
}
};
} // namespace mkwvk::aa_rows

111
gpu/vk/astc_opaque.h Normal file
View File

@@ -0,0 +1,111 @@
// (Triforce) Which ASTC blocks the GPU decoder (host_shaders/astc_decoder_comp_spv.h, Eden's) writes with alpha 255 in every
// texel, told from the block's 16 bytes alone: what lets a decoded ASTC texture be kept as BC1 instead of BC3 (vk_texture.cpp,
// tex_bc1.h). tests/test_tex_bc1.py
//
// The answer has to be the decoder's, not the ASTC specification's: a block the decoder takes for an error is written as
// (0, 0, 0, 0), alpha 0, and its endpoint modes with alpha may still decode to 255 everywhere. So this says yes only where the
// decoder's own code leaves no other outcome, and no everywhere else (a no keeps the texture BC3, as it was):
//
// - a void extent: the decoder's IsError takes one for an error when it is the HDR kind (bit 9) or bits 10 and 11 are not both
// set, and FillVoidExtentLDR writes the constant's alpha, bits 112 to 127, divided by 65535: 255 only for 0xFFFF. It does
// not look at the extent's coordinates, so neither does this.
// - any other block, the errors in DecompressBlock's order: the reserved block modes (bits 0 to 3 zero; bits 0 and 1 zero
// with bits 6 to 8 set), a weight grid larger than the block, four partitions with two planes, more than 64 weights, the
// weights' bits under 24 or over 96, and no room left for the colour data (its count of the remaining bits wraps).
// - then the colour endpoint modes. Modes 0, 1, 6, 8 and 9 (luminance and RGB, low dynamic range) give both endpoints the
// alpha 255 as a constant (ComputeEndpoints; BlueContract and ClampByte pass it through), and the decoder's interpolation
// of two equal 16-bit values, (65535 * (64 - w) + 65535 * w + 32) / 64, is 65535 for every weight, which it writes as
// 65535 / 65535 = 1. Modes 4, 5, 10, 12 and 13 carry alpha, and 2, 3, 7, 11, 14 and 15 are the HDR ones, which take the
// decoder's other path: no for all of those, without decoding them (such a block may be opaque all the same).
// One partition has its mode in bits 13 to 16. Several partitions share one mode when bits 23 and 24 are zero (the mode is
// bits 25 to 28); when they do not, the modes' other bits sit below the weights and are not looked for: no.
#pragma once
#include <cstdint>
namespace astc_opaque {
/// `n` bits (16 at most) of a block from bit `at`, the lowest first, as the decoder's StreamBits reads them
inline uint32_t bits(const uint8_t* b, unsigned at, unsigned n) {
uint32_t v = 0;
for (unsigned i = 0; i < 3 && at / 8 + i < 16; i++) v |= (uint32_t)b[at / 8 + i] << (8 * i);
return (v >> (at & 7)) & ((1u << n) - 1);
}
/// The bits `n` weights take in the weight range `index` (0 to 11: 2, 3, 4, 5, 6, 8, 10, 12, 16, 20, 24 and 32 levels), as
/// the decoder's GetBitLength counts them: the plain bits, and a trit for every 8/5 of a bit or a quint for every 7/3
inline unsigned weight_bits(unsigned n, unsigned index) {
static const uint8_t plain[12] = {1, 0, 2, 0, 1, 3, 1, 2, 4, 2, 3, 5};
static const uint8_t kind[12] = {0, 1, 0, 2, 1, 0, 2, 1, 0, 2, 1, 0}; // (1: with a trit, 2: with a quint)
unsigned total = n * plain[index];
if (kind[index] == 1) total += (n * 8 + 4) / 5;
if (kind[index] == 2) total += (n * 7 + 2) / 3;
return total;
}
/// Whether the decoder writes alpha 255 into every texel of this block of bw x bh texels (see the top of the file). False
/// for a block size the decoder is not given (under 4 or over 12 a side)
inline bool block(const uint8_t b[16], unsigned bw, unsigned bh) {
if (bw < 4 || bh < 4 || bw > 12 || bh > 12) return false;
const uint32_t mode = bits(b, 0, 11);
if ((mode & 0x1FF) == 0x1FC) { // (a void extent)
if (mode & 0x200) return false;
if (!(mode & 0x400) || !bits(b, 11, 1)) return false;
return b[14] == 0xFF && b[15] == 0xFF;
}
if (!(mode & 0xF)) return false;
if (!(mode & 3) && (mode & 0x1C0) == 0x1C0) return false;
// (the weight grid, by the decoder's FindLayout and DecodeBlockSize)
const unsigned a = (mode >> 5) & 3, b2 = (mode >> 7) & 3, b1 = (mode >> 7) & 1;
unsigned layout, w, h;
if (mode & 3) {
if (mode & 8) {
if (!(mode & 4)) layout = 2, w = a + 2, h = b2 + 8;
else if (mode & 0x100) layout = 4, w = b1 + 2, h = a + 2;
else layout = 3, w = a + 2, h = b1 + 6;
} else if (mode & 4) {
layout = 1, w = b2 + 8, h = a + 2;
} else {
layout = 0, w = b2 + 4, h = a + 2;
}
} else if (mode & 0x100) {
if (!(mode & 0x80)) layout = 9, w = a + 6, h = ((mode >> 9) & 3) + 6;
else if (mode & 0x20) layout = 8, w = 10, h = 6;
else layout = 7, w = 6, h = 10;
} else if (mode & 0x80) {
layout = 6, w = a + 2, h = 12;
} else {
layout = 5, w = 12, h = a + 2;
}
if (w > bw || h > bh) return false;
const bool dual = layout != 9 && (mode & 0x400);
const unsigned parts = bits(b, 11, 2) + 1;
if (parts == 4 && dual) return false;
const unsigned weights = w * h * (dual ? 2 : 1);
if (weights > 64) return false;
// (the weights' range, by the decoder's DecodeMaxWeight: bit 4 and two more bits, less 2, six further with bit 9)
unsigned range = ((mode >> 4) & 1) | (layout < 5 ? (mode & 3) << 1 : (mode & 0xC) >> 1);
if (range < 2) return false; // (cannot be: the reserved modes above)
range = range - 2 + (layout != 9 && (mode & 0x200) ? 6 : 0);
const unsigned wbits = weight_bits(weights, range);
if (wbits < 24 || wbits > 96) return false;
unsigned read, cem;
if (parts == 1) {
cem = bits(b, 13, 4), read = 17;
} else {
const uint32_t base = bits(b, 23, 6);
if (base & 3) return false; // (a mode a partition: not looked for)
cem = base >> 2, read = 29;
}
// (the decoder's last error, its count of the remaining bits wrapping. With one shared mode it cannot: 96 + 29 + 2)
if (wbits + read + (dual ? 2u : 0u) > 128) return false;
return cem == 0 || cem == 1 || cem == 6 || cem == 8 || cem == 9;
}
/// Whether every one of `n` blocks (16 bytes each, in any order) is one the decoder writes opaque
inline bool all(const uint8_t* blocks, uint64_t n, unsigned bw, unsigned bh) {
for (uint64_t i = 0; i < n; i++)
if (!block(blocks + 16 * i, bw, bh)) return false;
return true;
}
} // namespace astc_opaque

143
gpu/vk/barrier_rules.h Normal file
View File

@@ -0,0 +1,143 @@
// (Triforce) Where the replay records less than a full barrier between work the game left unordered: a part's split inside a
// pass, an event's signal, a dispatch right after another; and when held submits go to the queue (RX_VK_BATCH). Apart from
// Vulkan so that tests/test_barrier_rules.py can run the rules over recorded command streams and check every hazard the game's
// ordering asks for against what the recorded barriers order (and, a rule dropped at a time, that a hazard is then left open).
//
// What the game's ordering asks for (NVN's, as the replay keeps it): work after one of the game's barriers after the work before
// it; a draw after a dispatch and a dispatch after a draw (the console's 3D and compute work share the engine and drain at the
// switch); copies, clears and uniform updates after and before what they touch; the colour, depth and stencil outputs of draws
// in the order they were drawn, from pass to pass too; a draw that samples a target its pass wrote, or writes one its pass
// sampled, after the draws before (the replay ends the pass there itself: vk_draw.cpp draw_hazards); an event's word written
// after the work before its signal, and a wait's work after the work before the signal it waits for. A signal is no barrier on
// the console: the work after it in the same queue is not ordered after the work before it.
//
// 1. A part's split (RX_VK_SPLIT_BARRIER; vk_replay.cpp maybe_split). A long submission goes to the GPU in parts of 120 draws and
// dispatches (RX_VK_CHUNK) so that the GPU starts early. Each part ended its pass with a full barrier, and the next began with
// another: the GPU drained between two draws of one of the game's passes, about 27 times a frame. With the rule the open pass
// goes on in the next part: the part ends it, and the next begins it again with the same targets behind a barrier of the
// attachments' outputs alone (colour output and the depth and stencil tests, their writes before their reads and writes), and
// keeps what the pass's draws sampled and wrote, so that draw_hazards ends the pass where it would have without the split. That
// orders all that the same draws in one pass ordered (RX_VK_CHUNK=0): within a pass Vulkan orders the attachments' accesses
// alone. Only where nothing else was recorded at the split (the part's uniform updates, its write-backs, an event's write held:
// then the full barriers as before), and a pass open. A part's uploads keep their own full barriers (vk_texture.cpp upload_cmd,
// texture_uploads_recorded), and anything recorded or submitted between the part's submit and the pass's start again (another
// submitter's work on the queue) gives the next part a full barrier first. Unset: on with NVIDIA's driver; =pass: on any card;
// =full (or any other value): as before. ((Triforce, P4) A parallel replay's segments are never split: they are the parts there,
// each begun and ended with a full barrier (vk_replay.cpp record_segment), so nothing goes on across a segment's edge; the rule
// acts in the serial replay's parts, as before. "Recorded between" is the replaying thread's own commands and every submit to the
// queue (cmd_work.h work_between); RX_VK_BARRIER_CONTEXT=0: every thread's commands, as before)
// 2. An event's signal (RX_VK_EVENT_BARRIER; vk_replay.cpp NOP_SIGNAL_EVENT). Each signal ended the pass and drained the GPU before
// its word's write. With the rule nothing is recorded at the signal: its word's buffer is bound there (so its page is among the
// recording's write-backs, as before), and the write waits for the next full barrier on its command buffer (the game's next
// barrier, the next dispatch's or pass start's) and goes right after it, so it still follows all the work before the signal;
// a recording's end writes what is still held behind a full barrier first, before its write-backs copy the words to the game's
// memory (that barrier is recorded under the pools' lock, so the writes take none: their buffers were bound at the signals). The
// writes count as work for the next barrier's look, so the write-backs' barrier is never left out after them. The writes held
// for one barrier go in the order of their signals, a barrier of the transfers between each two, so the game sees them done in
// that order (two signals of one word too). A wait met in the replay's order whose event's write is still held gets a full
// barrier first: the waiting queue's work is then after the signalling queue's work before the signal, as the drain at the
// signal made it. The game sees the word later (at that barrier, not at the signal). NX2navi records nothing at a signal at all
// (its CPU writes the word once the submission is done). Unset: with the pass barrier's game mode (NVIDIA's driver,
// pass_barrier.h); =late: on any card; =full (or any other value): as before. Not with the parallel replay.
// 3. A dispatch right after another (RX_VK_DISPATCH_BARRIER; vk_draw.cpp dispatch). Off unless asked for: =game leaves out the
// full barrier of a dispatch whose command buffer's last command was the dispatch before it (no barrier of the game's, no copy,
// no draw between; not an indirect one, whose grid another dispatch may have written), as NX2navi's barrier_game does. The game
// orders its dependent dispatches with its own barriers on the console; whether it always does, only the game can show (a
// one-frame band or flicker where it does not), so it is a switch for the lab. (Triforce, wp14, P4) Beside the parallel replay too:
// "nothing between" is the command buffer's own count of its commands (cmd_work.h), and a segment is a command buffer of its own;
// with the global count the rule stood aside while the helper threads recorded, which at the Market is most frames
// (RX_VK_BARRIER_CONTEXT=0: so again). The event rule above still does (event_hold).
// 4. Held submits (RX_VK_BATCH; vk_replay.cpp submit_recording, submit_job). =1 (on by default with Mesa's RADV) holds every submit
// until something needs the GPU's results or the queue's order. =parts holds a whole submission's recording only while the game's
// submit is being replayed: a part (split off to start the GPU early) goes to the queue at once with what is held, so does the
// held when there are RX_VK_BATCH_MAX (4 here) of them, and what is held goes when the replay of the game's submit returns.
// Holding changes neither the command buffers nor their order on the queue, only how many vkQueueSubmit2 calls carry them.
#pragma once
#include <cstddef>
#include <cstdint>
#include <cstdlib>
#include <cstring>
namespace mkwvk::barrier_rules {
/// RX_VK_SPLIT_BARRIER: "pass" the rule on any card; "full" (or any other value) a full barrier at each split, as before; unset or
/// empty: on with NVIDIA's driver (vendor 0x10de). every_draw: RX_VK_BARRIER_EVERY_DRAW (an experiment that ends the pass at each
/// draw) keeps the old way
inline bool split_in_pass(const char* env, uint32_t vendor, bool every_draw) {
if (every_draw) return false;
if (env && *env) return std::strcmp(env, "pass") == 0;
(void)vendor; // ((Triforce, 11-16) off unless asked: the lab's Market had the same ~500 full barriers a frame with it on (the
return false; // helper threads record most frames there, and the rule stands aside for them) and no gain (90.6 / 89.2 fps
// against 92.8 / 90.3 with the old barriers)
}
/// Whether a part's split goes on in the same pass: the rule on, a pass open at the split, nothing recorded at the part's end (quiet:
/// the work count as it was after the pass's end) and no event's write held then (written behind a full barrier before the part's
/// write-backs, which copy its word to the game's memory: the GPU drained there all the same). ((Triforce, P4) no parallel replay's
/// argument: its segments are never split, maybe_split runs in the serial replay only)
inline bool split_carry(bool on, bool pass_open, bool quiet, bool writes_held) {
return on && pass_open && quiet && !writes_held;
}
/// Whether the next part starts with a full barrier: always when the pass is not carried, and when it is but something was recorded
/// between the part's submit and the pass's start again (into its command buffer, by this thread into another, or submitted to the
/// queue by any thread: cmd_work.h work_between)
inline bool split_start_full(bool carry, bool recorded_between) {
return !carry || recorded_between;
}
/// RX_VK_EVENT_BARRIER: "late" the rule on any card; "full" (or any other value) as before; unset or empty: with the pass barrier's
/// game mode (pass_barrier::game: NVIDIA's driver unless RX_VK_PASS_BARRIER says otherwise)
inline bool event_late(const char* env, bool pass_game) {
if (env && *env) return std::strcmp(env, "late") == 0;
(void)pass_game; // ((Triforce, 11-16) off unless asked, as the split rule above: no gain measured)
return false;
}
/// Whether a signal's write is held for the next full barrier: the rule on, no parallel replay (its segments' threads share the
/// command buffers), and room for it (most: the held writes a command buffer may have)
inline bool event_hold(bool on, bool parallel, int held, int most) {
return on && !parallel && held < most;
}
/// A wait the replay finds met: a full barrier first when the write of its event's signal is still held
inline bool wait_barrier(bool its_write_held) {
return its_write_held;
}
/// RX_VK_DISPATCH_BARRIER: "game" the barrier of a dispatch right after another left out; anything else, or unset: as before
inline bool dispatch_game(const char* env) {
return env && std::strcmp(env, "game") == 0;
}
/// Whether a dispatch records its full barrier. work_since: GPU work recorded on the command buffer since its start (vk_draw.cpp
/// work_since_barrier); no_order: RX_VK_NO_DISPATCH_ORDER; right_after: the command buffer's last command was a dispatch's (by the
/// command buffer's own count, cmd_work.h: with the parallel replay too)
inline bool dispatch_barrier(bool work_since, bool no_order, bool game, bool right_after, bool indirect) {
if (!work_since || no_order) return false;
return !(game && right_after && !indirect);
}
/// RX_VK_BATCH
enum class Batch { Off, Hold, Parts };
/// "parts": Parts; otherwise set: Hold where its number is not 0, Off where it is (as RX_VK_BATCH was read); unset: Hold with RADV,
/// Off elsewhere
inline Batch batch_mode(const char* env, bool radv) {
if (env) {
if (std::strcmp(env, "parts") == 0) return Batch::Parts;
return std::atoi(env) != 0 ? Batch::Hold : Batch::Off;
}
return radv ? Batch::Hold : Batch::Off;
}
/// Whether a recording is held rather than handed to the queue: Hold always; Parts only a whole submission's (not a part's), and
/// only inside the replay of a game's submit (in_submit: submit_job's)
inline bool batch_hold(Batch m, bool part, bool in_submit) {
if (m == Batch::Hold) return true;
return m == Batch::Parts && !part && in_submit;
}
/// Whether what is held goes to the queue now, after a submit was added to it (held: recordings held now, the added one too): Hold at
/// cap (RX_VK_BATCH_MAX, 32) as before; Parts at most (RX_VK_BATCH_MAX, 4) or when the added one was not held (a part: it goes with
/// those before it, in order)
inline bool batch_flush(Batch m, bool added_held, size_t held, size_t cap) {
if (m == Batch::Off) return true;
if (m == Batch::Parts && !added_held) return true;
return held >= cap;
}
} // namespace mkwvk::barrier_rules

87
gpu/vk/bugshot_pixels.h Normal file
View File

@@ -0,0 +1,87 @@
// (Triforce) The pixels of a bug capture's picture (vk_bugshot.cpp): the window's image as the GPU copied it out, into the
// R, G, B, 255 bytes a PNG is written from. Apart from Vulkan, so tests/test_bug_capture.py compiles it alone. The formats
// are the ones create_swapchain picks (vk_main.cpp): 8-bit RGBA or BGRA, UNORM or sRGB (the bytes are sRGB codes either
// way), and with the Insert menu's HDR a 10-bit HDR10 image, ST 2084 codes of Rec. 2020 light. That one is brought to
// sRGB with the picture's paper white as white, so a PNG viewer shows what an SDR screen would show of the game:
// highlights above white and colours outside sRGB are cut off.
#pragma once
#include <cmath>
#include <cstddef>
#include <cstdint>
#include <vector>
namespace bugshot {
enum Kind { NONE, RGBA8, BGRA8, A2B10G10R10, A2R10G10B10 };
/// What a window image's format is to this conversion, by its VkFormat number (vk_bugshot.cpp checks the numbers against
/// Vulkan's names); NONE: not one it reads
inline Kind kind_of(int vk_format) {
switch (vk_format) {
case 37: case 43: return RGBA8; // (R8G8B8A8_UNORM, R8G8B8A8_SRGB)
case 44: case 50: return BGRA8; // (B8G8R8A8_UNORM, B8G8R8A8_SRGB)
case 64: return A2B10G10R10; // (A2B10G10R10_UNORM_PACK32: red in the low ten bits)
case 58: return A2R10G10B10; // (A2R10G10B10_UNORM_PACK32: blue in the low ten bits)
default: return NONE;
}
}
/// ST 2084's curve the display's way: a code 0 to 1 as nits, 0 to 10000
inline double pq_to_nits(double code) {
const double m1 = 2610.0 / 16384.0, m2 = 2523.0 / 4096.0 * 128.0;
const double c1 = 3424.0 / 4096.0, c2 = 2413.0 / 4096.0 * 32.0, c3 = 2392.0 / 4096.0 * 32.0;
const double p = std::pow(code < 0 ? 0 : code > 1 ? 1 : code, 1.0 / m2);
return 10000.0 * std::pow((p > c1 ? p - c1 : 0) / (c2 - c3 * p), 1.0 / m1);
}
/// Linear light 0 to 1 as an sRGB code 0 to 255
inline uint8_t srgb8(double linear) {
const double x = linear < 0 ? 0 : linear > 1 ? 1 : linear;
const double e = x <= 0.0031308 ? 12.92 * x : 1.055 * std::pow(x, 1.0 / 2.4) - 0.055;
return (uint8_t)(e * 255.0 + 0.5);
}
/// n texels at px, four bytes each as the GPU wrote them, to R, G, B, 255 in place. hdr10: the image holds ST 2084 codes of
/// Rec. 2020 light (a 10-bit format only), and white_nits is the picture's paper white. False: not a format this reads
inline bool to_rgba8(uint8_t* px, size_t n, Kind kind, bool hdr10, double white_nits) {
if (kind == NONE) return false;
if (kind == RGBA8 || kind == BGRA8) {
for (size_t i = 0; i < n; i++) {
uint8_t* q = px + i * 4;
if (kind == BGRA8) {
const uint8_t b = q[0];
q[0] = q[2], q[2] = b;
}
q[3] = 255;
}
return true;
}
// (ten bits a channel. HDR: a table from a code to its light, white at 1, and one from light to an sRGB code, so a 4K
// picture is a few table reads a texel and not nine pow calls)
constexpr int kSteps = 4096;
std::vector<float> light(hdr10 ? 1024 : 0);
std::vector<uint8_t> code8(hdr10 ? kSteps + 1 : 0);
const double white = white_nits > 1 ? white_nits : 1;
for (size_t v = 0; v < light.size(); v++) light[v] = (float)(pq_to_nits((double)v / 1023.0) / white);
for (size_t v = 0; v < code8.size(); v++) code8[v] = srgb8((double)v / kSteps);
const auto out8 = [&](float x) { return code8[x <= 0 ? 0 : x >= 1 ? kSteps : (int)(x * kSteps + 0.5f)]; };
for (size_t i = 0; i < n; i++) {
uint8_t* q = px + i * 4;
const uint32_t v = (uint32_t)q[0] | (uint32_t)q[1] << 8 | (uint32_t)q[2] << 16 | (uint32_t)q[3] << 24;
const uint32_t lo = v & 1023, mid = (v >> 10) & 1023, hi = (v >> 20) & 1023;
const uint32_t r = kind == A2B10G10R10 ? lo : hi, g = mid, b = kind == A2B10G10R10 ? hi : lo;
if (!hdr10) { // (sRGB codes with ten bits: the nearest of eight)
q[0] = (uint8_t)((r * 255 + 511) / 1023), q[1] = (uint8_t)((g * 255 + 511) / 1023), q[2] = (uint8_t)((b * 255 + 511) / 1023);
} else { // (Rec. 2020 light to Rec. 709's primaries, ITU-R BT.2087's matrix, then sRGB's curve)
const float lr = light[r], lg = light[g], lb = light[b];
q[0] = out8(1.6605f * lr - 0.5876f * lg - 0.0728f * lb);
q[1] = out8(-0.1246f * lr + 1.1329f * lg - 0.0083f * lb);
q[2] = out8(-0.0182f * lr - 0.1006f * lg + 1.1187f * lb);
}
q[3] = 255;
}
return true;
}
} // namespace bugshot

129
gpu/vk/capture_behind.h Normal file
View File

@@ -0,0 +1,129 @@
/* (Triforce) A submit-time capture's copies behind the game's ordering point (vk_memory.cpp capture_make, RX_VK_CAPTURE_BEHIND).
With the replay on the submitting thread (RX_VK_ASYNC off, the default) the game's thread was held for the whole capture before
its replay began: the scan of the write marks, then the copy of every page they named. At the Castle Town Market that was
~0.35 ms a submit, six submits a frame: 2.1 ms of a 10.6 ms frame, outside the replay (the frame trace of 2026-11-15: scan 0.10
ms over ~1,240 ranges, ~2,000 pages copied, 7.8 MB). The scan is the ordering point: the pages written before the submit, their
marks taken. The copy need not be on the game's thread before the replay: the submitting thread replays the submission before it
returns to the game, so nothing it writes after its submit can reach the copy, and what reads the copy comes after the scan: the
replay's reads as submitted, the late pages' second copy and the texture shadow (take_capture), and the GPU's copies into the
mirrors once a recording is submitted. So the copy is cut in parts of consecutive pages that the capture's copy helpers take as
soon as the scan is done, while the submitting thread replays. A reader of a page whose part nobody has taken yet copies that part
itself; one whose part is being copied waits for it. Before a recording goes to the GPU, and before the submitting thread goes
back to the game, every part is done. Each page is copied once, by whoever claims its part.
Plain C++, no Vulkan: tests/test_capture_behind.py compiles it with stand-ins. */
#pragma once
#include <atomic>
#include <condition_variable>
#include <cstdint>
#include <memory>
#include <mutex>
#include <vector>
namespace mkwvk::capture_behind {
constexpr int64_t kPartPages = 32; // (128 KB a part: what a reader that needs one of its pages copies itself, ~10 us)
struct Part {
int64_t i0 = 0, i1 = 0; // (the capture's indices [i0, i1): consecutive, none of them copied ahead)
std::atomic<uint8_t> state{0}; // (0 nobody's yet, 1 being copied, 2 copied)
};
/// A capture's pages left to copy, in parts. An index is the page's place in the capture (its slot in the capture's buffer).
/// copy(i0, i1) is the caller's: it copies indices [i0, i1), and is run once for each part, by the thread that claims it
struct Work {
std::unique_ptr<Part[]> parts;
int nparts = 0;
std::vector<int32_t> part_of; // (by index: its part; -1: copied ahead, not one of this work's)
std::atomic<int> left{0}; // (the parts not copied yet)
std::mutex mu; // (the waits for a part another thread copies)
std::condition_variable cv;
/// The parts: the indices i < n with ahead[i] == 0 (ahead: null for none), in runs of at most part_pages cut at every index
/// copied ahead
void plan(const uint8_t* ahead, int64_t n, int64_t part_pages = kPartPages) {
if (part_pages < 1) part_pages = 1;
part_of.assign((size_t)(n > 0 ? n : 0), -1);
std::vector<std::pair<int64_t, int64_t>> runs;
for (int64_t i = 0; i < n;) {
if (ahead && ahead[i]) {
i++;
continue;
}
int64_t j = i + 1;
while (j < n && !(ahead && ahead[j]) && j - i < part_pages) j++;
runs.push_back({i, j});
i = j;
}
nparts = (int)runs.size();
parts.reset(new Part[(size_t)nparts]);
for (int k = 0; k < nparts; k++) {
parts[k].i0 = runs[(size_t)k].first, parts[k].i1 = runs[(size_t)k].second;
for (int64_t i = parts[k].i0; i < parts[k].i1; i++) part_of[(size_t)i] = k;
}
left.store(nparts, std::memory_order_release);
}
/// Part k is this thread's to copy (nobody had taken it)
bool claim(int k) {
uint8_t free = 0;
return parts[k].state.compare_exchange_strong(free, 1, std::memory_order_acq_rel);
}
/// Part k copied, by the thread that claimed it: its bytes are there for every thread that sees it so
void copied(int k) {
{
std::lock_guard lk(mu);
parts[k].state.store(2, std::memory_order_release);
left.fetch_sub(1, std::memory_order_acq_rel);
}
cv.notify_all();
}
bool done() const { return left.load(std::memory_order_acquire) == 0; }
/// Index i is copied (or was copied ahead)
bool ready(int64_t i) const {
if (i < 0 || (size_t)i >= part_of.size()) return true;
const int32_t k = part_of[(size_t)i];
return k < 0 || parts[k].state.load(std::memory_order_acquire) == 2;
}
/// Part k copied on return: by this thread when nobody had taken it, else by whoever took it, waited for. 1: this thread
/// copied it, 2: it waited, 0: it was copied already
template <class Copy> int make_ready(int k, Copy&& copy) {
if (parts[k].state.load(std::memory_order_acquire) == 2) return 0;
if (claim(k)) {
copy(parts[k].i0, parts[k].i1);
copied(k);
return 1;
}
std::unique_lock lk(mu);
cv.wait(lk, [&] { return parts[k].state.load(std::memory_order_acquire) == 2; });
return 2;
}
/// Index i copied on return (make_ready of its part)
template <class Copy> int ensure(int64_t i, Copy&& copy) {
if (i < 0 || (size_t)i >= part_of.size()) return 0;
const int32_t k = part_of[(size_t)i];
return k < 0 ? 0 : make_ready(k, copy);
}
/// Every part copied on return: those nobody had taken by this thread, in order, then the others waited for. The parts this
/// thread copied
template <class Copy> int finish(Copy&& copy) {
int made = 0;
for (int k = 0; k < nparts; k++)
if (parts[k].state.load(std::memory_order_acquire) == 0 && claim(k)) {
copy(parts[k].i0, parts[k].i1);
copied(k);
made++;
}
std::unique_lock lk(mu);
cv.wait(lk, [&] { return done(); });
return made;
}
};
/// Whether a capture's copies go behind: helpers to take them, enough pages for them (copy_split's own threshold: under it the
/// submitting thread copies all at once, cheaper than waking a helper), and few to copy ahead (the late pages and the texture
/// storage: take_capture reads those at once; many of them, as at a scene's load, and the capture is copied whole as before)
inline bool worth_behind(int helpers, int64_t pages, int64_t ahead, int64_t min_pages = 128, int64_t max_ahead = 512) {
return helpers > 0 && pages - ahead >= min_pages && ahead <= max_ahead;
}
} // namespace mkwvk::capture_behind

81
gpu/vk/checkpoint_kinds.h Normal file
View File

@@ -0,0 +1,81 @@
/* (Triforce) The kinds of RX_VK_CHECKPOINTS markers (vk_draw.cpp checkpoint, checkpoint_mark) and what lies after each, for the
device lost's report. The owner's test11 and test12 and two lab runs of five lost the device at the title the same way: the
top and the bottom of the pipe both at "write-back of pool ...", the next marker the next frame's first dispatch, never
reached. Between those two markers there was a pool's write-back copies, the write-back diff pass, the end of that recording,
the whole present (its recording, the blit to the window's image after its acquire, the menu) and the next recording's start
and uploads: none of them had a marker, and the report could not say which the GPU stopped in. Now each of those has one, and
the report says what the work after the last marker reached is (gap). Plain functions: tests/test_checkpoint_kinds.py. */
#pragma once
#include <cstdint>
namespace mkwvk::checkpoint_kinds {
/// A marker's kind as the report names it; its key follows the name in the report (a program, a pool, a texture, a recording's
/// timeline value, a swapchain image's index or a chunk count, as the name says)
inline const char* name(char c) {
switch (c) {
case 'd': return "draw of program";
case 'c': return "dispatch of program";
case 'i': return "INDIRECT dispatch of program";
case 'm': return "mirror sync of pool";
case 'b': return "write-back of pool";
case 'u': return "upload of texture";
case 'g': return "GPU unswizzle of texture";
case 'w': return "write-back diff pass, chunks";
case 'r': return "recording begun, value";
case 'v': return "uploads of the recording, value";
case 'z': return "recording ended, value";
case 'p': return "present blit begun, image";
case 'M': return "menu drawn over the present, image";
case 'P': return "present blit ended, image";
default: return "?";
}
}
/// What the GPU had to do after a marker of this kind, up to the next marker recorded: where a device lost whose last marker
/// reached at the top of the pipe is this one stopped
inline const char* gap(char c) {
switch (c) {
case 'd': case 'c': case 'i':
return "that draw or dispatch itself (its marker is set just before it), or what its command buffer records after it before the "
"next marker: barriers, copies, clears, queries";
case 'm': return "that mirror sync's copies (guest memory -> the pool's VRAM mirror), in the recording's uploads";
case 'b':
return "that pool's write-back copies (its mirror -> guest memory, and the shadow's fill), then the write-back diff pass if "
"one follows (w): with no w after it, the end of the recording (z)";
case 'u': case 'g': return "that texture's upload or unswizzle itself";
case 'w':
return "the write-back diff pass (one compute dispatch over every pool's chunks, reading mirrors and shadows and writing guest "
"memory by device address), then its barrier and the end of the recording (z)";
case 'r': return "the recording's first commands: its barrier, and the replay's work before its first marked draw or dispatch";
case 'v': return "the recording's uploads: mirror pieces filled, mirror syncs, the page table's changes, texture uploads";
case 'z':
return "the next submission: its waits on the queue (the mirrors' new pieces bound, a swapchain image acquired) or its first "
"commands (r, v, or the present's p next)";
case 'p': return "the present's blit: the game's picture scaled into the window's image, the HDR pass and the bars, before the menu (M)";
case 'M': return "the menu's drawing (Dear ImGui) over the present, then its last barrier (P)";
case 'P': return "the present of that image, then the next submission's waits and first commands";
default: return "not known (a marker of no named kind)";
}
}
/// The two stages' last markers (sequence numbers; 0: none reached): one sentence of what they mean together
inline const char* reading(uint32_t top, uint32_t bottom) {
if (!top) return "no marker was reached at the top of the pipe";
if (bottom == top)
return "every command up to the last marker reached is done and nothing after it began: the GPU stopped in what follows that "
"marker (below), or in a wait before the next one";
if (bottom < top)
return "the work between the bottom's last marker and the top's was running: the GPU stopped in it (a draw or dispatch that "
"did not end, or a fault)";
return "the bottom of the pipe is past the top: the markers are not in order (several queues or threads), read them with care";
}
/// (Triforce, P4) The two stages' last markers are of two threads' rings (vk_draw.cpp checkpoint_next: each recording thread
/// numbers its own): their numbers say nothing of each other's order
inline const char* apart() {
return "the two are of two threads' recordings (each numbers its own markers: the replay thread's, a helper thread's, the "
"present's): their numbers say nothing of their order, read each with the markers around it (below)";
}
} // namespace mkwvk::checkpoint_kinds

128
gpu/vk/cmd_work.h Normal file
View File

@@ -0,0 +1,128 @@
// (Triforce) What the replay's barrier rules ask of the command buffer they record into, kept per command buffer: how many
// commands went into it, and where its last full barrier, its last draw pass's end and its last dispatch were (vk_work.h counts
// the commands; vk_texture.cpp full_barrier, pass_start_barrier and pass_resume_barrier, vk_draw.cpp dispatch read and set the
// marks). Apart from Vulkan's calls so that tests/test_barrier_rules.py, test_pass_barrier.py and test_dispatch_rule_threads.py can
// run it.
//
// After NX2navi's per-context state (runtime/vk_exec.h VkExec.recorded, barrier_at, bar_dirty, bar_passes, bar_dispatches;
// vk_exec.c barrier_to_why, chunk_record, group_put): there a recording context counts its own commands, so its rules hold inside
// the worker threads' chunks. Here one count served every thread (work_epoch, an atomic add on every command from every recording
// thread), so each rule stood aside while the Replay helper threads recorded (most frames at the Market: about 500 full barriers a
// frame, no pass start in its fragment stages only). A command buffer is recorded by one thread at a time (Vulkan asks for that),
// and its state here is only touched by whoever records into it, so nothing in it needs to be atomic; the table that finds it is.
// Another thread's commands go into its own command buffers and leave this one's marks alone; a command buffer two threads take
// turns at (a parallel replay's segment init command buffer, vk_texture.cpp InitRedirect) has every turn's commands counted in it.
//
// RX_VK_BARRIER_CONTEXT=0: the one count of every thread's commands, and the rules standing aside while the helper threads record,
// as before (vk_texture.cpp rules_aside).
#pragma once
#include <vulkan/vulkan_core.h>
#include <atomic>
#include <cstddef>
#include <cstdint>
#include <cstdlib>
#include <cstring>
namespace mkwvk {
extern std::atomic<uint64_t> work_epoch; // (vk_texture.cpp: RX_VK_BARRIER_CONTEXT=0's count, every thread's commands)
inline thread_local uint64_t work_mine = 0; // (this thread's commands, into any command buffer: work_between)
inline std::atomic<uint64_t> work_submits{0}; // (every queue submit through vk_work.h, any thread's: work_between)
/// RX_VK_BARRIER_CONTEXT=0: the global count, as before; anything else, or unset: per command buffer. Read at the first command
/// (0: not read yet, 1: per command buffer, 2: global)
inline std::atomic<int> work_way{0};
inline bool work_global() {
int w = work_way.load(std::memory_order_relaxed);
if (!w) [[unlikely]] {
const char* e = std::getenv("RX_VK_BARRIER_CONTEXT");
w = e && std::strcmp(e, "0") == 0 ? 2 : 1;
work_way.store(w, std::memory_order_relaxed);
}
return w == 2;
}
/// One command buffer's state. The marks are counts of it (work_at) taken right after the command they mark; a mark equal to the
/// count now: nothing was recorded into it since
struct CmdWork {
uint64_t n = 0; // (its commands, and its begins: work_begun)
uint64_t barrier_at = UINT64_MAX; // (right after its last full barrier: full_barrier leaves out a second one there)
uint64_t pass_end_at = UINT64_MAX; // (right after its last draw pass ended, unless the replay split that pass for a hazard)
uint64_t dispatch_at = UINT64_MAX; // (right after its last dispatch: RX_VK_DISPATCH_BARRIER)
bool known = true; // (false: the stand-in for a command buffer the table had no room for, never right after anything)
};
/// The table: a slot a command buffer, taken at its first command or begin by whichever thread comes first and kept for good (the
/// replay makes its command buffers once: a few hundred at most). Full: cmd_work_none, which no rule relaxes on
constexpr size_t kCmdWorkSlots = 1024;
struct alignas(64) CmdWorkSlot { // (a cache line each: two threads recording two command buffers share none)
std::atomic<VkCommandBuffer> cmd{VK_NULL_HANDLE};
CmdWork w;
};
inline CmdWorkSlot cmd_work_slots[kCmdWorkSlots];
inline thread_local CmdWork cmd_work_none{0, UINT64_MAX, UINT64_MAX, UINT64_MAX, false}; // (vk_texture.cpp full_barrier tells it once)
inline CmdWork* cmd_work_find(VkCommandBuffer cmd) {
if (cmd == VK_NULL_HANDLE) return &cmd_work_none;
const size_t h = (size_t)(((uint64_t)(uintptr_t)cmd * 0x9E3779B97F4A7C15ull) >> 54); // (10 bits: kCmdWorkSlots)
for (size_t k = 0; k < kCmdWorkSlots; k++) {
CmdWorkSlot& s = cmd_work_slots[(h + k) % kCmdWorkSlots];
VkCommandBuffer at = s.cmd.load(std::memory_order_acquire);
if (at == VK_NULL_HANDLE && s.cmd.compare_exchange_strong(at, cmd, std::memory_order_acq_rel)) return &s.w;
if (at == cmd) return &s.w; // (its own, or taken by another thread just now)
}
return &cmd_work_none;
}
/// cmd's state (the last one this thread asked for is at hand: most commands go into the command buffer of the one before)
inline thread_local VkCommandBuffer cmd_work_last = VK_NULL_HANDLE;
inline thread_local CmdWork* cmd_work_last_w = nullptr;
inline CmdWork& cmd_work(VkCommandBuffer cmd) {
if (cmd != cmd_work_last || !cmd_work_last_w) [[unlikely]] cmd_work_last_w = cmd_work_find(cmd), cmd_work_last = cmd;
return *cmd_work_last_w;
}
/// The count a mark is taken in: the command buffer's own; RX_VK_BARRIER_CONTEXT=0, every thread's (work_epoch), as before
inline uint64_t work_at(const CmdWork& w) {
return work_global() ? work_epoch.load(std::memory_order_relaxed) : w.n;
}
/// Whether nothing was recorded into the command buffer since the mark was taken
inline bool work_right_after(const CmdWork& w, uint64_t mark) {
return w.known && mark == work_at(w);
}
/// A command that does or orders GPU work, into cmd (vk_work.h's macros; by hand after a library recorded into cmd itself)
inline void note_work(VkCommandBuffer cmd) {
// (RX_VK_BARRIER_CONTEXT=0: an add, not a load and a store: a thread preempted between the two would store a stale count over
// another thread's, which could make the count equal the one a barrier was recorded at with work recorded since)
if (work_global()) work_epoch.fetch_add(1, std::memory_order_relaxed);
else cmd_work(cmd).n++;
work_mine++;
}
inline VkCommandBuffer work_on(VkCommandBuffer cmd) {
note_work(cmd);
return cmd;
}
/// cmd begun (vkBeginCommandBuffer): a new recording starts afresh, whatever its handle held before (its last command may have
/// been a full barrier, and the work submitted since then is not after it)
inline VkCommandBuffer work_begun(VkCommandBuffer cmd) {
CmdWork& w = cmd_work(cmd);
w.n++;
w.barrier_at = w.pass_end_at = w.dispatch_at = UINT64_MAX;
return cmd;
}
/// A submit to the queue (vkQueueSubmit2), any thread's
inline VkQueue work_submitted(VkQueue q) {
work_submits.fetch_add(1, std::memory_order_relaxed);
return q;
}
/// Whether anything was recorded or submitted between two points of this thread's recording: equal before and after, nothing.
/// What crosses command buffers asks this (vk_replay.cpp maybe_split: a pass going on in the next part; finish_recording: the
/// early tail's write-back): this thread's own commands, into any command buffer, and every submit to the queue, any thread's
/// (another thread's recording reaches the queue between two of this thread's command buffers only by a submit).
/// RX_VK_BARRIER_CONTEXT=0: every thread's commands, as before
inline uint64_t work_between() {
return work_global() ? work_epoch.load(std::memory_order_relaxed) : work_mine + work_submits.load(std::memory_order_relaxed);
}
} // namespace mkwvk

333
gpu/vk/debug_names.h Normal file
View File

@@ -0,0 +1,333 @@
// (Triforce) The names on the Debug tab's Game flags, Cutscenes and Scan amiibo pages (vk_menu.cpp draws them), and what
// a page keeps: the list in groups, the choice, the cards shown. No Vulkan, no ImGui and no file here:
// tests/test_debug_names.py compiles this header and runs it on bytes of its own making.
//
// The game knows a flag (a Bool of its GameData) by the MurmurHash3 of its name and nothing else; its files list the
// Bools by hash (GameData/GameDataList.Product.<n>.byml.zs) and use their names in the event flows
// (Event/EventFlow/<event>.bfevfl.zs). So the flags' names are every text of the event flows whose hash is a Bool's, and
// the events are the event flows that hold a script (a flowchart: the others are a cutscene's timelines, which a script
// plays). After NX2navi's F9 panel, which found its names the same way; no line of it is here, and no name: both lists
// are made in memory from the player's own game files each run, and written nowhere.
#pragma once
#include <algorithm>
#include <cstdint>
#include <cstring>
#include <string>
#include <vector>
#include "oot_debug_link.h"
namespace mkwvk::debug_names {
/// A BYML file's Data.Bool[i].Hash values, sorted, with no number twice: the Bools of a GameDataList (either byte order;
/// the game's is the big-endian one). false when the bytes are not such a file. Nothing outside the bytes is read
inline bool gamedata_bools(const std::vector<unsigned char>& b, std::vector<uint32_t>& out) {
out.clear();
const size_t n = b.size();
if (n < 16 || !((b[0] == 'B' && b[1] == 'Y') || (b[0] == 'Y' && b[1] == 'B'))) return false;
const bool big = b[0] == 'B';
bool bad = false; // (a read outside the file: everything read after it is 0, and the file is refused)
auto u8 = [&](size_t o) -> uint32_t { return o < n ? b[o] : (bad = true, 0u); };
auto u24 = [&](size_t o) -> uint32_t {
if (o > n || n - o < 3) return bad = true, 0u;
return big ? (uint32_t)b[o] << 16 | (uint32_t)b[o + 1] << 8 | b[o + 2] : (uint32_t)b[o + 2] << 16 | (uint32_t)b[o + 1] << 8 | b[o];
};
auto u32 = [&](size_t o) -> uint32_t {
if (o > n || n - o < 4) return bad = true, 0u;
return big ? (uint32_t)b[o] << 24 | (uint32_t)b[o + 1] << 16 | (uint32_t)b[o + 2] << 8 | b[o + 3]
: (uint32_t)b[o + 3] << 24 | (uint32_t)b[o + 2] << 16 | (uint32_t)b[o + 1] << 8 | b[o];
};
// the keys' table: 0xc2, how many, an offset each (and one past the last) from the table's start, the texts
const size_t keys = u32(4), root = u32(12);
if (bad || !keys || u8(keys) != 0xc2) return false;
const uint32_t key_count = u24(keys + 1);
auto key_index = [&](const char* name) -> int64_t {
const size_t len = strlen(name);
for (uint32_t i = 0; i < key_count && !bad; i++) {
const size_t at = keys + (size_t)u32(keys + 4 + 4 * (size_t)i);
if (!bad && at < n && n - at > len && !memcmp(b.data() + at, name, len + 1)) return i;
}
return -1;
};
const int64_t k_data = key_index("Data"), k_bool = key_index("Bool"), k_hash = key_index("Hash");
if (bad || k_data < 0 || k_bool < 0 || k_hash < 0) return false;
// a dictionary: 0xc1, how many, then eight bytes each: the key's index (three bytes), the value's type, the value
auto member = [&](size_t dict, int64_t key, uint32_t* type) -> uint32_t {
if (u8(dict) != 0xc1) return bad = true, 0u;
const uint32_t count = u24(dict + 1);
for (uint32_t i = 0; i < count && !bad; i++) {
const size_t e = dict + 4 + 8 * (size_t)i;
if (u24(e) == (uint32_t)key) return *type = u8(e + 3), u32(e + 4);
}
return *type = 0, 0u;
};
uint32_t type = 0;
const size_t data = member(root, k_data, &type);
if (bad || type != 0xc1) return false;
const size_t bools = member(data, k_bool, &type);
if (bad || type != 0xc0 || u8(bools) != 0xc0) return false;
// an array: 0xc0, how many, a type byte each (to a multiple of four), then four bytes each
const uint32_t count = u24(bools + 1);
const size_t values = bools + 4 + (((size_t)count + 3) & ~(size_t)3);
if (bad || count > n / 4) return false;
out.reserve(count);
for (uint32_t i = 0; i < count; i++) {
if (u8(bools + 4 + i) != 0xc1) return out.clear(), false;
const uint32_t hash = member(u32(values + 4 * (size_t)i), k_hash, &type);
if (bad || (type != 0xd3 && type != 0xd1)) return out.clear(), false; // (an unsigned number, or a signed one)
if (hash) out.push_back(hash);
}
std::sort(out.begin(), out.end());
out.erase(std::unique(out.begin(), out.end()), out.end());
return !out.empty();
}
/// An event flow that holds a script: a little-endian BFEVFL whose header counts a flowchart at least (0x20)
inline bool event_has_script(const std::vector<unsigned char>& f) {
return f.size() >= 0x24 && !memcmp(f.data(), "BFEVFL\0\0", 8) && f[0xc] == 0xff && f[0xd] == 0xfe && (f[0x20] | f[0x21] << 8) >= 1;
}
/// The texts of a file that are flags' names: every run of a name's characters, 3 to OOT_NAME_MAX of them with a 0 after,
/// whose hash is one of bools (sorted). Added to out (the same name more than once: make_list sorts that out)
inline void flag_names(const std::vector<unsigned char>& f, const std::vector<uint32_t>& bools, std::vector<std::string>& out) {
const size_t n = f.size();
for (size_t i = 0; i < n;) {
if (!oot_name_char(f[i])) {
i++;
continue;
}
size_t end = i;
while (end < n && oot_name_char(f[end])) end++;
const size_t len = end - i;
if (end < n && !f[end] && len >= 3 && len <= OOT_NAME_MAX &&
std::binary_search(bools.begin(), bools.end(), oot_name_hash((const char*)f.data() + i, len)))
out.emplace_back((const char*)f.data() + i, len);
i = end;
}
}
/// The event an event flow's file is: its name without ".bfevfl.zs" and without a version before that (".110"); "" when
/// the file is not an event flow's or its name not an event's
inline std::string event_of_file(const std::string& file) {
static const char kEnd[] = ".bfevfl.zs";
const size_t end = sizeof kEnd - 1;
if (file.size() <= end || file.compare(file.size() - end, end, kEnd)) return "";
std::string name = file.substr(0, file.size() - end);
const size_t dot = name.rfind('.');
if (dot != std::string::npos && dot + 1 < name.size() && name.find_first_not_of("0123456789", dot + 1) == std::string::npos) name.resize(dot);
return oot_name_ok(name.c_str()) ? name : "";
}
// ---- a page's list: the names sorted, each once, in groups by what they begin with ----
struct Group {
std::string title;
int first = 0, count = 0; // (its names: [first, first + count) of the list)
bool begins = false; // (its names all begin with its title: not Other, and not a list without groups)
};
struct List {
std::vector<std::string> names;
std::vector<uint32_t> hashes; // (each name's, for the flags)
std::vector<Group> groups; // (never empty while there are names)
};
constexpr int kGroupMin = 6; // (a beginning that fewer names share has no group of its own: they are in "Other")
/// What a name begins with: its text before the first _ or . (the whole name when it has neither)
inline std::string name_prefix(const std::string& name) {
const size_t cut = name.find_first_of("_.");
return cut == std::string::npos || cut == 0 ? name : name.substr(0, cut);
}
/// The list of some names (any order, any name any number of times). grouped false: one group of them all (the amiibo
/// folder's files)
inline List make_list(std::vector<std::string> names, bool grouped = true, const char* one_title = "All") {
List list;
std::sort(names.begin(), names.end());
names.erase(std::unique(names.begin(), names.end()), names.end());
names.erase(std::remove(names.begin(), names.end(), std::string()), names.end());
if (names.empty()) return list;
std::vector<std::pair<std::string, std::string>> keyed; // (the group's title, the name)
keyed.reserve(names.size());
if (!grouped) {
for (std::string& name : names) keyed.emplace_back(one_title, std::move(name));
} else {
// (sorted, so the names of one beginning are together: counted in one pass)
std::vector<std::string> prefixes;
prefixes.reserve(names.size());
for (const std::string& name : names) prefixes.push_back(name_prefix(name));
std::vector<std::string> counted = prefixes;
std::sort(counted.begin(), counted.end());
for (size_t i = 0; i < names.size(); i++) {
const auto range = std::equal_range(counted.begin(), counted.end(), prefixes[i]);
const bool own = range.second - range.first >= kGroupMin;
// ("\x7f" sorts Other after every group of a beginning: a name's characters are all below it)
keyed.emplace_back(own ? prefixes[i] : std::string("\x7f"), std::move(names[i]));
}
std::sort(keyed.begin(), keyed.end());
}
for (size_t i = 0; i < keyed.size(); i++) {
if (list.groups.empty() || keyed[i].first != keyed[i - 1].first) {
Group g;
g.title = keyed[i].first == "\x7f" ? "Other" : keyed[i].first;
g.begins = grouped && keyed[i].first != "\x7f";
g.first = (int)i;
list.groups.push_back(std::move(g));
}
list.groups.back().count++;
list.hashes.push_back(oot_name_hash(keyed[i].second.data(), keyed[i].second.size()));
list.names.push_back(std::move(keyed[i].second));
}
return list;
}
/// A name as its card shows it: without the beginning its group is named for, and the _ or . after that (the group's
/// title says it once for all of them, and the room goes to where the names differ). The whole name in Other, in a list
/// without groups, and where nothing would be left
inline std::string card_label(const List& list, int group, int index) {
const std::string& name = list.names[index];
const Group& g = list.groups[group];
if (!g.begins || name.size() <= g.title.size() + 1) return name;
return name.substr(g.title.size() + 1);
}
/// A text that does not fit, cut in its middle: a third of what fits from its start, "..." and the rest from its end (the
/// game's names tell each other apart at their ends). fits(text): whether a text fits; it must hold for a shorter one when
/// it holds for a longer. The text itself when it fits, "..." when nothing more does. UTF-8 is cut between characters
template <class Fits>
std::string elide_middle(const std::string& text, Fits fits) {
if (fits(text)) return text;
const size_t n = text.size();
auto cut = [&](size_t keep) { // (keep bytes of the text at most: a head and a tail on character boundaries)
size_t head = keep / 3, tail = n - (keep - head);
while (head > 0 && ((unsigned char)text[head] & 0xc0) == 0x80) head--;
while (tail < n && ((unsigned char)text[tail] & 0xc0) == 0x80) tail++;
return text.substr(0, head) + "..." + text.substr(tail);
};
size_t lo = 0, hi = n ? n - 1 : 0; // (the most bytes kept that fit: lo fits or is 0, more than hi does not)
while (lo < hi) {
const size_t mid = (lo + hi + 1) / 2;
if (fits(cut(mid))) lo = mid;
else hi = mid - 1;
}
return cut(lo);
}
// ---- a page: a list, the group shown, the name chosen in it, and the cards as last drawn ----
struct Page {
List list;
int group = 0; // (the group shown)
int choice = 0; // (the name chosen: its place in list.names, inside the group)
int columns = 1, capacity = 1; // (as last drawn: the cards in a row, on a page)
int shown = 0; // (the page of cards shown: the choice's, or another the wheel turned to)
bool empty() const { return list.names.empty(); }
int begin() const { return empty() ? 0 : list.groups[group].first; }
int end() const { return empty() ? 0 : list.groups[group].first + list.groups[group].count; }
int pages() const { return std::max(1, (end() - begin() + std::max(1, capacity) - 1) / std::max(1, capacity)); }
/// The cards shown: [first, last) of list.names
int first() const { return begin() + std::clamp(shown, 0, pages() - 1) * std::max(1, capacity); }
int last() const { return std::min(end(), first() + std::max(1, capacity)); }
/// The page of cards the choice is on
void reveal() { shown = empty() ? 0 : (choice - begin()) / std::max(1, capacity); }
/// A new list (the files were read, or the folder listed again): the choice kept by its name when the new list has it
void set_list(List next) {
const std::string was = !empty() && choice >= 0 && choice < (int)list.names.size() ? list.names[choice] : std::string();
list = std::move(next);
group = 0, choice = 0, shown = 0;
if (empty()) return;
const auto at = std::find(list.names.begin(), list.names.end(), was);
if (!was.empty() && at != list.names.end()) {
choice = (int)(at - list.names.begin());
for (int g = 0; g < (int)list.groups.size(); g++)
if (choice >= list.groups[g].first && choice < list.groups[g].first + list.groups[g].count) group = g;
}
reveal();
}
/// Another group: dir -1 the one before, 1 the one after, round the ends; its first name chosen
void turn_group(int dir) {
if (empty()) return;
const int n = (int)list.groups.size();
group = ((group + dir) % n + n) % n;
choice = begin(), shown = 0;
}
/// The choice moved: by cards (Left and Right), rows (Up and Down), pages, or to the group's first or last name. It
/// stays in the group; the page shown follows it
void move(int cards, int rows, int by_pages, bool home, bool to_end) {
if (empty()) return;
const int step = cards + rows * std::max(1, columns) + by_pages * std::max(1, capacity);
choice = home ? begin() : to_end ? end() - 1 : std::clamp(choice + step, begin(), end() - 1);
reveal();
}
/// A card clicked: chosen, when it is one of the group's
bool choose(int index) {
if (empty() || index < begin() || index >= end()) return false;
choice = index;
return true;
}
/// The wheel: the page of cards shown, the choice where it is
void turn_shown(int by) { shown = std::clamp(shown + by, 0, pages() - 1); }
/// The hashes of the names shown now, and of the chosen one when the wheel turned its page away, at most OOT_WATCH_MAX:
/// the flags whose states the page asks for
std::vector<uint32_t> watched() const {
std::vector<uint32_t> hashes;
for (int i = first(); i < last() && (int)hashes.size() < OOT_WATCH_MAX; i++) hashes.push_back(list.hashes[i]);
if (!empty() && (choice < first() || choice >= last()) && (int)hashes.size() < OOT_WATCH_MAX) hashes.push_back(list.hashes[choice]);
return hashes;
}
};
/// The flags' states as the game's side last said them, for the cards shown
struct States {
std::vector<uint32_t> hashes; // (the list asked for)
std::string states; // (a character each: 1, 0, - or ?; empty until an answer for this list came)
/// The list asked for changed: true when a new W line is to be sent. The states known are forgotten then
bool ask(const std::vector<uint32_t>& now) {
if (now == hashes) return false;
hashes = now, states.clear();
return true;
}
/// A line of states: kept when it answers the list asked for
bool take(const char* line) {
std::string got(hashes.size(), '?');
if (!oot_states_parse(line, oot_watch_sum(hashes.data(), (int)hashes.size()), (int)hashes.size(), got.data())) return false;
states = std::move(got);
return true;
}
/// A flag's state: 1, 0, - (the save data has no such flag) or ? (not known: no save loaded, or no answer yet)
char of(uint32_t hash) const {
for (size_t i = 0; i < hashes.size() && i < states.size(); i++)
if (hashes[i] == hash) return states[i];
return '?';
}
};
/// The lines a page waits to send, a byte a read (HOS_OPT_DEBUG_ASK, HOS_OPT_AMIIBO_ASK): each ends with a line feed,
/// 0 while there is none. The game's side reads them only while the game runs, so of the lines that only say what the
/// page shows now (those beginning with `latest`: W) the newest alone is kept, and no more than kLinesMax wait
struct Lines {
static constexpr int kLinesMax = 32;
std::vector<std::string> waiting;
size_t at = 0; // (how much of waiting[0] was read)
void push(const std::string& line, char latest = 0) {
for (size_t i = at ? 1 : 0; latest && i < waiting.size();) { // (an older one not yet begun: dropped)
if (!waiting[i].empty() && waiting[i][0] == latest) waiting.erase(waiting.begin() + (long)i);
else i++;
}
if ((int)waiting.size() < kLinesMax) waiting.push_back(line);
}
int take() {
if (waiting.empty()) return 0;
if (at < waiting[0].size()) return (unsigned char)waiting[0][at++];
waiting.erase(waiting.begin()), at = 0;
return '\n';
}
/// The same line as the last one waiting: a key held on a request repeats it, and one of them is enough
bool repeats(const std::string& line) const { return !waiting.empty() && waiting.back() == line; }
/// The lines waiting are given up but those beginning with `keep` and one already half read (the Debug tools were
/// turned off: what was asked of the game and not yet taken is not done after all; a W still says what is shown)
void drop_but(char keep) {
for (size_t i = at ? 1 : 0; i < waiting.size();) {
if (waiting[i].empty() || waiting[i][0] != keep) waiting.erase(waiting.begin() + (long)i);
else i++;
}
}
};
} // namespace mkwvk::debug_names

42
gpu/vk/debug_pending.h Normal file
View File

@@ -0,0 +1,42 @@
// (Triforce) The Inventory page's change asked of the game and not yet taken by it (vk_menu.cpp keeps it in cheat_action;
// host/hos_main.c takes it at Link's update, which does not run while the game's own Options screen has the game paused).
// The arithmetic is here so that tests/test_debug_pending.py can run it.
//
// An action is OOT_CHEAT_ITEM_ACTION + item * 256 + op: op 0..127 adds op - 64 to the item's amount, op 200 unlocks it (1)
// or relocks it (0). Only one action waits at a time. Before, each press replaced the last: ten presses of Right on an
// amount in the paused Options screen made it one more, and the card showed the amount as it was; a second press of Unlock
// unlocked it all the same. Now a press adds to the same item's waiting change, and the card shows what it will be.
#pragma once
#include <algorithm>
#include "oot_cheat_items.h"
namespace mkwvk::debug_pending {
/// The change to item's amount the waiting action asks (0: none, or the action is not that item's amount)
inline int amount_delta(int pending, int item) {
const int base = OOT_CHEAT_ITEM_ACTION + item * 256;
return pending >= base && pending < base + 128 ? pending - base - 64 : 0;
}
/// Whether the waiting action is item's unlock or relock
inline bool toggles(int pending, int item) { return pending == OOT_CHEAT_ITEM_ACTION + item * 256 + 200; }
/// The action after a step (dir 1 or -1) on item's amount, count what the game last said it has (0 or more): the item's
/// waiting change and the step together, kept to what the game allows (0 to 999) and to what one action says (-64 to 63).
/// Another item's waiting action is replaced, as before
inline int amount_step(int pending, int item, int count, int dir) {
const int delta = std::clamp(amount_delta(pending, item) + dir, std::max(-64, -count), std::min(63, 999 - count));
return OOT_CHEAT_ITEM_ACTION + item * 256 + 64 + delta;
}
/// The action after a press of Unlock or Relock on item: a second press before the game took the first takes it back (0)
inline int toggle(int pending, int item) { return toggles(pending, item) ? 0 : OOT_CHEAT_ITEM_ACTION + item * 256 + 200; }
/// What item's card shows: count, what the game last said, with the waiting change made (count < 0, unknown: as it is)
inline int shown(int pending, int item, int count) {
if (count < 0) return count;
if (toggles(pending, item)) return count > 0 ? 0 : 1;
return std::clamp(count + amount_delta(pending, item), 0, 999);
}
} // namespace mkwvk::debug_pending

41
gpu/vk/debug_tab_layout.h Normal file
View File

@@ -0,0 +1,41 @@
// (Triforce) The Debug tab's value cells: how wide they are, and whether the tools stand in two columns (vk_menu.cpp
// draw_debug_rows measures the texts and draws; the arithmetic is here so that tests/test_debug_tab_layout.py can run it).
//
// The tab has the Debug tools switch across its top, the tools under their groups' headings in one column or two, and the
// Developer rows across its foot. The cells of what stands under one another are one width, so their left edges line up:
// a column's tools alike, and the switch and the Developer rows, which span the columns, alike. A Developer row of more
// than two values asks more than Off and On: before, it alone was wider than the rows around it.
#pragma once
#include <algorithm>
namespace mkwvk::debug_tab {
struct Widths {
bool two = false; // (the tools in two columns)
float cell[2] = {}; // (a column's tools' cells; both the same in one column)
float span = 0; // (the switch's and the Developer rows' cells)
};
/// col_w: a column's width when there are two; room: the tab's width. name_need: the longest name of each column with its
/// room in the name's cell; cell_full and cell_tight: what the column's widest cell asks, and its narrowest. span_need: what
/// the widest of the switch's and the Developer rows' cells asks.
/// Two columns only when each holds its longest name whole beside its cells at their narrowest; the cells are then as wide
/// as the name leaves them, up to what they ask, and the spanning cells as wide as the second column's (they end at the
/// same edge) or as they ask. In one column every cell is one width: what the widest of all asks, less only when the
/// longest name needs the room, and never under the narrowest
inline Widths widths(float col_w, float room, const float name_need[2], const float cell_full[2], const float cell_tight[2], float span_need) {
Widths w;
w.two = col_w >= name_need[0] + cell_tight[0] && col_w >= name_need[1] + cell_tight[1];
if (w.two) {
for (int k = 0; k < 2; k++) w.cell[k] = std::max(cell_tight[k], std::min(cell_full[k], col_w - name_need[k]));
w.span = std::max(span_need, w.cell[1]);
} else {
const float widest = std::max({cell_full[0], cell_full[1], span_need});
const float narrowest = std::max(cell_tight[0], cell_tight[1]);
w.cell[0] = w.cell[1] = w.span = std::max(std::min(widest, room - std::max(name_need[0], name_need[1])), narrowest);
}
return w;
}
} // namespace mkwvk::debug_tab

36
gpu/vk/demote_watch.h Normal file
View File

@@ -0,0 +1,36 @@
/* (Triforce) When a pool's trial in guest memory ends early (vk_memory.cpp demote_trials). A pool the CPU and the GPU both write
is tried out of the mirrors: its pieces (or the whole pool) are bound in guest memory, 10 frames to settle, then up to 90
measured, and the GPU's time a frame is compared with the window before. Only the measured frames were judged, and only after
ten of them: in the owner's log (RTX 5090, 2026-11-15, test4) a 16 MB pool took the GPU 106 ms a frame in guest memory
against 7.4 before, and the game's thread waited for that GPU in every capture (memory sync 99 ms a submit, 18 ms a frame
over 120 presents): some twenty frames of 100 ms, the same again at its second trial, and a 32 MB pool's two trials at 39 ms
against 7.7. A fresh install tries every such pool (a pool that failed twice is kept in triforce_mirror_keep.txt for later
runs), which is what a benchmark plays. So a trial far over its baseline ends as soon as a few frames say so, settling or
measured, the way a slower one ends: mirrored again and measured once more that way, and judged against that (phase 4), so
an early end that was wrong costs a second measure, not the pool. Of the 84 trials in the owner's five logs the four costly
ones were 4.7 to 19 times their baseline; every other was under 2.3 times but one, 3.6 times, whose mirrored-again frames were
as slow (its baseline was off; it stayed in guest memory): under this rule's line too. RX_VK_DEMOTE_QUICK=0: as before. Plain
functions: tests/test_demote_watch.cpp checks them. */
#pragma once
#include <cstdint>
#include <cstdlib>
namespace mkwvk::demote_watch {
constexpr uint64_t kFrames = 3; // (the frames since the trial's start, or its measure's, before one is judged: the GPU's time comes
// in as submissions finish, a frame or two late, so the first frames weigh less than they cost)
constexpr double kTimes = 3.0; // (over kTimes the baseline and kOverMs more: see above. The trial's own rule, twice the
constexpr double kOverMs = 5.0; // baseline and 1 ms after 10 measured frames, stays as it was)
/// Whether a trial whose frames so far took ms_a_frame of GPU time each, over frames of them, ends now: far slower than base_ms
inline bool gross(double base_ms, double ms_a_frame, uint64_t frames) {
return base_ms > 0 && frames >= kFrames && ms_a_frame > base_ms * kTimes + kOverMs;
}
/// RX_VK_DEMOTE_QUICK: on unless "0"
inline bool quick_on(const char* env) {
return !(env && *env && std::atoi(env) == 0 && env[0] == '0');
}
} // namespace mkwvk::demote_watch

113
gpu/vk/desc_keep.h Normal file
View File

@@ -0,0 +1,113 @@
// (Triforce) Per-binding descriptor reuse (vk_draw.cpp build_descriptors), apart from Vulkan so that tests/test_desc_keep.py can run
// it. NX2navi's per-binding reuse (runtime/vk_exec_draw.c pb_take and pb_keep, chrg6's) is the model; the rules below are this
// renderer's own, from what each of its descriptors is resolved from.
//
// A draw whose program, bindings, texture slots and uniform updates are all as the draw before it keeps that draw's descriptors whole
// (vk_draw.cpp same_desc). In the owner's test7 session that was 0 to 349 draws a frame of 2,300 to 2,870, and "descriptors" was 1.5
// to 2.2 ms of a replay of 7.4 to 9.8 ms: a draw that binds one uniform buffer of its own, or anything bound anywhere, built every
// descriptor again, each texture through the views' map and the sampler table, each buffer through its pool. Now each descriptor of
// a graphics program keeps its last result beside what it was resolved from, for the last kSlots programs a recording thread drew,
// and a descriptor whose inputs and stamps are all the same takes that result instead of resolving again. What a result stands on,
// by its kind (stamps_of: the stamps are read before the descriptor is resolved, so that one moved by the resolution itself, or by
// another thread meanwhile, makes the next one resolve again):
// - a sampled texture (TEX): its handle, view type and depth compare as inputs; the slot's version (the slot registered anew), the
// samplers' generation (a sampler registration's contents changed), the destroy generation (a texture destroyed or made again:
// a kept view would name freed memory, issue 1126), the dummy generation (a frame left a texture for later: a dummy is looked
// up again) and the texture epoch (a view's texture is touched once a submission: texture_view_of does it on its first look);
// - a user uniform buffer (UBO, c3 on): its address and size; the recording, the stream's inline uniform updates (ub_generation:
// an update into the range is a copy with it in) and the bind stamp (gpu_bind_stamp: a pool gone or imported, a mirror made, the
// next submission). Only a pool's or a mirror's range is kept: a copy into the recording's stream (an overlay, an unaligned range,
// a range read where it was bound) is made again each time, as it is today;
// - a stage's own constants (C1): the recording (they are uploaded once a recording, RecordingRes::c1_at);
// - a storage buffer the program only reads (SSBO, a single one, not a table): its address and size; the recording and the bind
// stamp. A pool's or a mirror's range only, as for UBO.
// The rest is resolved every time as before: the driver's banks (c0, c2: kept by their stages' generations already), storage buffers
// the program writes (each binding stamps its pages for the write-back), storage buffer tables, texel buffers and storage images.
// A failed resolution (no sampler, no buffer) is never kept: the next draw tries again.
#pragma once
#include <array>
#include <cstddef>
#include <cstdint>
#include <vector>
namespace desc_keep {
enum Kind : uint8_t { kNone = 0, kTex, kUbo, kC1, kSsbo };
/// The stamps results stand on, read where a descriptor is about to be resolved (only those of its kind are read: stamps_of)
struct Stamps {
uint64_t rec = 0; // the recording (vk_draw.cpp desc_rec: moved at each recording's begin)
uint64_t bind = 0; // gpu_bind_stamp
uint64_t ub = 0; // ub_generation
uint64_t samp = 0; // sampler_registrations_generation
uint64_t destroy = 0; // texture_destroy_generation
uint64_t dummy = 0; // texture_dummy_generation
uint64_t epoch = 0; // texture_epoch
};
/// The stamps a result of that kind stands on, in a fixed order (unused ones 0)
inline std::array<uint64_t, 4> stamps_of(Kind k, const Stamps& s) {
switch (k) {
case kTex: return {s.samp, s.destroy, s.dummy, s.epoch};
case kUbo: return {s.rec, s.bind, s.ub, 0};
case kC1: return {s.rec, 0, 0, 0};
case kSsbo: return {s.rec, s.bind, 0, 0};
default: return {~0ull, ~0ull, ~0ull, ~0ull};
}
}
/// One descriptor's last resolution: what it was resolved from, and its result (a buffer: handle, offset, range; a texture:
/// sampler, view as texture_view_of gave it (null: the dummy is drawn), the Texture). flags: a TEX's slot was a registered one
struct Elem {
uint8_t kind = kNone;
uint8_t flags = 0;
uint32_t version = 0; // (TEX: the slot's version)
uint64_t in[2]{};
std::array<uint64_t, 4> at{};
uint64_t out[3]{};
};
/// Whether e's result serves a descriptor of kind k with those inputs (version: a TEX's slot version, 0 for the rest), now
inline bool take(const Elem& e, Kind k, uint64_t in0, uint64_t in1, uint32_t version, const Stamps& now) {
return k != kNone && e.kind == k && e.in[0] == in0 && e.in[1] == in1 && e.version == version && e.at == stamps_of(k, now);
}
/// A descriptor resolved: kept with its inputs and the stamps read before it was resolved (before)
inline void keep(Elem& e, Kind k, uint64_t in0, uint64_t in1, uint32_t version, const Stamps& before, uint64_t out0,
uint64_t out1, uint64_t out2, uint8_t flags = 0) {
e.kind = k, e.flags = flags, e.version = version;
e.in[0] = in0, e.in[1] = in1;
e.at = stamps_of(k, before);
e.out[0] = out0, e.out[1] = out1, e.out[2] = out2;
}
/// Not kept (a failed resolution, or a result that is not to be taken again)
inline void drop(Elem& e) { e.kind = kNone; }
/// (RX_VK_DESC_KEEP=check) Whether a descriptor resolved again came out as its kept result
inline bool same(const Elem& e, uint64_t out0, uint64_t out1, uint64_t out2, uint8_t flags = 0) {
return e.out[0] == out0 && e.out[1] == out1 && e.out[2] == out2 && e.flags == flags;
}
/// A recording thread's kept results: the last programs drawn, one a slot by the program's address
inline constexpr size_t kSlots = 16;
struct Prog {
const void* prog = nullptr;
std::vector<Elem> e; // (by descriptor, in build_descriptors' order)
};
inline size_t slot_of(const void* prog) {
const uintptr_t p = (uintptr_t)prog;
return (size_t)((p >> 6) ^ (p >> 13)) % kSlots;
}
/// prog's kept results, n descriptors: emptied when its slot held another program's, or another count of them
inline Prog& prog_of(std::array<Prog, kSlots>& t, const void* prog, size_t n) {
Prog& p = t[slot_of(prog)];
if (p.prog != prog || p.e.size() != n) {
p.prog = prog;
p.e.assign(n, Elem{});
}
return p;
}
} // namespace desc_keep

95
gpu/vk/desc_sig.h Normal file
View File

@@ -0,0 +1,95 @@
// (Triforce) The whole-draw skip by the signature of a draw's descriptor inputs (vk_draw.cpp desc_sig_walk and draw), apart from
// Vulkan so that tests/test_desc_sig.py can run it. NX2navi's bind_sig (runtime/vk_exec_draw.c) is the model; its code is not
// chrg6's, so this is written anew from what it does, on this renderer's own rules (desc_keep.h).
//
// A draw keeps the descriptors the draw before it pushed only while nothing at all was bound anew, no inline uniform update came and
// no texture slot changed (vk_draw.cpp same_desc): 381 of 2,830 draws a frame in the owner's test18. The others build every
// descriptor again, even when what they read is what the draw before them read: a uniform buffer bound for another stage, a texture
// bound at a slot the program does not sample. Through the per-binding reuse (desc_keep.h) each of their descriptors then takes the
// result it had, and the push finds nothing changed: the build and the compare were the work, 2.0 to 3.3 ms a frame.
//
// Now such a draw first reads its signature: what each of its descriptors would be resolved from (desc_keep.h's inputs: a texture's
// handle and its slot's version, a user uniform buffer's address and size, a read-only storage buffer's address and size words, a
// driver bank's stage generation), and the stamps desc_keep.h's rules give each kind it has (stamps_of), read before the inputs.
// When it equals the signature of the descriptors at the graphics bind point, which are this program's, the build and the push are
// left out: a build would take every one of them from what was kept (each input and stamp is the same) and push nothing.
// That holds only where every descriptor of the last build was kept, so the signature stands for
// - push programs only (a set's bind point state is not followed), not the mesh fallback, not RenoDX's;
// - programs whose every descriptor is of a kind the reuse keeps (shape_of): sampled textures, user uniform buffers (c3 to c18),
// a stage's constants (c1), single read-only storage buffers, and the driver's banks (c0, c2), which their own cache keeps by
// the stage's generations. Not storage buffers a program writes (each build stamps their pages for the write-back), tables,
// texel buffers or storage images;
// - builds that dropped nothing (armed): a failed resolution, a copy in the recording's stream (an inline update's overlay, an
// unaligned range) or an unbound buffer make the next draw build again.
// The textures a hit samples that the GPU wrote (pass hazards: gfx_sampled) are read again from the last build's textures, as a
// build would. A signature is the bind point's only once that draw's descriptors were bound: a draw left out between its build and
// its bind leaves the last one in place.
// RX_VK_DESC_SIG=0: every such draw builds, as before. =check: a hit builds too, and one whose descriptors or sampled textures come
// out otherwise than what is pushed is counted and logged ("vk: descriptor signature check").
#pragma once
#include <array>
#include <cstddef>
#include <cstdint>
#include <vector>
#include "desc_keep.h"
namespace desc_sig {
/// The kinds of descriptor a program has, one bit each by desc_keep::Kind (bit kTex, kUbo, kC1, kSsbo); kBank its driver banks
enum : uint8_t { kBank = 1u << 7 };
constexpr uint8_t bit(desc_keep::Kind k) { return (uint8_t)(1u << k); }
/// What a program's descriptors are, from its stages' lists (made once a program: they do not change once it is translated)
struct Shape {
const void* prog = nullptr;
bool ok = false; // (every descriptor of a kind the signature stands for)
uint8_t kinds = 0;
};
inline constexpr size_t kShapes = 64;
inline size_t shape_slot(const void* prog) {
const uintptr_t p = (uintptr_t)prog;
return (size_t)((p >> 6) ^ (p >> 14)) % kShapes;
}
/// The signature's head: the program, the recording (c1 and the banks live in its streams), and the stamps of each kind it has, in
/// desc_keep.h's order (the same stamps a kept result of that kind stands on)
inline void put_head(std::vector<uint64_t>& out, const void* prog, uint8_t kinds, const desc_keep::Stamps& now) {
out.push_back((uint64_t)(uintptr_t)prog);
out.push_back(now.rec);
out.push_back(kinds);
for (desc_keep::Kind k : {desc_keep::kTex, desc_keep::kUbo, desc_keep::kC1, desc_keep::kSsbo})
if (kinds & bit(k))
for (uint64_t v : desc_keep::stamps_of(k, now)) out.push_back(v);
}
/// A signature, and what its build left besides the descriptors: the textures it sampled through a view (draw_sampled keeps those
/// the GPU wrote) and the dummy counts it added (the frame statistics')
struct Sig {
const void* prog = nullptr;
bool armed = false; // (every descriptor of that build was kept: a draw with the same signature may take them all)
std::vector<uint64_t> w;
std::vector<const void*> tex;
int64_t dummies = 0, dummies_known = 0;
};
/// Whether a draw of prog whose signature is now may keep the descriptors bound, which bound's draw built
inline bool hit(const Sig& bound, const Sig& now, const void* prog) {
return bound.armed && bound.prog == prog && now.prog == prog && !now.w.empty() && bound.w == now.w;
}
/// now's draw built its descriptors and they were bound: its signature is the bind point's (armed: none of them dropped)
inline void commit(Sig& bound, Sig& now, bool armed) {
std::swap(bound.w, now.w);
std::swap(bound.tex, now.tex);
bound.prog = now.prog, bound.armed = armed;
bound.dummies = now.dummies, bound.dummies_known = now.dummies_known;
now.prog = nullptr, now.w.clear();
}
/// Descriptors bound that no signature stands for (a program the skip does not take, a build that dropped one)
inline void forget(Sig& bound) {
bound.prog = nullptr, bound.armed = false;
bound.w.clear();
}
} // namespace desc_sig

74
gpu/vk/dlssg_pace.h Normal file
View File

@@ -0,0 +1,74 @@
// (Triforce) Who holds the frame limit while NVIDIA DLSS frame generation runs: Reflex's sleep, where the game's frame
// starts, or the game's own wait (host/nn_vi.c hos_vi_native_acquire). After NX2navi's runtime/dlssg_pace.h, whose rule
// this is. A function of what the game's presenting thread sees and nothing else, so tests/test_dlssg_pace.py drives
// it without a GPU (plain C: the test compiles it as C and as C++).
//
// The period counts the frames the game draws (the Game frame rate cap's, the host's own). Reflex holds it only while
// every one of these is so, and the game's own wait holds the whole period otherwise: the caller allows it (the
// Reflex frame limit row on, frames being generated now, a Streamline whose Reflex sleeps here and refused no limit);
// there is a period (the caller passes 0 while no frame is generated: a menu, a load, a movie); a present went out,
// and within DLSSG_HOLD_IDLE_NS (a minimized window gives the driver no present to time a frame by: Reflex's sleep is
// left out then, vk_dlssg.cpp frame_start); the limit Reflex should hold is the one Streamline last took; and the
// frames have not shown that Reflex does not hold it (for the rest of the run then).
//
// The frames are checked over windows of DLSSG_PACE_WINDOW presents Reflex held to one period and one multiplier.
// Over half of them sooner than three quarters of the period: nothing holds the game, and the game's own wait takes
// the limit for the run. Over half of them about the multiplier times the period apart (within half a period): the
// driver counted the frames on screen, the generated ones too, so Reflex is given the period over the multiplier from
// then on; the same again (a GPU that only managed that rate) gives the limit to the game's own wait for the run. A
// frame a load or a hitch held counts as neither.
#pragma once
#include <stdint.h>
#define DLSSG_HOLD_IDLE_NS 200000000ull // no present for this long: Reflex's sleep is left out
#define DLSSG_PACE_WINDOW 60 // presents over which Reflex's hold of the limit is checked
enum { DLSSG_PACE_SAME, DLSSG_PACE_FAST, DLSSG_PACE_SHOWN, DLSSG_PACE_SLOW }; // what the last window found
typedef struct {
uint64_t last_ns; // the newest frame start Reflex held, 0 when the game's own wait held it
uint64_t last_period; // its period
int last_mult; // and its multiplier
int window, fast, slow; // frames counted in the window, those too soon, those about the multiplier periods apart
int shown; // Reflex counts the frames on screen: it is given the period over the multiplier
int bad; // Reflex did not hold the limit: the game's own wait holds it for the rest of the run
} DlssgPace;
// now: the clock where the game's frame starts. period_ns: the period of the cap in effect (0: no frame is generated
// now, or no limit). mult: frames on screen per drawn frame now. hosted_ns: the clock after the newest present, 0
// before the first. allowed: Reflex may hold the limit. applied_us: the limit Streamline last took. *want_us: the
// limit Reflex should hold, 0 none. *found: what a window that ended here found (DLSSG_PACE_*). Returns 1 when Reflex
// holds this frame's limit: the game's own wait steps aside, and only then.
static inline int dlssg_pace_step(DlssgPace *p, uint64_t now, uint64_t period_ns, int mult, uint64_t hosted_ns,
int allowed, uint32_t applied_us, uint32_t *want_us, int *found) {
*found = DLSSG_PACE_SAME;
if (mult < 1) mult = 1;
if (!p->bad && p->last_ns && now >= p->last_ns && p->last_period == period_ns && p->last_mult == mult) {
uint64_t d = now - p->last_ns, m = (uint64_t)mult;
if (d < period_ns / 4 * 3) p->fast++;
else if (m > 1 && d * 2 > period_ns * (2 * m - 1) && d * 2 < period_ns * (2 * m + 1)) p->slow++;
if (++p->window >= DLSSG_PACE_WINDOW) {
if (p->fast > DLSSG_PACE_WINDOW / 2) p->bad = 1, *found = DLSSG_PACE_FAST;
else if (p->slow > DLSSG_PACE_WINDOW / 2 && p->shown) p->bad = 1, *found = DLSSG_PACE_SLOW;
else if (p->slow > DLSSG_PACE_WINDOW / 2) p->shown = 1, *found = DLSSG_PACE_SHOWN;
p->window = p->fast = p->slow = 0;
}
} else {
p->window = p->fast = p->slow = 0; // a window counts only frames Reflex held to one period and multiplier
}
uint64_t us = 0;
// a present stamped after now went out while this thread read the clock: as recent as one can be
int live = hosted_ns && (hosted_ns >= now || now - hosted_ns < DLSSG_HOLD_IDLE_NS);
if (allowed && period_ns && !p->bad && live) {
uint64_t per = p->shown ? period_ns / (uint64_t)mult : period_ns;
us = (per + 500) / 1000;
if (!us) us = 1;
if (us > 0x7fffffffu) us = 0; // no limit Reflex could be given: the game's own wait keeps it
}
*want_us = (uint32_t)us;
int held = us && applied_us == (uint32_t)us;
p->last_ns = held ? now : 0;
p->last_period = held ? period_ns : 0;
p->last_mult = mult;
return held;
}

42
gpu/vk/file_whole.h Normal file
View File

@@ -0,0 +1,42 @@
// (Triforce, handoff §8) A whole file at once, for file_writers.h: written as <path>.part and put in the old one's place, so
// that a write cut short (the process ended in it, a full disk) leaves the old file or the new one, never part of one
#pragma once
#include "file_writers.h"
#include <cstdio>
#include <string>
#ifdef _WIN32
#ifndef NOMINMAX
#define NOMINMAX
#endif
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN
#endif
#include <windows.h>
#endif
namespace file_writers {
/// rename over an existing file, as each platform does it
inline bool replace_file(const std::string& from, const std::string& to) {
#ifdef _WIN32
return MoveFileExA(from.c_str(), to.c_str(), MOVEFILE_REPLACE_EXISTING) != 0;
#else
return std::rename(from.c_str(), to.c_str()) == 0;
#endif
}
/// The whole file: <path>.part written and closed, then put in path's place; false (and no .part left) when any of it fails,
/// and then path is as it was. Inside a Writing of its own (refused once the end has begun)
inline bool write_whole(const std::string& path, const void* data, size_t n) {
Writing w;
if (!w) return false;
const std::string part = path + ".part";
FILE* f = std::fopen(part.c_str(), "wb");
if (!f) return false;
bool ok = !n || std::fwrite(data, 1, n, f) == n;
ok = std::fclose(f) == 0 && ok;
ok = ok && replace_file(part, path);
if (!ok) std::remove(part.c_str());
return ok;
}
inline bool write_whole(const std::string& path, const std::string& text) { return write_whole(path, text.data(), text.size()); }
} // namespace file_writers

96
gpu/vk/file_writers.h Normal file
View File

@@ -0,0 +1,96 @@
// (Triforce, handoff §8) The renderer's file writes made on threads of their own, and the process's end. The window's close ends
// the game through ExitProcess (host/hos_relaunch.c), which stops every other thread wherever it is: a thread in the middle of
// a file (a bug capture's picture, the pipeline cache's save in play, the order of the pipelines, a hang report, the prepared
// record, the predicted pipelines' list, a debug picture or the frame log) left part of it. Now:
// Writing one write, from its open to its close or rename, as a scope: counted while it runs, and refused once the end has
// begun (on any thread but the one ending the process: its own last saves go on)
// stop the process is about to end (HOS_OPT_EXITING, vk_replay.cpp option_set): no write begins from here on, and the
// ones under way are waited for, up to a bound (2 s; RX_VK_WRITERS_WAIT_MS=<ms>, 0: no wait, as before). A write
// that is still going then is cut by the exit as before; each such file is one write_whole makes, or one
// whose readers take its whole records only, so it is the old file or the new one, or a list one record short
// write_whole (file_whole.h) a whole file written as <path>.part and put in the old one's place: the old file or the new
// one, never part of one, whenever the process ends
// The threads themselves are not joined: they are detached and keep no state that outlives the process; what is waited for
// is their write, the only part of them an end can damage. The standard library alone (file_whole.h has the
// Windows part): tests/test_file_writers.py compiles it alone
#pragma once
#include <algorithm>
#include <atomic>
#include <chrono>
#include <cstdio>
#include <cstdlib>
#include <string>
#include <thread>
namespace file_writers {
inline std::atomic<bool> g_stopping{false};
inline std::atomic<int> g_busy{0}; // (writes under way on other threads than the ending one)
inline std::atomic<int> g_refused{0}; // (writes not begun because the end had)
inline thread_local bool t_ending = false; // (the thread that called stop: its writes are the exit's own)
inline thread_local int t_inside = 0; // (Writings this thread holds: one inside another is part of it, never refused)
/// One write: `file_writers::Writing w; if (!w) return;` before the file is opened, and the scope ends after its close (and
/// rename). The busy count goes up before the stop flag is read and stop sets the flag before it reads the count (both
/// sequentially consistent): a write either sees the stop and does not begin, or stop sees it and waits for it. One begun
/// inside another on the same thread (a save's own small files) goes on: the outer one is waited for whole
class Writing {
public:
Writing() {
if (t_ending || t_inside > 0) {
on_ = true;
t_inside++, nested_ = true;
return;
}
g_busy.fetch_add(1);
if (g_stopping.load()) {
g_busy.fetch_sub(1);
g_refused.fetch_add(1, std::memory_order_relaxed);
return;
}
on_ = counted_ = true;
t_inside++;
}
~Writing() {
if (counted_ || nested_) t_inside--;
if (counted_) g_busy.fetch_sub(1);
}
Writing(const Writing&) = delete;
Writing& operator=(const Writing&) = delete;
explicit operator bool() const { return on_; }
private:
bool on_ = false, counted_ = false, nested_ = false;
};
/// The end has begun: a loop that writes ends at its next turn
inline bool stopping() { return g_stopping.load(std::memory_order_relaxed); }
/// The bound of stop's wait: 2 s, or RX_VK_WRITERS_WAIT_MS (0: no wait at all, no write refused: as before this)
inline int exit_wait_ms() {
const char* e = getenv("RX_VK_WRITERS_WAIT_MS");
return e && *e ? std::max(0, atoi(e)) : 2000;
}
struct Stopped {
int left = 0; // (writes still under way when the wait ended: cut by the exit)
double waited_ms = 0; // (how long the wait took)
int refused = 0; // (writes not begun since the end began, so far)
};
/// The process is about to end, on this thread: no write begins on another thread from here on, and the ones under way are
/// waited for, up to ms. ms 0: nothing is stopped (the old behaviour)
inline Stopped stop(int ms) {
Stopped s;
if (ms <= 0) return s;
t_ending = true;
g_stopping.store(true);
const auto t0 = std::chrono::steady_clock::now();
const auto until = t0 + std::chrono::milliseconds(ms);
while (g_busy.load() > 0 && std::chrono::steady_clock::now() < until) std::this_thread::sleep_for(std::chrono::milliseconds(1));
s.left = g_busy.load();
s.waited_ms = std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - t0).count();
s.refused = g_refused.load();
return s;
}
} // namespace file_writers

59
gpu/vk/foliage_shadow.h Normal file
View File

@@ -0,0 +1,59 @@
// (Triforce) Foliage in the sun's far shadow cascades drawn with one face culled (vk_draw.cpp draw_cull, for dynamic_state;
// RX_VK_FOLIAGE_SHADOW_CULL, off unless set), apart from Vulkan so that tests/test_foliage_shadow.py can run its rule.
//
// The game draws its foliage cards (grass, ferns, leaves) double-sided with mesh programs, into the shadow cascades too: the
// sun's three cascades are the layers of one depth array, 1600 x 1600 in the owner's logs. Seen from the sun the stacked,
// alpha-tested cards cover the same texels many times over: NX2navi counted about 1,000 million fragments in one such draw of
// the far cascade. With the switch, a draw of a mesh program (native, or through the compute fallback) into a cascade from the
// given layer on, that culls neither face, is drawn with its back faces culled (its front ones with an "f"): half the cards'
// fragments there. Cull mode is dynamic state, so no other pipeline is made. A cascade here is Triforce's own test for one,
// as for the cascades' depth bias in dynamic_state: depth only, square, wider than 768 (the characters' own shadow maps, 768
// and less, are never touched).
//
// The behaviour of NX2navi's NX2NAVI_FOLIAGE_SHADOW_CULL (epicmeme546's, on there from layer 1), written anew. It is not the same
// picture: a card that faces away from the sun no longer casts its shadow in those cascades. Where cards stand in crossed pairs
// or clumps others cover the same texels, but a lone card's shadow can go, so it is off here: a switch to compare, at
// RX_VK_FOLIAGE_SHADOW_CULL=1 as NX2navi has it.
//
// Not done: NX2navi's second half, NX2NAVI_FOLIAGE_SHADOW_VRS (one fragment shaded per 2 x 2 pixels in those cascades, so the
// leaves' alpha test runs a quarter as often). It needs VK_KHR_fragment_shading_rate enabled on the device, the shading rate as
// dynamic state in every mesh pipeline and its libraries (a change to every mesh pipeline's creation, which the driver's cache
// then misses) and vkCmdSetFragmentShadingRateKHR, which takes pointers, carried by the recorder (vk_recorder.h, vk_work.h);
// none of that can be checked without the game, and its picture is not the same either (a coarser cut-out of the leaves).
#pragma once
#include <cstdint>
#include <cstdlib>
#include <cstring>
namespace mkwvk::foliage_shadow {
constexpr int kNone = 0, kFront = 1, kBack = 2; // (NVN's faces: NVN_FACE_NONE, _FRONT, _BACK)
constexpr uint32_t kCascadeMin = 769; // (a cascade is wider than 768: vk_draw.cpp dynamic_state's test)
struct Rule {
int from = -1; // (the first cascade layer it applies to; -1: off)
bool front = false; // (the front faces culled, not the back ones)
};
/// RX_VK_FOLIAGE_SHADOW_CULL=<layer>[f]: from that layer of the cascades on (the game has three, 0 to 2; 1 is NX2navi's), the back
/// faces culled, the front ones with an "f" after the number. Unset, empty or not starting with a digit: off
inline Rule parse(const char* e) {
Rule r;
if (!e || *e < '0' || *e > '9') return r;
r.from = std::atoi(e);
r.front = std::strchr(e, 'f') != nullptr;
return r;
}
/// The face a draw culls: the game's (cull, NVN's face), or under the rule the one it names, for a draw of a mesh program into a
/// cascade (no colour target, a depth target width x height, square and wider than 768, the depth view's first layer at
/// rule.from or past it) that culls neither face
inline int cull(const Rule& rule, int game, bool mesh_program, int colour_targets, bool depth, uint32_t width, uint32_t height,
uint32_t layer) {
if (rule.from < 0 || game != kNone || !mesh_program || colour_targets != 0 || !depth) return game;
if (width != height || width < kCascadeMin || layer < (uint32_t)rule.from) return game;
return rule.front ? kFront : kBack;
}
} // namespace mkwvk::foliage_shadow

23
gpu/vk/fps_cap.h Normal file
View File

@@ -0,0 +1,23 @@
/* The Game frame rate cap row's "Monitor's rate" (vk_menu.cpp), as plain code: the cap a display's refresh rate gives.
host/nn_vi.c fps_max asks the renderer for it at the start (HOS_OPT_DISPLAY_CAP, vk_replay.cpp's option), and the row
says the same number beside its name. After NX2navi's fps_limit = display. No Vulkan, no menu:
tests/test_fps_cap_monitor.py checks it. */
#pragma once
namespace mkwvk::fps_cap {
/// The row's value for Monitor's rate (frame_rate_cap= in the options file): after 60, 90, 120, 144, 165 and 240 (0 to
/// 5); 45, 40 and 30 are -1 to -3
constexpr int kMonitor = 6;
/// The caps the game's pacing and its time step take (host/nn_vi.c): a display under 30 Hz or over 240 gets the nearest
constexpr int kLowest = 30, kHighest = 240;
/// The cap for a display's refresh rate in mHz (59,940 is 60, 143,856 is 144): its rate rounded to whole frames a
/// second, kLowest to kHighest. 0 when the rate is not known (mhz 0 or less): the host then keeps the console's 60.
inline int of_display(int mhz) {
if (mhz <= 0) return 0;
const int hz = (int)(((long long)mhz + 500) / 1000);
return hz < kLowest ? kLowest : hz > kHighest ? kHighest : hz;
}
} // namespace mkwvk::fps_cap

46
gpu/vk/frame_trace.h Normal file
View File

@@ -0,0 +1,46 @@
// (Triforce, wp20-onepct) The per-frame trace's split of a frame (RX_VK_FRAME_TRACE=1: triforce_frames.csv beside the game, one
// line a frame, vk_replay.cpp status_tick). Its first columns say what the replay did (the draws' parts, sampled and summed over
// every thread that records: the parallel replay's helpers too); they could not say what made a slow frame that was not the
// draws': the lab's Market frames over the 99th percentile were 7-21% explained by them (r37, r38, r41). These columns, after
// the first ones, split every frame the way the stall line splits a frame of 50 ms or more (stall_watch.h): the replay's own
// time (its submits and the present phase, on every thread that replays: on the game's threads with the synchronous replay),
// the present's own, the dispatches and the queue submits within the replay, the replay thread's waits for its helpers'
// segments, and what is left: outside the replay and the present, the game's own update. Of that, the game's threads spend
// some in our work: their fence waits and the captures at their submits; and the waits for write-backs still on the GPU (the
// captures' late pages, the reads and bindings put off), with the part of them after the writer's recording was done on the
// GPU (the submissions' timeline held back by work stopped at an event wait: vk_replay.cpp vk_timeline_wait), and the queue
// submits made while a background import ran. Plain functions: tests/test_frame_trace.py
#pragma once
#include <cstddef>
#include <cstdio>
namespace mkwvk::frame_trace {
/// One frame's split, in milliseconds
struct Split {
double interval = 0; // (present to present)
double replay = 0; // (the submits and the present phase, every thread that replays: the stall line's "replay")
double present = 0; // (the present's own: the swapchain's image, its fence, the present call)
double dispatch = 0, qsubmit = 0; // (in replay: the dispatches, the queue submits)
double par_wait = 0; // (in replay: the replay thread's waits for the helper threads' segments)
double game_wait = 0; // (the game's fence waits, every thread)
double captures = 0; // (the submit-time captures on the game's threads)
double wb_wait = 0; // (the waits for write-backs still on the GPU, wherever they were: memory sync's capture, a read)
double wb_held = 0; // (of them, the part after the writer's recording was done: the timeline held, not the GPU)
double qs_bg = 0; // (the queue submits made while a background import ran)
};
/// Outside the replay and the present: the game's own update, its waits for the replay and the GPU among it. As the stall line's
inline double outside(const Split& s) { return s.interval - s.replay - s.present; }
/// The columns, after the trace's first ones (their names; the line's values in this order: line)
constexpr const char* kColumns =
"replay_ms,present_ms,dispatch_ms,qsubmit_ms,par_wait_ms,outside_ms,game_wait_ms,captures_ms,wb_wait_ms,wb_held_ms,qs_bg_ms";
/// The values for kColumns, comma-separated, no line end: snprintf's count
inline int line(const Split& s, char* out, size_t cap) {
return std::snprintf(out, cap, "%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f", s.replay, s.present, s.dispatch,
s.qsubmit, s.par_wait, outside(s), s.game_wait, s.captures, s.wb_wait, s.wb_held, s.qs_bg);
}
} // namespace mkwvk::frame_trace

139
gpu/vk/framegen_rows.h Normal file
View File

@@ -0,0 +1,139 @@
/* (Triforce) The Insert menu's Frame generation row (vk_menu.cpp, opts.framegen) with its last choice, AMD FSR 4, and what
AMD FSR 4's frame generation (vk_fsr4.cpp, fsr4fg) decides without a GPU: where the choice is offered, which of the
versions AMD's library lists is taken, when a frame is a camera cut, when a context still fits and when it is let go,
and the order it is taken down in. Plain functions, no Vulkan, no menu: tests/test_framegen_fsr4.py checks them. */
#pragma once
#include <cmath>
#include <cstdint>
#include <cstdio>
namespace mkwvk::framegen_rows {
/// opts.framegen (saved as frame_generation): 0 Off, 1 Auto, 2 to 8 the multiplier, as before (an options file from before
/// keeps its meaning), then 9: AMD FSR 4, one frame made by AMD's library between each two of the game's
enum { OFF = 0, AUTO = 1, kMaxMult = 8, FSR4 = 9, kRowMax = FSR4 };
/// Why AMD FSR 4 cannot make the frames here (the row says it); FG_OK when it can. FG_NOT_YET: not asked, the device is not
/// made yet
enum Why { FG_OK, FG_NOT_BUILT, FG_ENV_OFF, FG_NOT_YET, FG_NO_LIBRARY, FG_NO_FILE, FG_NO_MODEL, FG_FAILED, FG_WHY_COUNT };
/// The row's last value on this PC: AMD FSR 4 where it is offered, 8x elsewhere (the row as it was)
constexpr int row_hi(bool fsr4_offered) { return fsr4_offered ? (int)FSR4 : (int)kMaxMult; }
/// What the row shows, and what runs, for a value: itself, or 2x for a saved AMD FSR 4 on a PC where it is not offered
/// (one made frame a frame either way, there by the renderer's own interpolation). The saved value stays for a PC where it is
constexpr int shown(int v, bool fsr4_offered) { return v == FSR4 && !fsr4_offered ? 2 : v; }
/// The frames shown for each of the game's: 1 at Off, auto_most with Auto (the display's refresh over 30 fps), the
/// multiplier, and 2 with AMD FSR 4 (its library makes the one frame between two)
constexpr int multiplier(int v, int auto_most) {
return v == FSR4 ? 2 : v == AUTO ? auto_most : v < 1 ? 1 : v > kMaxMult ? (int)kMaxMult : v;
}
/// Whether AMD's library is asked for the made frame: the row at AMD FSR 4, where it can, and not beside NVIDIA DLSS frame
/// generation, whose frames go in at the present itself (dlssg: vk_dlssg.cpp says it runs in this session)
constexpr bool fsr4_asked(int v, Why why, bool dlssg) { return v == FSR4 && why == FG_OK && !dlssg; }
/// A version as AMD's library names it ("4.0.2"; one the graphics driver brings carries a star, "4.0.2*"): false when the
/// name does not start with three numbers
inline bool parse_version(const char* name, unsigned v[3]) {
v[0] = v[1] = v[2] = 0;
return name && std::sscanf(name, "%u.%u.%u", &v[0], &v[1], &v[2]) == 3;
}
/// Of the frame generation versions AMD's library lists for this GPU, the one contexts are made with: the newest FSR 4, the
/// first of equal ones; -1 when it lists none. An FSR 3.1 frame generation alone is not FSR 4: it never stands in under
/// FSR 4's name, as vk_fsr4.cpp's upscaler takes no FSR 3.1 either. any (RX_VK_FSR4_FG=1, for trying the path on a card
/// without FSR 4): the newest of whatever is listed
inline int pick_version(const char* const* names, int n, bool any) {
int best = -1;
unsigned bv[3] = {0, 0, 0};
for (int i = 0; i < n; i++) {
unsigned v[3];
if (!parse_version(names[i], v) || (!any && v[0] != 4)) continue;
const bool newer = v[0] != bv[0] ? v[0] > bv[0] : v[1] != bv[1] ? v[1] > bv[1] : v[2] > bv[2];
if (best < 0 || newer) best = i, bv[0] = v[0], bv[1] = v[1], bv[2] = v[2];
}
return best;
}
/// The camera in the world, from the view matrix's three rows as the game has them (row major, view = R * world + t: the
/// rows are its right, up and back axes, the last column the translation). The position is -R^T t
struct Pose {
float pos[3]{}, right[3]{}, up[3]{}, fwd[3]{};
};
inline Pose pose_of(const float v[12]) {
Pose p;
for (int i = 0; i < 3; i++) {
p.right[i] = v[i], p.up[i] = v[4 + i], p.fwd[i] = -v[8 + i];
p.pos[i] = -(v[i] * v[3] + v[4 + i] * v[7] + v[8 + i] * v[11]);
}
return p;
}
/// A camera cut between two frames frame_ms apart: the camera moved over 5 units, or turned over 10 degrees while it
/// moved over half a unit, for each 60th of a second (a slower frame may move in proportion, up to four times; the game's
/// world is in metres, its near plane at 0.1). A made frame between the two pictures of a cut would blend two views that
/// have nothing to do with each other: such a frame starts a new history and none is made. The thresholds are NX2navi's,
/// from its camera traces of the same game (a median of 0.05 m and 0.08 degrees a frame in play, area loads of 5 to
/// 180 m in one frame)
inline bool camera_cut(const Pose& before, const Pose& now, float frame_ms) {
const float scale = !(frame_ms > 0.0f) ? 1.0f : frame_ms < 16.667f ? 1.0f : frame_ms > 66.667f ? 4.0f : frame_ms / 16.667f;
float move2 = 0.0f, dot = 0.0f, la = 0.0f, lb = 0.0f;
for (int i = 0; i < 3; i++) {
const float d = now.pos[i] - before.pos[i];
move2 += d * d, dot += before.fwd[i] * now.fwd[i], la += before.fwd[i] * before.fwd[i], lb += now.fwd[i] * now.fwd[i];
}
if (!std::isfinite(move2) || !std::isfinite(dot) || !(la > 0.0f) || !(lb > 0.0f)) return true; // (no camera to compare)
const float move = std::sqrt(move2);
const float cosine = dot / std::sqrt(la * lb);
const float turn = std::acos(cosine < -1.0f ? -1.0f : cosine > 1.0f ? 1.0f : cosine); // (radians)
constexpr float kTenDegrees = 0.17453293f;
return move > 5.0f * scale || (turn > kTenDegrees * scale && move > 0.5f * scale);
}
/// What a present with the row at AMD FSR 4 shows between the game's last frame and this one
enum Made {
MADE, ///< AMD's library made the frame
OWN, ///< it did not take this present (no scene's depth and motion: a menu, a movie; not available; failed): the
///< renderer's own interpolation makes the frame, as at 2x
SKIP ///< a new history (the first frame, a gap, a camera cut): AMD's library was given the frame and no frame is made
};
/// A frame generation context as it was made: the presented picture's size, the largest scene it takes, and whether the
/// scene's far plane is at infinity (a context is made for one kind of depth)
struct ContextKey {
uint32_t width = 0, height = 0, max_render_width = 0, max_render_height = 0;
bool infinite_depth = false;
};
/// Whether a context made as c takes a frame of w x h with a scene of rw x rh: else it is made again
constexpr bool context_fits(const ContextKey& c, uint32_t w, uint32_t h, uint32_t rw, uint32_t rh, bool infinite_depth) {
return c.width == w && c.height == h && rw <= c.max_render_width && rh <= c.max_render_height &&
c.infinite_depth == infinite_depth;
}
/// The context for such a frame: the scene is never larger than the picture in play, and a larger one still gets a context
constexpr ContextKey context_for(uint32_t w, uint32_t h, uint32_t rw, uint32_t rh, bool infinite_depth) {
return ContextKey{w, h, rw > w ? rw : w, rh > h ? rh : h, infinite_depth};
}
/// The presents a context is kept after its last frame: 120 once the row has left AMD FSR 4 (stepping through the row's
/// choices passes it: nothing is made and let go again within two seconds), 2 after a failure. Never at the present of
/// its last frame: that frame's commands are recorded and not yet on the queue
constexpr uint64_t kIdlePresents = 120, kFailedPresents = 2;
constexpr bool release_due(bool have, bool failed, uint64_t presents, uint64_t used_at) {
return have && presents >= used_at && presents - used_at >= (failed ? kFailedPresents : kIdlePresents);
}
/// Taking AMD's frame generation down, in the order that is safe, as one function the real code (vk_fsr4.cpp) and
/// tests/test_framegen_fsr4.py both run. ops.stop: no call into AMD's library starts after it (the lock a call holds is
/// taken, so one in progress has returned). At the process's exit (HOS_OPT_EXITING, host/hos_relaunch.c) that is all:
/// there is no swapchain or thread of AMD's to stop (the renderer presents the made frames itself), and what the context
/// holds goes with the process, as the upscaler's does; AMD's library is not called again, so nothing of it runs under
/// ExitProcess. In play (the row left AMD FSR 4, another size, a failure): the GPU is past every command buffer that
/// used the context (ops.wait_gpu), then AMD's context goes (ops.destroy_context) before the image it wrote into
/// (ops.destroy_images), never the other way round
template <class Ops> void teardown(Ops& ops, bool exiting) {
ops.stop();
if (exiting) return;
ops.wait_gpu();
ops.destroy_context();
ops.destroy_images();
}
} // namespace mkwvk::framegen_rows

132
gpu/vk/game_language.h Normal file
View File

@@ -0,0 +1,132 @@
/* The Insert menu's Game language row (vk_menu.cpp), as plain code: what language.txt in the game folder chooses and offers,
and the file's text with another language chosen. No Vulkan, no menu: tests/test_game_language.py checks it.
The game's language is the host's to read, at the start (host/nn_applet.c desired_language): HOS_LANGUAGE, else the
file's first line that is not blank or a comment and whose first word is one of the game's codes, in any case; English
(US) when there is none. Setup's update keeps a folder's choice by the same line, and takes it only when the line is the
code alone, spelled as the game spells it (nx2/make_game.py folder_language). The file lists the codes in its comments
("# pt-BR Portuguese (Brazil)"), as the host and Setup write it.
The row offers the languages the file itself names, as its choice or in that list, that the game has; it writes the
choice as the code alone on the choice's line, and no code the file does not name. */
#pragma once
#include <cstddef>
#include <string>
#include <string_view>
namespace mkwvk::game_language {
struct Language {
const char* code;
const char* name;
};
/// The game's languages: host/nn_applet.c's table and nx2/make_game.py's LANGUAGES, in their order (the test holds the
/// three together)
constexpr Language kLanguages[] = {
{"en-US", "English (US)"}, {"en-GB", "English (UK)"}, {"fr", "French (France)"}, {"fr-CA", "French (Canada)"},
{"de", "German"}, {"it", "Italian"}, {"es", "Spanish (Spain)"}, {"es-419", "Spanish (Latin America)"},
{"pt-BR", "Portuguese (Brazil)"}, {"nl", "Dutch"}, {"pl", "Polish"}, {"ru", "Russian"},
{"ja", "Japanese"}, {"ko", "Korean"}, {"zh-Hans", "Chinese (Simplified)"}, {"zh-Hant", "Chinese (Traditional)"},
};
constexpr int kCount = sizeof kLanguages / sizeof kLanguages[0];
constexpr int kDefault = 0; // (English (US): the host's answer when the file chooses nothing)
inline unsigned bit(int language) { return 1u << language; }
/// A code's place in kLanguages, in any case (the host compares so: "pt-br" is pt-BR); -1 when the game has no such code
inline int find(std::string_view code) {
for (int i = 0; i < kCount; i++) {
const std::string_view k = kLanguages[i].code;
if (k.size() != code.size()) continue;
bool same = true;
for (size_t c = 0; c < k.size() && same; c++) {
const auto lower = [](char x) { return x >= 'A' && x <= 'Z' ? (char)(x - 'A' + 'a') : x; };
same = lower(k[c]) == lower(code[c]);
}
if (same) return i;
}
return -1;
}
/// What a language.txt says
struct File {
int chosen = -1; ///< the language its choice line names (kLanguages' index); -1: it has no such line
unsigned offered = 0; ///< a bit a language the file names: its choice, and each "# <code> ..." line
size_t at = 0, len = 0; ///< the choice line's text to replace: after a byte order mark, without its line ending
bool crlf = false; ///< its lines end with CR LF
};
/// The first word of s (after spaces and tabs), up to a space, a tab or a line's end
inline std::string_view word(std::string_view s) {
size_t a = 0;
while (a < s.size() && (s[a] == ' ' || s[a] == '\t')) a++;
size_t b = a;
while (b < s.size() && s[b] != ' ' && s[b] != '\t' && s[b] != '\r' && s[b] != '\n') b++;
return s.substr(a, b - a);
}
/// The file as the host reads it, line by line: what a line longer than its 79 bytes holds past them is not a line's start,
/// a byte order mark may open any line, and the choice is the first line that is neither blank nor a comment and names a
/// language of the game (a line that names none is passed over, as there)
inline File read(std::string_view text) {
File f;
f.crlf = text.find("\r\n") != std::string_view::npos;
for (size_t start = 0; start < text.size();) {
size_t end = text.find('\n', start);
if (end == std::string_view::npos) end = text.size();
size_t stop = end; // (the line without its ending)
if (stop > start && text[stop - 1] == '\r') stop--;
std::string_view line = text.substr(start, stop - start);
size_t lead = 0; // (bytes of the line before what the host looks at)
if (line.size() >= 3 && (unsigned char)line[0] == 0xEF && (unsigned char)line[1] == 0xBB && (unsigned char)line[2] == 0xBF) lead = 3;
// (the host's buffer: 79 bytes of the line, cut at a zero byte. It keeps 31 bytes of the word, more than any code has: a
// longer word names no language there or here)
std::string_view seen = line.substr(0, 79);
if (const size_t zero = seen.find('\0'); zero != std::string_view::npos) seen = seen.substr(0, zero);
seen = seen.substr(lead < seen.size() ? lead : seen.size());
size_t blank = 0;
while (blank < seen.size() && (seen[blank] == ' ' || seen[blank] == '\t')) blank++;
const std::string_view rest = seen.substr(blank);
if (!rest.empty() && rest[0] == '#') { // (a comment: a language it lists, "# <code> <name>")
const int listed = find(word(rest.substr(1)));
if (listed >= 0) f.offered |= bit(listed);
} else if (!rest.empty() && f.chosen < 0) {
const int named = find(word(rest));
if (named >= 0) f.chosen = named, f.offered |= bit(named), f.at = start + lead, f.len = stop - (start + lead);
}
start = end + 1;
}
return f;
}
/// The file's text with `language` chosen: its choice line replaced by the code alone (as Setup's update reads it), every
/// other byte as it was; with no choice line, the code on a line of its own at the end. The text as it is when the file
/// does not name that language: never a code it does not list
inline std::string write(std::string_view text, int language) {
const File f = read(text);
if (language < 0 || language >= kCount || !(f.offered & bit(language))) return std::string(text);
const std::string_view code = kLanguages[language].code;
std::string out(text);
if (f.chosen >= 0) {
out.replace(f.at, f.len, code);
return out;
}
const char* const eol = f.crlf ? "\r\n" : "\n";
if (!out.empty() && out.back() != '\n') out += eol;
out += code;
out += eol;
return out;
}
/// The languages a file offers, in kLanguages' order, into out (kCount of room): how many. None unless the language in
/// effect (the file's choice, else the host's English (US)) is among them: the row then has nothing sound to show
inline int offered(const File& f, int* out) {
const int now = f.chosen >= 0 ? f.chosen : kDefault;
if (!(f.offered & bit(now))) return 0;
int n = 0;
for (int i = 0; i < kCount; i++)
if (f.offered & bit(i)) out[n++] = i;
return n;
}
} // namespace mkwvk::game_language

73
gpu/vk/gate_wait.h Normal file
View File

@@ -0,0 +1,73 @@
// (Triforce) A submission whose command buffer begins with a wait for an event the game signals from the CPU: the submit waits for
// the signal (RX_VK_GATE_WAIT_MS at most) before its capture and its replay, after NX2navi's gate_wait_ms (runtime/vk_exec.c
// vk_exec_gate_wait and gate_first_wait).
//
// OoT's queues order some of their work with events the CPU signals once it has written what the work reads. A submission that
// meets such a wait unmet is taken (its capture of the game's memory at the submit) and replayed up to the wait, where it stops; once
// the CPU signals, the order thread (vk_replay.cpp sync_watch, a millisecond at a time) or the next submit or present lets it go on,
// with a second capture (resume_queue, RX_VK_RESUME_CAPTURE). NX2navi found the signal 1 to 2 ms after the submit call in about
// every other frame in Hyrule Field, and made the submit call wait for it, before its copies, as the console's GPU waits at the
// wait itself: the capture is then taken after the game has written the frame's data, and the frame is replayed in the submit call
// and on the submitting thread (54.5 fps instead of 53 there, the longest frame 33 ms instead of 46). Triforce meets it far less
// often (the "captures at a CPU event's wait" of the order line: 100 to 170 in a lab run of 26,000 frames at the Market), and a gate
// that runs out holds the submitting thread for the whole bound, so it is off unless asked for, for the lab to measure.
// The gate waits only where the replay would stop: the first op of the submission's first command buffer that is not state the
// replay only stores (bound programs, buffers, textures, targets, viewports...) is a wait for an event, the wait is not met in the
// replay's order now, and the CPU has signalled that event before (vk_replay.cpp cpu_events). It takes no lock while it waits (the
// thread that will signal may need the order lock first), and looks at the event's word in the game's memory, which the CPU's signal
// writes. It ends at the signal, or at the bound, and the submission then goes as before. After kOuts waits that ran out, once they
// are more than a tenth of the waits, it is off for the run (the signal is then one that comes only after the submitting thread goes
// on, and every wait would be a stall).
// RX_VK_GATE_WAIT_MS unset or 0: off, as before; 1 to 100: the bound in milliseconds (NX2navi's default is 10)
// Apart from Vulkan and the stream's op numbers so that tests/test_gate_wait.py can run the rules and the gate over authored streams.
#pragma once
#include <cstdint>
#include <cstdlib>
#include <cstring>
#if defined(__x86_64__) || defined(_M_X64) || defined(__i386__) || defined(_M_IX86)
#include <immintrin.h>
#endif
namespace mkwvk::gate_wait {
constexpr int kMaxOps = 256; // (the ops looked at before the wait, at most: NX2navi's)
constexpr int64_t kOuts = 4; // (waits that ran out before the gate may go off for the run)
/// RX_VK_GATE_WAIT_MS: the bound in milliseconds, 0 (off) to 100; unset or not a number: 0
inline int ms(const char* env) {
if (!env || !*env) return 0;
const int v = std::atoi(env);
return v < 0 ? 0 : v > 100 ? 100 : v;
}
/// The command buffer's first op that is not state (records {uint32 op, uint32 size}, sizes 8-aligned, from p): the wait it begins
/// with, or null. kind(op): 1 state the replay only stores (looked past), 2 a wait for an event, 0 anything else (a draw, a copy, a
/// signal, a barrier, the end: the gate does not wait)
template <class K>
const uint8_t* first_wait(const uint8_t* p, K kind) {
for (int k = 0; k < kMaxOps; k++) {
uint32_t h[2];
std::memcpy(h, p, sizeof h);
if (h[1] < 8 || (h[1] & 7)) return nullptr;
const int what = kind(h[0]);
if (what == 2) return p;
if (what != 1) return nullptr;
p += h[1];
}
return nullptr;
}
/// Whether the gate goes off for the run, after `gates` waits of which `outs` ran out
inline bool off_after(int64_t gates, int64_t outs) {
return outs >= kOuts && outs * 10 > gates;
}
/// A pause of the host core while it looks (the waiting thread holds no virtual core: the signaller runs)
inline void pause() {
#if defined(__x86_64__) || defined(_M_X64) || defined(__i386__) || defined(_M_IX86)
_mm_pause();
#endif
}
} // namespace mkwvk::gate_wait

242
gpu/vk/gpl_rules.h Normal file
View File

@@ -0,0 +1,242 @@
// (Triforce) The rules of the fast pipeline link (VK_EXT_graphics_pipeline_library; vk_draw.cpp graphics_pipeline,
// warm_libraries, prepare_run), apart from Vulkan so that tests/test_gpl_rules.py runs them. Off unless the hidden Debug
// tab's Fast pipeline link row or RX_VK_GPL=1 switches it on: nobody has run the game with it.
//
// What "the driver step" was, and why the link stayed off "until the driver step is safe". A draw can only be linked from
// its program's two shader libraries if the driver has them compiled; compiled in the draw they stall it for a compile
// (0.2 to 1.1 s measured on NX2navi), so a draw asks for them under VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED
// and goes without when they are cold. They are warm only if the start's "driver's part" made them: the step between the
// translation and the pipelines (the start-up screen's "Compiling shaders", vk_draw.cpp warm_libraries), where every
// kept program variant's two libraries are compiled once, on every processor, into the pipeline cache. That step was not
// safe to leave on:
// - it hands every shader of the game to the driver's compiler, and a driver's compiler faults on some of them (amdvlk64
// on a mesh program, NVIDIA 615.78 on one vertex library at the same point of every preparation, NX2navi's issue 30;
// the lab's start of 11-13 that hung with the driver's own lock held after such a fault);
// - what it kept of a start that died in it was one empty file, <prepared>.warming: "this step began and did not end".
// The next start then left the whole step out and wrote it down as done for that driver, for good. One shader the
// driver cannot compile, a start closed from the task manager, a power cut: each left every library cold, and with
// cold libraries no draw is linked, while each new pipeline still has two libraries built behind it on the workers
// its whole build waits for. The link then costs and gives nothing;
// - a fault the guard catches in that step starts the game again (driver_fault_restart), into exactly that.
// NX2navi's equivalent step is its library warm (runtime/vk_pipeline.c lw_worker, lb_warm_prepared). What makes it safe
// there, and here now: while a worker has a variant in the driver's hands, a journal on disk names it. A start that finds
// a journal left over puts the variants it names on a skip list for this driver, which the step leaves out from then on
// (each is built at its first draw, as without libraries), and goes on with the step: what the last start compiled comes
// back from the driver's own cache in a moment. Only after kTries starts in a row that began the step and did not end it
// is the step left out, as it was after the first. NX2navi's journal and skip list are not its owner's lines alone, so
// nothing of them is copied: this is written from what they do.
//
// And the rules of the link itself, which were in graphics_pipeline's lines:
// - a draw is linked only from libraries that are all there; a cold or failed library, or a link the driver refuses,
// leaves the entry not ready and the whole build to finish it: what a draw does with an entry that is not ready is
// what it does without libraries, nothing more is drawn with it;
// - the worker's whole (optimised) build always follows a link, and takes its place when it is made; if it fails, the
// linked pipeline stays, as before;
// - the linked pipeline a whole build replaced is given back to the driver (it was kept for the run: NX2navi measured
// 491 MiB of driver memory a session for its replaced links), but only kRetirePresents presents and kRetireMs later:
// a draw recorded with it may still be on its way to the GPU. RX_VK_GPL_KEEP_LINKS=1: kept, as before.
#pragma once
#include <atomic>
#include <cstdint>
#include <cstdio>
#include <deque>
#include <mutex>
#include <string>
#include <vector>
namespace gpl_rules {
// ---- an entry: linked first, then the whole build over it
/// The draw's thread, for a new entry: link is what the libraries linked to, "none" when a library was cold or failed or
/// the driver refused the link. True: the entry is ready with it. False: nothing changed, the whole build makes it ready
template <class Entry, class Pipeline>
bool linked(Entry& e, Pipeline link) {
if (link == Pipeline{}) return false;
e.fast.store(link, std::memory_order_relaxed);
e.pipeline.store(link, std::memory_order_relaxed);
e.ready.store(true, std::memory_order_release);
return true;
}
/// The worker, the whole build over: whole is "none" when it failed. The entry is ready from here on: with the whole
/// build, or with the link it had (the build failed), or with nothing (it failed and there was no link: the draws are left
/// out, as before). Returns the link the whole build replaced, for retire(); "none" when there is nothing to give back
template <class Entry, class Pipeline>
Pipeline built(Entry& e, Pipeline whole) {
Pipeline replaced{};
if (!(whole == Pipeline{})) {
e.pipeline.store(whole, std::memory_order_release);
replaced = e.fast.exchange(Pipeline{}, std::memory_order_acq_rel);
}
e.ready.store(true, std::memory_order_release);
return replaced;
}
// ---- the replaced links, given back later
inline constexpr uint64_t kRetirePresents = 600;
inline constexpr uint64_t kRetireMs = 10000;
/// The links whole builds replaced, until both waits are over (any thread adds, the present's thread takes)
template <class Pipeline>
class Retired {
public:
void add(Pipeline p, uint64_t present, uint64_t ms) {
std::lock_guard lk(mu_);
q_.push_back({p, present, ms});
}
/// Those replaced kRetirePresents presents and kRetireMs before now, the oldest first: the caller destroys them
std::vector<Pipeline> due(uint64_t present, uint64_t ms) {
std::vector<Pipeline> out;
std::lock_guard lk(mu_);
while (!q_.empty() && present >= q_.front().present + kRetirePresents && ms >= q_.front().ms + kRetireMs) {
out.push_back(q_.front().p);
q_.pop_front();
}
return out;
}
size_t waiting() {
std::lock_guard lk(mu_);
return q_.size();
}
private:
struct One {
Pipeline p;
uint64_t present, ms;
};
std::mutex mu_;
std::deque<One> q_;
};
// ---- the driver's part of the preparation: its journal and its skip list
/// A program variant (Program::key, Program::variant)
struct Id {
uint64_t key = 0, variant = 0;
bool operator==(const Id& o) const { return key == o.key && variant == o.variant; }
};
inline constexpr uint32_t kJournalMagic = 0x4A574654; // ("TFWJ")
inline constexpr uint32_t kSkipMagic = 0x53574654; // ("TFWS")
inline constexpr int kSlots = 64; // (the workers of the preparation at most: prepare_threads.h)
inline constexpr uint32_t kTries = 3; // (starts in a row that may begin the step and not end it)
/// What a start found of the last one's journal
struct Left {
bool found = false; // (a journal was there: the last start began the step and did not end it)
uint32_t tries = 0; // (the starts in a row that did so, the last one counted; 0: none, or another driver's)
std::vector<Id> in_hand; // (the variants its workers had in the driver's hands when it ended)
};
/// The journal at path as the last start left it. stamp: what the driver's part is made for now (vk_draw.cpp warmed_stamp).
/// A file of another stamp is another driver's or other shaders': found, with nothing to count and no variant to blame. A
/// file without the header is an older build's empty marker: one start, no variant named
inline Left journal_left(const std::string& path, uint64_t stamp) {
Left l;
FILE* f = fopen(path.c_str(), "rb");
if (!f) return l;
l.found = true;
uint32_t head[2]{};
uint64_t was = 0;
if (fread(head, 4, 2, f) != 2 || fread(&was, 8, 1, f) != 1 || head[0] != kJournalMagic) {
l.tries = 1;
} else if (was == stamp) {
l.tries = head[1] ? head[1] : 1;
for (int i = 0; i < kSlots; i++) {
uint64_t r[2];
if (fread(r, 8, 2, f) != 2) break;
if (r[0] || r[1]) l.in_hand.push_back({r[0], r[1]});
}
}
fclose(f);
return l;
}
/// Whether a start that found l goes on with the step (true) or leaves it out for this driver, as every start did before
inline bool goes_on(const Left& l) { return !l.found || l.tries < kTries; }
/// The journal of a step under way. Every mark is on its way to the disk before the driver is called
class Journal {
public:
Journal() = default;
Journal(const Journal&) = delete;
Journal& operator=(const Journal&) = delete;
~Journal() { close(); }
/// tries: the starts in a row that began the step, this one counted
bool open(const std::string& path, uint64_t stamp, uint32_t tries) {
std::lock_guard lk(mu_);
if (f_) fclose(f_);
f_ = fopen(path.c_str(), "wb");
if (!f_) return false;
const uint32_t head[2] = {kJournalMagic, tries};
const uint64_t zero[2 * kSlots]{};
const bool ok = fwrite(head, 4, 2, f_) == 2 && fwrite(&stamp, 8, 1, f_) == 1 &&
fwrite(zero, 8, (size_t)(2 * kSlots), f_) == (size_t)(2 * kSlots) && fflush(f_) == 0;
if (!ok) fclose(f_), f_ = nullptr;
return ok;
}
/// Worker slot's variant, about to go to the driver ({0, 0}: it came back). A slot past the last has no line
void mark(int slot, Id id) {
if (slot < 0 || slot >= kSlots) return;
std::lock_guard lk(mu_);
if (!f_) return;
const uint64_t r[2] = {id.key, id.variant};
if (fseek(f_, 16 + 16 * slot, SEEK_SET) == 0 && fwrite(r, 8, 2, f_) == 2) fflush(f_);
}
void close() {
std::lock_guard lk(mu_);
if (f_) fclose(f_), f_ = nullptr;
}
private:
std::mutex mu_;
FILE* f_ = nullptr;
};
/// The skip list at path for this stamp (another stamp's, or no file: empty)
inline std::vector<Id> skip_read(const std::string& path, uint64_t stamp) {
std::vector<Id> ids;
FILE* f = fopen(path.c_str(), "rb");
if (!f) return ids;
uint32_t head[2]{};
uint64_t was = 0;
if (fread(head, 4, 2, f) == 2 && fread(&was, 8, 1, f) == 1 && head[0] == kSkipMagic && was == stamp) {
uint64_t r[2];
while (ids.size() < 65536 && fread(r, 8, 2, f) == 2) ids.push_back({r[0], r[1]});
}
fclose(f);
return ids;
}
/// The skip list at path with more added (each variant once), written whole. Returns the list, as skip_read would give it
inline std::vector<Id> skip_add(const std::string& path, uint64_t stamp, const std::vector<Id>& more) {
std::vector<Id> ids = skip_read(path, stamp);
bool grew = false;
for (const Id& id : more) {
bool have = false;
for (const Id& o : ids) have = have || o == id;
if (!have) ids.push_back(id), grew = true;
}
if (!grew) return ids;
const std::string tmp = path + ".tmp";
FILE* f = fopen(tmp.c_str(), "wb");
if (!f) return ids;
const uint32_t head[2] = {kSkipMagic, 0};
bool ok = fwrite(head, 4, 2, f) == 2 && fwrite(&stamp, 8, 1, f) == 1;
for (const Id& id : ids) {
const uint64_t r[2] = {id.key, id.variant};
ok = ok && fwrite(r, 8, 2, f) == 2;
}
ok = fclose(f) == 0 && ok;
if (ok) {
remove(path.c_str()); // (rename does not replace on Windows)
ok = rename(tmp.c_str(), path.c_str()) == 0;
}
if (!ok) remove(tmp.c_str());
return ids;
}
} // namespace gpl_rules

34
gpu/vk/hang_watch.h Normal file
View File

@@ -0,0 +1,34 @@
// The hang watch's judgement, apart from its thread, the clock and the stacks (vk_hangdump.cpp). After NX2navi's hang
// watch (its runtime/hang_watch.c hw_step): a report when no frame is shown for some seconds, once a stall, a few a run.
// tests/test_hang_watch.py
#pragma once
#include <cstdint>
namespace mkwvk::hangwatch {
constexpr int kReports = 3; // (at most this many a run: a game that stalls again and again says so three times)
struct Watch {
int64_t presents = -1; // the presents counted at the last look
int still = 0; // judged seconds running without one
bool told = false; // this stall has its report
int reports = 0;
};
/// One second's look. presents: the frames shown so far. judged: whether a second without one counts (the game's window
/// in front and not minimized: the game may stop showing frames behind another window, and a watch asked for by name
/// judges always). True when a report is due now: `limit` judged seconds running without a present, once per stall (a
/// new one needs a present first), at most kReports a run.
inline bool step(Watch& w, int64_t presents, bool judged, int limit) {
if (presents != w.presents || !judged) {
w.presents = presents, w.still = 0, w.told = false;
return false;
}
if (++w.still < limit || w.told || w.reports >= kReports) return false;
w.told = true;
w.reports++;
return true;
}
} // namespace mkwvk::hangwatch

59
gpu/vk/helper_spin.h Normal file
View File

@@ -0,0 +1,59 @@
// (Triforce, the third performance audit's P6, after NX2navi's replay workers: runtime/vk_exec.c rw_main's idle spin and the
// submitting thread's spin for its chunks, epicmeme546, 3010a637, 7534193b, 7eee4817, ffa599ef) A bounded spin before a sleep
// on a condition variable, for the parallel replay (vk_replay.cpp): a helper thread with no segment to record spins for the
// next before it sleeps (par_worker_main), and the replay thread spins for a segment a helper records before it sleeps for it
// (replay_handle). What each waits for is published where the spin can read it without the lock (ParPool::queued, Job::done),
// and the condition variable's wait with its predicate follows the spin as before: the spin only makes the wait shorter when
// what it waits for comes within it, it never decides anything. The audit's trace: the helpers began a segment 0.262 ms after
// they were given it, and the replay thread waited 1.74 ms a frame for them (r22-trace). tests/test_helper_spin.py
//
// RX_VK_HELPER_SPIN_US=<us>: the spin's bound (100, NX2navi's; at most 1000000); 0: no spin, straight to the wait as before
#pragma once
#include <chrono>
#include <cstdint>
#include <cstdlib>
#if defined(__x86_64__) || defined(_M_X64) || defined(__i386__) || defined(_M_IX86)
#include <immintrin.h>
#endif
namespace helper_spin {
/// The bound in nanoseconds that RX_VK_HELPER_SPIN_US's text gives (null: unset)
inline int64_t bound_ns(const char* text) {
if (!text || !*text) return 100000;
const long long us = std::atoll(text);
return us <= 0 ? 0 : us >= 1000000 ? 1000000000LL : us * 1000;
}
/// RX_VK_HELPER_SPIN_US, read once
inline int64_t spin_ns() {
static const int64_t v = bound_ns(std::getenv("RX_VK_HELPER_SPIN_US"));
return v;
}
inline void pause() {
#if defined(__x86_64__) || defined(_M_X64) || defined(__i386__) || defined(_M_IX86)
_mm_pause();
#endif
}
/// Spins until done() holds or ns have passed; whether done() held. ns <= 0: done() once, no spin. *spent (when given): the
/// time spun
template <class Done>
bool until(Done&& done, int64_t ns, int64_t* spent = nullptr) {
if (spent) *spent = 0;
if (done()) return true;
if (ns <= 0) return false;
const auto t0 = std::chrono::steady_clock::now();
const auto end = t0 + std::chrono::nanoseconds(ns);
bool held = false;
for (unsigned i = 1;; i++) {
pause();
if ((held = done())) break;
if (!(i & 7) && std::chrono::steady_clock::now() >= end) {
held = done();
break;
}
}
if (spent) *spent = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now() - t0).count();
return held;
}
} // namespace helper_spin

125
gpu/vk/import_ahead.h Normal file
View File

@@ -0,0 +1,125 @@
/* (Triforce) The game's memory imported ahead of its first binding (vk_memory.cpp import_ahead_*). A pool's memory is imported
into Vulkan (VK_EXT_external_memory_host) at the first binding that reaches it, by 16 MB chunk; the import is the driver's
page locking, 8 to 17 ms a chunk on the owner's RTX 5090, and a binding the GPU writes, or a texture's storage read by the
GPU's unswizzle, is imported there and then on the replay's thread: seven in one dispatch made a submit of 79 ms in the
owner's test4 log, and the "unswizzled on the GPU" uploads of 22 to 32 ms were imports of their storage. The game's pools sit
at the same addresses from run to run (triforce_mirror_keep.txt relies on that too), so the ranges a run imported on its
own thread are kept in triforce_imports_ahead.txt beside the game, and in the next runs each is imported by the background
import thread as soon as the game's pool table holds memory there, before a binding wants it. A binding that comes while it is
under way waits for it, as for any import in the background. Plain functions: tests/test_import_ahead.cpp checks them.
RX_VK_IMPORT_AHEAD=1 or 0: on or off on any card; unset: on with NVIDIA's driver only, as the background imports of a
binding's reads are (RX_VK_ASYNC_IMPORT: background imports slowed an AMD card's own, vk_memory.cpp gpu_buffer_bind). */
#pragma once
#include <algorithm>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <vector>
namespace mkwvk::import_ahead {
constexpr size_t kMax = 256; // (ranges kept in the file at most)
constexpr size_t kPerLook = 4; // (ranges sent to the background thread at one look, 16 under way at most: a look comes every 8
// submissions too, and the test9 session's queue submits stalled beside batches of 16, made
// one after the other; each range is sent once a session, vk_memory.cpp)
/// RX_VK_IMPORT_AHEAD: 1 on, 0 off, unset (or anything else): on for NVIDIA's driver (vendor 0x10de) only, as before test12.
/// ((Triforce, 11-15) Off by default since test12: in the lab, at the title, with test12's binaries, the GPU hung after the
/// write-back of pool 0x1072000000 (a mirror in pieces) in 6 of 12 runs with the imports ahead on and in 0 of 10 with
/// RX_VK_IMPORT_AHEAD=0 (lazy write-back, GPL, BC1, Reflex, the sampler cache and the helper threads cleared, one by one);
/// the ahead list held 0x1072000000..0x1077000000, imported on the background thread while that pool's mirror was in use.
/// The owner's test10 never hung: its RAM was low all session, which pauses the imports ahead. Back on once the race is
/// found and closed. vendor: kept for that, unused)
/// ((Triforce, 11-15, later) A race that fits is closed: the host decommitted the game's freed pages under an import the background
/// thread had locked and not yet put in the table (vk_memory.cpp bg_held, memory_guest_imported; tests/test_bg_import_held.py), and
/// the imports ahead make that window wide where the game frees pages. Still off until a lab A/B with RX_VK_IMPORT_AHEAD=1 at the
/// title on a build with that fix hangs no more; the log's "freed under a background import" lines say whether the window was hit)
/// ((Triforce, 11-15, the lab A/B) On again: with that fix, 10 of 10 lab runs at the title with the imports ahead on were clean
/// and each hit the window 2 to 16 times ("freed under a background import"); with the fix undone (RX_VK_BG_IMPORT_HOLD=0)
/// 4 of 10 hung and others showed the title's video garbled. Runs E:\TriforceDev\lab-runs\holdA*, holdB*)
inline bool on(const char* env, uint32_t vendor) {
if (env && env[0] == '1' && !env[1]) return true;
if (env && env[0] == '0' && !env[1]) return false;
return vendor == 0x10DE;
}
/// (Triforce, test8) A range imported ahead, [lo, hi), on chunk bounds: what an import on a binding's own thread took, its
/// span over the game's pools across chunk ends included (vk_memory.cpp span_game_pools). The first version kept 16 MB chunks
/// and imported each on its own: a binding across their ends then made them one import again on its own thread ("not in the
/// background: an import beside it grows", 6 of the 13 still made so in the owner's test8 session, and the 64 MB imports that
/// took in four of them)
struct Range {
uint64_t lo = 0, hi = 0;
bool operator==(const Range& o) const { return lo == o.lo && hi == o.hi; }
};
/// The file's lines: "lo hi" in hex (a range), or "lo" alone (a chunk, as the first version wrote them); lines that are no such
/// numbers, and ranges not on chunk bounds, are left out. Sorted, none inside another, at most kMax
inline std::vector<Range> parse(const char* text, uint64_t chunk) {
std::vector<Range> out;
if (!chunk) return out;
for (const char* p = text; p && *p;) {
char* end = nullptr;
const unsigned long long lo = std::strtoull(p, &end, 16);
unsigned long long hi = 0;
bool ok = end != p && lo && lo % chunk == 0;
if (ok && *end == ' ') {
const char* q = end + 1;
hi = std::strtoull(q, &end, 16);
ok = end != q && hi > lo && hi % chunk == 0 && hi - lo <= (1ull << 30);
} else {
hi = lo + chunk;
}
ok = ok && (*end == '\n' || *end == '\r' || !*end);
if (ok) out.push_back({lo, hi});
while (*p && *p != '\n') p++;
if (*p) p++;
}
std::sort(out.begin(), out.end(), [](const Range& a, const Range& b) { return a.lo != b.lo ? a.lo < b.lo : a.hi > b.hi; });
std::vector<Range> kept; // (none inside another)
for (const Range& r : out)
if (kept.empty() || r.hi > kept.back().hi) kept.push_back(r);
if (kept.size() > kMax) kept.resize(kMax);
return kept;
}
/// [lo, hi) into the list (sorted, none inside another): true when the list changed (the caller writes the file again). Ranges
/// inside the new one go; a new one inside a kept one changes nothing; a full list takes nothing new
inline bool add(std::vector<Range>& list, uint64_t lo, uint64_t hi) {
if (hi <= lo) return false;
for (const Range& r : list)
if (r.lo <= lo && hi <= r.hi) return false;
const size_t before = list.size();
std::erase_if(list, [&](const Range& r) { return lo <= r.lo && r.hi <= hi; });
if (list.size() == before && list.size() >= kMax) return false;
list.insert(std::lower_bound(list.begin(), list.end(), lo, [](const Range& r, uint64_t v) { return r.lo < v; }), Range{lo, hi});
return true;
}
/// The file's text for list
inline std::vector<char> text(const std::vector<Range>& list) {
std::vector<char> out;
char line[48];
for (const Range& r : list) {
const int n = std::snprintf(line, sizeof line, "%llx %llx\n", (unsigned long long)r.lo, (unsigned long long)r.hi);
out.insert(out.end(), line, line + n);
}
return out;
}
/// The ranges of list that the game's pools (sorted by lo) reach now and wanted(range) allows (nothing imported over it,
/// nothing under way in it), at most n of them
template <class Pools, class Wanted>
std::vector<Range> due(const std::vector<Range>& list, const Pools& pools, size_t n, Wanted wanted) {
std::vector<Range> out;
for (const Range& r : list) {
if (out.size() >= n) break;
auto g = std::lower_bound(pools.begin(), pools.end(), r.hi, [](const auto& p, uint64_t v) { return p.lo < v; });
if (g == pools.begin() || std::prev(g)->hi <= r.lo) continue; // (no pool starting below its end reaches into it)
if (wanted(r)) out.push_back(r);
}
return out;
}
} // namespace mkwvk::import_ahead

42
gpu/vk/ini_merge.h Normal file
View File

@@ -0,0 +1,42 @@
// (Triforce, handoff §9) triforce_options.ini has more than one writer: the game's Insert menu (vk_menu.cpp save, the whole
// file at every change) and whoever else changes it while the game runs (the owner in a text editor, a tool, a launcher of
// another make). The menu wrote the values it held, so a line changed outside since the game read the file was put back as the
// game had it at its next save: that change was lost, however long before the save it came. Now each line the game writes is
// merged by its key: the game's value where the game changed it since it last read or wrote the file (its base), else what the
// file has now. Two changes made within a second, one in the game and one outside, both stay; one row changed on both sides is
// the game's (the later writer). The game itself takes an outside change at its next start, as a file read at the start is.
// Plain logic: tests/test_options_writers.py compiles it alone
#pragma once
#include <map>
#include <string>
namespace ini_merge {
using Map = std::map<std::string, std::string>;
/// A value as lines are compared: without its line end and the spaces around it
inline std::string trim(const std::string& v) {
size_t a = 0, b = v.size();
while (a < b && (v[a] == ' ' || v[a] == '\t')) a++;
while (b > a && (v[b - 1] == '\n' || v[b - 1] == '\r' || v[b - 1] == ' ' || v[b - 1] == '\t')) b--;
return v.substr(a, b - a);
}
/// One line of the file into the map, when it is key=value (a key of something before the '=')
inline void add_line(Map& m, const std::string& line) {
const size_t eq = line.find('=');
if (eq == std::string::npos || eq == 0) return;
m[line.substr(0, eq)] = trim(line.substr(eq + 1));
}
/// The value the game writes for key: its own (game) where it changed it since base was taken (base: the game's values as of
/// its last read or write of the file; a key not in base is the game's), else the file's as it is now (disk), and the game's
/// where the file has no such line. *taken: set when the file's was taken over the game's
inline std::string pick(const std::string& key, const std::string& game, const Map& base, const Map& disk, bool* taken) {
const auto b = base.find(key);
if (b == base.end() || b->second != trim(game)) return game; // (changed in the game, or new to the file)
const auto d = disk.find(key);
if (d == disk.end() || d->second == b->second) return game;
if (taken) *taken = true;
return d->second;
}
} // namespace ini_merge

99
gpu/vk/lab_ab.h Normal file
View File

@@ -0,0 +1,99 @@
/* (Triforce, lab) In-run A/B. Two runs of one build differ by more than most changes are worth (the other programs on the GPU,
which cores the threads landed on, the scene's state), so a change's effect is measured inside ONE run: the frames go in
phases of RX_VK_LAB_AB_FRAMES presents (300): none, knob 1, none, knob 2, ... and around again, and the frame trace
(RX_VK_FRAME_TRACE, vk_replay.cpp status_tick) carries the phase's knob in its last column (0: none, k: the k-th knob;
negative for a phase's first RX_VK_LAB_AB_SKIP frames (30), which a reader leaves out). perf\lowend\lab_ab.py compares each
knob's phases with the none-phases on both sides of them.
RX_VK_LAB_AB=<name>[:<value>],<name>[:<value>],... (at most 16)
Knobs joined by '+' are one knob that turns each of them on ("reclast+tlrec+rbsettle").
A knob named d_<site> is a delay: a busy wait of <value> microseconds at that site, at most once a frame (delay(), below):
what the frame's time answers to a part of it taking longer, which is whether that part is on the frame's critical path
(1 ms in for 1 ms of delay) or has slack (nothing). Any other name is a switch the code reads with on("<name>").
Not for players: nothing reads these without the variable. */
#pragma once
#include <atomic>
#include <chrono>
#include <cstdint>
#include <cstdlib>
#include <cstring>
namespace mkwvk::lab_ab {
struct Knob {
char name[64];
int64_t value;
};
inline Knob knobs[16];
inline int n_knobs = -1; // (-1: the variable not read yet)
inline std::atomic<int> active{0}; // (0: none; k: knobs[k - 1])
inline std::atomic<uint64_t> frame_no{0}; // (presents counted here)
inline std::atomic<uint64_t> fired[16]; // (the frame each delay last waited in)
inline std::atomic<uint32_t> pres_thread{0}; // (the thread that presents: the game's Presentation thread, in the plain mode)
inline int64_t phase_frames = 300, skip_frames = 30;
inline void init() {
if (n_knobs >= 0) return;
int n = 0;
const char* e = getenv("RX_VK_LAB_AB");
while (e && *e && n < 16) {
const char* end = strchr(e, ',');
const size_t len = end ? (size_t)(end - e) : strlen(e);
char item[96] = {};
memcpy(item, e, len < sizeof item - 1 ? len : sizeof item - 1);
char* colon = strchr(item, ':');
knobs[n].value = colon ? atoll(colon + 1) : 1;
if (colon) *colon = 0;
strncpy(knobs[n].name, item, sizeof knobs[n].name - 1);
if (knobs[n].name[0]) n++;
e = end ? end + 1 : nullptr;
}
if (const char* f = getenv("RX_VK_LAB_AB_FRAMES")) phase_frames = atoll(f) > 0 ? atoll(f) : 300;
if (const char* s = getenv("RX_VK_LAB_AB_SKIP")) skip_frames = atoll(s);
n_knobs = n;
}
/// Once a present, on the presenting thread: the phase this frame belongs to (the trace's column)
inline int phase_tick() {
init();
if (!n_knobs) return 0;
const uint64_t f = frame_no.fetch_add(1) + 1;
const uint64_t phase = f / (uint64_t)phase_frames; // (0 none, 1 knob 1, 2 none, 3 knob 2, ...)
const int k = (phase & 1) ? (int)((phase / 2) % (uint64_t)n_knobs) + 1 : 0;
active.store(k, std::memory_order_relaxed);
const bool settling = (int64_t)(f % (uint64_t)phase_frames) < skip_frames;
return settling ? -(k + 1) : k; // (-1: none, settling; -(k + 1): knob k, settling)
}
/// Is this switch on in this phase: the phase's knob is it, or names it among several joined by '+' ("reclast+tlrec":
/// both on in that knob's phases, so a combination is one knob)
inline bool on(const char* name) {
if (n_knobs <= 0) return false;
const int k = active.load(std::memory_order_relaxed);
if (k <= 0) return false;
const size_t len = strlen(name);
for (const char* p = knobs[k - 1].name; p && *p;) {
const char* plus = strchr(p, '+');
const size_t n = plus ? (size_t)(plus - p) : strlen(p);
if (n == len && !strncmp(p, name, len)) return true;
p = plus ? plus + 1 : nullptr;
}
return false;
}
inline int64_t value_of(const char* name, int64_t otherwise) {
if (n_knobs <= 0) return otherwise;
const int k = active.load(std::memory_order_relaxed);
return k > 0 && !strcmp(knobs[k - 1].name, name) ? knobs[k - 1].value : otherwise;
}
/// The delay site <site> (knob "d_<site>"): a busy wait of its microseconds, once a frame
inline void delay(const char* site) {
if (n_knobs <= 0) return;
const int k = active.load(std::memory_order_relaxed);
if (k <= 0 || knobs[k - 1].name[0] != 'd' || knobs[k - 1].name[1] != '_' || strcmp(knobs[k - 1].name + 2, site)) return;
const uint64_t f = frame_no.load(std::memory_order_relaxed);
if (fired[k - 1].exchange(f, std::memory_order_relaxed) == f) return;
const auto until = std::chrono::steady_clock::now() + std::chrono::microseconds(knobs[k - 1].value);
while (std::chrono::steady_clock::now() < until) {
}
}
} // namespace mkwvk::lab_ab

70
gpu/vk/layer_split.h Normal file
View File

@@ -0,0 +1,70 @@
/* (Triforce) A new layered ASTC texture's first upload spread over a few frames, some of its layers each (vk_texture.cpp
texture_touch, split_take). The GPU decodes an ASTC texture (and encodes it again as BC3 or BC1 where the renderer keeps it
so) in the upload command buffer of the recording that first draws it. In the owner's test7 session one such texture,
1365 x 1365 x 90 layers with 11 levels (57 MB of ASTC, some 890 MB of texels), was the upload of an area change's frame
("slow upload 35.0 ms ... ASTC (BC3)"), that frame's upload command buffer took the GPU 114.8 ms, and the next frame's
capture waited 117.2 ms for it. The frame's upload budget (new_texture_later) counts bytes of storage, and always lets a
frame's first upload through: one texture alone made the backlog. Now such a texture, while the hold of the last picture may
leave textures for later, is uploaded a part of its layers a frame: its draws sample the dummy and its frames are held until
the last part is in, as for a new texture left whole for the next frame. At the hold's bound, or for a dispatch, what is left
goes at once. Plain functions: tests/test_layer_split.cpp checks them. RX_VK_UPLOAD_SPLIT_MB=<MB of texels a frame> (128),
0: whole, as before. */
#pragma once
#include <algorithm>
#include <cstdint>
#include <cstdlib>
#include <utility>
#include <vector>
namespace mkwvk::layer_split {
using Ranges = std::vector<std::pair<uint64_t, uint64_t>>;
constexpr uint32_t kMaxParts = 6; // (frames at most: upload_owed::kLaterMax is 8, and a part is a frame)
/// RX_VK_UPLOAD_SPLIT_MB: the texels' MB a part may decode, 0: no split (unset or no number: 128)
inline uint64_t budget(const char* env) {
if (!env || !*env) return 128ull << 20;
char* end = nullptr;
const double v = std::strtod(env, &end);
if (end == env || *end || v < 0) return 128ull << 20;
return (uint64_t)(v * 1048576.0);
}
/// The texels an ASTC texture decodes to (RGBA8, every level and layer)
inline uint64_t texel_bytes(uint32_t w, uint32_t h, uint32_t levels, uint32_t layers) {
uint64_t n = 0;
for (uint32_t l = 0; l < std::max(1u, levels); l++) n += (uint64_t)std::max(1u, w >> l) * std::max(1u, h >> l) * 4;
return n * std::max(1u, layers);
}
/// The parts a texture of these texels and layers is uploaded in (1: whole)
inline uint32_t parts(uint64_t texels, uint32_t layers, uint64_t per_part) {
if (!per_part || layers < 2 || texels <= per_part) return 1;
const uint64_t p = (texels + per_part - 1) / per_part;
return (uint32_t)std::min<uint64_t>({p, (uint64_t)kMaxParts, (uint64_t)layers});
}
/// The layers of `remaining` (ranges of the storage, whole layers of layer_size bytes, in order) split into this frame's part,
/// the first `per_frame` layers of them, and the rest
inline void take(const Ranges& remaining, uint64_t layer_size, uint32_t per_frame, Ranges& part, Ranges& rest) {
part.clear(), rest.clear();
if (!layer_size) return;
uint64_t left = (uint64_t)std::max(1u, per_frame) * layer_size;
for (const auto& [lo, hi] : remaining) {
if (hi <= lo) continue;
if (!left) { rest.push_back({lo, hi}); continue; }
const uint64_t n = std::min(left, hi - lo);
part.push_back({lo, lo + n});
left -= n;
if (lo + n < hi) rest.push_back({lo + n, hi});
}
}
/// The layers a part takes: the texture's layers over its parts, rounded up
inline uint32_t per_frame(uint32_t layers, uint32_t n_parts) {
return n_parts ? (layers + n_parts - 1) / n_parts : layers;
}
} // namespace mkwvk::layer_split

203
gpu/vk/mem_parts.h Normal file
View File

@@ -0,0 +1,203 @@
// (Triforce) What the process holds in RAM, in parts, for the log's "vk: memory, the parts" line (vk_memory.cpp
// memory_parts_tick, every 600 presents) and the rules that react to the PC's RAM running short. Apart from Vulkan and
// Windows so that tests/test_mem_parts.py runs it alone.
//
// Why: a 32 GB PC ran out of RAM in play (v0.4.0-test7: "RAM short (525 to 850 of 32069 MB available)" 40 times, the
// working set 20.9 GB, frames of 102 and 245 ms outside the replay and the present while Windows paged), and the log
// said only the process's totals. Its private bytes are no guide to the RAM (they grew with the VRAM, 29.7 -> 42.5 GB
// while the device's blocks grew 1.4 -> 11.3 GB: the driver's video memory is charged there), the working set is. The
// working set is the game's own memory (the guest window: its heap and regions, committed at once, in RAM as the game
// touches it) and the rest: the driver's shader modules, pipelines and pipeline cache, the allocator's blocks in system
// memory, the heap. The line gives each part that is known, and the game's share of the working set by a sample of
// its pages (sample_plan, below): one page in every so many of the committed window asked whether it is in RAM.
// (11-16) The rest is split the same way, by what the address space says each run is (Kind): the program's code, the
// driver's, other code, mapped files and views, write-combined or uncached memory (what a driver maps for the GPU), and
// the heaps and other private memory. A session of v0.4.0-test9 had 9.9 GB of working set outside the game's memory
// all through play, with the RAM short, and nothing in the line said what it was.
#pragma once
#include <cstdint>
#include <cstdio>
#include <string>
#include <vector>
namespace mem_parts {
/// What a run of the address space is (kNone: not counted: free, reserved, or no access)
enum Kind : uint8_t { kNone = 0, kGuest, kExe, kDriverCode, kOtherCode, kMapped, kGpuVisible, kPrivate, kKinds };
/// A committed run, [lo, hi), and what it is
struct Span {
uint64_t lo = 0, hi = 0;
uint8_t kind = kGuest;
};
/// The runs between lo and hi that query(at, &base, &size, &kind) counts, which answers for the region that holds at
/// (VirtualQuery's answer: its base, its size from there, and what it is, kNone for what is not counted); false ends the
/// walk. [skip_lo, skip_hi) is stepped over without a query (the guest window, which has a walk of its own). At most
/// max_queries regions are asked (the walk is a system call a region), and runs of one kind that touch are joined.
/// *queries: how many were asked; *whole: false when max_queries stopped the walk before hi
template <class Query>
std::vector<Span> kind_spans(uint64_t lo, uint64_t hi, Query&& query, size_t max_queries, uint64_t skip_lo = 0,
uint64_t skip_hi = 0, size_t* queries = nullptr, bool* whole = nullptr) {
std::vector<Span> out;
uint64_t at = lo;
size_t n = 0;
while (at < hi && n < max_queries) {
if (at >= skip_lo && at < skip_hi) { // (no query spent on it)
at = skip_hi;
continue;
}
n++;
uint64_t base = 0, size = 0;
uint8_t kind = kNone;
if (!query(at, &base, &size, &kind) || !size) {
at = hi; // (the end of the walk: whole as far as anything answers)
break;
}
uint64_t end = base + size > base ? base + size : hi; // (no wrap past the top)
if (end <= at) {
at = hi;
break;
}
if (skip_hi > skip_lo && at < skip_lo && end > skip_lo) end = skip_lo; // (the skipped window is not counted)
const uint64_t a = base > at ? base : at, b = end < hi ? end : hi;
if (kind != kNone && a < b) {
if (!out.empty() && out.back().hi == a && out.back().kind == kind) out.back().hi = b;
else out.push_back({a, b, kind});
}
at = end;
}
if (queries) *queries = n;
if (whole) *whole = at >= hi;
return out;
}
/// The committed runs between lo and hi, from query(at, &base, &size, &committed) (the guest window's walk)
template <class Query>
std::vector<Span> committed_spans(uint64_t lo, uint64_t hi, Query&& query, size_t max_queries, size_t* queries = nullptr,
bool* whole = nullptr) {
return kind_spans(
lo, hi,
[&](uint64_t at, uint64_t* base, uint64_t* size, uint8_t* kind) {
bool committed = false;
if (!query(at, base, size, &committed)) return false;
*kind = committed ? kGuest : kNone;
return true;
},
max_queries, 0, 0, queries, whole);
}
/// Pages to ask about: one every stride bytes of each committed run (stride a whole number of pages, at least one, made
/// larger until the runs' pages are at most max_samples strides), each standing for the bytes up to the next (the run's
/// last for what is left). Each run has its first page asked, so there are at most max_samples + spans.size() of them
struct Plan {
uint64_t committed = 0, stride = 0;
std::vector<uint64_t> at, bytes;
std::vector<uint8_t> kind; // (each sample's run's)
};
inline Plan sample_plan(const std::vector<Span>& spans, uint64_t page, size_t max_samples) {
Plan p;
if (!page) page = 4096;
for (const Span& s : spans) p.committed += s.hi - s.lo;
if (!max_samples) max_samples = 1;
const uint64_t pages = (p.committed + page - 1) / page;
const uint64_t every = pages > max_samples ? (pages + max_samples - 1) / max_samples : 1;
p.stride = every * page;
for (const Span& s : spans) // (a run shorter than the stride counts all of itself by its first page)
for (uint64_t a = s.lo; a < s.hi; a += p.stride) {
p.at.push_back(a);
p.bytes.push_back(s.hi - a < p.stride ? s.hi - a : p.stride);
p.kind.push_back(s.kind);
if (s.hi - a <= p.stride) break; // (and no step past the top of the address space)
}
return p;
}
/// What the pages found in RAM stand for, in bytes (resident[i] answers for plan.at[i])
inline uint64_t resident_bytes(const Plan& p, const std::vector<bool>& resident) {
uint64_t r = 0;
for (size_t i = 0; i < p.at.size() && i < resident.size(); i++)
if (resident[i]) r += p.bytes[i];
return r;
}
/// The same, by the samples' kinds, into out[kKinds] (added to what is there)
inline void resident_by_kind(const Plan& p, const std::vector<bool>& resident, uint64_t out[kKinds]) {
for (size_t i = 0; i < p.at.size() && i < resident.size(); i++)
if (resident[i] && p.kind[i] < kKinds) out[p.kind[i]] += p.bytes[i];
}
/// The parts the line gives, in bytes (-1: not known on this platform or in this run)
struct Parts {
int64_t working_set = -1, private_bytes = -1;
int64_t guest_committed = -1, guest_resident = -1;
int64_t by_kind[kKinds] = {-1, -1, -1, -1, -1, -1, -1, -1}; // (the rest in RAM by what it is; -1: not sampled)
int64_t modules = 0, module_bytes = 0; // (program variants' shader modules alive, and their SPIR-V)
int64_t packed_stages = 0, packed_bytes = 0; // (stages kept packed instead, and what that takes)
int64_t pcache_bytes = -1; // (the pipeline cache as last read or saved; -1: let go)
int64_t pipelines = 0, libraries = 0; // (graphics pipelines and stage libraries kept)
int64_t drawn = -1; // (of those pipelines, how many a draw has looked up; -1: not known)
int64_t released = 0; // (kept pipelines no draw used, given back for the RAM)
int64_t system_blocks = -1; // (the allocator's blocks in system memory)
int64_t ram_avail = -1, ram_total = -1;
};
inline std::string mb(int64_t b) {
if (b < 0) return "?";
char t[32];
std::snprintf(t, sizeof t, "%lld", (long long)(b >> 20));
return t;
}
/// The line, without its "vk: " head
inline std::string line(const Parts& p) {
std::string s = "memory, the parts (MB): working set " + mb(p.working_set) + " (private " + mb(p.private_bytes) + ")";
s += "; the game's own memory " + mb(p.guest_committed) + " committed, " + mb(p.guest_resident) + " of it in RAM";
if (p.working_set >= 0 && p.guest_resident >= 0)
s += "; the rest of the working set " + mb(p.working_set > p.guest_resident ? p.working_set - p.guest_resident : 0);
if (p.by_kind[kPrivate] >= 0) {
static const char* const names[kKinds] = {"", "", "the program's code", "the driver's code", "other code",
"mapped files and views", "write-combined or uncached memory",
"the heaps and other private memory"};
s += " (sampled:";
for (int k = kExe; k < kKinds; k++) s += std::string(k == kExe ? " " : ", ") + names[k] + " " + mb(p.by_kind[k]);
s += ")";
}
char t[160];
std::snprintf(t, sizeof t, " (%lld modules", (long long)p.modules);
s += ": shader modules " + mb(p.module_bytes) + t;
std::snprintf(t, sizeof t, "; %lld stages packed in ", (long long)p.packed_stages);
s += t + mb(p.packed_bytes) + ")";
s += p.pcache_bytes < 0 ? ", the pipeline cache let go" : ", the pipeline cache " + mb(p.pcache_bytes);
std::snprintf(t, sizeof t, ", %lld pipelines and %lld stage libraries kept", (long long)p.pipelines, (long long)p.libraries);
s += t;
if (p.drawn >= 0) {
std::snprintf(t, sizeof t, " (%lld of the pipelines drawn; %lld no draw used given back)", (long long)p.drawn,
(long long)p.released);
s += t;
}
s += ", the allocator's blocks in system memory " + mb(p.system_blocks);
s += "; RAM " + mb(p.ram_avail) + " of " + mb(p.ram_total) + " available";
return s;
}
// ---- The PC's RAM running short: what the back end does about it that costs play nothing (vk_memory.cpp ram_watch) ----
/// Short: under 1.5 GB available, the imports' rule (vk_memory.cpp imports_give_back), whatever the PC has
constexpr uint64_t kShortBytes = 1536ull << 20;
inline bool short_now(uint64_t avail) { return avail < kShortBytes; }
/// Low: under 3 GB available, or a tenth of the RAM on a PC of more than 30 GB. The pipeline builder behind the game
/// (vk_draw.cpp predict_background) rests then: what the driver keeps of each pipeline it builds is RAM
inline bool low_now(uint64_t avail, uint64_t total) {
const uint64_t tenth = total / 10;
return avail < (tenth > (3ull << 30) ? tenth : (3ull << 30));
}
/// Whether a copy of bytes fits in RAM now with 2 GB to spare: the pipeline cache's save in play fetches the whole cache into
/// memory (2 to 3 GB with the stage libraries in it), and with less than that available it is what Windows pages out
inline bool copy_fits(uint64_t avail, uint64_t bytes) { return avail > bytes && avail - bytes > (2ull << 30); }
/// (11-16) Whether the precompile keeps every pipeline it builds for the session (vk_draw.cpp precompile_keeps_all): only where
/// the RAM the game can have, what is available plus what the process holds already (own), is 22 GB or more, on a PC of
/// 20 GB or more. The figure is v0.4.0-test9's, a 32 GB PC that started with 18.4 GB free: the game's own memory reached
/// 9.7 GB in RAM, the rest of the working set 9.9 GB, of which the precompile's 14,535 graphics and 1,033 compute
/// pipelines took about 3.3 GB (the working set before and after it); 9.7 + 6.6 + 3.3 GB, and 2 GB for Windows to page
/// with. The PC's total alone says nothing of what its other programs hold
constexpr uint64_t kKeepAllBytes = 22ull << 30;
inline bool keep_all_pipelines(uint64_t total, uint64_t avail, uint64_t own) {
return total >= (20ull << 30) && avail + own >= kKeepAllBytes;
}
} // namespace mem_parts

View File

@@ -7,7 +7,9 @@
namespace mkwvk::menu_restart {
constexpr int kMax = 16; // (rows that take a restart: the menu has ten; a row past kMax would have no slot)
// (rows that take a restart: the menu has about twenty; a row past kMax would have no slot. At most the bits of pending()'s
// mask: 32)
constexpr int kMax = 32;
struct State {
int n = 0;

127
gpu/vk/menu_sound_events.h Normal file
View File

@@ -0,0 +1,127 @@
// (Triforce) The options' own sounds, the game's interface sounds, after NX2navi's menu sounds: which sound a tick of the
// menu asks for, and where the sounds wait until the host's sound thread takes them (vk_api.h HOS_OPT_MENU_SOUND;
// host/menu_sound.c reads the game's files and plays them).
//
// Which sound, as NX2navi has it: the options opening and closing; another tab; the cursor on another row, card or button,
// and a value changed with Left and Right; a press on something (confirm, or a click); a step back. The menu's input is
// one long function (vk_menu.cpp menu_tick) that changes its state in many places, so the sound is told from what the
// tick changed: the state is taken before and after it (State), and with the keys the tick saw (Input) it names one
// sound at most (sound_of). The mouse resting on a row moves the cursor without a sound, as NX2navi's does.
//
// Inside the game's own Options screen the options are tabs beside the game's own list (vk_menu.cpp game_options_shown):
// going between that list and our tabs with L and R is a tab's move, not an opening; and Back on one of our tabs is the
// game's Back too (host/nn_hid.c), which leaves the screen with the game's own sound, so ours is not played over it.
// While the game's own list is shown the input and the sounds are the game's.
//
// The queue: the menu's thread puts, the host's sound thread takes, no lock (one of each). It holds kQueue sounds; one
// more is dropped. A sound older than kStaleNs when it is taken is not played: with no sound thread to take them
// (the game has not opened its sound yet, or stopped it) the presses of a while ago would otherwise all sound at once.
//
// No ImGui and no Vulkan: tests/test_menu_sound_events.py compiles it alone.
#pragma once
#include <atomic>
#include <cstdint>
namespace menu_sound {
// (vk_api.h HOS_MENU_SOUND_*: vk_menu.cpp holds them against these)
constexpr int kNone = 0, kOpen = 1, kClose = 2, kCursor = 3, kDecide = 4, kCancel = 5, kTabLeft = 6, kTabRight = 7;
// (State::page: the tabs' rows, a page over them, and the restart question, whose two buttons L and R choose between)
constexpr int kPageRows = 0, kPageAsk = 5;
// (State::mode: what the chosen row or card waits for)
constexpr int kModeNone = 0,
kModeListen = 1, // the Controls page: the new input of a binding
kModeEdit = 2, // Left and Right act on it each press: an item's amount, a flag's On and Off
kModeConfirm = 3; // Restart or Quit: its second press
/// What the options show, as far as their sounds go
struct State {
bool open = false; // the options are open
bool embedded = false; // ...as tabs inside the game's own Options screen
int page = kPageRows; // what is shown: the tabs' rows, or a page over them (vk_menu.cpp page; the Motion page is one too)
int tab = 0; // the tab, of the tabs' rows
int group = 0; // a page's section, which L and R turn: the inventory's, the teleport list's, a name page's
int cursor = 0; // the row, card, cell or button chosen
int mode = kModeNone;
int64_t value = 0; // what the chosen row is set to
bool operator==(const State&) const = default;
};
/// What a tick's input was, after the mouse's clicks were made the keys they stand for
struct Input {
bool toggle = false; // the keys that open and close the options over the game (Insert, F1, a gamepad's Home)
bool back = false; // Esc, the button B is bound to, a right click
bool confirm = false; // Enter, the button A is bound to, a click on what acts by one
bool tab_left = false, tab_right = false; // L and R, Q and E, Tab, the tab strip's arrows
bool left = false, right = false; // Left and Right, a click on a value's left or right half
bool move = false; // Up, Down, Page Up, Page Down, Home, End
bool clear = false; // Delete, Backspace (a binding taken away)
bool click = false; // the left mouse button went down
bool drag = false; // the left mouse button is held on a slider
};
/// The sound of a tick that found the options as `b` and left them as `a`: kNone, or one of the seven
inline int sound_of(const State& b, const State& a, const Input& in) {
if (!b.open && !a.open) return kNone; // (closed: the keys are the game's)
if (!b.open) // (opened: over the game by its keys, or from the game's own list by L or R, which is a tab's move)
return in.toggle ? kOpen : a.tab == 0 ? kTabRight : kTabLeft;
if (!a.open) {
// (closed by a press that acted: a map to travel to, an event, Restart now, an age switched, the question's Later)
if (in.confirm || in.left || in.right) return kDecide;
if (!b.embedded || in.toggle) return kClose;
// (our tabs inside the game's screen: on to the game's own list with L or R, or by a click on its tab, the first)
if (in.tab_left || in.tab_right) return in.tab_left ? kTabLeft : kTabRight;
// (Back is the game's Back too and has the game's sound; and the game's screen may have gone by itself)
return in.click && !in.back ? kTabLeft : kNone;
}
const bool changed = !(a == b);
if (b.mode == kModeListen) { // (a binding waited for its input: a key pressed meanwhile is that input, not the menu's)
if (a.mode == kModeListen) return kNone;
return a.value != b.value ? kDecide : in.back ? kCancel : kNone; // (bound; given up; or nothing came in time)
}
if ((in.back || in.toggle) && changed) return kCancel; // (out of a page, a mode, or to the restart question)
const bool tabs = in.tab_left || in.tab_right;
if (tabs && changed && b.page != kPageAsk) return in.tab_left ? kTabLeft : kTabRight;
// (a click on a tab, or on a page's section: by where it is from the one before)
if (in.click && a.page == b.page && a.tab != b.tab) return a.tab > b.tab ? kTabRight : kTabLeft;
if (in.click && a.page == b.page && a.group != b.group) return a.group > b.group ? kTabRight : kTabLeft;
// (a page opened, by a row that may have taken the press for itself; a page left, by a Back its tab took for itself)
if (a.page != b.page) return a.page == kPageRows ? kCancel : kDecide;
if (in.confirm) return kDecide;
if (in.click && (changed || in.left || in.right)) return kDecide;
const bool arrows = in.move || in.left || in.right || in.clear || tabs;
if (arrows && changed) return kCursor;
if ((in.left || in.right) && a.mode == kModeEdit && b.mode == kModeEdit) return kCursor; // (acts each press)
if (in.drag && a.value != b.value) return kCursor; // (a slider drawn along)
return kNone;
}
constexpr int kQueue = 16;
constexpr int64_t kStaleNs = 250000000; // (a quarter second)
struct Queue {
struct Entry { int sound; int64_t ns; };
Entry entries[kQueue] = {};
std::atomic<unsigned> head{0}, tail{0}; // (taken up to head, put up to tail)
/// The menu's thread: a sound asked for at now_ns. false when it is none of the seven, or the queue is full
bool put(int sound, int64_t now_ns) {
if (sound <= kNone || sound > kTabRight) return false;
const unsigned t = tail.load(std::memory_order_relaxed);
if (t - head.load(std::memory_order_acquire) >= kQueue) return false;
entries[t % kQueue] = {sound, now_ns};
tail.store(t + 1, std::memory_order_release);
return true;
}
/// The host's sound thread: the oldest sound waiting that is not stale, 0 when there is none
int take(int64_t now_ns) {
for (;;) {
const unsigned h = head.load(std::memory_order_relaxed);
if (h == tail.load(std::memory_order_acquire)) return kNone;
const Entry e = entries[h % kQueue];
head.store(h + 1, std::memory_order_release);
if (now_ns - e.ns <= kStaleNs) return e.sound;
}
}
};
} // namespace menu_sound

135
gpu/vk/mesh_ahead.h Normal file
View File

@@ -0,0 +1,135 @@
// (Triforce) The indirect mesh draws of a pass with their records prepared at the first of them, after NX2navi's mesh_ahead
// (runtime/vk_exec_draw.c mesh_ring_entry, runtime/vk_exec.c op_linear and vk_exec_mesh_run).
//
// NVN's indirect mesh draw reads {count, first} and Vulkan's {x, y, z}: each one's records are copied into a ring as {count, 1, 1}
// before it (vk_draw.cpp mesh_indirect_convert). A copy cannot be recorded inside a pass, so every indirect mesh draw ended its pass
// and the next draw began it again behind a barrier, three barriers and a fill and a copy between (OoT's foliage, grass and leaves:
// about 16 a frame at the Market, 45 in NX2navi's village). Now the first indirect mesh draw of a run copies the records of the ones
// after it as well, in the same barriers; those find their entry and stay in the pass. A run is the indirect mesh draws after it with
// only ops between that store state, draw, or reach the CPU alone (the counter reports, which the replay writes as it records, and the
// render conditions, which it reads): nothing between runs outside a pass or writes memory where the GPU could, so no work of the
// game's between can write the records after they were copied (the culling compute that writes them is a dispatch, which ends the run;
// a draw's shader writes a later draw's records only with the game's barrier between, which ends it too, as on the console, where the
// GPU's front end reads them early). An entry is taken only by the draw it was copied for: its op, the same record address, count,
// stride and word; the replay clears the table at any other op (and at each command buffer's start), and a draw the replay did not
// draw (its render condition) leaves its entry unused. At most kMax draws and kMaxBytes of ring a run, so the ring (4 MB) never comes
// round inside one. RX_VK_MESH_AHEAD=0: each draw copies its own, as before.
//
// Apart from Vulkan and the stream's op numbers so that tests/test_mesh_ahead.py can run the scan and the table over authored streams.
#pragma once
#include <cstdint>
#include <cstring>
namespace mkwvk::mesh_ahead {
constexpr int kMax = 32; // (the draws after the first whose records it copies, at most)
constexpr int kMaxOps = 2048; // (the ops looked at after it, at most: NX2navi's MESH_RUN_OPS)
constexpr uint64_t kMaxBytes = 1ull << 20; // (a run's ring bytes, at most: a quarter of vk_draw.cpp's kMeshRing)
/// RX_VK_MESH_AHEAD: on unless "0"
inline bool on(const char* env) {
return !(env && env[0] == '0' && !env[1]);
}
/// What an indirect mesh draw's conversion copies: its op in the stream (where the replay meets it), its records' address, how many,
/// their stride and the word that holds the count
struct Draw {
const uint8_t* op = nullptr;
uint64_t indirect = 0;
uint32_t n = 0, stride = 0, word = 0;
};
inline bool same(const Draw& a, const Draw& b) {
return a.op == b.op && a.indirect == b.indirect && a.n == b.n && a.stride == b.stride && a.word == b.word;
}
/// What the conversion copies for an indirect mesh draw of the stream's max and stride (the game's ints): its records, 1 to 4096 of
/// them (as many as the draw may draw), 8 bytes apart when the stride is not given
inline Draw draw_of(const uint8_t* op, uint64_t indirect, int32_t max, int32_t stride, uint32_t word) {
Draw d;
d.op = op, d.indirect = indirect, d.word = word;
d.n = (uint32_t)(max < 1 ? 1 : max > 4096 ? 4096 : max);
d.stride = stride > 0 ? (uint32_t)stride : 8u;
return d;
}
/// A draw's ring bytes (vk_draw.cpp's: {count, 1, 1} a record, 16-aligned)
inline uint64_t ring_bytes(uint32_t n) {
return ((uint64_t)n * 12 + 15) & ~15ull;
}
/// The ops after the one at `at` (records {uint32 op, uint32 size}, sizes 8-aligned), up to `end` (null: none): the indirect mesh draws
/// among them while every op between belongs in a run, into out, at most max; at most kMaxOps ops looked at. kind(op, record): 0 the
/// run ends there (the stream's end too), 1 it may be in a run, 2 an indirect mesh draw (in the run, and collected)
template <class K>
int scan(const uint8_t* at, const uint8_t* end, const uint8_t** out, int max, K kind) {
uint32_t h[2];
std::memcpy(h, at, sizeof h);
if (h[1] < 8 || (h[1] & 7)) return 0;
const uint8_t* p = at + h[1];
int n = 0;
for (int ops = 0; ops < kMaxOps && n < max && p != end; ops++) {
std::memcpy(h, p, sizeof h);
if (h[1] < 8 || (h[1] & 7)) break;
const int k = kind(h[0], p);
if (!k) break;
if (k == 2) out[n++] = p;
p += h[1];
}
return n;
}
/// How many of the run's draws after the first get entries: in order, while the ring bytes of the first and theirs stay within kMaxBytes
/// (and no more than the table holds)
inline int fit(const Draw* next, int n, uint32_t first_n) {
if (n > kMax) n = kMax;
uint64_t bytes = ring_bytes(first_n);
int k = 0;
while (k < n && bytes + ring_bytes(next[k].n) <= kMaxBytes) bytes += ring_bytes(next[k].n), k++;
return k;
}
/// What the replay knows of the indirect mesh draw it is about to draw (set before draw(), vk_replay.cpp; taken by the conversion):
/// its op, and the draws after it in its run (none when they are in the table already)
struct Pending {
struct Next { // (as the stream has it: draw_of makes the conversion's numbers)
const uint8_t* op;
uint64_t indirect;
int32_t max, stride;
};
const uint8_t* op = nullptr;
Next next[kMax];
int n = 0;
};
/// The records copied ahead: the run's draws after its first, in order, each with its ring entry
struct Table {
struct Entry {
Draw d;
uint64_t at = 0;
};
Entry e[kMax];
int n = 0, i = 0; // (entries, and the first not taken)
void clear() { n = i = 0; }
bool holds(const uint8_t* op) const {
for (int k = i; k < n; k++)
if (e[k].d.op == op) return true;
return false;
}
/// The ring entry copied for d (its op, and the same records): true, and the entries before it are let go (draws the replay did not
/// draw); false when there is none or it does not match, and the table is cleared
bool take(const Draw& d, uint64_t* at) {
for (int k = i; k < n; k++)
if (e[k].d.op == d.op) {
if (!same(e[k].d, d)) break;
*at = e[k].at;
i = k + 1;
return true;
}
clear();
return false;
}
};
inline thread_local Pending t_pending;
inline thread_local Table t_table;
} // namespace mkwvk::mesh_ahead

112
gpu/vk/mod_cheat_rows.h Normal file
View File

@@ -0,0 +1,112 @@
// (Triforce) The System tab's rows for the cheats of the mods' cheat files: what the host names (host/mod_cheats.c), what
// the rows show, and what goes back to the host when the player switches one (vk_api.h HOS_OPT_MODCHEAT_*).
//
// The cheats are the host's: it reads the files at the start and runs the ones that are on, once a game frame. The menu
// has a row for each, with the file's own name: the row table is fixed, so it holds kSlots rows for them that are not
// shown until the host has named its cheats, and a file's cheats past kSlots have no row (the host's log says so). A
// name arrives a byte a call, as the save rows' lines do, with the line the row shows under its help after a line feed.
// A row is On | Off like the menu's toggles; a master code and a cheat that cannot run here have a row that cannot be
// changed. Which cheats are on is kept by the host (triforce_cheats.txt), not in the options file.
//
// The host's calls (text, add, done, take) come on the game's thread, the rows are drawn and pressed on the renderer's:
// names and lines are written before count says the rows are there and never after; the switches are atomics.
//
// No ImGui and no Vulkan: tests/test_mod_cheat_rows.py compiles it alone.
#pragma once
#include <atomic>
#include <cstdint>
#include <cstring>
namespace mod_cheat_rows {
constexpr int kSlots = 96;
constexpr int kName = 80; // (a cheat's name is at most 63 bytes; " (cheat)" after one that is a row's of the menu's own)
constexpr int kLine = 200; // (the line under the row's help)
constexpr int kOn = 1, kAlways = 2, kRefused = 4; // (vk_api.h HOS_MODCHEAT_*)
struct List {
char names[kSlots][kName] = {};
char lines[kSlots][kLine] = {};
unsigned char bits[kSlots] = {};
std::atomic<int> on[kSlots] = {};
std::atomic<int> count{0}; // (the rows shown: 0 until the host's list is whole)
// the host's side
char in[kName + kLine] = {};
int in_n = 0, added = 0;
int told[kSlots] = {}; // (what the host knows of each switch)
/// HOS_OPT_MODCHEAT_TEXT: the next byte of the row being named
void text(int byte) {
if (byte > 0 && byte < 256 && in_n + 1 < (int)sizeof in) in[in_n++] = (char)byte;
}
/// HOS_OPT_MODCHEAT_ADD: the bytes so far are the next row's name and, after a line feed, its line. taken: names a
/// row may not have (rows of the menu's own that act by their name): such a cheat is shown with " (cheat)" after it
void add(int row_bits, const char* const* taken = nullptr, int n_taken = 0) {
in[in_n] = 0;
in_n = 0;
if (count.load(std::memory_order_acquire) || added >= kSlots) return; // (shown already, or no row left)
const int slot = added++;
char* feed = strchr(in, '\n');
if (feed) *feed = 0;
int n = (int)strlen(in);
if (n > 63) { // (not in the middle of a letter)
n = 63;
while (n && ((unsigned char)in[n] & 0xC0) == 0x80) n--;
}
memcpy(names[slot], in, (size_t)n);
names[slot][n] = 0;
if (!n) strcpy(names[slot], "(no name)");
for (int i = 0; i < n_taken; i++)
if (!strcmp(names[slot], taken[i])) strcat(names[slot], " (cheat)");
if (feed) {
size_t ln = strlen(feed + 1);
if (ln > kLine - 1) ln = kLine - 1;
memcpy(lines[slot], feed + 1, ln);
lines[slot][ln] = 0;
for (char* p = lines[slot]; *p; p++)
if (*p == '\n') *p = ' ';
}
bits[slot] = (unsigned char)(row_bits & (kOn | kAlways | kRefused));
const int is_on = (row_bits & kAlways) || ((row_bits & kOn) && !(row_bits & kRefused));
on[slot].store(is_on, std::memory_order_relaxed);
told[slot] = is_on;
}
/// HOS_OPT_MODCHEAT_COUNT: the list is whole: the rows are shown from now on
void done(int n) {
in_n = 0;
if (count.load(std::memory_order_acquire)) return;
count.store(n < 0 ? 0 : n < added ? n : added, std::memory_order_release);
}
int shown() const { return count.load(std::memory_order_acquire); }
/// A row that cannot be changed: a master code, or a cheat that cannot run here
bool fixed(int slot) const { return slot < 0 || slot >= kSlots || (bits[slot] & (kAlways | kRefused)); }
bool is_on(int slot) const { return slot >= 0 && slot < kSlots && on[slot].load(std::memory_order_relaxed); }
/// What a row that cannot be changed shows in its value's place
const char* fixed_text(int slot) const { return slot >= 0 && slot < kSlots && (bits[slot] & kRefused) ? "Not supported" : "Always on"; }
const char* line(int slot) const { return slot >= 0 && slot < kSlots ? lines[slot] : ""; }
/// The slot of a row by its name's address (the rows' names are this list's own), -1 for any other row
int slot_of(const char* name) const {
const uintptr_t first = (uintptr_t)names[0], at = (uintptr_t)name;
return at >= first && at - first < sizeof names ? (int)((at - first) / kName) : -1;
}
/// A press on a row: Left is On and Right is Off, as they are drawn; confirm flips. true when the switch changed
bool press(int slot, bool left, bool right, bool confirm) {
if (slot < 0 || slot >= shown() || fixed(slot)) return false;
const int was = on[slot].load(std::memory_order_relaxed);
const int now = confirm ? !was : left ? 1 : right ? 0 : was;
if (now == was) return false;
on[slot].store(now, std::memory_order_relaxed);
return true;
}
/// HOS_OPT_MODCHEAT_ASK: the next switch that is not as the host knows it, (slot + 1) * 2 + its state; 0 none
int take() {
const int n = shown();
for (int slot = 0; slot < n; slot++) {
const int now = on[slot].load(std::memory_order_relaxed);
if (fixed(slot) || now == told[slot]) continue;
told[slot] = now;
return (slot + 1) * 2 + (now ? 1 : 0);
}
return 0;
}
};
} // namespace mod_cheat_rows

454
gpu/vk/occ_answer.h Normal file
View File

@@ -0,0 +1,454 @@
// (Triforce, 11-17) Samples-passed reports answered from occlusion queries (RX_VK_OCC_QUERY=1, vk_replay.cpp "occlusion
// queries"): the bookkeeping, header only so that tests/test_occ_answer.py checks it without a device.
//
// The game resets NVN's samples-passed counter, draws an object's proxy and reports the count (its render job f_80084de0,
// the report from near lr 0x800876dc); its CPU reads the count back once it has waited for the frame, and does not draw an
// object whose count is 0. The replay writes every report as it passes it, in stream order, long before the GPU has run
// the draws it counts, and has always written 4096 there (everything visible). It still does: that placeholder is what
// anything reading the record early sees, as before. With RX_VK_OCC_QUERY=1 the draws between a reset and its report are
// also counted by a Vulkan occlusion query, and once the recording that holds them is done on the GPU the count is
// written into the same record, over the placeholder, as NX2navi writes it at completion. The game's report records
// rotate (a ring of them: an address stands for another object at its next report), so a count is written only into the
// record of the report it counted, and only while no later report has been written there (Writers), and only over the
// value the replay left in the word (a compare-and-write: the game's CPU re-arms its records between frames). A wrong 0
// hides an object the player should see, so a count goes in only when it is whole; otherwise the record keeps its 4096.
// The game also predicates draws on these words (render conditions, NOP_RENDER_COND): as NX2navi decides them
// (runtime/vk_exec_xfer.c cond_enabled), a word whose newest report is a samples-passed one with its count still to come is
// unknown, and the condition renders; a word whose count was written is read from memory, where a real 0 can skip the draw.
//
// Counter one counting, from a reset to its report, over the queries its draws began: the replay's (one per NVN
// queue's state, under replay_mu) or a parallel replay's segment's (the helper thread's: a segment starts with
// none, what came before it is another thread's). Matched to its recording by the recording's serial (each
// begin of a slot takes a new one), never by the command buffer's handle, which a slot reuses
// Writers per report address, the newest report written there (its place in the stream, the value it left, whether its
// count is pending, written or not to come): what a count read back may be written over, what a condition reads
// Cond a render condition's answer from its two words' sources (cond_decide)
// complete a done recording's counts into their records (through the caller's compare-and-write), or why not
// Stats what happened, for the line every 600 presents
// Sites the first 64 report addresses with their counts, for the lab to find the one RX_VK_OCC_QUERY_SKIP keeps at 4096
#pragma once
#include <algorithm>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <unordered_map>
#include <unordered_set>
#include <vector>
namespace mkwvk::occ {
constexpr uint64_t kVisible = 4096; // (the placeholder every report gets at replay; it stays unless its whole count comes)
constexpr int64_t kLateAge = 8; // (presents: a count read back this long after its report is not written)
constexpr int64_t kKeepAge = 16; // (presents: a report address not reported again for longer leaves Writers)
/// RX_VK_OCC_QUERY_SKIP=<addr>[,<addr>...]: report addresses (hex with 0x, or decimal) that keep the fixed answer.
/// Separated by ',', ';' or spaces; anything else ends the list
inline std::vector<uint64_t> parse_addrs(const char* e) {
std::vector<uint64_t> v;
while (e && *e) {
while (*e == ',' || *e == ';' || *e == ' ') e++;
if (!*e) break;
char* end = nullptr;
const unsigned long long a = strtoull(e, &end, 0);
if (end == e) break;
v.push_back((uint64_t)a);
e = end;
}
std::sort(v.begin(), v.end());
v.erase(std::unique(v.begin(), v.end()), v.end());
return v;
}
inline bool listed(const std::vector<uint64_t>& sorted, uint64_t addr) {
return std::binary_search(sorted.begin(), sorted.end(), addr);
}
/// Whether a knob list (RX_VK_LAB_AB's knobs, each maybe several names joined by '+') names this switch anywhere
inline bool knob_names(const char* knob, const char* name) {
const size_t len = strlen(name);
for (const char* p = knob; p && *p;) {
const char* plus = strchr(p, '+');
const size_t n = plus ? (size_t)(plus - p) : strlen(p);
if (n == len && !strncmp(p, name, len)) return true;
p = plus ? plus + 1 : nullptr;
}
return false;
}
/// The samples of the queries [first, first + n) of a block read with VK_QUERY_RESULT_64_BIT |
/// VK_QUERY_RESULT_WITH_AVAILABILITY_BIT ({samples, available} per query). False when one is not available
inline bool sum_range(const uint64_t* pairs, uint32_t first, uint32_t n, uint64_t* out) {
uint64_t sum = 0;
for (uint32_t i = first; i < first + n; i++) {
if (!pairs[2 * i + 1]) return false;
sum += pairs[2 * i];
}
*out = sum;
return true;
}
/// A measured report: its address, its queries in its recording's block, its place in the stream (the same in the dry pass
/// that wrote its placeholder and on the thread that recorded its draws) and the present it was recorded at
struct Rep {
uint64_t addr = 0;
uint32_t first = 0, n = 0;
uint64_t pos = 0;
int64_t present = 0;
};
/// One samples-passed counting. The replay calls reset() at a SAMPLES_PASSED reset, draw_enter() as a draw goes into draw(),
/// begin() where a query may begin (the draw's pass open; may_begin() first, cheaply), draw_issued() for each draw recorded,
/// report() at the report. A query is begun inside a pass and ended at the pass's end or the report, so the queries of one
/// count follow one another in the block of the recording that holds them (first.. the block's next). The recording is
/// known by its serial (a new one each time a slot is begun): a draw or report in another recording, or in the same slot
/// begun again, loses the count
struct Counter {
enum State { kUnknown, kCounting, kReported };
enum Outcome { kMeasured, kNoReset, kIncomplete, kDouble };
State state = kUnknown;
int slot = -1; // (the recording slot whose block the queries are in; -1: none, the count is lost)
uint64_t serial = 0; // (that slot's recording: occ_slot_begin's serial; 0: none)
uint32_t first = 0; // (the count's first query in the block)
int open = -1; // (the query begun and not ended, its index in the block)
bool lost = false; // (a draw in another recording, or the block full)
uint32_t ops = 0, issued = 0; // (draws that went into draw() while counting, and those it recorded)
enum Why { kNone, kSkipped, kOtherCmd, kFull };
Why why = kNone;
/// A parallel replay's segment starts (or ends), or the replay's counting is given up: nothing is counting
void segment() { *this = Counter{}; }
bool counting(bool keep) const { return state == kCounting || (state == kReported && keep); }
/// A reset: counting from the block's next query of slot's recording serial_ (slot -1 or serial 0: the command buffer
/// is no recording's, the count is lost)
void reset(int slot_, uint64_t serial_, uint32_t next) {
const bool none = slot_ < 0 || !serial_;
state = kCounting, slot = none ? -1 : slot_, serial = none ? 0 : serial_, first = next, open = -1, lost = none;
ops = issued = 0;
why = none ? kOtherCmd : kNone;
}
void draw_enter(bool keep) {
if (counting(keep)) ops++;
}
void draw_issued(bool keep) {
if (counting(keep)) issued++;
}
/// Whether a query might begin at a draw (before the recording's serial is looked up)
bool may_begin(bool keep) const { return counting(keep) && open < 0 && !lost; }
/// A draw in an open pass of the recording serial_: true when a query is to begin (the caller takes the block's next
/// index and sets open); a draw in another recording loses the count
bool begin(bool keep, uint64_t serial_) {
if (!may_begin(keep)) return false;
if (serial_ != serial) {
lost = true, why = kOtherCmd;
return false;
}
return true;
}
void full() { lost = true, why = kFull; }
/// The report, in the recording serial_ (the open query already ended by the caller): what it measured. *keep_now: the
/// game reported twice without a reset, which NVN counts on from the reset (cumulative): from then on the counting goes
/// on past a report. A report in another recording than the reset's is incomplete
Outcome report(bool keep, uint64_t serial_, uint32_t next, Rep* rep, bool* keep_now) {
*keep_now = false;
if (state == kUnknown) return kNoReset;
if (state == kReported && !keep) { // (the draws since the first report went uncounted: none of this count is whole)
*keep_now = true;
lost = true, why = kSkipped;
return kDouble;
}
if (!lost && serial_ != serial) lost = true, why = kOtherCmd;
const bool complete = !lost && ops == issued;
if (!complete && !lost) why = kSkipped;
state = kReported;
if (!complete) return kIncomplete;
rep->first = first, rep->n = next - first;
return kMeasured;
}
};
/// Per report address, the newest samples-passed report the replay wrote there (in stream order: placed) and where its
/// count stands: NX2navi's Pending table (runtime/vk_exec_xfer.c: seq, written, unknown, "the last report to addr
/// decides"). A count read back goes into its record only while its report is the newest there (the game's records rotate,
/// and a later report at the same address is another object's, whose placeholder stays), and a render condition reads a
/// word whose newest report's count is still to come as unknown (Src kPendingSrc: renders, NX2navi's cond_enabled "rx ==
/// 2"). A record rewritten in the stream after its report (a small buffer clear the replay applies in stream order, or a
/// report of another counter type) leaves the table: its count is not written there, and a condition reads memory, as before
class Writers {
public:
enum Verdict { kWrite, kSuperseded, kUnknown, kFixed };
enum State : uint8_t {
kPending, // (its count is to be written and has not been yet)
kWritten, // (its count is in the record: completion wrote it)
kFinal // (no count comes: skip list, knob off, a clear flag, not counted whole, lost, late, the CPU wrote it)
};
enum Src { kNone, kPendingSrc, kWrittenSrc, kFinalSrc }; // (what a render condition reading the word finds)
struct Placed {
uint64_t pos = 0;
int64_t present = 0;
uint64_t value = 0; // (what the replay, or a count written since, put in the word: a count is written only over it)
State state = kFinal;
};
/// A report's placeholder `value` written at the replay, at pos in the stream (the newest there from now on). write:
/// its count is to be written once read back (not on RX_VK_OCC_QUERY_SKIP, the lab knob on, not a clear flag)
void placed(uint64_t addr, uint64_t pos, int64_t present, bool write, uint64_t value) {
m_[addr] = Placed{pos, present, value, write ? kPending : kFinal};
page_[page(addr)] = 1, page_[page(addr + 7)] = 1;
}
bool is_report(uint64_t addr) const { return m_.count(addr) != 0; }
/// What a count read back for the report at (addr, pos) may do. (Two reports at one address with no draw between share
/// a place; in stream order the last one's count is written last, over the first one's)
Verdict verdict(uint64_t addr, uint64_t pos) const {
auto it = m_.find(addr);
if (it == m_.end()) return kUnknown; // (not reported for kKeepAge presents, or rewritten in the stream)
if (it->second.pos > pos) return kSuperseded; // (reported again since: another object's record now)
if (it->second.pos != pos) return kUnknown;
return it->second.state == kFinal ? kFixed : kWrite;
}
/// The value the report at addr left in its word (the placeholder, or the count written since)
uint64_t value(uint64_t addr) const {
auto it = m_.find(addr);
return it == m_.end() ? 0 : it->second.value;
}
/// The count of the report at (addr, pos) is in its record now
void wrote(uint64_t addr, uint64_t pos, uint64_t count) {
auto it = m_.find(addr);
if (it != m_.end() && it->second.pos == pos) it->second.state = kWritten, it->second.value = count;
}
/// No count comes for the report at (addr, pos) (not counted whole, lost, late, never sent, the CPU wrote the record):
/// a condition reads its word from memory from now on, as before
void settle(uint64_t addr, uint64_t pos) {
auto it = m_.find(addr);
if (it != m_.end() && it->second.pos == pos && it->second.state == kPending) it->second.state = kFinal;
}
/// A report of another counter type at addr (a timestamp: base behaviour) is the newest there now
bool erase(uint64_t addr) { return m_.erase(addr) != 0; }
/// A small buffer clear of [lo, hi) applied in stream order (NOP_CLEAR_BUF): the records whose count word it reaches leave
/// the table (NX2navi's clear_small and clr_mask, by whole records: memory holds the clear's value from now on, and the
/// count of a report before it is not written over it). Report addresses are 4-byte aligned at least. Returns how many
int cleared(uint64_t lo, uint64_t hi) {
if (hi <= lo || m_.empty() || (!page_[page(lo)] && !page_[page(hi - 1)])) return 0;
int n = 0;
for (uint64_t k = ((lo >= 7 ? lo - 7 : 0) + 3) & ~3ull; k < hi; k += 4) n += (int)m_.erase(k);
return n;
}
/// What a render condition finds in the word at addr: no samples-passed report (memory, as before), its count pending
/// (unknown: renders), written (memory holds it), or none to come (memory). A count pending longer than kLateAge
/// presents is not written any more (complete: late): memory
Src word(uint64_t addr, int64_t now) const {
auto it = m_.find(addr);
if (it == m_.end()) return kNone;
switch (it->second.state) {
case kPending: return now - it->second.present > kLateAge ? kFinalSrc : kPendingSrc;
case kWritten: return kWrittenSrc;
case kFinal: break;
}
return kFinalSrc;
}
/// The addresses not reported for more than `age` presents go (a count still to come for them is not written)
void prune(int64_t now, int64_t age) {
for (auto it = m_.begin(); it != m_.end();)
it = now - it->second.present > age ? m_.erase(it) : std::next(it);
}
size_t size() const { return m_.size(); }
private:
// (the pages a report went into, by a hash of the page, as NX2navi's rep_pages: a clear elsewhere looks nothing up)
static uint32_t page(uint64_t a) { return (uint32_t)(((a >> 12) * 0x9E3779B97F4A7C15ull) >> 48); }
std::unordered_map<uint64_t, Placed> m_;
std::vector<uint8_t> page_ = std::vector<uint8_t>(1u << 16);
};
/// A render condition (NOP_RENDER_COND: mode 0 renders when the counter words at addr and addr + 16 are equal, mode 1 when
/// they differ) over the words' sources (Writers::word), after NX2navi's cond_enabled: a samples-passed count not done yet
/// renders; a written one, as any other word, is read from memory (base_enabled: the replay's answer from memory). active:
/// RX_VK_OCC_QUERY=1 or the lab knob's phase (else the answer is the base one, whatever the table says); cond_on: false
/// with RX_VK_OCC_COND=0, every condition on a samples-passed word renders
struct Cond {
enum Kind { kNotSamples, kBase, kUnknown, kForced, kWrittenOn, kWrittenOff, kMemory };
bool enabled;
Kind kind;
};
inline Cond cond_decide(bool base_enabled, Writers::Src sx, Writers::Src sy, bool active, bool cond_on) {
if (sx == Writers::kNone && sy == Writers::kNone) return {base_enabled, Cond::kNotSamples};
if (!active) return {base_enabled, Cond::kBase};
if (!cond_on) return {true, Cond::kForced};
if (sx == Writers::kPendingSrc || sy == Writers::kPendingSrc) return {true, Cond::kUnknown};
if (sx == Writers::kWrittenSrc || sy == Writers::kWrittenSrc)
return {base_enabled, base_enabled ? Cond::kWrittenOn : Cond::kWrittenOff};
return {base_enabled, Cond::kMemory};
}
inline const char* src_name(Writers::Src s) {
return s == Writers::kPendingSrc ? "pending" : s == Writers::kWrittenSrc ? "written" : s == Writers::kFinalSrc ? "memory, no count to come" : "memory";
}
/// What happened since the last line (under the caller's lock)
struct Stats {
enum At { kAtWait, kAtBegin, kAtPresent, kAtReuse };
// at the replay
int64_t reports = 0; // samples-passed reports written (the placeholder, 4096 as before)
int64_t in_segments = 0; // of them by the dry pass of a parallel replay
int64_t skipped = 0; // on RX_VK_OCC_QUERY_SKIP: no count to be written
int64_t fixed = 0; // the lab knob occq off in this phase: no count to be written
int64_t flagged = 0; // the count word a clear flag (NVN_COUNTER_WRITTEN): left alone
int64_t rewritten = 0; // records rewritten in the stream after their report (a small clear, another counter type)
int64_t measured = 0, no_reset = 0, incomplete = 0, doubles = 0; // counted whole by queries, and those not
int64_t inc_skipped = 0, inc_other_cmd = 0, inc_full = 0; // why not whole
// render conditions that read a samples-passed report's word (Cond::Kind), and the work they skipped
int64_t conds = 0, cond_base = 0, cond_unknown = 0, cond_forced = 0, cond_written_on = 0, cond_written_off = 0, cond_memory = 0;
int64_t draws_skipped = 0, clears_skipped = 0;
// at completion
int64_t written = 0, zeros = 0; // counts written into their records (of them 0: the object hidden)
int64_t lost = 0; // a query not available once its recording was done: 4096 stays
int64_t dropped = 0; // their recording never went to the GPU
int64_t late = 0; // read back more than kLateAge presents after the report: not written
int64_t superseded = 0; // the record reported again first (the ring came round): not written
int64_t unknown = 0; // the record no longer known (pruned, or rewritten in the stream): not written
int64_t kept_fixed = 0; // its report placed with no count to be written (skip list, knob off)
int64_t rearmed = 0; // the word no longer held the replay's value (the CPU wrote the record since): not written
int64_t lat_sum = 0, lat_ge1 = 0, lat_ge2 = 0; // presents from the report to the write, over written
int64_t at[4] = {}; // recordings read: at a game's fence wait, a recording's begin, a present, the slot's reuse
int64_t addrs = 0; // distinct report addresses since the last line
void cond(Cond::Kind k) {
if (k == Cond::kNotSamples) return;
conds++;
(k == Cond::kBase ? cond_base : k == Cond::kUnknown ? cond_unknown : k == Cond::kForced ? cond_forced
: k == Cond::kWrittenOn ? cond_written_on : k == Cond::kWrittenOff ? cond_written_off : cond_memory)++;
}
/// The line, into out
int line(char* out, size_t size, int64_t presents, size_t tracked) const {
return snprintf(out, size,
"vk: occlusion queries (RX_VK_OCC_QUERY), %lld presents: %lld samples-passed reports, 4096 written at "
"the replay as before (%lld by a parallel replay's dry pass; %lld skipped (RX_VK_OCC_QUERY_SKIP), %lld "
"lab knob occq off, %lld a clear flag; %lld records rewritten in the stream after their report); %lld "
"counted whole, %lld without their reset on the recording thread, %lld incomplete (%lld draws left out, "
"%lld over two recordings, %lld block full), %lld reported twice without a reset. At completion: %lld "
"counts written (%lld of them 0), %lld lost (not available), %lld never sent, %lld late, %lld superseded "
"(the record reported again first), %lld no longer known, %lld kept 4096 (skip list or knob off), %lld "
"not written: the record no longer held the replay's value; mean latency %.2f presents, %lld written "
"after the next present, %lld after two; recordings read at: %lld a game's fence wait, %lld a "
"recording's begin, %lld a present, %lld the slot's reuse. Render conditions on samples-passed words: "
"%lld (%lld as before: knob occq off; %lld rendered: a count pending (unknown); %lld rendered: "
"RX_VK_OCC_COND=0; %lld rendered from a written count; %lld skipped from a written count; %lld from "
"memory, no count to come); skipped by them: %lld draws and dispatches, %lld clears; %lld addresses "
"this period, %zu tracked",
(long long)presents, (long long)reports, (long long)in_segments, (long long)skipped, (long long)fixed,
(long long)flagged, (long long)rewritten, (long long)measured, (long long)no_reset,
(long long)incomplete, (long long)inc_skipped, (long long)inc_other_cmd, (long long)inc_full,
(long long)doubles, (long long)written, (long long)zeros, (long long)lost, (long long)dropped,
(long long)late, (long long)superseded, (long long)unknown, (long long)kept_fixed, (long long)rearmed,
written ? (double)lat_sum / (double)written : 0.0, (long long)lat_ge1, (long long)lat_ge2,
(long long)at[kAtWait], (long long)at[kAtBegin], (long long)at[kAtPresent], (long long)at[kAtReuse],
(long long)conds, (long long)cond_base, (long long)cond_unknown, (long long)cond_forced,
(long long)cond_written_on, (long long)cond_written_off, (long long)cond_memory,
(long long)draws_skipped, (long long)clears_skipped, (long long)addrs, tracked);
}
};
/// A done recording's reports (in stream order) into their records. pairs: its block read with VK_QUERY_RESULT_64_BIT |
/// VK_QUERY_RESULT_WITH_AVAILABILITY_BIT over the `used` queries it took (readable false: the read failed). A count is
/// written only when every query of it is available, it is read back at most kLateAge presents after its report, and its
/// report is still the newest at its address with a count to be written there (Writers::verdict); then write(addr, expect,
/// count) puts it in the record only while the word still holds expect, what the replay (or an earlier count of the same
/// place) left there, and says whether it did: the game's CPU re-arms its records between frames, and a count read back
/// late must not land over that (NX2navi treats a report as pending until its value is written; here the write itself is
/// a compare-and-write). Anything else leaves the record as it is, and the report's count is settled (no count comes: a
/// condition reads memory). measured(addr, count): each whole count read back (the sites)
template <class Write, class Measured>
void complete(const std::vector<Rep>& reps, const uint64_t* pairs, uint32_t used, bool readable, Writers& w, int64_t now,
Stats& st, Write&& write, Measured&& measured) {
for (const Rep& p : reps) {
uint64_t sum = 0;
const bool inside = p.n <= used && p.first <= used - p.n; // (its queries in the block the recording took)
if (!readable || !inside || !sum_range(pairs, p.first, p.n, &sum)) {
st.lost++;
w.settle(p.addr, p.pos);
continue;
}
measured(p.addr, sum);
if (now - p.present > kLateAge) {
st.late++;
w.settle(p.addr, p.pos);
continue;
}
switch (w.verdict(p.addr, p.pos)) {
case Writers::kSuperseded: st.superseded++; continue;
case Writers::kUnknown: st.unknown++; continue;
case Writers::kFixed: st.kept_fixed++; continue;
case Writers::kWrite: break;
}
if (!write(p.addr, w.value(p.addr), sum)) {
st.rearmed++;
w.settle(p.addr, p.pos);
continue;
}
w.wrote(p.addr, p.pos, sum);
const int64_t lat = std::max<int64_t>(0, now - p.present);
st.written++, st.zeros += sum ? 0 : 1, st.lat_sum += lat, st.lat_ge1 += lat >= 1, st.lat_ge2 += lat >= 2;
}
}
/// A recording's reports that never went to the GPU: no count comes for them
inline void dropped(const std::vector<Rep>& reps, Writers& w, Stats& st) {
for (const Rep& p : reps) w.settle(p.addr, p.pos);
st.dropped += (int64_t)reps.size();
}
/// The first 64 report addresses the replay saw, with how often each was reported, what its queries counted and how many
/// counts went into it
struct Sites {
struct Site {
uint64_t addr = 0;
int64_t reports = 0, measured = 0, written = 0;
uint64_t last = 0, lo = UINT64_MAX, hi = 0;
};
static constexpr int kMost = 64;
Site s[kMost];
int n = 0;
Site* at(uint64_t addr, bool add) {
for (int i = 0; i < n; i++)
if (s[i].addr == addr) return &s[i];
if (!add || n == kMost) return nullptr;
s[n].addr = addr;
return &s[n++];
}
void reported(uint64_t addr) {
if (Site* x = at(addr, true)) x->reports++;
}
void measured(uint64_t addr, uint64_t count) {
if (Site* x = at(addr, false)) x->measured++, x->last = count, x->lo = std::min(x->lo, count), x->hi = std::max(x->hi, count);
}
void wrote(uint64_t addr) {
if (Site* x = at(addr, false)) x->written++;
}
/// Lines of eight sites each: "addr reports/measured last [lo..hi] written"
template <class Log>
void log(Log&& out) const {
for (int b = 0; b < n; b += 8) {
char line[1024];
int k = snprintf(line, sizeof line, "vk: occlusion query sites %d-%d of %d (address: reports/measured, last count "
"[lowest..highest], counts written):", b + 1, std::min(n, b + 8), n);
for (int i = b; i < std::min(n, b + 8) && k < (int)sizeof line - 96; i++) {
const Site& x = s[i];
k += snprintf(line + k, sizeof line - k, " %#llx: %lld/%lld %llu [%llu..%llu] %lld;", (unsigned long long)x.addr,
(long long)x.reports, (long long)x.measured, (unsigned long long)x.last,
(unsigned long long)(x.measured ? x.lo : 0), (unsigned long long)x.hi, (long long)x.written);
}
out(line);
}
}
};
/// The distinct addresses reported since the last line (bounded)
struct Distinct {
std::unordered_set<uint64_t> seen;
void note(uint64_t addr) {
if (seen.size() < 65536) seen.insert(addr);
}
int64_t take() {
const int64_t n = (int64_t)seen.size();
seen.clear();
return n;
}
};
} // namespace mkwvk::occ

138
gpu/vk/oot_debug_link.h Normal file
View File

@@ -0,0 +1,138 @@
/* (Triforce, OoT debug tools) The Debug tab's Game flags and Cutscenes pages (gpu/vk/vk_menu.cpp, gpu/vk/debug_names.h) and
the game's side (host/hos_main.c): the lines that go between them, a byte a call (HOS_OPT_DEBUG_ASK one way,
HOS_OPT_DEBUG_TEXT and HOS_OPT_DEBUG_SAID the other: gpu/vk/vk_api.h). C and C++; tests/test_debug_link.py runs it.
The game knows a flag (a Bool of its GameData) by the MurmurHash3 of its name alone, and an event by the name of its
file. Neither list is in this repository: the pages read the names from the player's own game files, in memory.
The pages ask:
W <hash> <hash> ... the flags the page shows now, at most OOT_WATCH_MAX, eight hex digits each: their states are
sent twice a second from then on (a W with none: no more)
F <hash> <0|1> the flag cleared or set, through the game's own setter
E <name> the event of that name started, as one of the game's actors starts it
The game's side says:
OOT_SAID_FLAGS "<sum> <states>": the watch list's oot_watch_sum, then a character a flag in the list's order:
1 set, 0 not, - the save data has no such flag, ? no save is loaded
OOT_SAID_STATUS what became of an F or an E, in words, for the page's foot */
#ifndef OOT_DEBUG_LINK_H
#define OOT_DEBUG_LINK_H
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
enum { OOT_WATCH_MAX = 64, OOT_NAME_MAX = 95, OOT_LINE_MAX = 4 + 9 * OOT_WATCH_MAX };
enum { OOT_SAID_FLAGS = 1, OOT_SAID_STATUS = 2 };
/* MurmurHash3 (32-bit, seed 0) of a name's bytes, as the game's GameData getters and setters take it */
static inline uint32_t oot_name_hash(const char *text, size_t n) {
uint32_t h = 0;
size_t i = 0;
for (; i + 4 <= n; i += 4) {
uint32_t k = (uint32_t)(uint8_t)text[i] | (uint32_t)(uint8_t)text[i + 1] << 8 | (uint32_t)(uint8_t)text[i + 2] << 16 |
(uint32_t)(uint8_t)text[i + 3] << 24;
k *= 0xcc9e2d51u, k = k << 15 | k >> 17, k *= 0x1b873593u;
h ^= k, h = h << 13 | h >> 19, h = h * 5 + 0xe6546b64u;
}
uint32_t k = 0;
if ((n & 3) == 3) k ^= (uint32_t)(uint8_t)text[i + 2] << 16;
if ((n & 3) >= 2) k ^= (uint32_t)(uint8_t)text[i + 1] << 8;
if (n & 3) k ^= (uint8_t)text[i], k *= 0xcc9e2d51u, k = k << 15 | k >> 17, k *= 0x1b873593u, h ^= k;
h ^= (uint32_t)n;
h ^= h >> 16, h *= 0x85ebca6bu, h ^= h >> 13, h *= 0xc2b2ae35u, h ^= h >> 16;
return h;
}
/* a character of a flag's or an event's name: the game's names are letters, digits, _ . and - */
static inline int oot_name_char(int ch) {
return (ch >= 'a' && ch <= 'z') || (ch >= 'A' && ch <= 'Z') || (ch >= '0' && ch <= '9') || ch == '_' || ch == '.' || ch == '-';
}
/* a name of 1 to OOT_NAME_MAX such characters */
static inline int oot_name_ok(const char *name) {
size_t n = 0;
while (name[n] && n <= OOT_NAME_MAX && oot_name_char((unsigned char)name[n])) n++;
return n > 0 && n <= OOT_NAME_MAX && !name[n];
}
/* what pairs a line of states with the list it answers: the flags' hashes in their order */
static inline uint32_t oot_watch_sum(const uint32_t *hashes, int n) {
uint32_t sum = 0x9e3779b9u + (uint32_t)n;
for (int i = 0; i < n; i++) sum = (sum << 5 | sum >> 27) ^ hashes[i];
return sum;
}
/* eight hex digits at *at, which moves past them: 1 when there were eight */
static inline int oot_hex8(const char **at, uint32_t *value) {
uint32_t v = 0;
for (int i = 0; i < 8; i++) {
const int ch = (unsigned char)(*at)[i];
const int digit = ch >= '0' && ch <= '9' ? ch - '0' : ch >= 'a' && ch <= 'f' ? ch - 'a' + 10 : -1;
if (digit < 0) return 0;
v = v << 4 | (uint32_t)digit;
}
*at += 8, *value = v;
return 1;
}
typedef struct OotAsk {
int kind; /* 'W', 'F' or 'E' */
uint32_t hash; /* F: the flag */
int value; /* F: 0 or 1 */
int count; /* W: how many */
uint32_t watch[OOT_WATCH_MAX]; /* W: the flags */
char name[OOT_NAME_MAX + 1]; /* E: the event */
} OotAsk;
/* A line of the pages taken apart: 1 when it is one of the three, whole and well formed (anything else is dropped) */
static inline int oot_ask_parse(const char *line, OotAsk *ask) {
memset(ask, 0, sizeof *ask);
const char *at = line + 1;
if (line[0] == 'W') {
while (*at) {
if (*at++ != ' ' || ask->count == OOT_WATCH_MAX || !oot_hex8(&at, &ask->watch[ask->count])) return 0;
ask->count++;
}
ask->kind = 'W';
return 1;
}
if (line[0] == 'F') {
if (*at++ != ' ' || !oot_hex8(&at, &ask->hash) || !ask->hash || *at++ != ' ' || (*at != '0' && *at != '1') || at[1]) return 0;
ask->value = *at == '1';
ask->kind = 'F';
return 1;
}
if (line[0] == 'E') {
if (*at++ != ' ' || !oot_name_ok(at)) return 0;
memcpy(ask->name, at, strlen(at) + 1);
ask->kind = 'E';
return 1;
}
return 0;
}
/* The pages' lines written (out of OOT_LINE_MAX at least): their length, 0 when the line cannot be made */
static inline int oot_ask_watch(char *out, size_t cap, const uint32_t *hashes, int n) {
if (n < 0 || n > OOT_WATCH_MAX || cap < 2 + 9 * (size_t)n) return 0;
int used = snprintf(out, cap, "W");
for (int i = 0; i < n; i++) used += snprintf(out + used, cap - (size_t)used, " %08x", (unsigned)hashes[i]);
return used;
}
static inline int oot_ask_flag(char *out, size_t cap, uint32_t hash, int value) {
return hash && cap >= 13 ? snprintf(out, cap, "F %08x %d", (unsigned)hash, value ? 1 : 0) : 0;
}
static inline int oot_ask_event(char *out, size_t cap, const char *name) {
return oot_name_ok(name) && cap >= strlen(name) + 3 ? snprintf(out, cap, "E %s", name) : 0;
}
/* A line of states read: 1 when it answers the list of that sum and has n states of the four characters; they are in
states then */
static inline int oot_states_parse(const char *line, uint32_t sum, int n, char *states) {
uint32_t said;
const char *at = line;
if (!oot_hex8(&at, &said) || said != sum || *at++ != ' ' || (int)strlen(at) != n) return 0;
for (int i = 0; i < n; i++)
if (at[i] != '0' && at[i] != '1' && at[i] != '-' && at[i] != '?') return 0;
memcpy(states, at, (size_t)n);
return 1;
}
#endif

60
gpu/vk/pace_rule.h Normal file
View File

@@ -0,0 +1,60 @@
// (Triforce) Who holds a frame of the game to the Game frame rate cap: RivaTuner Statistics Server, NVIDIA Reflex or
// the game's own wait, and what RivaTuner is asked to hold (gpu/vk/vk_pace.cpp, where the game's frame starts). Plain
// C with no system call, so that tests/test_rtss_limit.py runs every combination of rows through it.
//
// The owner's rule: RivaTuner is the game's limiter whenever it can hold the game. It holds presents, and it counts
// every one, the frames frame generation makes too. So what it can hold depends on where the presents come from:
//
// No frame generation: one present for each of the game's frames, on the replay's thread. RivaTuner's limit is the
// cap, and holding the presents holds the game through the present queue.
//
// The Frame generation row's fixed multipliers (2x to 8x, AMD FSR 3.1's 2x and AMD FSR 4's frame): the display thread
// (vk_framegen.cpp display_main) presents every frame of every set, the multiplier's number for each of the game's.
// RivaTuner's limit is the cap times the multiplier, and holding those presents holds the game: the display thread
// then takes the game's frames at the cap, and the replay's thread waits for it (framegen_record) as the game waits
// for the replay. The display thread's own pacing of the made frames has to yield meanwhile (pace_display_yields):
// it spaces the presents by the game's measured frame time, which under RivaTuner's hold is RivaTuner's own spacing
// read back; it can only ever fall behind that, never catch up, and each hitch would leave the game slower for good.
//
// The Frame generation row's Auto: a frame for each refresh of the screen, at that refresh's time, however many the
// game makes (a game frame is given none, one or several). The presents follow the screen, not the game, so no limit
// on them holds the game to its cap: RivaTuner is kept out (RTSS_OUT_AUTO) and the game's own wait holds the frames.
//
// NVIDIA DLSS frame generation: the game's one present goes into NVIDIA's Streamline, which shows the made frames
// and the game's own from inside its closed library, on a pace of its own that cannot be told to yield, and shows
// one frame for each of the game's while it holds generation back (a menu, a load, a window not in front). What a
// hold on each of its presents does to that pacing cannot be worked out from this code, so RivaTuner is kept out
// (RTSS_OUT_DLSSG) and the game's own wait, or Reflex with its row, holds the frames. HOS_RTSS_DLSSG=1 lets RivaTuner
// try there all the same, with the cap times the multiplier in effect and every safeguard of rtss_decide.h.
#pragma once
#include <stdint.h>
#include "rtss_decide.h"
enum { PACE_GAME = 0, PACE_REFLEX = 1, PACE_RTSS = 2 };
enum {
PACE_FG_OFF, // no frame generation: a present for each of the game's frames
PACE_FG_FIXED, // the display thread shows mult frames for each of the game's
PACE_FG_AUTO, // the display thread shows a frame for each refresh of the screen
PACE_FG_DLSSG, // NVIDIA's Streamline shows mult frames for each of the game's (1 while it holds generation back)
};
// What RivaTuner is asked to hold where the game's frame starts: the limit on the frames it counts, every frame
// presented, in millihertz; or why it is kept out (RTSS_OUT_*); RTSS_UNSET while the host has told no period yet.
// period_ns: the period the host paces the game's frames by (the cap in effect). fg, mult: where the presents come
// from and how many for each of the game's frames now. dlssg_try: HOS_RTSS_DLSSG=1.
static inline int64_t pace_rtss_want(uint64_t period_ns, int fg, int mult, int dlssg_try) {
if (!period_ns) return RTSS_UNSET;
if (fg == PACE_FG_AUTO) return RTSS_OUT_AUTO;
if (fg == PACE_FG_DLSSG && !dlssg_try) return RTSS_OUT_DLSSG;
return rtss_cap_mhz(period_ns) * (fg == PACE_FG_OFF || mult < 1 ? 1 : mult);
}
// Who holds this frame. rtss: RivaTuner holds it (rtss_frame_holds of the worker's word, with no movie playing and the
// frames' own check passed). reflex: Reflex would hold it (vk_dlssg.cpp pace), which is asked only when RivaTuner does
// not: with RivaTuner as the limiter Reflex is given no frame limit, whatever the Reflex frame limit row says. Exactly
// one of the three for every frame: the host waits when, and only when, the answer is PACE_GAME.
static inline int pace_owner(int rtss, int reflex) { return rtss ? PACE_RTSS : reflex ? PACE_REFLEX : PACE_GAME; }
// The display thread's own pacing of made frames stands aside: RivaTuner holds the frames it presents.
static inline int pace_display_yields(int owner, int fg) { return owner == PACE_RTSS && fg == PACE_FG_FIXED; }

54
gpu/vk/pass_barrier.h Normal file
View File

@@ -0,0 +1,54 @@
// (Triforce) The barrier a draw pass starts with (vk_texture.cpp pass_start_barrier), apart from Vulkan so that
// tests/test_pass_barrier.py can run its rule.
//
// Before every draw pass the replay records a barrier: what the work before it wrote is made visible to it. A full one (all
// commands, all memory) makes the GPU drain: the new pass's vertex, task and mesh work waits until the pass before has drawn its
// last fragment. The game mode (RX_VK_PASS_BARRIER=game) keeps that wait for the new pass's fragment stages only (its fragment
// shaders, depth and stencil tests and colour output), where a draw pass comes right after another with nothing recorded
// between into its command buffer: no barrier of the game's (nvnCommandBufferBarrier is a full barrier and ends the pass), no copy,
// clear, uniform update or dispatch. ((Triforce, P4) by the command buffer's own count, cmd_work.h: what other command buffers get
// meanwhile is not between the two passes on the queue; a recording's uploads go before its commands, behind full barriers of their
// own.) The new pass's vertex, task and mesh work may then overlap the earlier pass's fragments, as on the
// console's GPU, where NVN asks for a barrier before rendering results are read. What changes order is only a vertex, task or
// mesh stage that reads or writes what the pass before touched with no barrier from the game between; every fragment-side access
// stays ordered after all earlier work, the writes and the reads (so a clear or a draw that writes a target an earlier pass
// sampled waits for that pass).
//
// Not where the replay itself ended the pass to keep a draw or a clear apart from its own pass's work (vk_draw.cpp draw_hazards,
// clear_after_draws, note_pass_split): those passes start with a full barrier, as before, since the draw that follows may read the
// pass's targets in its vertex or mesh stage. Not with RX_VK_BARRIER_EVERY_DRAW (an experiment that ends the pass at each draw for a
// full barrier there). ((Triforce, P4) With the parallel replay too: a segment is a command buffer of its own, begun with a full
// barrier, so its first pass start is a full one; RX_VK_BARRIER_CONTEXT=0, the one count of every thread's commands: not while the
// helper threads record, as before.)
//
// On by default with NVIDIA's driver only, where NX2navi turned its own game-only barriers on (its barrier_game, "about a tenth off
// the graphics card's time for a frame", 34 screenshots the same on an RTX 5090); nothing in Triforce's notes records a fault of
// this one, and none of the owner's runs had it on. NX2navi's rule is broader: between two passes it orders the attachments alone
// (no shader reads), and between two dispatches nothing. An attachment-only barrier leaves a target's write unordered with the
// shader reads of it before: across two passes, the fault Triforce's notes record within one (vk_draw.cpp clear_after_draws: a
// clear while draws before it still sampled its target, a band of the picture for a frame). So it is not taken; the dispatches'
// barriers are vk_draw.cpp dispatch's, not this rule's.
#pragma once
#include <cstdint>
#include <cstring>
namespace mkwvk::pass_barrier {
/// RX_VK_PASS_BARRIER: "game" the game mode on any card; "full" (or any other value) a full barrier at every pass start, as
/// before; unset or empty: the game mode with NVIDIA's driver (vendor 0x10de), full on every other card. every_draw:
/// RX_VK_BARRIER_EVERY_DRAW is set, which keeps every pass start full
inline bool game(const char* env, uint32_t vendor, bool every_draw) {
if (every_draw) return false;
if (env && *env) return std::strcmp(env, "game") == 0;
return vendor == 0x10DE;
}
/// Whether a pass start gets the fragment-stage barrier: the game mode, and a draw pass the last thing on the command buffer (the
/// pass ended on this command buffer, its count as it was then, and not a pass the replay split). ((Triforce, P4) no parallel
/// replay's argument: vk_texture.cpp folds RX_VK_BARRIER_CONTEXT=0's standing aside into game)
inline bool relaxed(bool game, bool after_pass) {
return game && after_pass;
}
} // namespace mkwvk::pass_barrier

114
gpu/vk/pcache_bound.h Normal file
View File

@@ -0,0 +1,114 @@
// (Triforce) The bound on the pipeline cache's file (mkw_vk_pipelines.bin; vk_draw.cpp pcache_init, pipeline_cache_save),
// apart from Vulkan so that tests/test_pcache_bound.py can run it. As NX2navi's (runtime/vk_cache_file.h and its cache_cap).
//
// The file is read into memory whole at every start and handed to the driver, which keeps its own copy for the session, and
// every save fetches all of it again. Nothing bounded it: a player's file reached 4.1 GB on a 32 GB PC whose log said "RAM
// short" all session, the lab's 3.9 GB. The rule of 11-14 (the file is kept for one set of translated shaders) stops the
// growth from update to update; what one set of shaders can reach on some driver, nothing stops. So:
// - a file over the bound is not read at all: its size is asked before a byte of it is, so it never takes its size in
// memory. The cache starts empty, the driver's own cache serves the pipelines, and the first save writes a new file
// over the old one;
// - data over the bound is not written: the file keeps its last save, which is under the bound and is read at the next
// start; what was built after that save is compiled again then (from the driver's own cache, where it has one). A file
// the next start would refuse is never made;
// - a file at or under the bound is read and written as before, byte for byte (in pieces now: no single read call has
// to take gigabytes), and one whose read ends early is no cache at all, never part of one.
// The bound: RX_VK_PCACHE_MAX_MB megabytes when set (0: none, as before); unset, an eighth of the PC's RAM, 2048 MB at
// least and 8192 MB at most (16 GB: 2048, 32 GB: 4096, 64 GB and more: 8192). Files measured: 647 MB fresh and 1146 MB
// with the precompile's 13,000 pipelines (vk_draw.cpp's notes), 1631 MB in an install on the lab's PC: all under the
// least of them. Never below 2048: a bound under what one run builds would have the data refused at every save, and every
// start compile again (NX2navi's 512 MB did that on an RTX 2080 Ti). The RAM, not what is free of it at the start: the
// same file is not read at one start and refused at the next.
#pragma once
#include <algorithm>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <new>
#include <stdexcept>
#include <vector>
namespace pcache_bound {
inline constexpr uint64_t kLeastMb = 2048, kMostMb = 8192;
/// A read asks the C library for this much at a time
inline constexpr size_t kPiece = (size_t)64 << 20;
/// The bound in bytes, 0: none. env: RX_VK_PCACHE_MAX_MB (null: unset); ram_bytes: the PC's RAM, 0 when it is not known
/// (the least then). A value that is no number counts as unset, never as "no bound"
inline uint64_t bytes(const char* env, uint64_t ram_bytes) {
if (env && *env) {
char* end = nullptr;
const unsigned long long mb = strtoull(env, &end, 10);
if (end != env && !*end && env[0] != '-') return mb > (UINT64_MAX >> 20) ? 0 : (uint64_t)mb << 20;
}
return std::clamp<uint64_t>((ram_bytes >> 20) / 8, kLeastMb, kMostMb) << 20;
}
/// Whether a file or data of that size is within the bound (0: none)
inline bool within(uint64_t size, uint64_t bound) { return !bound || size <= bound; }
enum class Read {
Ok,
None, // (no such file: a first start)
Unsized, // (its length could not be asked)
Short, // (not longer than a pipeline cache's header)
OverBound, // (longer than the bound: not a byte of it read)
NoMemory, // (no block of its size)
Cut, // (the read ended early: nothing of it is used)
};
inline const char* why(Read r) {
switch (r) {
case Read::Unsized: return "its size cannot be asked";
case Read::Short: return "it is shorter than a pipeline cache's header";
case Read::OverBound: return "it is over the bound";
case Read::NoMemory: return "there is no memory for a block of its size";
case Read::Cut: return "reading it ended early";
default: return "";
}
}
/// The whole file into out, or nothing: out is empty unless the answer is Ok. *len: the file's length whenever it could be
/// asked, read or not. least: a file of at most this many bytes is refused; bound: one longer than this too (0: none)
inline Read read(const char* path, uint64_t least, uint64_t bound, std::vector<uint8_t>& out, uint64_t* len) {
out.clear();
*len = 0;
FILE* f = fopen(path, "rb");
if (!f) return Read::None;
#ifdef _WIN32
int64_t n = _fseeki64(f, 0, SEEK_END) ? -1 : _ftelli64(f);
if (n >= 0 && _fseeki64(f, 0, SEEK_SET)) n = -1;
#else
int64_t n = fseeko(f, 0, SEEK_END) ? -1 : (int64_t)ftello(f);
if (n >= 0 && fseeko(f, 0, SEEK_SET)) n = -1;
#endif
Read r = Read::Ok;
if (n >= 0) *len = (uint64_t)n;
if (n < 0) r = Read::Unsized;
else if ((uint64_t)n <= least) r = Read::Short;
else if (!within((uint64_t)n, bound)) r = Read::OverBound;
else if ((uint64_t)n > SIZE_MAX) r = Read::NoMemory;
else {
try {
out.resize((size_t)n);
} catch (const std::bad_alloc&) {
r = Read::NoMemory;
} catch (const std::length_error&) {
r = Read::NoMemory;
}
size_t got = 0;
while (r == Read::Ok && got < out.size()) {
const size_t piece = std::min(out.size() - got, kPiece);
const size_t had = fread(out.data() + got, 1, piece, f);
got += had;
if (had != piece) r = Read::Cut;
}
}
fclose(f);
if (r != Read::Ok) out.clear(), out.shrink_to_fit();
return r;
}
} // namespace pcache_bound

67
gpu/vk/pcache_gate.h Normal file
View File

@@ -0,0 +1,67 @@
// (Triforce) The pipeline cache let go in play when the PC's RAM runs short (vk_draw.cpp pcache_let_go): the gate every
// pipeline creation with the cache passes, so that the cache is destroyed only once no creation is using it. Apart from
// Vulkan so that tests/test_pcache_gate.py runs it alone, with threads. (11-16: the letting go is off by default,
// RX_VK_PCACHE_RAM=1: in v0.4.0-test9 it gave back nothing the working set showed; the gate stays, passed as ever)
//
// Why: the driver holds the whole cache in RAM for the run, as it was read from mkw_vk_pipelines.bin (2.6 to 2.7 GB on a
// 32 GB PC with the stage libraries in it, v0.4.0-test5 to test7; the lab measured as much more RAM in use at the title
// with a 3.9 GB file than with a fresh one). In play it serves the pipelines built there, and the driver's own cache on
// disk serves most of those too (NVIDIA's: ~460 of ~12,900 precompiled pipelines found in ours). When the PC has under
// 1.5 GB of RAM available, Windows pages, and a frame waits 100-250 ms for it; the cache is then worth less than the RAM.
// Vulkan's rule: a cache may be destroyed once no command that takes it runs (pipelines made with it do not depend on it).
// Every creation that may run in play takes the cache through Gate::Use; Gate::close turns new ones away (they create
// without a cache) and waits for those inside. Sequentially consistent atomics: a Use that entered before close saw no
// creation inside has either left or sees the gate closed.
#pragma once
#include <atomic>
#include <cstdint>
namespace pcache_gate {
class Gate {
public:
/// In: true, the caller may use the cache until leave(); false, the gate is closed (the caller goes without a cache)
bool enter() {
users_.fetch_add(1);
if (closed_.load()) {
users_.fetch_sub(1);
return false;
}
return true;
}
void leave() { users_.fetch_sub(1); }
/// Closed from now on, and returns once nobody is inside; wait() is called while somebody is (a sleep). Once only:
/// false when it was closed already
template <class Wait>
bool close(Wait&& wait) {
if (closed_.exchange(true)) return false;
while (users_.load()) wait();
return true;
}
bool closed() const { return closed_.load(); }
int64_t inside() const { return users_.load(); }
private:
std::atomic<int64_t> users_{0};
std::atomic<bool> closed_{false};
};
/// The cache for one creation: cache itself while the gate is open, none once it is closed (or when none was asked for).
/// Held for as long as the creation runs: a temporary in the call's full expression does (Use(gate, pcache).cache)
template <class Cache>
struct Use {
Gate& gate;
Cache cache;
bool in;
Use(Gate& g, Cache want) : gate(g), cache(want), in(want != Cache{} && g.enter()) {
if (!in) cache = Cache{};
}
~Use() {
if (in) gate.leave();
}
Use(const Use&) = delete;
Use& operator=(const Use&) = delete;
};
} // namespace pcache_gate

69
gpu/vk/pcache_play.h Normal file
View File

@@ -0,0 +1,69 @@
// (Triforce) The pipeline cache's play file (vk_draw.cpp pcache_init, cached_graphics, cached_compute, pipeline_cache_save),
// apart from Vulkan so that tests/test_pcache_play.py runs its rules. NX2navi's play cache (NX2NAVI_PIPELINE_CACHE_PLAY,
// runtime/vk_pipeline.c pipe_create_in, cache_load, cache_save; chrg6, at 62cc3b13), written for Triforce's cache.
//
// Why. mkw_vk_pipelines.bin holds what the preparation built and what earlier runs made: 2.6 GB on the owner's PC with the
// stage libraries. A file can only be written whole, so a save after a few new pipelines fetched all of it from the driver
// into memory and wrote it again: the owner's test18 start wrote 2,683,916 KB right after its precompile, for 116 KB new,
// on the start-up's own thread. In play the 64 MB growth rule (pipeline_cache_save, 11-14) already waits for more; the start's
// precompile and the game's close did not.
//
// What. Once the preparation is done, what a run compiles goes into a second cache with a file of its own,
// mkw_vk_pipelines_play.bin beside the first (with its own .for, the same shaders' stamp): a creation with the main cache asks
// it first with VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED_BIT (the main cache or the driver's own gives the
// pipeline at once), and only when that answers VK_PIPELINE_COMPILE_REQUIRED goes to the play cache with the creation's own
// flags: one there from an earlier run comes back, one asked of the caches only is answered by it too, the rest is compiled
// into it (create). A save writes the play file when it changed, and the main file only when something was compiled into the
// main cache since its last save (the preparation, which builds into it as before) (save_what). The play file is read back at
// the next start; once it has reached kFoldMb (RX_VK_PCACHE_FOLD_MB) it is merged into the main cache at that start, the
// session builds into the main cache as before, its first save writes the main file and the play file is removed (fold_due).
// Without pipelineCreationCacheControl, or RX_VK_PCACHE_PLAY_FILE=0, there is no play cache: everything as before.
// A cache changes how long a pipeline takes to make, never the pipeline: nothing of this can change a picture.
#pragma once
#include <cstdint>
namespace pcache_play {
inline constexpr uint64_t kFoldMb = 64; // (NX2navi's NX2NAVI_PIPELINE_CACHE_PLAY_MB default)
/// Whether a creation goes through the play cache: with the main cache (not none, not the gate's none), a play cache there,
/// and no preparation under way (it builds into the main cache)
inline bool routes(bool main_cache, bool play_cache, bool preparing) {
return main_cache && play_cache && !preparing;
}
/// At the start, the play file read: whether it is folded into the main cache now (fold_bytes 0: never)
inline bool fold_due(uint64_t play_bytes, uint64_t fold_bytes) {
return fold_bytes && play_bytes >= fold_bytes;
}
/// What a save writes: the play file (there is a play cache, its data's size changed since its last write) and the main
/// file (there is no play cache, or pipelines were compiled into the main cache since its last save)
struct SaveWhat {
bool play = false, main = false;
};
inline SaveWhat save_what(bool play_cache, uint64_t play_now, uint64_t play_saved, int64_t main_built) {
SaveWhat w;
w.play = play_cache && play_now && play_now != play_saved;
w.main = !play_cache || main_built > 0;
return w;
}
/// One creation (create(cache, info, out) makes it, as vkCreate*Pipelines with one info). route: routes(). fail_bit:
/// VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED_BIT, compile_required: VK_PIPELINE_COMPILE_REQUIRED. *into_play is set
/// when the play cache was asked (it made or answered the creation)
template <class Info, class Cache, class Pipeline, class Result, class Create>
Result create(bool route, Cache main, Cache play, const Info* ci, Pipeline* out, uint32_t fail_bit, Result compile_required,
Create make, bool* into_play) {
if (!route) return make(main, ci, out);
Info asked = *ci;
asked.flags |= fail_bit;
Result r = make(main, &asked, out);
if (r != compile_required) return r;
*out = Pipeline{};
if (into_play) *into_play = true;
return make(play, ci, out);
}
} // namespace pcache_play

542
gpu/vk/perf_machine.cpp Normal file
View File

@@ -0,0 +1,542 @@
// (Triforce) The performance overlay's machine numbers (perf_machine.h): a thread that samples them twice a second while
// the FPS counter row is on Full, and stops when it leaves Full. The OS's counters for the CPU, the memory and the GPU's
// load (Windows: GetSystemTimes, GetProcessTimes, GlobalMemoryStatusEx, K32GetProcessMemoryInfo, and PDH for the thermal
// zones and the GPU's engines and memory, as Task Manager reads them; Linux: /proc), and the GPU vendor's driver library
// for its temperature: NVML on NVIDIA, ADL's PM log on AMD, both loaded at run time from the system's folders
// and kept for the process while they have the GPU. Nothing of it is linked: on a PC without them (an Intel GPU, an old driver) their numbers stay
// -1 and the overlay shows "-"; a vendor library with no device of the rendering GPU is shut down and let go at once.
// Nothing here runs on the presenting thread but start, stop and latest, which only take the lock (start makes the
// thread); the thread runs at the lowest priority.
//
// (Triforce: from NX2navi runtime/perfmon.c (nvml_open, nvml_sample, adl_open, adl_sample, sampler, perfmon_mode),
// runtime/platform/win32.c (pdh_load, pdh_query, pdh_items, plat_load_open, plat_cpu_load, plat_gpu_load, plat_mem_info),
// runtime/platform/linux.c (plat_cpu_load, plat_mem_info) and runtime/sysinfo.c (sysinfo_cpu_name), at 62cc3b13.) Contains
// code Copyright (c) 2026 Revivedx and the Savage Wind Recomp contributors, Copyright (c) 2026 Peiikai (MIT), and AMD
// Display Library declarations (AdapterInfo, ADLSingleSensorData, ADLPMLogDataOutput, the ADL_PMLOG_* sensor numbers and
// the ADL2_* signatures) Copyright (c) 2016 - 2022 Advanced Micro Devices, Inc. (MIT): the notice is in perf_machine.h.
#include "perf_machine.h"
#include <chrono>
#include <condition_variable>
#include <cstdlib>
#include <mutex>
#include <string>
#include <system_error>
#include <thread>
#include <vector>
#ifdef _WIN32
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN
#endif
#ifndef NOMINMAX
#define NOMINMAX
#endif
#include <windows.h>
#include <pdh.h>
#include <pdhmsg.h>
#include <psapi.h>
#if defined(_M_X64) || defined(_M_IX86) || defined(__x86_64__) || defined(__i386__)
#include <intrin.h>
#define PERF_CPUID_MS 1
#endif
#else
#include <dlfcn.h>
#include <sys/resource.h>
#include <time.h>
#include <unistd.h>
#ifdef __linux__
#include <sys/syscall.h>
#endif
#if defined(__x86_64__) || defined(__i386__)
#include <cpuid.h>
#define PERF_CPUID_GNU 1
#endif
#endif
namespace mkwvk::perf_machine {
namespace {
void (*g_log)(const char*) = nullptr;
void note(const char* line) {
if (g_log) g_log(line);
}
// ---- a library some PCs lack (a GPU vendor's): a bare name from the system's folders only
void* lib_open(const char* name) {
#ifdef _WIN32
const bool bare = !std::strchr(name, '\\') && !std::strchr(name, '/');
return (void*)LoadLibraryExA(name, nullptr, bare ? LOAD_LIBRARY_SEARCH_SYSTEM32 : 0);
#else
return dlopen(name, RTLD_NOW | RTLD_LOCAL);
#endif
}
void* lib_sym(void* lib, const char* name) {
#ifdef _WIN32
return lib ? (void*)GetProcAddress((HMODULE)lib, name) : nullptr;
#else
return lib ? dlsym(lib, name) : nullptr;
#endif
}
/// A library that has nothing for the rendering GPU, let go (NVML or ADL on a PC whose game renders on another card)
void lib_close(void* lib) {
#ifdef _WIN32
if (lib) FreeLibrary((HMODULE)lib);
#else
if (lib) dlclose(lib);
#endif
}
// ---- the CPU's name: CPUID's brand string, without the spaces around it
void cpu_brand(char* out, size_t n) {
out[0] = 0;
unsigned r[12] = {};
#if defined(PERF_CPUID_MS)
int top[4];
__cpuid(top, (int)0x80000000);
if ((unsigned)top[0] < 0x80000004u) return;
for (int i = 0; i < 3; i++) __cpuid((int*)&r[i * 4], (int)(0x80000002u + (unsigned)i));
#elif defined(PERF_CPUID_GNU)
if (__get_cpuid_max(0x80000000, nullptr) < 0x80000004u) return;
for (unsigned i = 0; i < 3; i++) __get_cpuid(0x80000002u + i, &r[i * 4], &r[i * 4 + 1], &r[i * 4 + 2], &r[i * 4 + 3]);
#else
(void)n;
return;
#endif
char b[49];
std::memcpy(b, r, 48);
b[48] = 0;
const char* p = b;
while (*p == ' ') p++;
snprintf(out, n, "%s", p);
for (char* e = out + std::strlen(out); e > out && e[-1] == ' '; e--) e[-1] = 0;
}
// ---- NVML (NVIDIA): its structs as the library lays them out, no NVIDIA header
struct NvmlUtil { unsigned gpu, memory; };
struct NvmlMem { unsigned long long total, free, used; };
struct Nvml {
bool tried = false, ok = false;
void* dev = nullptr;
int (*util)(void*, NvmlUtil*) = nullptr;
int (*temp)(void*, int, unsigned*) = nullptr;
int (*mem)(void*, NvmlMem*) = nullptr;
int (*clock)(void*, int, unsigned*) = nullptr;
} nv;
template <class F> F sym(void* lib, const char* name) { return (F)lib_sym(lib, name); }
/// Once, on the sampler's thread: the device whose name is the rendering GPU's, else none (a PC that renders on another
/// vendor's GPU: NVML's numbers would be another card's). With none NVML is shut down and let go: it is kept only for the
/// device it samples
void nvml_open(const char* gpu) {
nv.tried = true;
#ifdef _WIN32
static const char* const names[] = {"nvml.dll", "C:\\Program Files\\NVIDIA Corporation\\NVSMI\\nvml.dll"};
#else
static const char* const names[] = {"libnvidia-ml.so.1"};
#endif
void* lib = nullptr;
for (const char* n : names)
if (!lib) lib = lib_open(n);
if (!lib) return;
const auto init = sym<int (*)()>(lib, "nvmlInit_v2");
const auto count = sym<int (*)(unsigned*)>(lib, "nvmlDeviceGetCount_v2");
const auto handle = sym<int (*)(unsigned, void**)>(lib, "nvmlDeviceGetHandleByIndex_v2");
const auto name = sym<int (*)(void*, char*, unsigned)>(lib, "nvmlDeviceGetName");
nv.util = sym<int (*)(void*, NvmlUtil*)>(lib, "nvmlDeviceGetUtilizationRates");
nv.temp = sym<int (*)(void*, int, unsigned*)>(lib, "nvmlDeviceGetTemperature");
nv.mem = sym<int (*)(void*, NvmlMem*)>(lib, "nvmlDeviceGetMemoryInfo");
// ((Triforce, 11-17) no power reading (nvmlDeviceGetPowerUsage): the owner's call, power polling is reported to cost
// frames and stutter on some cards; the overlay's GPU row has no watts)
nv.clock = sym<int (*)(void*, int, unsigned*)>(lib, "nvmlDeviceGetClockInfo");
const auto shutdown = sym<int (*)()>(lib, "nvmlShutdown");
const bool up = init && count && handle && name && init() == 0;
unsigned n = 0;
if (up && count(&n) == 0)
for (unsigned i = 0; i < n && !nv.dev; i++) {
void* d = nullptr;
char nm[96] = "";
if (handle(i, &d) != 0 || name(d, nm, sizeof nm) != 0) continue;
if (name_match(gpu, nm)) nv.dev = d;
}
nv.ok = nv.dev != nullptr;
if (up) note(nv.ok ? "NVML has the rendering GPU: its temperature" : "NVML has no device of the rendering GPU's name");
if (nv.ok) return;
if (up && shutdown) shutdown(); // (nvmlInit's own count of users given back, then the library)
lib_close(lib);
nv.util = nullptr, nv.temp = nullptr, nv.mem = nullptr, nv.clock = nullptr;
}
void nvml_sample(Machine& m) {
if (!nv.ok) return;
NvmlUtil u;
NvmlMem mem;
unsigned v;
if (m.gpu_pct < 0 && nv.util && nv.util(nv.dev, &u) == 0) m.gpu_pct = (float)u.gpu; // (the system's counters first)
if (nv.mem && nv.mem(nv.dev, &mem) == 0) {
if (m.vram_used_mb < 0) m.vram_used_mb = (float)(mem.used >> 20);
m.vram_total_mb = (float)(mem.total >> 20);
}
if (nv.temp && nv.temp(nv.dev, 0 /* NVML_TEMPERATURE_GPU */, &v) == 0) m.gpu_temp_c = (float)v;
if (nv.clock && nv.clock(nv.dev, 0 /* NVML_CLOCK_GRAPHICS */, &v) == 0) m.gpu_clock_mhz = (float)v;
}
// ---- ADL (AMD): the PM log the driver keeps for Radeon RX 5000 and later (the numbers Adrenalin's overlay shows); an
// older GPU answers with an error and its numbers stay -1. Windows only
#ifdef _WIN32
constexpr int kAdlMaxPath = 256, kAdlSensors = 256;
enum {
ADL_PMLOG_CLK_GFXCLK = 1,
ADL_PMLOG_TEMPERATURE_EDGE = 8,
ADL_PMLOG_INFO_ACTIVITY_GFX = 19,
ADL_PMLOG_TEMPERATURE_HOTSPOT = 27,
};
struct AdlAdapterInfo { // (AdapterInfo, the Windows layout)
int iSize, iAdapterIndex;
char strUDID[kAdlMaxPath];
int iBusNumber, iDeviceNumber, iFunctionNumber, iVendorID;
char strAdapterName[kAdlMaxPath], strDisplayName[kAdlMaxPath];
int iPresent, iExist;
char strDriverPath[kAdlMaxPath], strDriverPathExt[kAdlMaxPath], strPNPString[kAdlMaxPath];
int iOSDisplayIndex;
};
struct AdlSensor { int supported, value; };
struct AdlPmLog { int size; AdlSensor sensors[kAdlSensors]; };
typedef void*(__stdcall* AdlAlloc)(int);
struct Adl {
bool tried = false, ok = false;
int index = 0;
void* ctx = nullptr;
int (*pmlog)(void*, int, AdlPmLog*) = nullptr;
AdlPmLog log; // (the sampler's buffer)
} adl;
void* __stdcall adl_alloc(int n) { return std::malloc((size_t)n); }
/// Once, on the sampler's thread, when NVML has not the GPU: the adapter of the rendering GPU's name whose PM log answers
/// (ADL lists one index per display output of a GPU). With none ADL's context is destroyed and the library let go
void adl_open(const char* gpu) {
adl.tried = true;
void* lib = lib_open("atiadlxx.dll");
if (!lib) return;
const auto create = sym<int (*)(AdlAlloc, int, void**)>(lib, "ADL2_Main_Control_Create");
const auto destroy = sym<int (*)(void*)>(lib, "ADL2_Main_Control_Destroy");
const auto count = sym<int (*)(void*, int*)>(lib, "ADL2_Adapter_NumberOfAdapters_Get");
const auto info = sym<int (*)(void*, AdlAdapterInfo*, int)>(lib, "ADL2_Adapter_AdapterInfo_Get");
adl.pmlog = sym<int (*)(void*, int, AdlPmLog*)>(lib, "ADL2_New_QueryPMLogData_Get");
int n = 0;
const bool up = create && count && info && adl.pmlog && create(adl_alloc, 1, &adl.ctx) == 0;
AdlAdapterInfo* a = up && count(adl.ctx, &n) == 0 && n > 0 && n <= 256 ? (AdlAdapterInfo*)std::calloc((size_t)n, sizeof *a) : nullptr;
if (a) {
for (int i = 0; i < n; i++) a[i].iSize = sizeof *a;
if (info(adl.ctx, a, n * (int)sizeof *a) == 0) {
for (int i = 0; i < n && !adl.ok; i++) {
if (a[i].iVendorID != 1002 || !name_match(gpu, a[i].strAdapterName)) continue; // (ADL writes 0x1002 as decimal 1002)
std::memset(&adl.log, 0, sizeof adl.log);
adl.log.size = sizeof adl.log;
if (adl.pmlog(adl.ctx, a[i].iAdapterIndex, &adl.log) != 0) continue;
adl.index = a[i].iAdapterIndex;
adl.ok = true;
}
}
std::free(a);
}
if (up) note(adl.ok ? "ADL has the rendering GPU's PM log: its temperature" : "ADL has no PM log for the rendering GPU");
if (adl.ok) return;
if (up && destroy) destroy(adl.ctx); // (the context ADL2_Main_Control_Create made, then the library)
adl.ctx = nullptr, adl.pmlog = nullptr;
lib_close(lib);
}
void adl_sample(Machine& m) {
if (!adl.ok) return;
std::memset(&adl.log, 0, sizeof adl.log);
adl.log.size = sizeof adl.log;
if (adl.pmlog(adl.ctx, adl.index, &adl.log) != 0) return;
const AdlSensor* v = adl.log.sensors;
if (m.gpu_pct < 0 && v[ADL_PMLOG_INFO_ACTIVITY_GFX].supported) m.gpu_pct = (float)v[ADL_PMLOG_INFO_ACTIVITY_GFX].value;
if (v[ADL_PMLOG_TEMPERATURE_EDGE].supported) m.gpu_temp_c = (float)v[ADL_PMLOG_TEMPERATURE_EDGE].value;
else if (v[ADL_PMLOG_TEMPERATURE_HOTSPOT].supported) m.gpu_temp_c = (float)v[ADL_PMLOG_TEMPERATURE_HOTSPOT].value;
// ((Triforce, 11-17) the card's power draw is not read: see nvml_open)
if (v[ADL_PMLOG_CLK_GFXCLK].supported) m.gpu_clock_mhz = (float)v[ADL_PMLOG_CLK_GFXCLK].value;
}
#else
struct Adl { bool tried = false, ok = false; } adl;
void adl_open(const char*) { adl.tried = true; }
void adl_sample(Machine&) {}
#endif
// ---- the OS's counters
#ifdef _WIN32
typedef PDH_STATUS(WINAPI* PdhOpenQueryW_t)(LPCWSTR, DWORD_PTR, PDH_HQUERY*);
typedef PDH_STATUS(WINAPI* PdhAddEnglishCounterW_t)(PDH_HQUERY, LPCWSTR, DWORD_PTR, PDH_HCOUNTER*);
typedef PDH_STATUS(WINAPI* PdhCollectQueryData_t)(PDH_HQUERY);
typedef PDH_STATUS(WINAPI* PdhGetFormattedCounterArrayW_t)(PDH_HCOUNTER, DWORD, LPDWORD, LPDWORD, PPDH_FMT_COUNTERVALUE_ITEM_W);
typedef PDH_STATUS(WINAPI* PdhCloseQuery_t)(PDH_HQUERY);
struct Pdh {
PdhOpenQueryW_t open = nullptr; // (set only when every one of them is there)
PdhAddEnglishCounterW_t add = nullptr;
PdhCollectQueryData_t collect = nullptr;
PdhGetFormattedCounterArrayW_t get = nullptr;
PdhCloseQuery_t close = nullptr;
};
const Pdh& pdh() { // (pdh.dll, loaded at the first use, kept)
static const Pdh p = [] {
Pdh r;
void* l = lib_open("pdh.dll");
if (!l) return r;
r.add = sym<PdhAddEnglishCounterW_t>(l, "PdhAddEnglishCounterW");
r.collect = sym<PdhCollectQueryData_t>(l, "PdhCollectQueryData");
r.get = sym<PdhGetFormattedCounterArrayW_t>(l, "PdhGetFormattedCounterArrayW");
r.close = sym<PdhCloseQuery_t>(l, "PdhCloseQuery");
const auto open = sym<PdhOpenQueryW_t>(l, "PdhOpenQueryW");
if (r.add && r.collect && r.get && r.close) r.open = open;
return r;
}();
return p;
}
/// A PDH query with its wildcard counters ("\Object(*)\Counter"); q null when it could not be opened
struct PdhQuery {
PDH_HQUERY q = nullptr;
PDH_HCOUNTER c[2] = {nullptr, nullptr};
void open(const wchar_t* c0, const wchar_t* c1) {
const Pdh& p = pdh();
if (!p.open || p.open(nullptr, 0, &q) != ERROR_SUCCESS) {
q = nullptr;
return;
}
if (p.add(q, c0, 0, &c[0]) != ERROR_SUCCESS) c[0] = nullptr;
if (!c1 || p.add(q, c1, 0, &c[1]) != ERROR_SUCCESS) c[1] = nullptr;
if (!c[0] && !c[1]) close();
}
void close() {
if (q) pdh().close(q);
q = nullptr;
}
};
/// A counter's valid instances: their names (narrowed: the instance names are ASCII) and values. False when there is no
/// valid data yet (a rate counter needs two collections)
struct Items {
std::vector<std::string> names;
std::vector<const char*> ptrs;
std::vector<double> values;
};
bool pdh_items(PDH_HCOUNTER c, Items& out) {
out.names.clear(), out.ptrs.clear(), out.values.clear();
DWORD size = 0, count = 0;
const Pdh& p = pdh();
if (!c || p.get(c, PDH_FMT_DOUBLE | PDH_FMT_NOCAP100, &size, &count, nullptr) != (PDH_STATUS)PDH_MORE_DATA || !size) return false;
std::vector<unsigned char> buf(size);
auto* items = (PDH_FMT_COUNTERVALUE_ITEM_W*)buf.data();
if (p.get(c, PDH_FMT_DOUBLE | PDH_FMT_NOCAP100, &size, &count, items) != ERROR_SUCCESS) return false;
for (DWORD i = 0; i < count; i++) {
if (items[i].FmtValue.CStatus != PDH_CSTATUS_VALID_DATA) continue;
std::string n;
for (const wchar_t* w = items[i].szName; w && *w; w++) n += (char)(*w < 128 ? *w : '?');
out.names.push_back(std::move(n));
out.values.push_back(items[i].FmtValue.doubleValue);
}
for (const std::string& n : out.names) out.ptrs.push_back(n.c_str());
return true;
}
uint64_t ft64(FILETIME f) { return (uint64_t)f.dwHighDateTime << 32 | f.dwLowDateTime; }
struct Sources {
CpuTimes cpu; // (the sample before's)
PdhQuery thermal, gpu; // (gpu: the engines' utilisation, the adapters' dedicated memory)
char luid[32] = ""; // (the rendering GPU's, luid_name; empty: any)
Items items;
};
void sources_open(Sources& s, const Gpu& g) {
if (g.has_luid) luid_name(g.luid, s.luid, sizeof s.luid);
s.thermal.open(L"\\Thermal Zone Information(*)\\Temperature", nullptr);
s.gpu.open(L"\\GPU Engine(*)\\Utilization Percentage", L"\\GPU Adapter Memory(*)\\Dedicated Usage");
char line[160];
snprintf(line, sizeof line, "Windows' counters: GPU engines %s (%s), thermal zones %s", s.gpu.q ? "open" : "not there",
s.luid[0] ? s.luid : "any adapter", s.thermal.q ? "open" : "not there");
note(line);
}
void sources_close(Sources& s) { s.thermal.close(), s.gpu.close(); }
void sample_os(Sources& s, Machine& m) {
FILETIME idle, kern, user, cr, ex, pk, pu;
if (GetSystemTimes(&idle, &kern, &user) && GetProcessTimes(GetCurrentProcess(), &cr, &ex, &pk, &pu)) {
CpuTimes now;
now.idle = ft64(idle), now.kernel = ft64(kern), now.user = ft64(user), now.proc = ft64(pk) + ft64(pu);
cpu_load(s.cpu, now, &m.cpu_pct, &m.cpu_game_pct);
s.cpu = now;
}
const Pdh& p = pdh();
// (the firmware's ACPI zones, near the CPU on most boards: the hottest under 150 degrees, from kelvin)
if (s.thermal.q && p.collect(s.thermal.q) == ERROR_SUCCESS && pdh_items(s.thermal.c[0], s.items))
for (double k : s.items.values) {
const double c = k - 273.15;
if (c > m.cpu_temp_c && c < 150) m.cpu_temp_c = (float)c;
}
if (s.gpu.q && p.collect(s.gpu.q) == ERROR_SUCCESS) {
if (pdh_items(s.gpu.c[0], s.items))
m.gpu_pct = engine_load(s.items.ptrs.data(), s.items.values.data(), (int)s.items.values.size(), s.luid);
if (pdh_items(s.gpu.c[1], s.items))
m.vram_used_mb = adapter_memory_mb(s.items.ptrs.data(), s.items.values.data(), (int)s.items.values.size(), s.luid);
}
MEMORYSTATUSEX ms{};
ms.dwLength = sizeof ms;
if (GlobalMemoryStatusEx(&ms) && ms.ullTotalPhys) {
m.ram_total_mb = (float)(ms.ullTotalPhys >> 20);
m.ram_used_mb = (float)((ms.ullTotalPhys - ms.ullAvailPhys) >> 20);
}
PROCESS_MEMORY_COUNTERS pmc{};
pmc.cb = sizeof pmc;
if (K32GetProcessMemoryInfo(GetCurrentProcess(), &pmc, sizeof pmc)) m.ram_game_mb = (float)(pmc.WorkingSetSize >> 20);
}
#else
struct Sources {
CpuTimes cpu; // (/proc/stat's busy jiffies in kernel's place with idle 0, its total in user's; the process's in jiffies)
};
void sources_open(Sources&, const Gpu&) {}
void sources_close(Sources&) {}
void sample_os(Sources& s, Machine& m) {
unsigned long long v[10] = {};
int n = 0;
if (FILE* f = std::fopen("/proc/stat", "r")) {
n = std::fscanf(f, "cpu %llu %llu %llu %llu %llu %llu %llu %llu %llu %llu", &v[0], &v[1], &v[2], &v[3], &v[4], &v[5],
&v[6], &v[7], &v[8], &v[9]);
std::fclose(f);
}
timespec ts{};
clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &ts);
const long tick = sysconf(_SC_CLK_TCK);
if (n >= 8 && tick > 0) {
uint64_t total = 0;
for (int i = 0; i < 8; i++) total += v[i]; // (guest time is already inside user time)
const uint64_t idle = v[3] + v[4];
CpuTimes now; // (cpu_load's terms: kernel + user the total, idle the idle)
now.kernel = total, now.user = 0, now.idle = idle;
now.proc = (uint64_t)(((double)ts.tv_sec + ts.tv_nsec / 1e9) * (double)tick);
cpu_load(s.cpu, now, &m.cpu_pct, &m.cpu_game_pct);
s.cpu = now;
}
if (FILE* f = std::fopen("/proc/meminfo", "r")) {
char line[128];
unsigned long long total = 0, avail = 0, k;
while (std::fgets(line, sizeof line, f)) {
if (std::sscanf(line, "MemTotal: %llu kB", &k) == 1) total = k;
else if (std::sscanf(line, "MemAvailable: %llu kB", &k) == 1) avail = k;
}
std::fclose(f);
if (total) m.ram_total_mb = (float)(total >> 10), m.ram_used_mb = (float)((total - std::min(total, avail)) >> 10);
}
if (FILE* f = std::fopen("/proc/self/statm", "r")) {
unsigned long long size, resident;
if (std::fscanf(f, "%llu %llu", &size, &resident) == 2)
m.ram_game_mb = (float)((resident * (uint64_t)sysconf(_SC_PAGESIZE)) >> 20);
std::fclose(f);
}
}
#endif
// ---- the sampler: runs while want is set, twice a second
struct State {
std::mutex mu; // (everything here)
std::condition_variable cv;
bool want = false, alive = false;
int live = 0; // (the sampler threads made and not yet ended: live_samplers)
Gpu gpu;
Machine latest;
};
State& st() { // (never destroyed: a sampler still asleep at the process's end holds its lock)
static State* s = new State;
return *s;
}
void clear(Machine& m) { // (the numbers, not the names)
Machine e;
std::memcpy(e.gpu_name, m.gpu_name, sizeof e.gpu_name);
std::memcpy(e.cpu_name, m.cpu_name, sizeof e.cpu_name);
e.samples = m.samples;
m = e;
}
/// The sampler's thread below the game's own (the replay's and the presenting threads): its counters can wait in the driver
void lowest_priority() {
#ifdef _WIN32
SetThreadPriority(GetCurrentThread(), THREAD_PRIORITY_LOWEST);
#elif defined(__linux__)
setpriority(PRIO_PROCESS, (id_t)syscall(SYS_gettid), 19); // (Linux: a nice of a thread's own, by its id)
#endif
}
void sampler() {
lowest_priority();
State& s = st();
Gpu gpu;
{
std::lock_guard lk(s.mu);
gpu = s.gpu;
}
Machine m;
snprintf(m.gpu_name, sizeof m.gpu_name, "%s", gpu.name);
cpu_brand(m.cpu_name, sizeof m.cpu_name);
if (!nv.tried) nvml_open(gpu.name);
if (!nv.ok && !adl.tried) adl_open(gpu.name);
Sources src;
sources_open(src, gpu);
std::unique_lock lk(s.mu);
while (s.want) {
lk.unlock();
clear(m);
sample_os(src, m);
if (gpu.vram_total_mb > 0) m.vram_total_mb = gpu.vram_total_mb;
nvml_sample(m);
adl_sample(m);
m.samples++;
lk.lock();
s.latest = m;
if (s.want) s.cv.wait_for(lk, std::chrono::milliseconds(500), [&] { return !s.want; });
}
s.alive = false; // (in the same locked part as the last look at want: a start after it makes a new thread)
lk.unlock();
sources_close(src);
lk.lock();
s.live--; // (this thread's last step: live_samplers)
}
} // namespace
void start(const Gpu& gpu, void (*log)(const char* line)) {
State& s = st();
std::lock_guard lk(s.mu);
s.want = true;
if (s.alive) return;
g_log = log;
s.gpu = gpu;
s.latest = Machine{}; // ("-" until the first sample, not the numbers of the time before)
s.alive = true;
s.live++;
// (a thread the system will not make: no sampler, and the panel's machine numbers stay "-" (latest, emptied above). Never
// the exception, which would reach menu_tick on the game's thread; the next start tries again)
try {
std::thread(sampler).detach();
} catch (const std::system_error&) {
s.alive = false;
s.live--;
note("its sampler's thread could not be made: the machine's numbers stay \"-\"");
}
}
void stop() {
State& s = st();
std::lock_guard lk(s.mu);
s.want = false;
s.cv.notify_all();
}
Machine latest() {
State& s = st();
std::lock_guard lk(s.mu);
return s.latest;
}
int live_samplers() {
State& s = st();
std::lock_guard lk(s.mu);
return s.live;
}
} // namespace mkwvk::perf_machine

177
gpu/vk/perf_machine.h Normal file
View File

@@ -0,0 +1,177 @@
// (Triforce) The performance overlay's machine numbers (vk_menu.cpp draw_perf, the FPS counter row on Full): the GPU's
// load and temperature (no power: removed 11-17, the owner's call), the video memory the card has in use and in all, the CPU's load, temperature and this
// game's share of it, the system memory in use, in all and the game's, and the GPU's and the CPU's names. Sampled twice a
// second on a thread of their own (perf_machine.cpp) while the row is on Full, since some of the calls take milliseconds:
// the presenting thread only reads the latest sample. A number the machine does not give is -1, and the overlay shows "-"
// (an AMD or Intel card has no NVIDIA library, an old Radeon no PM log, a board no thermal zone, Linux no temperatures).
//
// (Triforce: from NX2navi runtime/perfmon.c, runtime/perfmon.h, runtime/platform/win32.c plat_cpu_load, plat_gpu_load and
// plat_mem_info, runtime/platform/linux.c and runtime/sysinfo.c sysinfo_cpu_name, at 62cc3b13.) That code contains code
// Copyright (c) 2026 Revivedx and the Savage Wind Recomp contributors, Copyright (c) 2026 Peiikai (MIT: the performance
// overlay of github.com/Revivedx/SavageWindRecomp, its NVML calls and sampling loop), and its AMD Display Library
// declarations Copyright (c) 2016 - 2022 Advanced Micro Devices, Inc. (MIT: github.com/GPUOpen-LibrariesAndSDKs/display-library).
// Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
// documentation files (the "Software"), to deal in the Software without restriction, including without limitation the
// rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit
// persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and
// this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS
// PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
// HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
//
// The parts below with no clock and no system call of their own (cpu_load, luid_name, engine_load, adapter_memory_mb,
// cell_text, local_heap_mb, name_match) are what tests/test_perf_overlay.cpp runs; it runs the sampler too, for a GPU of no
// vendor's name.
#pragma once
#include <algorithm>
#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <cstring>
namespace mkwvk::perf_machine {
struct Machine {
char gpu_name[96] = ""; // (the GPU the game renders on, as Vulkan names it)
char cpu_name[64] = ""; // (the CPU's brand string, CPUID)
float gpu_pct = -1; // (the busiest engine type of the rendering GPU, as Task Manager shows it; else NVML's or ADL's)
float gpu_temp_c = -1; // (NVML; ADL's edge temperature, else its hotspot)
float vram_used_mb = -1; // (the card's dedicated memory in use by every program: Windows' adapter counter, else NVML)
float vram_total_mb = -1; // (the card's: NVML, else the device-local heaps Vulkan reports, added up)
float gpu_clock_mhz = -1; // (the graphics clock: NVML, else ADL's; the VRAM row's third cell when the game's figure is unknown)
float cpu_pct = -1; // (the whole machine, percent of all logical processors, since the sample before)
float cpu_game_pct = -1; // (this process, the same percent)
float cpu_temp_c = -1; // (the hottest ACPI thermal zone: a zone near the CPU on most boards, not the package itself)
float ram_used_mb = -1, ram_total_mb = -1, ram_game_mb = -1; // (in use, installed, this process's working set)
int samples = 0; // (made since the sampler started: 0 until the first)
};
/// The GPU the game renders on, given when the sampler starts
struct Gpu {
char name[96] = "";
uint8_t luid[8] = {}; // (Vulkan's deviceLUID: Windows' counters name an adapter by it)
bool has_luid = false;
float vram_total_mb = -1;
};
/// Starts the sampler (the row went to Full) unless it runs; gpu is read then. log, or null: a line on what was found
/// (NVML, ADL, the counters), once a run
void start(const Gpu& gpu, void (*log)(const char* line));
/// Stops it (the row left Full): its thread ends at its next wake, within a moment
void stop();
/// The latest sample (all -1 before the first): a copy, under the sampler's lock for as long as the copy takes
Machine latest();
/// (for tests/test_perf_overlay.cpp) The sampler threads alive now: 1 while it runs, 0 once it has ended (a second start
/// while one runs makes no second)
int live_samplers();
// ---- the parts with no clock and no system call ----
/// The CPU's times as GetSystemTimes and GetProcessTimes give them (100 ns units; kernel includes idle), or /proc/stat's
/// (busy and total jiffies in kernel and idle's place) with the process's in the same unit
struct CpuTimes {
uint64_t idle = 0, kernel = 0, user = 0, proc = 0;
};
/// The machine's and this process's share of all logical processors between two readings, percent; -1 when there is no
/// reading before (before.kernel 0) or no time went by
inline void cpu_load(const CpuTimes& before, const CpuTimes& now, float* total_pct, float* game_pct) {
*total_pct = *game_pct = -1;
const uint64_t total = (now.kernel - before.kernel) + (now.user - before.user); // (kernel time includes idle time)
if (!before.kernel || !total || now.kernel < before.kernel || now.user < before.user) return;
const uint64_t idle = now.idle >= before.idle ? now.idle - before.idle : 0;
*total_pct = 100.0f * (float)(total - std::min(total, idle)) / (float)total;
*game_pct = 100.0f * (float)(now.proc >= before.proc ? now.proc - before.proc : 0) / (float)total;
}
/// The adapter's part of a counter's instance names, from Vulkan's deviceLUID (the LUID struct's memory: LowPart, then
/// HighPart): "luid_0x<high>_0x<low>", lower case
inline void luid_name(const uint8_t luid[8], char* out, size_t n) {
uint32_t lo, hi;
std::memcpy(&lo, luid, 4);
std::memcpy(&hi, luid + 4, 4);
snprintf(out, n, "luid_0x%08x_0x%08x", hi, lo);
}
/// The "luid_0x..._0x..." part of an instance name, lower case, into out (27 bytes at least); empty when it has none
inline void instance_luid(const char* name, char* out) {
const char* p = std::strstr(name, "luid_");
size_t n = 0;
while (p && p[n] && n < 26) out[n] = (char)(p[n] >= 'A' && p[n] <= 'Z' ? p[n] + 32 : p[n]), n++; // (luid_0x%08x_0x%08x: 26)
out[n] = 0;
}
/// The GPU's load from "\GPU Engine(*)\Utilization Percentage": instances "pid_<n>_luid_0x.._0x.._phys_<n>_eng_<n>_engtype_<type>"
/// with each process's share of an engine. Per adapter and engine type the shares add up; the load is the busiest type,
/// 100 at most. luid: the adapter's (luid_name), or empty for any. -1 when no instance counts
inline float engine_load(const char* const* names, const double* values, int n, const char* luid) {
enum { kTypes = 32 };
struct { char luid[32], type[32]; double sum; } t[kTypes];
int nt = 0;
for (int i = 0; i < n; i++) {
char lu[32];
instance_luid(names[i], lu);
if (luid[0] && std::strcmp(lu, luid)) continue;
const char* ty = std::strstr(names[i], "engtype_");
ty = ty ? ty + 8 : "";
int k = 0;
while (k < nt && (std::strcmp(t[k].luid, lu) || std::strncmp(t[k].type, ty, 31))) k++;
if (k == nt) {
if (nt == kTypes) continue;
std::memcpy(t[k].luid, lu, sizeof lu);
std::strncpy(t[k].type, ty, 31);
t[k].type[31] = 0;
t[k].sum = 0;
nt++;
}
t[k].sum += values[i];
}
float load = -1;
for (int k = 0; k < nt; k++)
if (t[k].sum > load) load = (float)(t[k].sum > 100 ? 100 : t[k].sum);
return load;
}
/// The adapter's dedicated memory in use from "\GPU Adapter Memory(*)\Dedicated Usage" (instances
/// "luid_0x.._0x.._phys_<n>", bytes), MB: the instances of luid added up (-1 when they come to nothing), or with luid
/// empty the adapter that uses the most (-1 when there is none)
inline float adapter_memory_mb(const char* const* names, const double* values, int n, const char* luid) {
double sum = 0, best = -1;
for (int i = 0; i < n; i++) {
char lu[32];
instance_luid(names[i], lu);
if (luid[0] && !std::strcmp(lu, luid)) sum += values[i];
if (values[i] > best) best = values[i];
}
if (luid[0] ? sum <= 0 : best < 0) return -1;
return (float)((luid[0] ? sum : best) / 1048576.0);
}
/// A number of the overlay's as its panel writes it: fmt with x, or "-" when x is under 0 (the machine, its driver or the game
/// gives none: no NVIDIA or AMD library for this GPU, no thermal zone, no sample yet); true for "-", which the panel dims
/// (NX2navi's perf_panel.cpp cell)
inline bool cell_text(char* out, size_t n, const char* fmt, double x) {
if (x < 0) {
snprintf(out, n, "-");
return true;
}
snprintf(out, n, fmt, x);
return false;
}
/// The card's memory as Vulkan reports it, MB: the heaps with local_bit (VK_MEMORY_HEAP_DEVICE_LOCAL_BIT) added up, the
/// system memory a card can also reach left out; -1 when there is none (NX2navi's perf_gpu_once). Heap: Vulkan's
/// VkMemoryHeap, or any struct with its size and flags
template <class Heap> float local_heap_mb(const Heap* heaps, uint32_t n, uint32_t local_bit) {
uint64_t bytes = 0;
for (uint32_t i = 0; i < n; i++)
if (heaps[i].flags & local_bit) bytes += heaps[i].size;
return bytes ? (float)(bytes >> 20) : -1.0f;
}
/// Whether a vendor library's device name is the rendering GPU's: either within the other (NVIDIA's names match NVIDIA's
/// GPUs only), both not empty
inline bool name_match(const char* vulkan, const char* vendor) {
return vulkan[0] && vendor[0] && (std::strstr(vulkan, vendor) || std::strstr(vendor, vulkan));
}
} // namespace mkwvk::perf_machine

140
gpu/vk/perf_stats.h Normal file
View File

@@ -0,0 +1,140 @@
// (Triforce) The performance overlay's frame times (vk_menu.cpp, the FPS counter row on Full): a ring of the
// game's frame times, and what the overlay says of them: the mean, the longest, the median, the 99th percentile and
// the 1% and 0.1% lows over the last ten seconds, and the newest of them for the graph. After what NX2navi's overlay shows
// (its runtime/perfmon.c; none of its code is here: its 0.1% low and its rule of 100 frames for both lows are, computed
// here by the same nearest rank as the 1% low). Plain functions with no clock and no lock of their own, so that
// tests/test_perf_overlay.py runs them (tests/test_perf_overlay.cpp); the caller gives the times and keeps one thread in
// at a time.
#pragma once
#include <algorithm>
#include <cstdint>
namespace mkwvk::perf_stats {
constexpr int kFrames = 2048; // the frame times kept (a power of two)
constexpr int64_t kWindowNs = 10000000000ll; // the summary counts the frames of the last ten seconds
constexpr int64_t kGapNs = 1000000000ll; // held longer than this, a frame is a load or a pause: no frame time
constexpr int kLowFrames = 100; // under this many frames there is no 1% low yet, nor a 0.1% low
struct Ring {
uint32_t us[kFrames] = {}; // frame times in microseconds
uint32_t count = 0; // frame times written since the start: the newest is us[(count - 1) % kFrames]
int64_t last_ns = 0; // when the newest frame came; 0 before the first
};
/// A frame of the game came at now_ns: the time since the one before goes into the ring. The first frame, a frame
/// after a gap (kGapNs) and a clock that did not move on only start the next interval.
inline void note(Ring& r, int64_t now_ns) {
const int64_t d = r.last_ns ? now_ns - r.last_ns : 0;
r.last_ns = now_ns;
if (d <= 0 || d > kGapNs) return;
r.us[r.count++ % kFrames] = (uint32_t)((d + 500) / 1000);
}
/// How many frame times the ring holds
inline int held(const Ring& r) { return (int)std::min<uint32_t>(r.count, (uint32_t)kFrames); }
/// The newest n frame times (as many as there are), the oldest of them first, in milliseconds: the graph. Returns how
/// many were written.
inline int newest(const Ring& r, float* out, int n) {
const int m = std::min(std::max(n, 0), held(r));
for (int i = 0; i < m; i++) out[i] = (float)r.us[(r.count - (uint32_t)m + (uint32_t)i) % kFrames] / 1000.0f;
return m;
}
struct Summary {
int frames = 0; // the frame times counted
float avg_ms = 0; // their mean
float max_ms = 0; // the longest
float p50_ms = 0; // the median
float p99_ms = 0; // the 99th percentile: 99 of 100 frames took no longer
float fps = 0; // the frame rate of the mean
float low1_fps = -1; // the 1% low: the frame rate of the 99th percentile; -1 with fewer than kLowFrames frames
float low01_fps = -1; // (Triforce: from NX2navi's perfmon.c lows_summary) the 0.1% low: the frame rate of the 99.9th
// percentile (under 1000 frames, the longest); -1 with fewer than kLowFrames frames
};
/// Of n values sorted upwards, the one at percentile q (0 to 1), by the nearest rank: the smallest value that at least
/// that share of them does not exceed.
inline uint32_t percentile(const uint32_t* sorted, int n, double q) {
if (n <= 0) return 0;
int rank = (int)(q * n);
if ((double)rank < q * n) rank++; // (the ceiling)
return sorted[std::min(std::max(rank, 1), n) - 1];
}
/// What the overlay says of the newest frames: those whose times add up to window_ns, at least one.
inline Summary summarize(const Ring& r, int64_t window_ns = kWindowNs) {
Summary s;
static thread_local uint32_t v[kFrames];
const int have = held(r);
int n = 0;
uint64_t sum = 0;
for (int i = 0; i < have; i++) { // (the newest first)
const uint32_t us = r.us[(r.count - 1u - (uint32_t)i) % kFrames];
if (n && (int64_t)(sum + us) * 1000 > window_ns) break;
v[n++] = us, sum += us;
}
if (!n) return s;
std::sort(v, v + n);
s.frames = n;
s.avg_ms = (float)((double)sum / n / 1000.0);
s.max_ms = (float)v[n - 1] / 1000.0f;
s.p50_ms = (float)percentile(v, n, 0.50) / 1000.0f;
s.p99_ms = (float)percentile(v, n, 0.99) / 1000.0f;
s.fps = sum ? (float)(1e6 * n / (double)sum) : 0;
if (n >= kLowFrames && s.p99_ms > 0) s.low1_fps = 1000.0f / s.p99_ms;
if (n >= kLowFrames) {
const uint32_t p999 = percentile(v, n, 0.999);
if (p999) s.low01_fps = 1000000.0f / (float)p999;
}
return s;
}
// ---- (Triforce) the overlay's figures per frame (NX2navi's waits block: GPU frame prep, GPU busy per frame, Pipeline waits,
// Shader waits, Texture uploads), from running counts read twice a second (vk_menu.cpp perf_sample) ----
/// The running counts the figures come from, as of one sample: the game's frames, the replay's time to record them and the
/// GPU's in the recordings (ns; the GPU's stays 0 without timestamps), the textures uploaded, and the replay's waits for
/// pipelines and for translations (us, vk_draw.cpp draw_wait_split)
struct Counts {
int64_t frames = 0, prep_ns = 0, gpu_ns = 0, uploads = 0, pipe_us = 0, shader_us = 0;
};
/// The counts of the sample before (have: false until there is one)
struct Base {
bool have = false;
Counts at;
};
/// The figures per game frame since the sample before, ms or uploads a frame; -1 where there is none (the panel's "-")
struct PerFrame {
double prep_ms = -1, gpu_ms = -1, pipe_ms = -1, shader_ms = -1, uploads = -1;
};
enum Step {
STEP_FIRST, ///< no counts to go from (the first sample, or fresh): every figure "-"
STEP_NEW, ///< out has the figures since the sample before
STEP_KEEP, ///< no game frame since: the panel keeps what it shows (out untouched)
};
/// One sample's step. fresh: the panel was turned on now, so the counts of the sample before are of another time. Each figure
/// is its count's change over the frames' change; a count that went back is 0, and the GPU's time "-" when it did not move
/// on (no timestamps on this device). now becomes the next sample's base in every case
inline Step per_frame(Base& b, const Counts& now, bool fresh, PerFrame* out) {
const Counts was = b.at;
const bool had = b.have && !fresh;
b.have = true, b.at = now;
if (!had) {
*out = PerFrame{};
return STEP_FIRST;
}
const int64_t n = now.frames - was.frames;
if (n <= 0) return STEP_KEEP;
const auto per = [n](int64_t d, double unit) { return (double)(d > 0 ? d : 0) / unit / (double)n; };
out->prep_ms = per(now.prep_ns - was.prep_ns, 1e6);
out->gpu_ms = now.gpu_ns > was.gpu_ns ? per(now.gpu_ns - was.gpu_ns, 1e6) : -1;
out->pipe_ms = per(now.pipe_us - was.pipe_us, 1e3);
out->shader_ms = per(now.shader_us - was.shader_us, 1e3);
out->uploads = per(now.uploads - was.uploads, 1);
return STEP_NEW;
}
} // namespace mkwvk::perf_stats

237
gpu/vk/pipe_order.h Normal file
View File

@@ -0,0 +1,237 @@
/* (Triforce, wp18) The pipelines made ahead in the order the last sessions' draws first needed them (vk_draw.cpp
graphics_pipeline, order_met, warm_index_set, pipelines_warm, warm_variant, warm_hold, warm_link_take; the file
triforce_pipeline_order.bin beside the pipeline notes).
With the programs read ahead in the sessions' order (program_order.h) the save's load-in frame reads none of its programs
any more (lab round 37: "0 programs read from the cache by the draws, 298 taken read ahead"), but it still makes its
pipelines: "pipelines 86.5" ms of its 249 to 270 (86 to 92 in every run), and 178 ms summed over the helpers in the Market's
arrival frame. Where the precompile keeps no pipelines (the RAM spared), each pipeline of a load is new at its first draw,
which links it on its own thread from its stage libraries, those made from the driver's caches there too. The pipeline warm
(pipe_warm.h) makes them ahead for a program the read-ahead has read, from the pipeline notes: up to 32 a program variant,
every pipeline any session ever built with it. In round 37 it made 1964 links and the draws took 54: its bounds (512 links,
the oldest let go beyond; 1024 libraries no draw has had, none made beyond) were full well before the load, of pipelines the
load did not draw, and of the program order's read-ahead the newest, the farthest from the draws, were what it kept.
So each session keeps the pipelines its draws first needed (a new entry, or one the precompile made that no draw had), in
that order (Record: the first kKeysMax), and writes them beside the notes now and then: this session's first, then those of
the sessions before it has not met (program_order.h merge; the same form of file, its own magic). The next session finds
each one's note (Index) and from the title on, when the read-ahead has read a program variant, the warm makes the pipelines
the order names for it, and with an order only those (kPerVariant at most): their stage libraries, asked of the caches only,
and their links, held for the draws (Held). The draw that needs one takes its link as it takes the warm's now.
- At most kLinksMax links held: beyond, the one farthest ahead in the order goes, not the oldest (the nearest are the next
frames'). At most kLibsMax libraries the warm made that no draw has had: none made beyond.
- Where the draws are: the middle place of the last kRecent pipelines they needed first (Front, program_order.h's rule: one
met out of the order's turn moves nothing). A link held kBehind places behind it goes, and none is made there: that
session went elsewhere.
- A link that goes takes with it the libraries the warm made for it that no draw has had and no other link held uses
(let_go_libs), so that the libraries' bound does not fill with what the draws did not come to.
Nothing changes what a draw gets: the same pipeline from the same libraries, linked a moment earlier on another thread.
Plain functions: tests/test_pipe_order.cpp checks them. RX_VK_PIPELINE_ORDER=0: neither read nor written, the warm from the
notes as before. */
#pragma once
#include <algorithm>
#include <cstdint>
#include <deque>
#include <iterator>
#include <map>
#include <unordered_map>
#include <unordered_set>
#include <utility>
#include <vector>
#include "program_order.h"
namespace mkwvk::pipe_order {
constexpr size_t kKeysMax = program_order::kKeysMax; // (pipelines a session's order keeps, and the file)
constexpr size_t kPerVariant = 32; // (the order's pipelines made for a program variant, at most: as the notes')
constexpr size_t kLinksMax = 1024; // (links held for draws that have not come: a load's, beside the title's taken)
constexpr size_t kLibsMax = 2048; // (stage libraries the warm made that no draw has had: two a program variant of 1024)
constexpr int64_t kBehind = 512; // (a link held this many places behind where the draws are goes, and none is made there)
constexpr size_t kRecent = 16; // (where the draws are: the middle place of the last this many they needed first)
constexpr uint32_t kMagic = 0x50505446; // ("FTPP")
/// RX_VK_PIPELINE_ORDER: on unless "0"
inline bool on(const char* env) { return !(env && env[0] == '0' && !env[1]); }
/// The file: program_order.h's form, its own magic
inline std::vector<uint8_t> write(const std::vector<uint64_t>& keys) { return program_order::write(keys, kMagic); }
inline std::vector<uint64_t> read(const uint8_t* data, size_t n) { return program_order::read(data, n, kMagic); }
using program_order::merge;
using Record = program_order::Record;
/// One pipeline of the order: its place, its note's record in the notes file
struct Entry {
int64_t place = 0;
uint32_t rec = 0;
};
/// The order as the warm takes it: the pipelines with a note by program variant (the caller's key for it), each list in the
/// order, at most per a variant; and every key's place, for where the draws are
struct Index {
std::unordered_map<uint64_t, std::vector<Entry>> by_variant;
std::unordered_map<uint64_t, int64_t> place;
size_t named = 0, variants = 0; // (the pipelines with a note, and their variants, as built)
/// note(key, &variant, &rec): the note of key, false when the notes have none
template <class Note>
void build(const std::vector<uint64_t>& order, Note note, size_t per = kPerVariant) {
by_variant.clear(), place.clear(), named = 0;
for (size_t i = 0; i < order.size(); i++) {
const uint64_t k = order[i];
if (!k || !place.emplace(k, (int64_t)i).second) continue;
uint64_t variant = 0;
uint32_t rec = 0;
if (!note(k, &variant, &rec)) continue;
std::vector<Entry>& list = by_variant[variant];
if (list.size() >= per) continue;
list.push_back({(int64_t)i, rec});
named++;
}
variants = by_variant.size();
}
/// The variant's pipelines, taken out (each variant once a session); none when the order names none for it
std::vector<Entry> take(uint64_t variant) {
const auto it = by_variant.find(variant);
if (it == by_variant.end()) return {};
std::vector<Entry> out = std::move(it->second);
by_variant.erase(it);
return out;
}
/// key's place in the order; -1: it names none
int64_t at(uint64_t key) const {
const auto it = place.find(key);
return it == place.end() ? -1 : it->second;
}
};
/// Where the draws are in the order
struct Front {
std::deque<int64_t> recent; // (the places of the last kRecent pipelines the draws needed first)
int64_t furthest = -1; // (the furthest the middle of recent has been)
/// A draw needed the pipeline at place first (-1: one the order does not name, nothing moves): true when where the draws
/// are moved on
bool met(int64_t place) {
if (place < 0) return false;
recent.push_back(place);
if (recent.size() > kRecent) recent.pop_front();
std::vector<int64_t> s(recent.begin(), recent.end());
std::nth_element(s.begin(), s.begin() + s.size() / 2, s.end());
if (s[s.size() / 2] <= furthest) return false;
furthest = s[s.size() / 2];
return true;
}
/// Whether the draws went kBehind places past place (nothing made there, and what is held there goes)
bool passed(int64_t place) const { return furthest >= 0 && place < furthest - kBehind; }
/// The first place not passed
int64_t from() const { return furthest - kBehind; }
};
/// The links held for draws that have not come, by pipeline key, each with its place and the keys of the libraries it was
/// linked from. At most max: beyond, the one farthest ahead goes. Not thread-safe: the caller's lock
template <class Link>
class Held {
public:
struct One {
uint64_t key = 0;
int64_t place = 0;
Link link{};
uint64_t libs[4]{};
uint32_t n = 0;
};
explicit Held(size_t max = kLinksMax) : max_(max) {}
/// link held for key at place, linked from n libraries; into gone: the new one when key is held already, the farthest
/// ahead beyond max (the new one, when it is that). The caller destroys what goes (no draw has had it)
void put(uint64_t key, int64_t place, Link link, const uint64_t* libs, uint32_t n, std::vector<One>& gone) {
One o;
o.key = key, o.place = place, o.link = link, o.n = n > 4 ? 4 : n;
for (uint32_t i = 0; i < o.n; i++) o.libs[i] = libs[i];
if (place_of_.count(key)) {
gone.push_back(o);
return;
}
by_place_.emplace(std::make_pair(place, key), o);
place_of_.emplace(key, place);
while (by_place_.size() > max_) {
const auto last = std::prev(by_place_.end());
gone.push_back(last->second);
place_of_.erase(last->first.second);
by_place_.erase(last);
}
}
/// The link held for key, taken out (a draw's from now on): false when none is
bool take(uint64_t key, One& out) {
const auto it = place_of_.find(key);
if (it == place_of_.end()) return false;
const auto at = by_place_.find({it->second, key});
out = at->second;
by_place_.erase(at);
place_of_.erase(it);
return true;
}
/// Those placed before at, taken out into gone
void behind(int64_t at, std::vector<One>& gone) {
const auto end = by_place_.lower_bound({at, 0});
for (auto it = by_place_.begin(); it != end;) {
gone.push_back(it->second);
place_of_.erase(it->first.second);
it = by_place_.erase(it);
}
}
/// Those linked from library lib, taken out into gone (it is let go)
void drop_using(uint64_t lib, std::vector<One>& gone) {
for (auto it = by_place_.begin(); it != by_place_.end();) {
if (!linked_from(it->second, lib)) {
++it;
continue;
}
gone.push_back(it->second);
place_of_.erase(it->first.second);
it = by_place_.erase(it);
}
}
/// Whether a link at place would stay held: room left, or nearer than the farthest held (made, not let go at once)
bool room(int64_t place) const {
return by_place_.size() < max_ || (!by_place_.empty() && place < std::prev(by_place_.end())->first.first);
}
/// Whether a link held was linked from library lib
bool uses(uint64_t lib) const {
for (const auto& [at, o] : by_place_)
if (linked_from(o, lib)) return true;
return false;
}
/// Every link held, taken out into out, the nearest first
void all(std::vector<One>& out) {
for (const auto& [at, o] : by_place_) out.push_back(o);
by_place_.clear();
place_of_.clear();
}
size_t size() const { return by_place_.size(); }
private:
static bool linked_from(const One& o, uint64_t lib) {
for (uint32_t i = 0; i < o.n; i++)
if (o.libs[i] == lib) return true;
return false;
}
size_t max_;
std::map<std::pair<int64_t, uint64_t>, One> by_place_; // ((place, key): the nearest first)
std::unordered_map<uint64_t, int64_t> place_of_;
};
/// Of the libraries of the links that went (gone), those to let go with them, each once: the warm's own that no draw has had
/// (mine(lib)) and that no link still held uses (used(lib))
template <class One, class Mine, class Used>
std::vector<uint64_t> let_go_libs(const std::vector<One>& gone, Mine mine, Used used) {
std::vector<uint64_t> out;
std::unordered_set<uint64_t> seen;
for (const One& o : gone)
for (uint32_t i = 0; i < o.n; i++) {
const uint64_t lib = o.libs[i];
if (seen.insert(lib).second && mine(lib) && !used(lib)) out.push_back(lib);
}
return out;
}
} // namespace mkwvk::pipe_order

52
gpu/vk/pipe_release.h Normal file
View File

@@ -0,0 +1,52 @@
// (Triforce) The kept pipelines no draw has had, given back to the driver while the PC's RAM is short (vk_draw.cpp
// pipelines_release_tick). Apart from Vulkan so that tests/test_pipe_release.py runs it alone.
//
// Why: on a PC of 20 GB or more the precompile builds every pipeline the notes name before the title and keeps it for the
// session (vk_draw.cpp precompile_run). On v0.4.0-test9's 32 GB PC that was 14,535 graphics and 1,033 compute pipelines,
// and the working set grew by 3.6 GB over the precompile (5517 -> 9094 MB): about 235 KB a pipeline at most, its
// programs and notes in that too. The RAM was short all session after (900 to 1,500 MB available, 43 "RAM short" lines),
// with 9.9 GB of the working set outside the game's own memory. How many of those pipelines a session draws depends on
// where it goes; the ones no draw has had are the cheapest to give back: nothing recorded them, no thread's table holds
// them (only graphics_pipeline hands an entry out, and it marks the entry first), and the driver's own cache and ours
// build them again in well under a millisecond when a draw first needs one (the precompile measured 0.2-0.4 ms each
// with the caches warm), on a worker, the draw left out or waiting meanwhile, as for any pipeline new to a session.
// (The idea is NX2navi's pipe_release, which gives back the pipelines no draw has used while video memory is short; the
// code is this back end's own.)
//
// The rules: a pass at most every so many presents while the RAM is short (due), a batch at most a pass (take), and
// only entries the caller says may go: no draw has had them, they are built (ready, with a pipeline) and have no fast
// link. The caller holds the table's locks and destroys what take moved out.
#pragma once
#include <cstddef>
#include <cstdint>
#include <memory>
#include <vector>
namespace pipe_release {
constexpr size_t kBatch = 512;
/// Whether a pass is due: the RAM short now, and at least every presents since the last pass (last 0: none yet)
inline bool due(bool ram_short, uint64_t presents, uint64_t last, uint64_t every) {
return ram_short && (last == 0 || presents >= last + every);
}
/// Up to batch entries of map (key -> std::unique_ptr<Entry>) for which may_go(entry) says yes, taken out of it into out.
/// The others stay where they are. Returns how many were taken
template <class Map, class Entry, class MayGo>
size_t take(Map& map, size_t batch, std::vector<std::unique_ptr<Entry>>& out, MayGo&& may_go) {
size_t n = 0;
for (auto it = map.begin(); it != map.end() && n < batch;) {
if (it->second && may_go(*it->second)) {
out.push_back(std::move(it->second));
it = map.erase(it);
n++;
} else {
++it;
}
}
return n;
}
} // namespace pipe_release

199
gpu/vk/pipe_wait.h Normal file
View File

@@ -0,0 +1,199 @@
// (Triforce) What a draw does whose pipeline is not ready yet (vk_draw.cpp draw, graphics_pipeline), apart from Vulkan so that
// tests/test_pipe_wait.py runs it. NX2navi's pipeline-miss rules (runtime/vk_exec_draw.c prepare_draw_ex, frame_wait,
// last_wait, link_budget; runtime/vk_target_frames.h tfr_pass_recent; runtime/vk_pipeline.c lib_worker; by chrg6 and
// epicmeme546, at 62cc3b13) brought to Triforce's own design: the rules and their numbers are NX2navi's, the code is
// written for this renderer.
//
// What a draw did before, and still does by default: one whose pipeline a worker still builds was left out when its targets
// held a picture from an earlier frame (vk_draw.cpp targets_persistent) and its shaders write no memory, so that the next
// frame draws it; any other waited for its pipeline however long that took (RX_VK_PIPE_SKIP=0: every one waited for).
// Triforce links a pipeline from its program's stage libraries on the draw's own thread at its first miss (gpl_rules.h), so
// a draw whose libraries the driver has is never left out; what is not ready after that is a compile on a worker.
//
// The rules added, each with its switch:
// - kept(): a pass with a colour target drawn for the first time counts as one drawn once, whatever its depth target is
// (NX2navi's NX2NAVI_PIPE_SKIP=any). The game's item icon capture draws into a new colour target with the per-frame depth
// buffer, once: a draw left out there stayed missing from the icon until the item was captured again. Its draws wait, as
// for any target drawn once. RX_VK_PIPE_SKIP_NEW=0: the depth target alone keeps the pass, as before.
// - choose(), a full-screen draw (at most 6 vertices over half the pass or more: a lighting, fog or tone mapping pass) into
// kept targets: left out, its frame lacks the pass, a black or unlit picture with the interface intact. Where the hold of
// the last picture can take that frame (present_hold.h; the draws left out for a pipeline count as owed, vk_replay.cpp
// hold_step) it is left out and the frame is not shown, as before the hold existed for pipelines. Where it cannot (the hold
// at its bound, resting after one, or off) the draw waits for its pipeline from the frame's wait pool (NX2navi's
// NX2NAVI_PIPE_FRAME_WAIT_MS, kFrameWaitMs) and then from kLastWaitMs (NX2navi's FULLSCREEN_LAST_MS), and is left out only
// after both. NX2navi waits from the pool first and holds after; here the order is the other way: Triforce's link was tried
// at the miss already, so what the draw would wait for is a compile, and a held frame costs the game's thread nothing.
// RX_VK_PIPE_FRAME_WAIT_MS=<ms>: the pool; 0: left out, as before.
// - choose(), a draw into targets drawn once or a draw whose shaders write memory: waited for as before, unless
// RX_VK_PIPE_ONESHOT_BUDGET=1 (NX2navi's NX2NAVI_ONESHOT_BUDGET, on there): from the frame's wait pool, and left out once it
// is spent. Off here: a draw left out of a target the game draws once (a volume, a lookup table) may leave it wrong until
// the game draws it again, and a slow frame is better than a wrong one.
// - LinkBudget: the synchronous links of a frame (graphics_pipeline) take at most RX_VK_PIPE_LINK_MS (NX2navi's
// NX2NAVI_PIPE_LINK_MS, 6 there); past it a new pipeline is linked by a worker instead, its draws left out or waited for
// meanwhile. 0 (the default): every link on the draw's thread, as before. NX2navi's other half of that rule, the 3 ms a
// draw waits for one pipeline's link (NX2NAVI_PIPE_LINK_DRAW_MS), has no counterpart: its links are made by worker threads
// and the draw waits for them, while here the link is made in the draw.
// - libs_first(), a pipeline whose stage libraries were cold (or whose link the budget left to a worker): the worker builds
// those libraries first and links the pipeline from them, the entry ready with that link, and the whole optimised build
// follows as a job of its own (NX2navi's NX2NAVI_PIPE_FAST_WORKERS). Before, the whole build came first and the cold
// libraries after it, so the draws of such a pipeline waited for the whole build. The same objects are made either way.
// A draw that waits runs the first job itself (vk_jobs.cpp jobs_wait): the libraries and the link, not the whole build.
// RX_VK_PIPE_FAST_WORKERS=0: the whole build first, as before.
// A wait from a budget is vk_jobs.cpp jobs_wait_for: the pipeline's job goes to the front of the queue, and is not run on the
// draw's thread (a compile there would outlast the bound).
#pragma once
#include <algorithm>
#include <atomic>
#include <cstdint>
namespace pipe_wait {
inline constexpr int64_t kMs = 1000000;
inline constexpr int64_t kFrameWaitMs = 50; // (NX2navi's NX2NAVI_PIPE_FRAME_WAIT_MS default)
inline constexpr int64_t kLastWaitMs = 40; // (NX2navi's FULLSCREEN_LAST_MS)
inline constexpr int64_t kLinkMs = 6; // (NX2navi's NX2NAVI_PIPE_LINK_MS default; here 0 unless asked)
inline constexpr int64_t kMinWaitNs = kMs; // (a wait shorter than this is none: NX2navi counts its waits in whole ms)
inline constexpr int kFullVertices = 6; // (a full-screen triangle or quad, or a tile: NX2navi's count <= 6)
// ---- the pass: kept from an earlier frame, or drawn once
/// A target of the pass as kept() sees it: there, drawn in an earlier frame, or not drawn before this frame (drawn first in
/// this frame, or never)
struct Target {
bool there = false;
bool earlier = false;
};
inline Target target(bool there, uint64_t first_render_frame, uint64_t now) {
return {there, there && first_render_frame && first_render_frame < now};
}
/// Whether a draw into this pass may be left out while its pipeline builds: the game draws its targets frame after frame.
/// n colour targets c, the depth target d. new_colour_rule (RX_VK_PIPE_SKIP_NEW): a colour target not drawn before this
/// frame makes a pass whose depth target alone was drawn before one drawn once. Without it: any target drawn before keeps
/// the pass (targets_persistent, as before)
inline bool kept(const Target* c, int n, Target d, bool new_colour_rule) {
bool colour_earlier = false, colour_new = false;
for (int i = 0; i < n; i++) {
if (!c[i].there) continue;
colour_earlier |= c[i].earlier;
colour_new |= !c[i].earlier;
}
if (colour_earlier) return true;
return d.earlier && !(new_colour_rule && colour_new);
}
/// A full-screen draw: direct, at most kFullVertices vertices in one instance, over half the pass or more (the scissor's
/// area as the draw has it, the whole target when it has none)
inline bool full_screen(bool mesh, bool indirect, int count, int instances, uint64_t area, uint64_t pass_area) {
return !mesh && !indirect && count > 0 && count <= kFullVertices && instances <= 1 && pass_area &&
area * 2 >= pass_area;
}
// ---- a presented frame's budgets
/// What the draws of one presented frame may still spend (any thread; a new frame number fills it again). The draws of the
/// parallel replay's helper threads share it: a frame's total is approximate where two of them charge at once
class Pool {
public:
explicit Pool(int64_t per_frame_ns = 0) : per_frame_(per_frame_ns) {}
void per_frame(int64_t ns) { per_frame_ = ns; }
int64_t per_frame() const { return per_frame_; }
int64_t left(uint64_t frame) {
fill(frame);
return left_.load(std::memory_order_relaxed);
}
void charge(uint64_t frame, int64_t ns) {
fill(frame);
int64_t was = left_.load(std::memory_order_relaxed);
while (!left_.compare_exchange_weak(was, std::max<int64_t>(0, was - std::max<int64_t>(0, ns)), std::memory_order_relaxed)) {
}
}
private:
void fill(uint64_t frame) {
uint64_t was = frame_.load(std::memory_order_relaxed);
if (was != frame && frame_.compare_exchange_strong(was, frame, std::memory_order_relaxed))
left_.store(per_frame_, std::memory_order_relaxed);
}
int64_t per_frame_ = 0;
std::atomic<uint64_t> frame_{~0ull};
std::atomic<int64_t> left_{0};
};
/// The synchronous links of a frame (RX_VK_PIPE_LINK_MS): links() true while the frame's links have time left (always, with
/// no budget)
class LinkBudget {
public:
explicit LinkBudget(int64_t per_frame_ns = 0) : pool_(per_frame_ns) {}
bool on() const { return pool_.per_frame() > 0; }
bool links(uint64_t frame) { return !on() || pool_.left(frame) > 0; }
void spent(uint64_t frame, int64_t ns) {
if (on()) pool_.charge(frame, ns);
}
private:
Pool pool_;
};
// ---- the draw's choice
enum class Act { Skip, Wait, WaitAll };
enum Kind { K_NONE, K_ONESHOT, K_FULL, K_LAST };
struct Rules {
bool always_wait = false; // (RX_VK_PIPE_SKIP=0: every pipeline waited for)
bool oneshot_budget = false; // (RX_VK_PIPE_ONESHOT_BUDGET=1)
bool full_wait = true; // (RX_VK_PIPE_FRAME_WAIT_MS not 0)
};
struct Choice {
Act act = Act::Skip;
int64_t ns = 0; // (Wait: for at most this long, then left out)
Kind kind = K_NONE; // (Wait: which budget the time is charged to)
};
/// A draw whose pipeline is not ready. kept: its pass (kept()); writes: its shaders write memory; full: full_screen(); held:
/// the hold of the last picture would take this frame if it lacks a draw (present_hold.h DEFERS, the pipelines counted);
/// pool_ns, last_ns: what the frame's wait pool and its last wait have left
inline Choice choose(const Rules& r, bool kept, bool writes, bool full, bool held, int64_t pool_ns, int64_t last_ns) {
if (r.always_wait) return {Act::WaitAll, 0, K_NONE};
if (!kept || writes) {
if (!r.oneshot_budget) return {Act::WaitAll, 0, K_NONE};
if (pool_ns >= kMinWaitNs) return {Act::Wait, pool_ns, K_ONESHOT};
return {Act::Skip, 0, K_NONE};
}
if (!full || held || !r.full_wait) return {Act::Skip, 0, K_NONE};
if (pool_ns >= kMinWaitNs) return {Act::Wait, pool_ns, K_FULL};
if (last_ns >= kMinWaitNs) return {Act::Wait, last_ns, K_LAST};
return {Act::Skip, 0, K_NONE};
}
// ---- a pipeline whose libraries were cold: the worker's order
/// The job a draw may wait for (its entry's ready flag), for a pipeline the draw could not link: cold names the libraries the
/// draw found cold (0: not), link_later says the draw did not try (LinkBudget). link(cold) links from the libraries, naming
/// those still cold in cold (gpl_link); build_cold(i, key) builds one (gpl_build_cold); linked(pipeline) makes the entry
/// ready with a link (gpl_rules::linked); whole_later() queues the whole build over it as a job of its own, whole_now() builds
/// it here and makes the entry ready with it (gpl_rules::built). Returns true when the entry is ready with a link
template <class Pipeline, class Link, class BuildCold, class Linked, class WholeLater, class WholeNow>
bool libs_first(uint64_t cold0, uint64_t cold1, bool link_later, Link link, BuildCold build_cold, Linked linked,
WholeLater whole_later, WholeNow whole_now) {
uint64_t cold[2] = {cold0, cold1};
Pipeline l{};
if (link_later) l = link(cold);
if (l == Pipeline{} && (cold[0] || cold[1])) {
for (int i = 0; i < 2; i++)
if (cold[i]) build_cold(i, cold[i]);
uint64_t still[2] = {0, 0};
l = link(still);
}
if (!(l == Pipeline{}) && linked(l)) {
whole_later();
return true;
}
whole_now();
return false;
}
} // namespace pipe_wait

301
gpu/vk/pipe_warm.h Normal file
View File

@@ -0,0 +1,301 @@
// (Triforce) The pipelines of a load made behind its draws (vk_draw.cpp pipelines_warm, called by the program read-ahead's
// workers: vk_shader.cpp readahead_main). Apart from Vulkan so that tests/test_pipe_warm.py runs it alone.
//
// Why: v0.4.0-test16's load at Castle Town spent 59.6, 79.5 and 37.3 ms a frame in "pipelines" on the thread replaying
// (frames 8769, 8789 and 8793), with 267, 203 and 99 programs new to the draws and no waits for a pipeline. That is the fast
// link at each new pipeline's first draw: its two stage libraries made from the caches (the driver had all 1654 the session
// asked for, none cold) and their link (1.35 ms for 6.01 links a frame over that session, some 0.2 ms a link). That PC's
// precompile keeps no pipelines (the RAM spared: precompile_keeps_all), so every pipeline of a load is new at its first
// draw. The program read-ahead reads a load's programs on four workers before their draws, in the notes' order
// (program_readahead.h), and the notes hold each pipeline's whole state. So the worker that has just read a program, before
// any draw can have it, makes the stage libraries of the pipelines the notes name for it, asking the caches only, and links
// them. The draw finds the link made and goes on; the worker's whole build follows it as before.
//
// The rules, here and in vk_draw.cpp:
// - level (RX_VK_PIPELINE_WARM): 0 none, as before; 1 the stage libraries only; 2 (the default) the libraries and the links
// - only where the precompile keeps no pipelines, with the fast link on and asking the caches only (gpl_ask_only). A
// library the driver would have to compile is left to its draw, which hands it to a worker as before. Nothing is made
// while the RAM is low or short (may_warm)
// - at most kNotesPerVariant notes a program variant (the first in the notes' order), each variant once a session
// - what it holds is bounded: at most kLinksMax links made for draws that have not come (the oldest let go beyond: Held),
// at most kLibsMax stage libraries made that no draw has had (none made beyond). A library or link a draw has had is the
// draw's from then on, as if the draw had made it
// - while the RAM is short, what it holds goes back to the driver (vk_draw.cpp warm_let_go, from pipelines_release_tick):
// none of it was ever in a recording, and no warm has one of its libraries in hand then
// - ((Triforce, wp18) with the last sessions' order of pipelines (pipe_order.h, RX_VK_PIPELINE_ORDER), the pipelines that
// order names for a variant read and no other, their links held by their place in it with its own bounds; the notes'
// others are not made then, and the warm does not follow the draws)
#pragma once
#include <algorithm>
#include <cstddef>
#include <cstdint>
#include <cstdlib>
#include <deque>
#include <iterator>
#include <list>
#include <mutex>
#include <unordered_map>
#include <unordered_set>
#include <vector>
namespace pipe_warm {
constexpr size_t kNotesPerVariant = 32; // (notes warmed a program variant, at most)
constexpr size_t kLinksMax = 512; // (links held for draws that have not come: about one load's, as the shelf's 512 programs)
constexpr size_t kLibsMax = 1024; // (stage libraries made that no draw has had: two a program for the shelf's 512)
/// RX_VK_PIPELINE_WARM: unset 2; 0, 1 or 2 as given; below 0 is 0, above 2 is 2
inline int level(const char* env) {
if (!env || !*env) return 2;
const int v = atoi(env);
return v < 0 ? 0 : v > 2 ? 2 : v;
}
/// Whether a worker makes anything now: a level, the fast link on, asking the caches only, the RAM neither low nor short
inline bool may_warm(int level, bool fast_link, bool ask_only, bool ram_low, bool ram_short) {
return level > 0 && fast_link && ask_only && !ram_low && !ram_short;
}
/// The notes' records by program variant (the caller's key for it), at most per a variant, in the order they are added
inline void index_add(std::unordered_map<uint64_t, std::vector<uint32_t>>& index, uint64_t variant, uint32_t rec,
size_t per = kNotesPerVariant) {
std::vector<uint32_t>& recs = index[variant];
if (recs.size() < per) recs.push_back(rec);
}
/// Whether the warm may make one more stage library, with made of its own that no draw has had
inline bool lib_room(size_t made, size_t max = kLibsMax) { return made < max; }
/// The links held for draws that have not come, by pipeline key, each with the keys of the libraries it was linked from.
/// At most max: beyond, the oldest goes. Not thread-safe: the caller's lock
template <class Link>
class Held {
public:
struct One {
uint64_t key = 0;
Link link{};
uint64_t libs[4]{};
uint32_t n = 0;
};
explicit Held(size_t max = kLinksMax) : max_(max) {}
/// link held for key, linked from n libraries; what goes into gone: the new one when key is held already, the oldest
/// beyond max. The caller destroys what goes (no draw has had it)
void put(uint64_t key, Link link, const uint64_t* libs, uint32_t n, std::vector<Link>& gone) {
if (by_key_.count(key)) {
gone.push_back(link);
return;
}
One o;
o.key = key, o.link = link, o.n = n > 4 ? 4 : n;
for (uint32_t i = 0; i < o.n; i++) o.libs[i] = libs[i];
order_.push_back(o);
by_key_[key] = std::prev(order_.end());
while (order_.size() > max_) {
gone.push_back(order_.front().link);
by_key_.erase(order_.front().key);
order_.pop_front();
}
}
/// The link held for key, taken out (a draw's from now on): false when none is
bool take(uint64_t key, One& out) {
const auto it = by_key_.find(key);
if (it == by_key_.end()) return false;
out = *it->second;
order_.erase(it->second);
by_key_.erase(it);
return true;
}
/// ((Triforce, test18) the links held that were linked from library lib, taken out into gone: it is let go)
void drop_using(uint64_t lib, std::vector<Link>& gone) {
for (auto it = order_.begin(); it != order_.end();) {
bool uses = false;
for (uint32_t i = 0; i < it->n; i++) uses |= it->libs[i] == lib;
if (!uses) {
++it;
continue;
}
gone.push_back(it->link);
by_key_.erase(it->key);
it = order_.erase(it);
}
}
/// ((Triforce, wp18) whether a link held was linked from library lib: pipe_order.h let_go_libs)
bool uses(uint64_t lib) const {
for (const One& o : order_)
for (uint32_t i = 0; i < o.n; i++)
if (o.libs[i] == lib) return true;
return false;
}
/// Every link held, taken out into out, the oldest first
void all(std::vector<Link>& out) {
for (const One& o : order_) out.push_back(o.link);
order_.clear();
by_key_.clear();
}
size_t size() const { return order_.size(); }
private:
size_t max_;
std::list<One> order_; // (the oldest first)
std::unordered_map<uint64_t, typename std::list<One>::iterator> by_key_;
};
// ---- (Triforce, test18) Following the draws. In the owner's test18 session the warm made 2266 links and 2462 libraries and
// its draws took 276 of the links, of the 1659 pipelines the session drew (all but 15 of them noted). The read-ahead reads 32
// programs after each one the draws take, far more than a load takes, and the warm made everything it read: by the end of the
// load (the line at frame ~3540) the libraries no draw had had were at their bound of 1024, so nothing read after that had a
// library made or a link (the warm makes no link without all its libraries), and 1207 links had been let go as the oldest
// beyond 512, among them ones the load came to later. With RX_VK_PIPELINE_WARM_FOLLOW=1 (off by default since test19, below):
// - where the draws are (Front): the last kPoints places in the read-ahead's order (the notes', then the compute programs')
// the draws consumed (took a program read ahead, or read one themselves), each with its time;
// - a program read within kLinkAhead places after one of those consumed in the last kRecentMs has its libraries made and
// linked; one within kLibAhead its libraries only (cheap, and shared by its other pipelines: its draw links them itself, a
// fraction of a millisecond); one farther, or with no recent place before it (the draws stopped taking), nothing (want);
// - the libraries no draw has had are kept up to kLibsKeep, the oldest let go beyond (Made; the held links made from one
// go with it, Held::drop_using), so the bound never stops the warm as kLibsMax did;
// Off (the default): every program read is warmed whole and the libraries stop at kLibsMax, as before.
// tests/pipe_warm_sim.cpp runs both ways through a synthetic load.
//
// (Triforce, test18) The compute pipelines too (vk_draw.cpp warm_compute). test18's load spent 67.6 ms of its first frame in
// "dispatch": that PC's precompile builds no compute pipelines either (built once for the caches, then left to play), so the
// first dispatch of each compute program made its pipeline on the thread replaying, asked of the caches (a hit, but a
// creation all the same), or queued it and waited up to 50 ms. The read-ahead reads the compute programs play dispatched
// (triforce_computes.bin, after the notes' programs); the worker that has read one makes its pipeline the same way, of the
// caches only, and the dispatch that takes the program has it. One a program on the read-ahead's shelf (512 at most, and the
// game lists 286), destroyed when the shelf lets the program go, given back from the shelf while the RAM is short.
// RX_VK_COMPUTE_WARM=1 to have it; off by default since test19 (below): none, as before.
constexpr size_t kPoints = 16; // (the draws' latest places kept)
constexpr int64_t kLinkAhead = 64; // (places after a recent one: libraries and links)
constexpr int64_t kLibAhead = 192; // (places after a recent one: libraries only)
constexpr uint64_t kRecentMs = 2000; // (a place older than this is not where the draws are)
constexpr size_t kLibsKeep = 2048; // (following: the libraries no draw has had, kept; the oldest let go beyond)
/// On unless "0"
inline bool on(const char* env) { return !(env && env[0] == '0' && !env[1]); }
/// ((Triforce, test19) RX_VK_PIPELINE_WARM_FOLLOW and RX_VK_COMPUTE_WARM: off unless "1" (opt_in))
inline bool opt_in(const char* env) { return env && env[0] == '1' && !env[1]; }
// ---- (Triforce, test19) What the warm may cost. The lab's test19 tour (w19-new-1, the one run of four whose precompile kept
// no pipelines, so the one where the warm ran at all; the other three kept every pipeline, and there the warm and its switches
// do nothing) had the warm spend 2538 ms on the read-ahead's four workers, 6862 libraries and 3662 links made for 556 links
// taken, 3143 libraries let go as the oldest, and 2833 compute pipelines made for the 378 compute programs the game lists: the
// read-ahead's shelf let 7647 programs go and read many again, and the warm made them again. A worker warming is a worker not
// reading: that run's draws read 500 programs themselves, and waited 28 times (69 ms) for one being read. So:
// - at most kBudgetMsPerSecond of the workers' time a second goes to the warm, kBudgetBurstMs saved up at most (Budget): past
// it, a program read is parked as it is, its draws link it as before;
// - nothing is made twice in a session: a library the warm let go as the oldest is not made again by the warm (a draw that
// needs it makes it), a compute pipeline is made once a program variant (Once)
constexpr double kBudgetMsPerSecond = 250; // (a quarter of one worker)
constexpr double kBudgetBurstMs = 500;
/// The workers' time the warm may spend: per_second ms a second, burst saved up at most. Any thread
class Budget {
public:
explicit Budget(double per_second = kBudgetMsPerSecond, double burst = kBudgetBurstMs)
: per_second_(per_second), burst_(burst), tokens_(burst) {}
/// Whether the warm may begin something now (anything left of the budget)
bool allows(uint64_t now_ms) {
std::lock_guard lk(mu_);
refill(now_ms);
return tokens_ > 0;
}
/// ms of the workers' time spent (the budget may go below nothing: what is spent is spent)
void spent(double ms, uint64_t now_ms) {
std::lock_guard lk(mu_);
refill(now_ms);
tokens_ -= ms;
}
private:
void refill(uint64_t now_ms) {
if (last_ && now_ms > last_) tokens_ = std::min(burst_, tokens_ + (double)(now_ms - last_) * per_second_ / 1000.0);
last_ = now_ms;
}
std::mutex mu_;
double per_second_, burst_, tokens_;
uint64_t last_ = 0;
};
/// Keys done once a session: first(key) is true the first time only. Not thread-safe: the caller's lock
class Once {
public:
bool first(uint64_t key) { return done_.insert(key).second; }
bool done(uint64_t key) const { return done_.count(key) != 0; }
size_t size() const { return done_.size(); }
private:
std::unordered_set<uint64_t> done_;
};
enum class Want { none, libraries, links };
/// What the warm makes of a program that far (Front::ahead) after the nearest place the draws consumed lately
inline Want want(int64_t ahead, int64_t link_ahead = kLinkAhead, int64_t lib_ahead = kLibAhead) {
if (ahead < 0 || ahead > lib_ahead) return Want::none;
return ahead <= link_ahead ? Want::links : Want::libraries;
}
/// Where the draws are: their latest places in the read-ahead's order, with when. Any thread
class Front {
public:
void consumed(int64_t at, uint64_t ms) {
if (at < 0) return;
std::lock_guard lk(mu_);
points_[next_++ % kPoints] = {at, ms, true};
}
/// How far at lies after the nearest place at or before it consumed in the last recent_ms (0: that place); -1: none
int64_t ahead(int64_t at, uint64_t now_ms, uint64_t recent_ms = kRecentMs) const {
if (at < 0) return -1;
std::lock_guard lk(mu_);
int64_t best = -1;
for (const Point& p : points_) {
if (!p.set || p.at > at || (now_ms > p.ms && now_ms - p.ms > recent_ms)) continue;
if (best < 0 || at - p.at < best) best = at - p.at;
}
return best;
}
private:
struct Point {
int64_t at = 0;
uint64_t ms = 0;
bool set = false;
};
mutable std::mutex mu_;
Point points_[kPoints];
size_t next_ = 0;
};
/// The libraries the warm made, in the order it made them: the oldest still the warm's (still(key)) is the one let go.
/// Not thread-safe: the caller's lock
class Made {
public:
void add(uint64_t key) { q_.push_back(key); }
/// The oldest key still(key) says is the warm's, taken out into key; false when there is none. Keys a draw had since
/// are passed over and dropped
template <class Still>
bool oldest(uint64_t& key, Still still) {
while (!q_.empty()) {
key = q_.front();
q_.pop_front();
if (still(key)) return true;
}
return false;
}
/// The keys no longer the warm's dropped once the list is four times max (a draw had them)
template <class Still>
void trim(size_t max, Still still) {
if (q_.size() <= 4 * max) return;
std::deque<uint64_t> keep;
for (const uint64_t k : q_)
if (still(k)) keep.push_back(k);
q_.swap(keep);
}
size_t size() const { return q_.size(); }
private:
std::deque<uint64_t> q_;
};
} // namespace pipe_warm

39
gpu/vk/prepare_threads.h Normal file
View File

@@ -0,0 +1,39 @@
// (Triforce) How many workers the shader preparation before the title runs on (vk_draw.cpp: prepare_run, warm_libraries,
// state_list_prepare), apart from Vulkan so that tests/test_prepare_threads.py can run it.
//
// It was half the processors, 32 at most: "below normal, and half the CPUs: on all of them at High the driver's compiles
// froze the whole PC". That was a process started at High priority (scripts/run-vk.ps1 starts its test runs so): a thread
// below normal in such a process is still above every program of normal priority, and with one on each processor nothing
// else got a turn. A player's game is a process of normal priority, where a worker below normal gives way to everything
// else on the PC, and nothing of the game runs during the preparation: there the other half of the processors stood idle
// through a wait of minutes (165 s with 12 workers on a 24-thread PC). NX2navi prepares on every processor it is given.
//
// So: every processor, 64 at most, unless the process was started above normal priority (half, as before). Nothing
// bounded the memory a worker takes: each holds one program's translation at a time and the driver's compiler for one
// pipeline, and how much the driver's part is has not been measured here (the compute programs go first, so every worker
// is in the driver's compiler at once when the preparation begins). So the workers past half are given only what RAM is
// free when the preparation starts, 1 GB each; with the RAM not known, half. Never fewer than half: what it was.
// RX_VK_PREPARE_THREADS=<n> sets the number outright, 1 to 64.
#pragma once
#include <algorithm>
#include <cstdint>
namespace prepare_threads {
/// The free RAM a worker past half the processors must find when the preparation starts
inline constexpr uint64_t kRamAWorker = 1ull << 30;
/// What it was, and what the preparation behind the game is counted from: half the processors, 1 to 32
inline unsigned half(unsigned cpus) { return std::clamp(cpus / 2, 1u, 32u); }
/// cpus: the logical processors; ram_free: the physical memory free now, 0 when it is not known; above_normal: the process
/// was started above normal priority; asked: RX_VK_PREPARE_THREADS, 0 or less when it is not set
inline unsigned count(unsigned cpus, uint64_t ram_free, bool above_normal, int asked) {
if (asked > 0) return (unsigned)std::min(asked, 64);
const unsigned least = half(cpus), most = std::clamp(cpus, 1u, 64u);
if (above_normal || !ram_free) return least;
return (unsigned)std::clamp<uint64_t>(ram_free / kRamAWorker, least, most);
}
} // namespace prepare_threads

90
gpu/vk/present_hold.h Normal file
View File

@@ -0,0 +1,90 @@
// (Triforce) Hold the last picture while new textures come in (vk_replay.cpp do_present), apart from Vulkan so that
// tests/test_present_hold.py can run it.
//
// At an area change one frame took every new texture's upload at once: 523 ms for 119 uploads of 363 MB in the owner's test5
// session, 107 and 164 ms for some 400 in others. New textures now have a budget a frame too (vk_texture.cpp
// new_texture_later): once a frame's uploads took it, a new texture a draw samples is left for the next frame, and the draw
// samples the dummy in its place. Asked what should show meanwhile, the owner chose "hold the last picture": a frame drawn
// with any texture still owed its first bytes is not shown, the window gets the last complete picture again, and the game
// goes on stepping underneath. This is the rule of when a frame is held and when the hold ends:
//
// - a frame drawn with every texture it sampled complete is shown, and ends a hold;
// - a frame drawn with a texture owed its first bytes is held, while the hold is younger than its bound (RX_VK_HOLD_MS,
// 250 ms by default);
// - once a hold reaches its bound no texture is left for later any more (the flush) and the frame after that is shown
// whatever it holds, which ends the hold: a picture is never held for longer than the bound and the frame after it;
// - after a hold that reached its bound no hold begins, and nothing is left for later, for a rest of 120 presents (2 s at
// 60), twice as long after each such hold that follows (at most 1920), back to 120 once a hold ends by itself or the
// presents since the rest's end are more than 600: a card that cannot keep its textures (one at its video memory
// allowance, whose render targets are made again from memory frame after frame) gets its frames as before, not a frozen
// picture every few seconds.
#pragma once
#include <algorithm>
#include <cstdint>
namespace present_hold {
/// What the draws of the next frame may do with a new texture once the frame's uploads took their budget (vk_texture.cpp)
enum Mode {
AS_BEFORE = 0, // no hold: as before the hold (a new ASTC texture's own rule, upload_defer, draws the placeholder)
DEFERS = 1, // left for the next frame (the frame is then held)
FLUSH = 2, // the hold's bound: none left, every upload now
};
inline constexpr uint64_t kRest = 120, kRestMax = 1920, kQuiet = 600;
struct State {
bool holding = false;
int64_t since_ns = 0; // (the first held present's time)
int64_t frames = 0; // (this hold's frames so far)
bool flush = false; // (this hold reached its bound)
uint64_t rest_until = 0; // (no hold before this present)
uint64_t rest = kRest;
// (counts, for the log: all of the session)
int64_t holds = 0, held_frames = 0, held_ns = 0, longest_ns = 0, bounded = 0;
// (the hold that ended at the last decide, for its log line: frames, time, and whether at the bound; 0 frames: none)
int64_t ended_frames = 0, ended_ns = 0;
bool ended_bounded = false;
};
/// What the draws of the frame after present `present` may do (bound_ns 0: the hold is off)
inline Mode mode(const State& s, int64_t bound_ns, uint64_t present) {
if (bound_ns <= 0) return AS_BEFORE;
if (s.flush) return FLUSH;
return present >= s.rest_until ? DEFERS : AS_BEFORE;
}
/// The frame at present `present` (counted from 1), at `now_ns`: drawn with a texture owed its first bytes (owed) or with
/// every one complete. true: it is held (the window keeps the picture before it), false: it is shown
inline bool decide(State& s, int64_t bound_ns, bool owed, int64_t now_ns, uint64_t present) {
s.ended_frames = 0, s.ended_ns = 0, s.ended_bounded = false;
if (bound_ns <= 0 && !s.holding) return false;
if (s.holding) {
const int64_t held = std::max<int64_t>(0, now_ns - s.since_ns);
if (!owed || s.flush || bound_ns <= 0) { // (complete, or the frame after the bound: shown, the hold over)
s.holding = false;
s.held_ns += held, s.longest_ns = std::max(s.longest_ns, held);
s.ended_frames = s.frames, s.ended_ns = held, s.ended_bounded = s.flush;
if (s.flush) {
s.bounded++;
s.rest_until = present + s.rest;
s.rest = std::min(s.rest * 2, kRestMax);
} else {
s.rest = kRest;
}
s.flush = false;
return false;
}
if (held >= bound_ns) s.flush = true; // (this one still held; the next drawn with every upload made)
s.frames++, s.held_frames++;
return true;
}
if (!owed || present < s.rest_until) return false; // (complete, or resting: shown as it is)
if (present >= s.rest_until + kQuiet) s.rest = kRest;
s.holding = true, s.since_ns = now_ns, s.frames = 1, s.flush = false;
s.holds++, s.held_frames++;
return true;
}
} // namespace present_hold

32
gpu/vk/present_mode.h Normal file
View File

@@ -0,0 +1,32 @@
/* The swapchain's present mode (vk_main.cpp create_swapchain), as plain code: which of the modes a surface lists is asked
for, by the Insert menu's VSync row and the two switches. No Vulkan: tests/test_vsync_row.py checks it.
VSync on, the default, is the choice from before the row. Mailbox where the driver has it: the newest finished frame at
each refresh of the display, no tearing, and nothing held back (the game paces itself, host/nn_vi.c). Without mailbox,
immediate on Windows (AMD's driver has no mailbox, and FIFO held anything under 60 fps at 30) and FIFO elsewhere
(immediate on NVIDIA's X11 path turns vblank off for every output, so other monitors and programs tear).
VSync off asks for immediate wherever the surface lists it: a frame is on screen the moment it is presented, up to a
refresh sooner, and torn where that falls inside a refresh. It lifts no limit: the frame rate is the game's own cap
either way. Where On is immediate already (Windows without mailbox) Off changes nothing, and where the surface lists
no immediate Off is On.
RX_VK_FIFO (set) forces FIFO, and RX_VK_IMMEDIATE (set) asks for immediate, over the row, as before it. */
#pragma once
namespace mkwvk::present_mode {
enum Mode { kFifo = 0, kMailbox = 1, kImmediate = 2 };
/// The mode to ask for. mailbox, immediate: the surface lists them (FIFO always is there). vsync: the VSync row.
/// windows: a Windows build. force_fifo, want_immediate: RX_VK_FIFO and RX_VK_IMMEDIATE are set
inline Mode choose(bool mailbox, bool immediate, bool vsync, bool windows, bool force_fifo, bool want_immediate) {
if (force_fifo) return kFifo;
const Mode waits = mailbox ? kMailbox : kFifo;
const bool at_once = want_immediate || !vsync || (windows && waits == kFifo);
return at_once && immediate ? kImmediate : waits;
}
inline const char* name(int mode) { return mode == kMailbox ? "mailbox" : mode == kImmediate ? "immediate" : "fifo"; }
} // namespace mkwvk::present_mode

82
gpu/vk/present_rate.h Normal file
View File

@@ -0,0 +1,82 @@
// (Triforce) The presents a second the textures' waits go by (vk_texture.cpp texture_frame, texture_sweep), apart from
// Vulkan so that tests/test_present_rate.py can run it. After NX2navi's runtime/vk_tex_release.h (trl_rate_sample,
// trl_rate, trl_presents) and its vk_texture.c rate_tick.
//
// The eviction's waits count presents: how long a texture has gone unused before it may go (60, RX_VK_EVICT_MIN_UNUSED;
// 120 in a pass under the budget), how long ago the game let go of one (30, RX_VK_LET_GO_AFTER), how long one the GPU
// drew stays (600 near what the driver allows, else 3600: keep_rendered.h), the compaction's hold and its reserve (600
// and 1800: vram_compact.h). They were set at 60 presents a second: a second, half a second, ten seconds and a minute,
// ten and thirty seconds. Below 60 they last longer, at 30 twice as long, and the card short of video memory is the one
// that runs slow (past what the driver allows, the driver pages: 14 fps on an RTX 3060 Ti, 5 on an RTX 4050). There a
// texture stays out of the eviction's reach four times as long, and what the waits hold grows with their length: the
// game lets go and loads by the second, as the player moves. NX2navi measured it (an RTX 4070 SUPER at 2560x1440,
// 2026-11-14): its 120-present wait held 442 MiB at 60 presents a second and 2,105 MiB at 21, on a card then 600 MiB over
// its budget.
//
// So below 60 an age in presents counts for what it is at 60 a second (at60): the waits are the same seconds at 30 as at
// 60. The rate is that of the last 8 seconds, the highest of them, so that a slow second or a few (an area's load, a
// movie's start, a window not in front) shorten nothing and only 8 slow seconds in a row do; fewer than 15 a second count
// as 15. The rate is rounded up, so no wait is shorter than its seconds at 60. RX_VK_TEX_RATE_LOW=0: presents below 60,
// as before.
//
// (Triforce, after NX2navi's M19: vk_texture.c 177-181, vk_tex_release.h TRL_RATE_MAX, chrg6 69945636) Above 60 too: the
// waits were the presents they were there, so at 94 a second (the owner's Market) "60 presents unused" was 0.64 s and
// "30 presents after the game let it go" 0.32 s. NX2navi found its textures going three times as readily as meant at 85 to
// 240 a second, and the ones still on screen showing their stand-ins until they were made again. Now an age above 60 counts
// for what it is at 60 too, up to 480 a second (more count as 480): the waits are their seconds at any rate, and longer in
// presents above 60. RX_VK_TEX_RATE_HIGH=0: presents above 60, as before.
#pragma once
#include <algorithm>
#include <cstdint>
namespace present_rate {
/// A rate is a multiple of 60 presents a second in 1/256: 256 is 60 a second, 64 is 15, 2048 is 480
inline constexpr uint32_t k60 = 256, kLeast = 64, kMost = 2048;
/// The seconds whose highest rate counts
inline constexpr int kSeconds = 8;
/// The rate of `presents` in `ns`, rounded up: an age is then taken for no more than its seconds at 60, and a wait is never
/// shorter than at 60. low: 15 to 60 a second below 60 (false: 60 there, the rule before); high: up to 480 above 60 (false:
/// 60 there, the rule before)
inline uint32_t sample(uint64_t presents, uint64_t ns, bool low, bool high) {
if (!ns || (!low && !high)) return k60;
presents = std::min<uint64_t>(presents, 1ull << 24), ns = std::min<uint64_t>(ns, 1ull << 56); // (no overflow below)
const uint64_t r = (presents * 256ull * 1000000000ull + ns * 60 - 1) / (ns * 60);
const uint32_t least = low ? kLeast : k60, most = high ? kMost : k60;
return r < least ? least : r > most ? most : (uint32_t)r;
}
/// The rate of the last n samples (one a second): the highest of them. A sample not taken yet (0) counts as 60 a second:
/// the first n seconds of a run are never below it
inline uint32_t of(const uint32_t* ring, int n) {
uint32_t m = kLeast;
for (int i = 0; i < n; i++) m = std::max(m, ring[i] ? ring[i] : k60);
return m;
}
/// An age of n presents at the rate, in presents at 60 a second: the same wall time. Below 60 never less than n, above 60
/// never more
inline uint64_t at60(uint64_t n, uint32_t rate) {
return rate && rate != k60 ? n * k60 / rate : n;
}
/// The rate as the presents come: tick at each present with the count of presents and a clock in ns
struct Clock {
uint32_t ring[kSeconds] = {};
uint32_t next = 0, rate = k60;
uint64_t t0 = 0, f0 = 0;
bool begun = false;
/// True when a second's sample was taken (the rate may have changed)
bool tick(uint64_t presents, uint64_t now_ns, bool low, bool high) {
if (!begun) return begun = true, t0 = now_ns, f0 = presents, false;
if (now_ns - t0 < 1000000000ull) return false;
ring[next++ % kSeconds] = sample(presents - f0, now_ns - t0, low, high);
t0 = now_ns, f0 = presents;
rate = of(ring, kSeconds);
return true;
}
};
} // namespace present_rate

81
gpu/vk/present_smooth.h Normal file
View File

@@ -0,0 +1,81 @@
// (Triforce, after NX2navi's runtime/vk_backend.c pace_smooth: its behaviour, not its code, whose lines are epicmeme546's)
// The Frame pacing row on the plain present (vk_replay.cpp present_image), with RX_VK_PACE_SMOOTH=1: a picture of the
// game's goes out no sooner than seven eighths of the recent interval after the one before it. Plain C with no system
// call, so that tests/test_present_smooth.py runs it on frame times of its own (as C and as C++).
//
// The row ("When each frame is shown ... At an even interval: less stutter") only ever spaced the frames of frame
// generation's display thread (vk_framegen.cpp display_main, RX_VK_PACE); without frame generation every picture went out
// the moment the replay finished it. The game hands its frames over unevenly although each costs about the same: 12 to 28
// ms apart around a mean of 19 when the display thread was written, 13 to 31 ms around 22 in a village in NX2navi (the same
// game), and shown as they come that is a judder at a rate that would look even. Here only a picture that comes early
// waits, until seven eighths of the recent interval after the last one; a late one goes at once, and the rate stays what it
// was but for the early ones' few milliseconds. The recent interval follows each picture's by a quarter, so it follows a
// scene that gets lighter within about a second; an interval of PRESENT_SMOOTH_HITCH_NS or more, or more than three times
// the recent one (a load, a hitch; twice would take the village's 13/31 ms frames for hitches), is not the rate and is left out of it.
//
// ((Triforce, 11-16) A picture waits at most PRESENT_SMOOTH_MAX_NS (8 ms) and at most half the recent interval, and where
// the floor is further than that it waits that long, not to the floor. The owner's test16 held a picture 41 ms at the start
// (frames 60 to 70 ms apart, under the 100 ms hitch line, had become the recent interval, and the old bound was 50 ms) and
// up to 12 ms while their save loaded; in play the waits were 0.3 to 1.7 ms on average, which this leaves as they were.)
//
// On by default since the lab's A/B (vk_replay.cpp smooth_wanted; RX_VK_PACE_SMOOTH=0 off). The present was on the game's
// presenting thread here (NX2navi's has a present thread of its own), so a picture's wait also held the start of the
// game's next frame on that thread: hence the bounds. ((Triforce) With the present thread, RX_VK_PRESENT_THREAD (present_thread.h),
// the wait is that thread's, and the presenting thread waits only where its next picture comes before this one is out.)
#pragma once
#include <stdint.h>
#define PRESENT_SMOOTH_HITCH_NS 100000000ll // an interval this long or longer is a load or a hitch, not the rate
#define PRESENT_SMOOTH_MAX_NS 8000000ll // no picture is held longer than this (nor longer than half the recent interval)
typedef struct {
int64_t avg_ns; // the recent interval, 0 before the second picture
int64_t last_ns; // when the last picture went out, 0 before the first (and after present_smooth_reset)
} PresentSmooth;
// The time a picture ready at now goes out: now, or toward the floor (seven eighths of the recent interval after the last
// picture) where that is later, waiting at most PRESENT_SMOOTH_MAX_NS and half the recent interval
static inline int64_t present_smooth_at(const PresentSmooth* s, int64_t now) {
if (!s->last_ns || s->avg_ns <= 0) return now;
const int64_t soonest = s->last_ns + s->avg_ns / 8 * 7;
if (soonest <= now) return now;
int64_t most = s->avg_ns / 2;
if (most > PRESENT_SMOOTH_MAX_NS) most = PRESENT_SMOOTH_MAX_NS;
return soonest - now <= most ? soonest : now + most;
}
// ((Triforce) The Steady frame rate row, host/pace_steady.h) With the game's frames paced to a period they fit in (paced_ns,
// over 0), the floor is that period after the last picture less a 64th of it, in place of seven eighths of the recent interval:
// the frames start a period apart, but each one's work after its start still swings, and each picture going out as it is
// done leaves the presents as uneven as the work (the lab's Market: 10.6 ms frames, a millisecond either way). Held to the
// period, a picture goes out a period after the one before however early its frame was done; the 64th lets the pictures come
// back toward their frames' ends by that much a frame after a slow one, so that the delay it added does not stay. A late
// picture goes at once, as before, and none waits more than PRESENT_SMOOTH_MAX_NS or half the period. paced_ns 0:
// present_smooth_at
static inline int64_t present_smooth_paced_at(const PresentSmooth* s, int64_t now, int64_t paced_ns) {
if (paced_ns <= 0) return present_smooth_at(s, now);
if (!s->last_ns) return now;
const int64_t soonest = s->last_ns + paced_ns - paced_ns / 64;
if (soonest <= now) return now;
int64_t most = paced_ns / 2;
if (most > PRESENT_SMOOTH_MAX_NS) most = PRESENT_SMOOTH_MAX_NS;
return soonest - now <= most ? soonest : now + most;
}
// A picture went out at t: the recent interval moves a quarter of the way to its interval, an interval counted at most as one
// and a half times the recent one. ((Triforce, 11-16) The lab's Market after a tour of areas had re-upload bursts in about 13%
// of its frames (18-26 ms against 10.4); counted whole they pulled the recent interval to 15-18 ms, and 75% of the frames then
// waited 2.4-2.8 ms for a floor the steady frames never needed (~70 fps where ~80 was possible: triforce-frame-stability.md).
// Counted at most 1.5x, such bursts leave it within a tenth of the steady rate. (A median was tried: the shown intervals that
// waited for the floor pull it down, and the floor follows it down; the average keeps the frames' total time)
static inline void present_smooth_shown(PresentSmooth* s, int64_t t) {
if (s->last_ns && t > s->last_ns && t - s->last_ns < PRESENT_SMOOTH_HITCH_NS && (s->avg_ns <= 0 || t - s->last_ns <= 3 * s->avg_ns)) {
int64_t d = t - s->last_ns;
if (s->avg_ns > 0 && d > s->avg_ns + s->avg_ns / 2) d = s->avg_ns + s->avg_ns / 2;
s->avg_ns = s->avg_ns > 0 ? s->avg_ns + (d - s->avg_ns) / 4 : d;
}
s->last_ns = t;
}
// The floor is not in effect (switched off, frame generation presents, the row Off): the next picture starts it afresh
static inline void present_smooth_reset(PresentSmooth* s) { s->avg_ns = 0, s->last_ns = 0; }

75
gpu/vk/present_thread.h Normal file
View File

@@ -0,0 +1,75 @@
// (Triforce) The present thread (vk_replay.cpp present_image, RX_VK_PRESENT_THREAD): the last step of a present of the
// game's handed to a thread of its own, apart from Vulkan so that tests/test_present_thread.py can run it.
//
// A frame of OoT here is one chain on the game's presenting thread: its build, the replay, its poll for the frame's GPU
// event, then the present. present_image's last step, the pacing's wait for an early picture (present_smooth.h, up to 8 ms)
// and vkQueuePresentKHR (where the driver, or an overlay's frame limiter, may block), held that thread and with it the start
// of the next frame. Now the presenting thread records the blit and submits it as before (the frame fence, the swapchain's
// image, the overlays, the submit: on the queue in the same order, so the same picture), hands the rest over and goes on;
// whoever presents next first waits until the one handed over is out (settle), so one present is in hand at most and the
// presents go out in their order. NX2navi does the same with a thread of its own (runtime/vk_backend.c present_main, most of
// whose lines are not chrg6's: its behaviour only, none of its code); there the blit is the thread's as well, and every later
// submission waits for it. Here the blit stays where it was, so nothing else of the replay has to wait for the thread.
//
// Not for frame generation's display thread nor NVIDIA DLSS frame generation (presents of their own), nor with AMD
// Anti-Lag 2 (its present stage would come after the next frame's input stage); NVIDIA Reflex carries the frame's id with
// its submit's end (vk_reflex.cpp shown_id), so its present's markers are that frame's on either thread.
#pragma once
#include <stdint.h>
#include <chrono>
#include <condition_variable>
#include <mutex>
/// Whether a present goes out from the present thread. on: RX_VK_PRESENT_THREAD (on unless =0); games: one of the game's
/// own pictures (do_present, smooth_mine), not the start-up screen's nor frame generation's display thread's; framegen,
/// dlssg: frame generation or NVIDIA DLSS frame generation runs; antilag: AMD Anti-Lag 2 gives its stages
static inline int present_thread_use(int on, int games, int framegen, int dlssg, int antilag) {
return on && games && !framegen && !dlssg && !antilag;
}
namespace present_thread {
/// One present in hand at most. The presenting thread hands one over (hand) and goes on; the present thread takes it
/// (take), presents and says so (done); whoever presents next waits first until it is out (settle)
template <class Job>
struct Handoff {
std::mutex mu;
std::condition_variable cv;
Job job{};
bool pending = false; // (handed over, not taken yet)
bool busy = false; // (taken, not out yet)
int64_t handed = 0, taken = 0;
/// Before a present: until the one handed over before it is out. The time waited in ns; -1 when none was in hand
int64_t settle() {
std::unique_lock<std::mutex> lk(mu);
if (!pending && !busy) return -1;
const auto t0 = std::chrono::steady_clock::now();
cv.wait(lk, [this] { return !pending && !busy; });
return std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now() - t0).count();
}
/// A present for the thread; returns at once unless one is still in hand (the caller settled first: never)
void hand(const Job& j) {
std::unique_lock<std::mutex> lk(mu);
cv.wait(lk, [this] { return !pending && !busy; });
job = j, pending = true, handed++;
cv.notify_all();
}
/// The present thread's: the next one handed over
Job take() {
std::unique_lock<std::mutex> lk(mu);
cv.wait(lk, [this] { return pending; });
pending = false, busy = true, taken++;
return job;
}
/// The present thread's: the one taken is out
void done() {
std::lock_guard<std::mutex> lk(mu);
busy = false;
cv.notify_all();
}
};
} // namespace present_thread

84
gpu/vk/program_carry.h Normal file
View File

@@ -0,0 +1,84 @@
/* (Triforce) The program cache carried over an update of the translator (vk_shader.cpp disk_stamp and disk_init).
triforce_programs.bin keeps what translation made, a record a program variant, under a stamp: what translated it and for
what. The stamp named every file of the translator whole (nx2/build.py's HOOKSHOT_SHADER_STAMP: Eden, sirit, vk_shader.cpp,
vk_bank.cpp, vk_state.h, vk_api.h), so any edit of one threw every program away: a log line, the read-ahead, a host option's
number in vk_api.h (14 of the 17 commits that changed that stamp from v0.4.0 to a464979 were vk_api.h's alone, and none of
the 17 changed a translation). The owner's test7 and test8 started with "0 program keys kept" and prepared some 46,600
programs before the title (75.2 s in test7), and the pipeline cache, the pipeline notes and the prepare's records, which go
by the same stamp, went with them.
The rule. A record is used by a build of the translator other than the one that made it only when the two have the same
translation stamp (nx2/build.py translation_stamp, HOOKSHOT_TRANSLATION_STAMP): the same files, each taken without what
cannot change a translation, and nothing else left out:
- vk_api.h, the interface between the game and the renderer: no translation reads a constant, a macro or a type of its own
(and it includes rx_nvn.h before anything of its own). What a translation asks of the host goes through host->, the
guest's memory and the depth mode, and the answers are the host's whatever vk_api.h says (the depth mode is in the
stamp's run-time part below).
- The parts of vk_shader.cpp between an "Outside the translation stamp" mark and the next "Inside the translation stamp
again": the shader modules made of a translation's SPIR-V (made the same for a record read back as for a program just
translated, from SPIR-V they take const), the program cache's file and its list of records, the read-ahead and the
precompile's loads (where the records are and when they are read, never what one holds: program_cache_read, in the stamp,
reads each and checks its key). What a record holds, how it is written and read, the keys and the answers a record is
picked by all stay in the stamp.
- Comments, blank lines and the spaces at the ends of lines, in the C and C++ files.
The run-time part of the stamp is as before (disk_stamp: the depth mode, the GPU's mesh shaders and warps, the per-GPU
translation settings, the C++ library's layouts, the RX_VK_ and EDEN_ switches that change a translation). So a record is
never used by a build that would have made it otherwise, as far as the stamp of the whole translator could ever tell (what
it never covered, the host's side, the other gpu/vk headers, the compiler, this does not cover either), and nothing is
guessed at run time: the two stamps are the same, or every record is translated again. tests/test_program_carry.py holds
the rule: which edits change the stamp and which do not, that every translator file is the same C++ to clang's lexer as
the stamp takes it, that the marked parts touch nothing of a translation and give the rest only what it may take from
them, that vk_api.h is not read, and disk_init's verdicts on files it authors.
Then what vk_draw.cpp keeps by program_cache_stamp() (the pipeline cache, mkw_vk_pipelines.bin, by its .for; the pipeline
notes; the prepare's record, triforce_prepared.bin, and the driver's part of it; the predicted pipelines' list) is kept with
the programs, as at a start of the same build: no preparation before the title. The log's "program cache" line at the start
says whether the programs were kept, carried over the translator's update (and from which build), or translated again (and
how many the cache had); while programs of a cache thrown away are translated again, its lines count them.
RX_VK_PROGRAM_CARRY=0: the old rule, the stamp of the whole translator (a cache is good for one build of it). Plain
functions: tests/test_program_carry.py compiles them. */
#pragma once
#include <cstdint>
#include <cstdlib>
namespace mkwvk::program_carry {
/// RX_VK_PROGRAM_CARRY: on unless set to 0
inline bool on(const char* env) {
return !env || std::atoi(env) != 0;
}
/// The build of the translator, by its whole stamp (HOOKSHOT_SHADER_STAMP): the cache file's second word names the build that
/// last opened it (0 in files of before this rule)
inline uint32_t writer(uint64_t whole) {
const uint32_t w = (uint32_t)(whole ^ (whole >> 32));
return w ? w : 1;
}
enum class Verdict {
Fresh, // (no cache there, or not one of ours: a new one)
Kept, // (made for this stamp, and last opened by this build of the translator)
Carried, // (made for this stamp by another build of the translator: its translation is this one's)
Dropped, // (made for another stamp: every program is translated again)
};
/// A cache file by its head (read: whether its 16 bytes were there; magic, writer and stamp as they are in it), for this run's
/// stamp and build. Only a file made for this very stamp is used
inline Verdict verdict(bool read, uint32_t magic, uint32_t file_writer, uint64_t file_stamp, uint32_t want_magic, uint64_t stamp,
uint32_t writer) {
if (!read || magic != want_magic) return Verdict::Fresh;
if (file_stamp != stamp) return Verdict::Dropped;
return file_writer == writer ? Verdict::Kept : Verdict::Carried;
}
/// What the stamp is made from for the translator, two words: with the carry the translation stamp, without it (or in a build
/// that has no translation stamp, 0) the whole translator's, and which of the two (a cache of one rule is never one of the other)
inline void translator_words(bool carry, uint64_t translation, uint64_t whole, uint64_t out[2]) {
const bool by_translation = carry && translation != 0;
out[0] = by_translation ? translation : whole;
out[1] = by_translation ? 2u : 1u;
}
} // namespace mkwvk::program_carry

285
gpu/vk/program_order.h Normal file
View File

@@ -0,0 +1,285 @@
/* (Triforce) The programs read ahead in the order the last sessions first needed them (vk_shader.cpp program_cache_load,
readahead_main; the file triforce_program_order.bin beside the program cache).
The save's load-in frame is 250 to 330 ms in every lab run that loads saves\castle-town, 220 to 436 ms in the owner's test29
sessions, and of it the draws read 96 to 108 programs from the program cache themselves ("96 programs read from the cache by
the draws (73.4 ms), 202 taken read ahead"; the dispatches' program picks, 28 to 42 ms, are mostly compute programs read the
same way). The read-ahead (program_readahead.h) reads after each program a draw meets, in the pipeline notes' order: the
order the notes' pipelines were first built, by whichever session built them, which is not the order a load meets them in.
It begins with the load's first program, at most 32 ahead of the draws, and its shelf lets its oldest go when full (888 of
1024 programs read ahead let go unused by the owner's load), so a load's first frame outruns it.
So each session keeps the programs its draws needed from the cache, or found nowhere and translated, in the order it first
needed them (Record: the first kKeysMax), and writes them beside the cache now and then (this session's first, then those of
the sessions before it has not met: merge). The next session's read-ahead workers, from the first draw on (the title and the
file select take seconds), read those programs in that order whenever no draw asks them for anything, and keep them made
(Ahead) apart from the shelf, which does not let them go: at most kKeptMax at a time, each until a draw takes it or the draws
have gone kBehind places past it in the order (that session went elsewhere). Where the draws are is the middle place of the
last kRecent programs they needed for the first time, not the furthest: one program met out of the order's turn (a session
that first drew it later) does not let go the load's programs. The asks come first, as before. Nothing changes what a draw
gets: the same records, read the same way, taken as a program read ahead is taken.
Memory: a program read ahead keeps its stages' SPIR-V packed (spv_store.h, about 24 KB a stage) and no module: kKeptMax
programs are some 25 to 50 MB, beside the shelf's 512. The warm of their pipelines (pipe_warm.h) keeps its own bounds, made
for 512 programs ahead of the draws (kLinksMax, kLibsMax): kKeptMax is the same, so that what it makes for the load is not
let go before the load comes.
((Triforce, wp19) Kept until needed. The owner's test37 session took another path than the order's sessions (SDR where
they were HDR Native with RenoDX: other programs at the file select, some of them the order has far later), and its
load-in frame (8064, 318 ms) still read 115 programs on the draws' threads: "of the programs the draws read, 64 read ahead
and let go before, 51 not read ahead yet". Programs met out of the order's turn dragged where the draws are past the load's
places, so the load's programs read ahead went as "behind" and those not read yet were passed by; and the 512 kept filled
with programs that session never needed, so the reading stopped. tests/test_program_order.cpp has such a session: the rules
before drop 64 of its load's 115 and leave 51 unread. Now (keep_all): a program read in this order stays until a draw takes
it, whatever the draws do meanwhile; nothing is passed by; the bound is the whole order (kKeepAllMax, the file's 4096), so
the workers read all of it while the title and the file select stand still (seconds; a read is about a millisecond on each
of four workers); a program an ask of the draws read that this order names is kept with these, not on the shelf, which lets
its oldest go (this order passed it by while it was parked there); and only a RAM gone short lets them go (all of them:
vk_shader.cpp). Where the draws are decides nothing then of what is read or kept (only the warm's count, below), so no path
through the order can drag it. Memory: a kept program is some 40 to 50 KB packed (its stages and their Shader::Info): the
2411 of test37's order some 100 to 120 MB, the 4096 at most some 200 MB, on a 32 GB PC; none read while the RAM is low, all
given back when it is short.
The pipelines' warm (pipe_warm.h, pipe_order.h) makes a program's pipelines once, as it is read, within a window of its own
ahead of the draws: a program read far ahead would have none made. So the first kKeptMax kept and not taken are read with the
warm, as before; those beyond are read cold (no warm), and as the draws take warmed ones the nearest cold one is warmed then
(next_warm), in hand as a read is, so that the warm keeps the window it had.
(Triforce: a warmed program the draws never take held its room in that window for the whole session: an order recorded in
HDR replayed in SDR leaves some 300 of its first 512 never drawn, and the window shrank to the rest. So where the draws are
still moves on as by the rules before, and a kept program kBehind behind it is passed (Ahead::pass): kept, a draw may still
take it, but out of the window's count and not warmed. The window refills ahead of the draws. And the cold read ahead are
kept by place beside their set (cold_at), so that the nearest is found without a look at each under the read-ahead's lock,
which the draws take.)
RX_VK_PROGRAM_ORDER_KEEP=0: the rules before (512, where the draws are, kBehind).
Plain functions: tests/test_program_order.cpp checks them. RX_VK_PROGRAM_ORDER=0: neither read nor written, as before. */
#pragma once
#include <algorithm>
#include <climits>
#include <cstdint>
#include <cstring>
#include <deque>
#include <set>
#include <unordered_map>
#include <unordered_set>
#include <utility>
#include <vector>
namespace mkwvk::program_order {
constexpr size_t kKeysMax = 4096; // (programs a session's order keeps, and the file)
constexpr size_t kKeptMax = 512; // (programs read ahead in this order and not taken yet, at most: the shelf's and the warm's)
constexpr size_t kKeepAllMax = kKeysMax; // ((Triforce, wp19) the same, kept until needed: the whole order)
constexpr int64_t kBehind = 256; // (one kept this many places behind where the draws are goes, and none is read there)
constexpr size_t kRecent = 16; // (where the draws are: the middle place of the last this many they needed first)
constexpr int64_t kSaveEveryMs = 3000; // (the file written at most this often, when the session's order grew)
constexpr uint32_t kMagic = 0x4f505446; // ("FTPO")
constexpr uint32_t kVersion = 1;
/// RX_VK_PROGRAM_ORDER: on unless "0" ((Triforce, wp19) and RX_VK_PROGRAM_ORDER_KEEP, Ahead::keep_all, the same way)
inline bool on(const char* env) { return !(env && env[0] == '0' && !env[1]); }
/// A check word of the keys (FNV-1a of their bytes)
inline uint64_t check(const std::vector<uint64_t>& keys) {
uint64_t h = 0xCBF29CE484222325ull;
for (const uint64_t k : keys)
for (int i = 0; i < 8; i++) h = (h ^ ((k >> (8 * i)) & 0xFF)) * 1099511628211ull;
return h;
}
/// The file's bytes: magic, version, count (4 bytes each, and 4 spare), the keys, their check word. ((Triforce, wp18) magic:
/// another file of the same form, pipe_order.h's)
inline std::vector<uint8_t> write(const std::vector<uint64_t>& keys, uint32_t magic = kMagic) {
std::vector<uint8_t> out(16 + keys.size() * 8 + 8);
const uint32_t head[4] = {magic, kVersion, (uint32_t)keys.size(), 0};
std::memcpy(out.data(), head, 16);
if (!keys.empty()) std::memcpy(out.data() + 16, keys.data(), keys.size() * 8);
const uint64_t c = check(keys);
std::memcpy(out.data() + 16 + keys.size() * 8, &c, 8);
return out;
}
/// The keys of a file's bytes; none when they are not a whole file of this version (cut short, another file, a bad check word)
inline std::vector<uint64_t> read(const uint8_t* data, size_t n, uint32_t magic = kMagic) {
uint32_t head[4];
if (!data || n < 24) return {};
std::memcpy(head, data, 16);
if (head[0] != magic || head[1] != kVersion || head[2] > kKeysMax || n != 16 + (size_t)head[2] * 8 + 8) return {};
std::vector<uint64_t> keys(head[2]);
if (!keys.empty()) std::memcpy(keys.data(), data + 16, keys.size() * 8);
uint64_t c;
std::memcpy(&c, data + 16 + keys.size() * 8, 8);
return c == check(keys) ? keys : std::vector<uint64_t>{};
}
/// This session's keys first, then those of the sessions before it has not, max at most
inline std::vector<uint64_t> merge(const std::vector<uint64_t>& now, const std::vector<uint64_t>& before, size_t max = kKeysMax) {
std::vector<uint64_t> out;
std::unordered_set<uint64_t> seen;
for (const auto* list : {&now, &before})
for (const uint64_t k : *list) {
if (out.size() >= max) return out;
if (k && seen.insert(k).second) out.push_back(k);
}
return out;
}
/// A session's order: each program once, as first needed, the first kKeysMax
struct Record {
std::vector<uint64_t> keys;
std::unordered_set<uint64_t> seen;
size_t saved = 0; // (keys.size() at the last write)
bool add(uint64_t key) {
if (!key || keys.size() >= kKeysMax || !seen.insert(key).second) return false;
keys.push_back(key);
return true;
}
/// Whether to write now: it grew since the last write, and kSaveEveryMs passed (since_ms)
bool due(int64_t since_ms) const { return keys.size() > saved && since_ms >= kSaveEveryMs; }
};
/// The read-ahead in the sessions' order: where it is, where the draws are, and the programs it keeps made for them
struct Ahead {
bool keep_all = true; // ((Triforce, wp19) kept until needed, nothing passed by, the whole order; false: the rules before)
std::vector<uint64_t> order;
std::unordered_map<uint64_t, int64_t> place; // (key -> its place in order)
std::unordered_map<uint64_t, int64_t> kept; // (read and kept, not taken yet: key -> place)
std::unordered_set<uint64_t> chosen; // (chosen to be read, not kept yet)
std::unordered_set<uint64_t> cold; // ((Triforce, wp19) kept or chosen, read without the pipelines' warm yet)
std::set<std::pair<int64_t, uint64_t>> cold_at; // ((Triforce) the cold by place, the nearest first: no look at each)
std::unordered_set<uint64_t> passed; // ((Triforce, keep_all) kept, kBehind behind where the draws are: pass)
std::deque<int64_t> recent; // (the places of the last kRecent programs the draws needed first)
int64_t next = 0; // (the next place to read)
int64_t furthest = -1; // (where the draws are: the furthest the middle of recent has been)
void load(std::vector<uint64_t> keys) {
order = std::move(keys);
place.clear();
for (size_t i = 0; i < order.size(); i++) place.emplace(order[i], (int64_t)i);
kept.clear(), chosen.clear(), cold.clear(), cold_at.clear(), passed.clear(), recent.clear(), next = 0, furthest = -1;
}
/// ((Triforce, wp19) The kept and chosen read with the pipelines' warm: at most kKeptMax, the warm's window. ((Triforce) not
/// those passed: the draws went by them, and their room is the window's again)
size_t warmed() const {
const size_t all = kept.size() + chosen.size();
return all - std::min(cold.size() + passed.size(), all);
}
/// Whether key is (to be) read without the warm
bool is_cold(uint64_t key) const { return cold.count(key) != 0; }
/// ((Triforce) key (to be) read without the warm: cold, and by its place
void make_cold(uint64_t key) {
const auto it = place.find(key);
if (cold.insert(key).second && it != place.end()) cold_at.emplace(it->second, key);
}
/// ((Triforce) key no longer cold
void uncold(uint64_t key) {
if (!cold.erase(key)) return;
if (const auto it = place.find(key); it != place.end()) cold_at.erase({it->second, key});
}
/// ((Triforce, wp19) Whether a kept program read cold has its warm due (next_warm would give one). ((Triforce) the nearest
/// first: the only cold ones it looks past are those chosen and still being read, a worker's few)
bool warm_due() const {
if (cold_at.empty() || warmed() >= kKeptMax) return false;
for (const auto& [at, k] : cold_at)
if (kept.count(k)) return true;
return false;
}
/// ((Triforce, wp19) The nearest kept program read cold, now that the warm has room: its warm is due (no longer cold); 0: none
uint64_t next_warm() {
if (cold_at.empty() || warmed() >= kKeptMax) return 0;
for (const auto& [at, k] : cold_at)
if (kept.count(k)) {
const uint64_t best = k;
uncold(best);
return best;
}
return 0;
}
/// The programs kept (and chosen to be) at most
size_t max_kept() const { return keep_all ? kKeepAllMax : kKeptMax; }
/// Whether there is more to read: places left, and room among the kept
bool more() const { return next < (int64_t)order.size() && kept.size() + chosen.size() < max_kept(); }
/// Whether the order names key
bool names(uint64_t key) const { return place.count(key) != 0; }
/// The next key to read (wanted: still on disk, not read ahead already, not being read), chosen; 0: none now
template <class Wanted> uint64_t choose(Wanted wanted) {
while (more()) {
const int64_t at = next++;
if (!keep_all && at < furthest - kBehind) continue; // (the draws went past it)
const uint64_t k = order[(size_t)at];
if (!kept.count(k) && !chosen.count(k) && wanted(k)) {
chosen.insert(k);
if (keep_all && warmed() > kKeptMax) make_cold(k); // ((Triforce, wp19) beyond the warm's window: cold)
return k;
}
}
return 0;
}
/// Whether key was chosen here (to be kept apart from the shelf once read)
bool is_chosen(uint64_t key) const { return chosen.count(key) != 0; }
/// key, chosen here, read: kept (true), or not (false: nothing made, or taken by a draw meanwhile)
void read_done(uint64_t key, bool keep) {
if (chosen.erase(key) && keep) {
const int64_t at = place.count(key) ? place.at(key) : 0;
kept.emplace(key, at);
if (keep_all && at < furthest - kBehind) pass(key); // ((Triforce) the draws went by it while it was read)
} else {
uncold(key);
}
}
/// ((Triforce, wp19, keep_all only) key, which the order names, read by an ask of the draws and not chosen here: kept with
/// the order's (true), not on the shelf. Not one chosen or kept already, nor beyond the bound
bool keep(uint64_t key) {
const auto it = place.find(key);
if (!keep_all || it == place.end() || chosen.count(key) || kept.size() + chosen.size() >= max_kept()) return false;
if (!kept.emplace(key, it->second).second) return false;
if (it->second < furthest - kBehind) pass(key); // ((Triforce) behind where the draws are already)
return true;
}
/// ((Triforce, keep_all) key, kept, kBehind behind where the draws are: kept still (a draw may yet take it), but no longer
/// counted in the warm's window, and not warmed if it was read cold (the draws went by it)
void pass(uint64_t key) {
if (!kept.count(key)) return;
uncold(key);
passed.insert(key);
}
/// ((Triforce, wp19) key taken by a draw: no longer kept
void taken(uint64_t key) { kept.erase(key), uncold(key), passed.erase(key); }
/// ((Triforce, wp19) Every program kept, to let go, into out (the RAM short). ((Triforce) only their cold marks go: one
/// chosen and being read keeps its own, so that it is kept cold once read and warmed when the window reaches it)
void let_go_all(std::vector<uint64_t>& out) {
for (const auto& [k, at] : kept) out.push_back(k), uncold(k);
kept.clear(), passed.clear();
}
/// A draw needed key for the first time (took it, read it, or found it nowhere): where the draws are moves on, to the middle
/// place of the last kRecent. The keys kept that are now kBehind behind it, to let go, into out. ((Triforce, wp19) keep_all:
/// only key leaves the kept (taken); nothing is let go for where the draws are. ((Triforce) where the draws are moves on as
/// before, and the kept now kBehind behind it are passed (pass): a warmed program the draws never took (an order recorded
/// in HDR replayed in SDR: some 300 of its first 512) no longer holds the warm's window for the rest of the session)
void met(uint64_t key, std::vector<uint64_t>& out) {
kept.erase(key), uncold(key), passed.erase(key);
const auto it = place.find(key);
if (it == place.end()) return;
recent.push_back(it->second);
if (recent.size() > kRecent) recent.pop_front();
std::vector<int64_t> s(recent.begin(), recent.end());
std::nth_element(s.begin(), s.begin() + s.size() / 2, s.end());
if (s[s.size() / 2] <= furthest) return;
const int64_t was = furthest;
furthest = s[s.size() / 2];
if (keep_all) { // ((Triforce) nothing let go: the places now behind, each once, passed)
for (int64_t at = std::max<int64_t>(0, was - kBehind); at < furthest - kBehind; at++) pass(order[(size_t)at]);
return;
}
for (auto k = kept.begin(); k != kept.end();) {
if (k->second < furthest - kBehind) {
out.push_back(k->first);
k = kept.erase(k);
} else {
++k;
}
}
}
};
} // namespace mkwvk::program_order

140
gpu/vk/program_read_once.h Normal file
View File

@@ -0,0 +1,140 @@
/* (Triforce) A program a draw reads from the program cache itself, read with the shaders' lock let go (vk_shader.cpp
program_cache_load, readahead_take).
A draw that meets a program for the first time in a session and finds it neither in memory nor read ahead reads it from
triforce_programs.bin there and then: the record off the disk and its stages' shader modules made, 0.6 to 1.3 ms a program
in the stall lines of the owner's test29 ("108 programs read from the cache by the draws (63.3 ms)"). The program pick
(vk_shader.cpp program_pick, mesh_program_for) held shader_mu through it, and through a draw's wait (up to 50 ms) for a
program the read-ahead's worker was reading, so the replay's helper threads, each picking its own programs, queued behind
it: "the program picks waited 169.0 ms for the shaders' lock" at the Market's arrival (lab run r33-t28-s1-1, frame 17617),
58.4 ms at an area change in the owner's session 2.
Now the draw marks the program's key as being read, lets shader_mu go, reads into a list of its own, takes the lock again and
only then puts the variants in the program's list: a program is in that list only once whole and ready, as before. A second
draw of the same program waits for the first's result, not for the lock and not to read it again (claim). The record stays
listed in the cache's list until the read is done, so that every other reader of that list (the prepare in play, the
borrows, the read-ahead's choice) sees it kept, as before the read began. The same records are read the same way: nothing
changes what a draw gets.
The lock is let go only where the pick holds it (program_cache_load's draws: program_pick and mesh_program_for hold it whole)
and only for the read, and what the pick holds on to across it is not let go meanwhile: its program's list (the programs'
lists are never erased) and its stages' code, which only the program's release lets go (vk_shader.cpp program_released). A
release closes the gate (close): no read lets the lock go from then on, and the release waits until none has it let go.
Releases are rare (the game finalizes programs at area changes) and already wait for the replay (vk_replay.cpp
do_program_released holds replay_mu), so this costs nothing in play.
Locks: the caller's (shader_mu), then this one's own (mu_), never the other way: a reader waiting for another's read waits
on mu_ with the caller's let go, and takes the caller's again after letting mu_ go. Plain code, the caller's mutex a template
parameter: tests/test_program_read_once.py runs vk_shader.cpp's program_cache_load and readahead_take with threads against
a program cache whose reads are slow. RX_VK_PROGRAM_READ_UNLOCKED=0: read under the lock, as before. */
#pragma once
#include <atomic>
#include <chrono>
#include <condition_variable>
#include <cstdint>
#include <mutex>
#include <unordered_set>
namespace mkwvk::program_read_once {
/// RX_VK_PROGRAM_READ_UNLOCKED: on unless "0"
inline bool on(const char* env) { return !(env && env[0] == '0' && !env[1]); }
/// The keys being read by the draws themselves, and the windows in which a pick has the caller's lock (outer) let go
template <class Mutex> class Reads {
public:
explicit Reads(Mutex& outer) : outer_(outer) {}
/// (outer held) Whether a read may let outer go: none while a release waits (close)
bool may_let_go() const { return closed_ == 0; }
/// (outer held) key to be read by this caller: true, and it is marked as being read (done ends that). false: another
/// caller reads it now; it was waited for with outer let go, and outer is held again (its variants are in their list by
/// then, put there before done). *wait_ns: the wait, the lock's taking again included. (Outer is let go for this wait even
/// while a release waits: the read waited for needs outer to end, and the release waits for both)
bool claim(uint64_t key, std::atomic<int64_t>* wait_ns = nullptr) {
std::unique_lock lk(mu_);
if (keys_.insert(key).second) return true;
const auto t0 = std::chrono::steady_clock::now();
open_++; // (outer held: close() reads it under outer)
outer_.unlock();
cv_.wait(lk, [&] { return !keys_.count(key); });
lk.unlock();
outer_.lock();
leave();
if (wait_ns) *wait_ns += std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now() - t0).count();
return false;
}
/// (outer held) fn run with outer let go (a read, a wait for the read-ahead's worker), outer taken again after it, the
/// wait to take it counted in *relock_ns. Only where may_let_go(); fn must not take outer
template <class Fn> void unlocked(Fn&& fn, std::atomic<int64_t>* relock_ns = nullptr) {
open_++;
outer_.unlock();
struct Relock {
Reads& r;
std::atomic<int64_t>* ns;
~Relock() {
const auto t0 = std::chrono::steady_clock::now();
r.outer_.lock();
if (ns) *ns += std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now() - t0).count();
r.leave();
}
} relock{*this, relock_ns};
fn();
}
/// (outer held) key's read done (what it read is in its list by now): the callers waiting for it go on
void done(uint64_t key) {
{
std::lock_guard lk(mu_);
keys_.erase(key);
}
cv_.notify_all();
}
/// Whether key is being read by a caller now (the read-ahead's worker leaves it alone)
bool reading(uint64_t key) const {
std::lock_guard lk(mu_);
return keys_.count(key) != 0;
}
/// (outer not held) A release: from now on no read lets outer go, and this returns once none has it let go (outer not held
/// then). reopen() ends it
void close() {
std::unique_lock lk(outer_);
closed_++;
gate_.wait(lk, [&] { return open_ == 0; });
}
void reopen() {
std::lock_guard lk(outer_);
closed_--;
}
/// The picks with outer let go now (for the tests: read without outer)
int open() const { return open_.load(); }
private:
void leave() { // (outer held again)
if (--open_ == 0) gate_.notify_all();
}
Mutex& outer_;
mutable std::mutex mu_;
std::condition_variable cv_;
std::unordered_set<uint64_t> keys_; // (being read; mu_)
std::condition_variable_any gate_; // (open_ back to 0; waited on with outer)
std::atomic<int> open_{0}; // (picks with outer let go; changed under outer)
int closed_ = 0; // (releases waiting or under way; outer)
};
/// A release's close() for a scope
template <class R> struct Closed {
R& r;
explicit Closed(R& reads) : r(reads) { r.close(); }
~Closed() { r.reopen(); }
Closed(const Closed&) = delete;
Closed& operator=(const Closed&) = delete;
};
} // namespace mkwvk::program_read_once

110
gpu/vk/program_readahead.h Normal file
View File

@@ -0,0 +1,110 @@
/* (Triforce) Programs read ahead of their first draw from the program cache (vk_shader.cpp program_cache_load). A program
a draw meets first in a session is read from triforce_programs.bin there and then, on the draw's thread: the record off
the disk (the file is gigabytes, read where a record is wanted), its stages' shader modules made. In the owner's test4
logs that was the "programs" phase of the slow submits, up to 53 ms in one. The file holds the records in the order earlier
runs first made them, so the programs a scene's first draws need lie together in it: when a draw reads one, a worker reads
the next ones in the file's order and keeps them made, and the draws that come to them take them from it (waiting for one it
is reading, which is less than reading it). Nothing changes what a draw gets: the same records, read the same way. Plain
functions, apart from the file and Vulkan: tests/test_program_readahead.cpp checks them. RX_VK_PROGRAM_READAHEAD=<n>: the
records read after each (32), 0: none, as before. */
#pragma once
#include <algorithm>
#include <cstdint>
#include <cstdlib>
#include <deque>
#include <utility>
#include <vector>
// ((Triforce) and the draws' own reads, with the shaders' lock let go, and the read-ahead in the order the last sessions first
// needed their programs: vk_shader.cpp's include lines are in the translation stamp (program_carry.h), this header's are not)
#include "program_order.h"
#include "program_read_once.h"
namespace mkwvk::program_readahead {
constexpr int kDefault = 32; // (records read ahead after each program a draw read itself)
constexpr size_t kParkedMax = 512; // (programs read ahead and not taken yet, at most: each holds its shader modules)
constexpr size_t kAsksMax = 4; // (the newest asks only: an older one's place in the file is behind the play)
constexpr int kWaitMs = 50; // (a draw waits at most this for a program the worker is reading, then reads it itself)
/// RX_VK_PROGRAM_READAHEAD: the records read after each, 0 to 256 (unset or not a number: kDefault)
inline int count(const char* env) {
if (!env || !*env) return kDefault;
char* end = nullptr;
const long v = std::strtol(env, &end, 10);
if (end == env || *end) return kDefault;
return (int)std::clamp<long>(v, 0, 256);
}
/// The keys of the n records after offset `at` in the file's order (order: (offset, key) sorted by offset), the first n for
/// which wanted(key) holds (still on disk, not read ahead already, not being read)
template <class Wanted>
std::vector<uint64_t> next_keys(const std::vector<std::pair<int64_t, uint64_t>>& order, int64_t at, size_t n, Wanted wanted) {
std::vector<uint64_t> out;
auto it = std::upper_bound(order.begin(), order.end(), std::make_pair(at, UINT64_MAX));
for (; it != order.end() && out.size() < n; ++it)
if (wanted(it->second)) out.push_back(it->second);
return out;
}
/// An ask to read after offset at, kept with the newest kAsksMax (an ask for a place already asked for is not added again)
inline void ask(std::deque<int64_t>& asks, int64_t at) {
if (std::find(asks.begin(), asks.end(), at) != asks.end()) return;
asks.push_back(at);
while (asks.size() > kAsksMax) asks.pop_front();
}
/* (Triforce, test16) The order to read ahead in. The file's order is the order the records were written, and since the shaders
are prepared before the title (some 46,600 programs, test7 on) that is the preparation's, which says nothing of what a scene
draws: loading a save at the file select read 596 programs on the draws' threads in four frames (457.9, 337.4, 162.0 and
158.7 ms) and took 7 of them read ahead; the 512 read ahead in the menus before it were never taken, and the read-ahead
stopped with its shelf full of them. The pipeline notes (vk_draw.cpp, triforce_pipelines.bin) name the pipelines play built,
in the order earlier sessions first built them, each with its program: the programs an area's first frames draw were first
built together, so they lie together in the notes. So the order is the notes' (each program at its first note), and after
them the compute programs in the order play first dispatched them (triforce_computes.bin), the file's only for a program
neither names; and a full shelf lets its oldest go rather than stopping. RX_VK_PROGRAM_READAHEAD_NOTES=0: the file's order
alone, one thread, no compute programs, and stopping when full, as before. */
constexpr int kWorkers = 4; // (threads reading ahead, each with a handle of its own on the file: at a load the draws meet a new
// program every few hundred microseconds, and a read is about a millisecond)
/// RX_VK_PROGRAM_READAHEAD_NOTES: on unless "0"
inline bool notes_on(const char* env) { return !(env && env[0] == '0' && !env[1]); }
/// The notes' programs (keys, one a note, in the notes' order; 0: none) as an order to read in: each program once, at the index of
/// its first note
inline std::vector<std::pair<int64_t, uint64_t>> notes_order(const std::vector<uint64_t>& note_programs) {
std::vector<uint64_t> keys(note_programs);
std::sort(keys.begin(), keys.end());
keys.erase(std::unique(keys.begin(), keys.end()), keys.end());
std::vector<bool> taken(keys.size(), false);
std::vector<std::pair<int64_t, uint64_t>> out;
for (size_t i = 0; i < note_programs.size(); i++) {
const size_t at = (size_t)(std::lower_bound(keys.begin(), keys.end(), note_programs[i]) - keys.begin());
if (!note_programs[i] || taken[at]) continue;
taken[at] = true;
out.push_back({(int64_t)i, note_programs[i]});
}
return out;
}
/// The key -> index table of an order, sorted by key (index_of looks in it)
inline std::vector<std::pair<uint64_t, int64_t>> by_key(const std::vector<std::pair<int64_t, uint64_t>>& order) {
std::vector<std::pair<uint64_t, int64_t>> out;
out.reserve(order.size());
for (const auto& [at, k] : order) out.push_back({k, at});
std::sort(out.begin(), out.end());
return out;
}
/// key's index in the order by_key was made of, -1 when it does not name it
inline int64_t index_of(const std::vector<std::pair<uint64_t, int64_t>>& table, uint64_t key) {
auto it = std::lower_bound(table.begin(), table.end(), std::make_pair(key, INT64_MIN));
return it != table.end() && it->first == key ? it->second : -1;
}
/// How many of the oldest parked programs go so that `adding` more fit under max
inline size_t to_let_go(size_t parked, size_t adding, size_t max = kParkedMax) {
return parked + adding > max ? parked + adding - max : 0;
}
} // namespace mkwvk::program_readahead

42
gpu/vk/rect_clamp.h Normal file
View File

@@ -0,0 +1,42 @@
// (Triforce) A scissor's rectangle in a target without integer overflow. NVN's scissor is the game's x, y, width and
// height as ints; x + width in an int wrapped for a large width (a game that means "to the edge" may say INT_MAX): a
// clear's rectangle came out empty, and the draw's scissor wrapped again in end minus start, to an extent of two billion
// pixels past the target. Here the end is summed in 64 bits; a float scaled to
// pixels is clamped into an int's range before it is converted (a float past it converted to INT_MIN on x86).
// Plain functions: tests/test_rect_clamp.py compiles and runs them against the old expressions.
#pragma once
#include <algorithm>
#include <cstdint>
namespace rect_clamp {
struct Span {
int32_t at; // (the offset)
uint32_t n; // (the extent)
};
/// A draw's scissor along one axis (vk_draw.cpp dynamic_state): from max(0, x) to min(x + w, cap), empty when that is
/// nothing; the offset is not moved down to cap (as before: an offset past the target with no extent)
inline Span scissor(int32_t x, int32_t w, uint32_t cap) {
const int64_t x0 = std::max<int64_t>(0, x);
const int64_t x1 = std::min<int64_t>((int64_t)x + w, (int64_t)cap);
return {(int32_t)x0, (uint32_t)std::max<int64_t>(0, x1 - x0)};
}
/// A clear's rectangle along one axis (vk_draw.cpp clear_rect): both ends clamped into [0, cap]
inline Span clear(int32_t x, int32_t w, uint32_t cap) {
const int64_t x0 = std::clamp<int64_t>(x, 0, (int64_t)cap);
const int64_t x1 = std::clamp<int64_t>((int64_t)x + w, 0, (int64_t)cap);
return {(int32_t)x0, (uint32_t)std::max<int64_t>(0, x1 - x0)};
}
/// A float pixel coordinate as an int, truncated as a cast does, but clamped into an int's range first (NaN: 0)
inline int32_t px(float v) {
if (!(v == v)) return 0;
if (v >= 2147483648.0f) return INT32_MAX;
if (v <= -2147483648.0f) return INT32_MIN;
return (int32_t)v;
}
} // namespace rect_clamp

39
gpu/vk/replay_prio.h Normal file
View File

@@ -0,0 +1,39 @@
// (Triforce) The priority of the thread that replays the game's frames while it records them, after NX2navi's recorder_prio
// (runtime/vk_exec.c rec_claim, chrg6's lines). NX2navi keeps the thread that records at the highest scheduling priority, so that the
// scheduler prefers it over the game's other threads: preempted in the middle of a part, the submission came late for its frame
// (Kokiri Forest: 274 of 2,737 frames had a late submit). Here the replay runs on the game's own submitting thread (RX_VK_ASYNC off),
// at the game's priority (normal), pinned while it replays to the core kept for it (vk_replay.cpp ReplayPin, hos_reserved_cpus). That
// core is kept from the game's threads only: the renderer's helper threads and the capture's copy helpers (above normal), the shader
// workers and the driver's threads may all be put there, and the first two outrank a replay at normal priority. So a thread is raised
// for as long as it replays (the outermost ReplayPin: a submit's capture and replay, a present, a stopped queue let on by the order
// thread), and given its own priority back after: the game's code it runs between two replays keeps the game's priority. The replay
// thread of RX_VK_ASYNC=1 and the recorder thread of RX_VK_RECORDER=thread, which do nothing else, take the level for good (above
// normal at least, as they had).
// RX_VK_REPLAY_PRIO unset (or 2): highest (THREAD_PRIORITY_HIGHEST); 1: above normal; 0: the thread's priority left alone, as before
// Apart from Windows so that tests/test_replay_prio.py can run the rule and vk_replay.cpp's ReplayPin over stand-ins.
#pragma once
#include <cstdlib>
namespace mkwvk::replay_prio {
constexpr int kAbove = 1, kHighest = 2; // (Windows' THREAD_PRIORITY_ABOVE_NORMAL and THREAD_PRIORITY_HIGHEST)
/// The level a replay raises its thread to, from RX_VK_REPLAY_PRIO: 0 none (as before), kAbove, kHighest
inline int level(const char* env) {
if (!env || !*env) return kHighest;
const int v = std::atoi(env);
return v <= 0 ? 0 : v == 1 ? kAbove : kHighest;
}
/// Whether a replay raises its thread, whose priority is `now`, to `want`: only ever up (a thread above it, a time-critical one, or
/// one Windows could not tell, THREAD_PRIORITY_ERROR_RETURN, is left alone; nothing is given back for it then)
inline bool raise(int want, int now) {
return want > 0 && now < want;
}
/// The level of a thread of the renderer's own that only records (the replay thread of RX_VK_ASYNC=1, the recorder thread): `want`,
/// and above normal at least, as before
inline int own_thread(int want) {
return want > kAbove ? want : kAbove;
}
} // namespace mkwvk::replay_prio

412
gpu/vk/replay_threads.h Normal file
View File

@@ -0,0 +1,412 @@
// (Triforce) The Debug tab's Replay helper threads row: what its Automatic gives a PC, and what the row is when the options
// file does not say (vk_replay.cpp par_threads_max, vk_menu.cpp). tests/test_replay_threads_row.py
//
// The row's values: 0 Off, 1 to 7 the parallel replay tried on 2 to 8 threads (what RX_VK_PARALLEL_AUTO=<n> does), and 8
// Automatic: the thread count below, tried the same way.
//
// Automatic's rule, after NX2navi's replay_workers = auto (its vk_exec_replay_workers_auto: half the physical cores less one,
// none for four cores or fewer, none on AMD's Windows driver for RX 6000 and older, where players' games lost the device with
// workers on by default and the cause was never found), fitted to what this renderer measured (three threads at the Castle
// Town save, 67.1 fps against 58.8; kMaxBatch: more than three recording threads only took slots and submits):
// - four cores or fewer: off. The game's threads, the replay and the captures already share them (docs/design.md, "The CPU
// saturates first").
// - five to seven cores: 2 threads. Eight or more: 3.
// - AMD's driver for RDNA 1 and 2 (vk_vendor.h amd_older_compiler): off.
// The cores are taken as half the logical processors: a processor without SMT gets the smaller count.
//
// What the row is when the file has no replay_helper_threads: Automatic, as NX2navi has its workers and as the owner asked
// (2026-11-15). A file that says 0 means Off, as it did: one written since the row exists has the line, so only a new
// options file, or one from before the row, takes the new default. Off on the Debug tab (after a restart) or
// RX_VK_PARALLEL=0 is the replay thread alone, always.
//
// Automatic only says how many threads; the tuner (Tuner below, called by vk_replay.cpp par_tune_present) decides when: the
// game starts on the replay thread alone, and every 900 presents or so the other way is tried for about a hundred frames and
// kept when its frames are at least 4% shorter than before and after. So on a PC or in a scene where the threads do not help,
// they record only those trial frames. (Triforce, 11-15) And its rule (rule below): where the replay is plainly the frame's
// limit they go on without a trial, and where they cannot pay no trial is made. (Triforce, 11-16) And once a way has held,
// trials come ever rarer (up to some 14,400 presents apart) and shorter, and early only when the scene clearly changes (the
// back-off, at Tuner).
//
// The parallel replay was audited for this by reading (nothing could be run). What was found and is fixed, each with a
// check in tests/test_parallel_fixes.py:
// - a data race: vk_draw.cpp upscale_stub, temporal_rejected_network and typed_upscale (through taau_stand_in) used the
// texture cache and their own pipelines and history images without the recording lock, from a dispatch a segment's worker
// records. They take ParLock around those now, and never across the temporal upscaler's own calls.
// - an ordering hole: the dry pass marked what draws and clears of a pass write, not what a texture copy, a
// buffer-to-texture copy or a texture clear writes. A later segment's upload of the game's memory landed over the
// earlier segment's copy: one-frame black or stale textures, for good when the copy was made once. vk_xfer.cpp
// dry_copy_b2t, dry_copy_t2t and dry_clear_tex mark them, and the copies the CPU makes are made in the dry pass.
// - the page table's changes reached the queue in the segments' order, not in the order they were made (vk_memory.cpp
// table_pending is each thread's): a pool imported by a later segment and retired by an earlier one ended with its
// pages pointing at the retired pool. They are written once more, in order, after the last segment
// (memory_table_segments_end).
// - a temporal upscaler's hooks in the draws, and those of a frame generation that takes the scene's camera (AMD FSR's,
// NVIDIA DLSS's), are told things in the order the draws are recorded
// ("the first draw after the scene's last", the scene's camera). While one was on, the frame was recorded on the replay
// thread alone. (Triforce, 11-15) Now those hooks are told in the dry pass, in stream order, up to the first draw or
// dispatch that takes what they noted, and the command buffer from there is recorded in order after the segments before
// it; the skinned draws are named again in stream order where they are taken (stream_order.h, tests/test_stream_order.py).
// RX_VK_PARALLEL_TEMPORAL=0: the replay thread alone with them on, as before (vk_replay.cpp par_stream_order_only).
// - a texture's look without the lock could pass while another worker was still uploading it: touch_quick.h.
// - a flat texture's one-slice 3D copy got its first layout change in the maker's own uploads: through InitRedirect now,
// and copied again by a segment before the maker's.
//
// What is left, each a frame's flaw at most, none seen or ruled out in the game; RX_VK_PARALLEL=0 or the row's Off avoids
// them all:
// - a pool first bound, or retired, inside a frame recorded in segments: a segment earlier in the queue than the one that
// made the change does not have it until the frame's end, and a pool first bound there is read from the game's memory
// for that frame, its mirror made at the next (vk_memory.cpp gpu_buffer_bind, "wanted"). A frame of missing or wrong
// geometry, foliage or shadow where shaders follow pointers into that pool, at the moment the pool appears.
// - the dry pass does not mark a storage image a draw's program writes (only a dispatch's), nor what upscale_stub writes
// for a program that did not translate: such a texture can still be uploaded over by a later segment, as above.
// - a texture the GPU only cleared (the game's font cache) whose new letters a later segment uploaded: an earlier
// segment's draws of that frame do not have them yet. A letter a frame late.
// - the CPU's buffer-to-texture copies are made in the dry pass: a draw before such a copy in the same command buffer
// samples the texels after it.
// - the segments' page table changes, written once more in the recording after them (memory_table_segments_end), would
// undo a change of the same pages that a thread not recording wrote at once in the meantime (until those pages change
// again), as the replay thread's own deferred changes can in a serial replay. The threads that record defer theirs; whether
// another writes the table (under the pools' lock, pool_of) while a frame is recorded was not traced to the end.
// - the tuner's trials are short, and a scene's frame time moves by more than what they measure (vk_replay.cpp, above
// par_threads_max): it may change over, or back, for no gain. The rule's thresholds are fitted to four of the owner's
// sessions on one PC, not measured on others.
#pragma once
#include <algorithm>
#include <cmath>
#include <cstdint>
namespace replay_threads {
/// The row's value for Automatic, and the value the row has when the options file does not name it (see above)
inline constexpr int kRowAutomatic = 8;
inline constexpr int kDefaultRow = kRowAutomatic;
/// The threads Automatic records a frame's large command buffers on: 0, none (the replay thread alone), on every PC.
/// (Triforce, 11-17) It was 2 or 3 with five cores or more. The owner's playtests on a 12-core PC (24 logical, RTX, 2560x1440):
/// with helper threads on (Automatic's 3, or the row's 3 or 8) moving and jumping brought stretches of 10-25 s at 39-48 fps,
/// the game's own frame time going from about 4 to 10-13 ms and the game slowing then catching up, on the merged build and
/// on the build before it; with the row's None, twice (both builds), none of it, at the same frame rate otherwise (52-74).
/// The helper threads take the processors the game's own threads need while it streams and makes textures. The row's
/// counts are still there for a tester. logical, amd_older: as before, for the log's line (why)
constexpr int automatic(unsigned logical, bool amd_older) {
(void)logical, (void)amd_older;
return 0;
}
/// Why, for the log's line at the start
constexpr const char* why(unsigned logical, bool amd_older) {
(void)logical, (void)amd_older;
return "none: the helper threads slowed the game's own threads when moving (11-17); the Debug tab's row still sets a count";
}
/// The threads a value of the row stands for: 0 for Off and for anything the row does not have, one more than the value
/// for 1 to 7 (there is no "1 thread": that is the replay thread alone), and automatic_threads for Automatic
constexpr int of_row(int row, int automatic_threads) {
if (row == kRowAutomatic) return automatic_threads;
return row >= 1 && row <= 7 ? row + 1 : 0;
}
/// (Triforce, 11-15) One frame as the tuner sees it: vk_replay.cpp par_tune_present takes it from the replay's counters at each
/// present. Times in ms. What a trial's line says of each of its three parts, so that a session's log tells why the helper
/// threads' frames were longer or shorter (in the owner's test7 every trial kept the replay thread alone, and the log had only
/// the three means)
struct Frame {
double frame = 0; // present to present
double replay = 0; // the replay thread's own time in the frame's submissions (the replay time line's submit)
double gpu = 0; // the GPU's: the recordings' command buffers and their uploads, the segments' too (read as their slots
// come round again: a frame or two late)
double barriers = 0; // full barriers recorded
double draws = 0; // draws and dispatches
double cut = 0; // of them in command buffers large enough for two segments or more
double op = 0; // the draw path's time (draw() and dispatch(); the helper threads' too while they record)
// (the parallel replay's parts, vk_replay.cpp ParStats: zero on the replay thread alone)
double segmented = 0; // command buffers recorded in segments
double dry = 0, setup = 0, rec = 0, own = 0, waited = 0, submit = 0, first = 0;
bool parallel = false; // (recorded with the helper threads in use)
};
/// A part of a trial: the frames counted in it, summed
struct Part {
Frame sum;
int n = 0;
void add(const Frame& f) {
sum.frame += f.frame, sum.replay += f.replay, sum.gpu += f.gpu, sum.barriers += f.barriers, sum.draws += f.draws;
sum.cut += f.cut, sum.op += f.op, sum.segmented += f.segmented, sum.dry += f.dry, sum.setup += f.setup;
sum.rec += f.rec, sum.own += f.own, sum.waited += f.waited, sum.submit += f.submit, sum.first += f.first;
sum.parallel = sum.parallel || f.parallel;
n++;
}
Frame mean() const {
Frame m = sum;
const double k = n ? 1.0 / n : 0.0;
m.frame *= k, m.replay *= k, m.gpu *= k, m.barriers *= k, m.draws *= k, m.cut *= k, m.op *= k, m.segmented *= k;
m.dry *= k, m.setup *= k, m.rec *= k, m.own *= k, m.waited *= k, m.submit *= k;
m.first = sum.segmented > 0 ? sum.first / sum.segmented : 0.0; // (each command buffer's, not a frame's)
return m;
}
};
// (Triforce, 11-15) The rule: where the helper threads plainly pay, on without a trial; where they cannot, no trial; trials for
// the rest. From the owner's own logs (test4, test5, test7, test8; NX2navi's figures are another engine's):
// - A frame of OoT here is a chain: the game's Presentation thread replays it and then waits for its GPU work to end (its
// event, vk_replay.cpp RX_VK_EVENT_EARLY) before the next frame is built. So the helper threads shorten the frame by the
// time they take off the replay thread, but never below the GPU's time: in test8 they took 1.65 to 1.8 ms off the replay
// thread (7.63 to 5.84-5.98 ms a frame) and the frame did not get shorter, the GPU's time being 6.4 + 0.7 ms (its command
// buffers and uploads) of a 9.3 ms frame. The gain is at most the replay's time over the GPU's, less what the helpers add
// to the GPU and to its start (kGpuExtra, a guess the trials' parts lines will measure: 70 more full barriers a frame in
// test8, the first segment's own recording before anything reaches the GPU).
// - Where the replay is not most of the frame the game's own threads are its limit, and three more threads at above-normal
// priority take processor time from them: test7's trial 4, 11.08 ms frames with 6.3 ms of replay and 5.6 of GPU, was 12.31
// ms with them. kShareOn, kShareOff.
// - What the helpers take off the replay thread: the draws and dispatches in command buffers large enough to cut, at what
// one costs on the replay thread alone (its draw() and dispatch() time over their count), less what the replay thread still
// records of them (one share of `threads`), less the dry pass (kDryPerOp a draw: 0.44 ms a frame for ~2,300 in test8).
// It is cautious both ways: "on" counts on the gain less kGpuExtra, "no trial" only where even without it nothing is to be won.
// Against test4's heaviest scene (9.71 ms of replay, 6.4 of GPU, a 12.0 ms frame, 2,870 draws and dispatches) it says on: the
// trial there kept the helpers, and frames went from 75 to 86-90 fps with more draws. Against test7's trial 4 and the title it
// says no trial; test8's play (7.63 of 7.1, 9.34: a trial there kept them, for about 2%) and test7's busy scenes (9.6 of 8.4,
// 12.0) are trials.
// RX_VK_PARALLEL_RULE=0 (vk_replay.cpp): trials only, as before.
struct Verdict {
enum Kind { kTrial, kOn, kOff };
Kind kind = kTrial;
double saved = 0; // what the helper threads would take off the replay thread a frame (ms)
double gain = 0; // what the frame would get shorter by, at most (ms)
double share = 0; // the replay thread's share of the frame
const char* why = "";
};
inline constexpr double kGpuExtra = 0.5, kDryPerOp = 0.0002, kOnMs = 1.0, kOnPart = 0.10, kOffPart = 0.02;
// (the replay thread's share of the frame: 0.78 to 0.85 in the logs' scenes where the replay was the limit, 0.47 to 0.56 in
// test7's trial 4)
inline constexpr double kShareOn = 0.70, kShareOff = 0.60;
/// m: a window's mean frame on the replay thread alone; threads: the threads that would record segments (the helper threads,
/// and the replay thread when it records too)
inline Verdict rule(const Frame& m, int threads) {
Verdict v;
const auto is = [&v](Verdict::Kind kind, const char* why) {
v.kind = kind, v.why = why;
return v;
};
if (m.frame <= 0 || threads < 2) return is(Verdict::kOff, "no frames, or no helper threads");
v.share = m.replay / m.frame;
const double per_op = m.draws > 0 ? m.op / m.draws : 0.0;
v.saved = std::max(0.0, m.cut * per_op * (1.0 - 1.0 / threads) - m.cut * kDryPerOp);
// (cautious both ways: "on" counts on the gain less kGpuExtra; "no trial" only where even without it there is nothing to win)
const double over_gpu = m.replay - m.gpu;
v.gain = std::max(0.0, std::min(v.saved, over_gpu - kGpuExtra));
if (m.cut <= 0) return is(Verdict::kOff, "no command buffer large enough to cut");
if (v.share < kShareOff)
return is(Verdict::kOff, "the replay is not the frame's limit: the game's threads are, and the helpers would take from them");
if (over_gpu <= kOffPart * m.frame)
return is(Verdict::kOff, "the GPU's time is about as long as the replay's: the frame waits for it either way");
if (std::min(v.saved, over_gpu) <= kOffPart * m.frame) return is(Verdict::kOff, "less to win than a trial could tell");
if (v.share >= kShareOn && v.gain >= std::max(kOnMs, kOnPart * m.frame))
return is(Verdict::kOn, "the replay is the frame's limit and the helpers plainly take enough off it");
return is(Verdict::kTrial, "not plain either way: the trials decide");
}
/// The tuner (vk_replay.cpp par_tune_present calls step at each present, the replay thread's; it was inline there). Every
/// kEvery presents a trial: kTake frames as they are, kTake the other way (after kSettle to settle) and kTake as they were
/// again, and a change when the other way's mean frame was at least 4% shorter than both, so that a scene that changed under the
/// trial does not pass for a gain; frames over 60 ms (a load) left out; the next trial kEvery presents after a change, twice
/// that after none. With the rule (use_rule): between trials, on the replay thread alone, every kWindow presents the window's
/// mean frame is judged (rule()): on, the helper threads at once (a trial follows kEvery presents later, as after a change);
/// no trial, the next one put off while that holds; else nothing changes. A trial that takes the helpers back off holds the
/// rule's "on" for kHold presents: the measurement goes over the model
/// (Triforce, 11-16) The back-off (backoff; vk_replay.cpp par_backoff_on, RX_VK_PARALLEL_BACKOFF=0: off, every step as before).
/// In the owner's steady Market (lab runs f24-S1-1, -2, r25-base-1, -2) every trial kept the helper threads, the other way 13
/// to 19% longer, and each trial held the frame ~1.9 ms longer for its ~100 frames the other way, one every ~2,090 presents:
/// 16 to 31% of the frame time's variance there. So, once a way has held:
/// - each trial that keeps the way in use, its other way longer than both of the way's own parts, puts the next one off twice
/// as long as the one before (2 x kEvery after the first, as before, then 4 and 8 x kEvery, up to kBackoffMax presents:
/// ~2.7 minutes at 90 fps), and one whose other way was clearly longer (by kClear or more) counts twice. One too close to
/// tell (the other way between the two) puts the next off 2 x kEvery, as before, and the first after a change of scene or
/// way (fresh_) kEvery: right after a change the first trial is the one most often wrong;
/// - with the way held through two such steps (held_ 2 or more) a trial's other way is kTakeShort frames after its settling,
/// not kTake. A short trial that finds the other way shorter changes nothing itself: a full trial follows at once, and only
/// a full one changes the way (the shorter mean is the noisier one: it may ask for a trial, not decide one);
/// - the next trial comes early when the scene clearly changes, not on a clock: every kWindow presents between trials the
/// window's mean draws and dispatches, replay thread's time and GPU time are held against those of the first window after
/// the way was last set or tried (the times differ with the way in use); two windows in a row off by more than kSceneDraws
/// of the draws (and kSceneDrawsMin) or kScenePart of either time (and kScenePartMs), and the back-off starts again, the
/// next trial kWindow presents on but never sooner than kEvery after the last one ended, nor while the rule says no trial
/// (its no-trial verdicts hold as before). In the lab runs' frames.csv the draw counts alone moved so at every area's load
/// and arrival, and never in the 8,700 presents of the steady Market after its arrival (f24-S1-1, -2, r25-base-1, -2). A
/// trial whose first and last parts (the same way) moved so had the scene change under it: the windows after it are the
/// new scene's reference, so the back-off starts again there too, the next trial kEvery on.
/// A change of way, by a trial or by the rule, starts the back-off again. tests/test_replay_threads_tuner.py "backoff"
class Tuner {
public:
static constexpr int kEvery = 900, kTake = 80, kSettle = 24, kWindow = 120, kHold = 4 * kEvery;
static constexpr int kTakeShort = 40, kBackoffMax = 16 * kEvery;
static constexpr double kClear = 1.10, kSceneDraws = 0.25, kSceneDrawsMin = 200, kScenePart = 0.30, kScenePartMs = 1.0;
bool use_rule = true;
bool backoff = true; // (the back-off above; false: every step as before it)
int threads = 4; // (the threads that record segments: rule()'s)
struct Step {
bool flip = false; // change the way in use
bool trial = false; // a trial ended (a, b, c its means; ok: each part had enough frames; better: changed)
double a = 0, b = 0, c = 0;
bool ok = false, better = false;
int64_t trials = 0, changes = 0;
const Part* parts = nullptr; // (the trial's three parts)
bool ruled = false; // the rule judged a window
bool rule_new = false; // ...and its verdict differs from the last one, or it turned the helpers on
Verdict verdict;
Frame window; // (the window's mean frame it judged)
// (the back-off's)
bool short_trial = false; // (the trial that ended had kTakeShort frames the other way)
bool recheck = false; // (...and found the other way shorter: nothing changed, a full trial begins at once)
bool scene = false; // (the scene changed: the back-off starts again, the next trial brought forward)
bool moved = false; // (the scene changed under the trial that ended: the back-off starts again)
bool sure = false; // (the trial kept the way, the other way longer than both its parts: the back-off grows)
int64_t next = 0; // (the present the next trial may begin at: set at a trial's end and at a scene change)
};
/// f: the frame just presented; on: the helper threads in use for it; order_only: both ways are the replay thread alone
Step step(const Frame& f, bool on, bool order_only) {
Step s;
frames_++;
if (phase_ == 0) {
if (use_rule && !on) {
if (f.frame <= 60.0) win_.add(f);
if (++win_frames_ >= kWindow) {
if (win_.n >= kWindow * 3 / 4 && !order_only) {
s.ruled = true, s.window = win_.mean(), s.verdict = rule(s.window, threads);
const bool turn_on = s.verdict.kind == Verdict::kOn && frames_ >= hold_until_;
s.rule_new = s.verdict.kind != last_ || turn_on;
last_ = s.verdict.kind;
if (turn_on) {
s.flip = true;
next_trial_ = frames_ + kEvery;
if (backoff) held_ = 0, fresh_ = true, scene_restart();
} else if (s.verdict.kind == Verdict::kOff) {
next_trial_ = std::max(next_trial_, frames_ + kEvery);
no_trial_until_ = frames_ + kEvery;
}
}
win_ = Part{}, win_frames_ = 0;
if (s.flip) return s;
}
} else {
win_ = Part{}, win_frames_ = 0;
}
if (backoff) scene_look(f, s);
if (frames_ < next_trial_) return s;
if (order_only) return s;
phase_ = 1, n_ = 0;
short_ = backoff && held_ >= 2;
win_ = Part{}, win_frames_ = 0; // (the rule's next window begins after the trial)
sum_[0] = sum_[1] = sum_[2] = 0, cnt_[0] = cnt_[1] = cnt_[2] = 0;
part_[0] = part_[1] = part_[2] = Part{};
return s;
}
const int k = phase_ - 1;
const int take = phase_ == 2 && short_ ? kTakeShort : kTake; // (only the other way is shorter: the rest costs nothing)
n_++;
if (n_ > (phase_ == 1 ? 0 : kSettle) && f.frame <= 60.0) sum_[k] += f.frame, cnt_[k]++, part_[k].add(f);
if (n_ < (phase_ == 1 ? take : kSettle + take)) return s;
n_ = 0;
if (phase_ == 1) {
s.flip = true, phase_ = 2;
} else if (phase_ == 2) {
s.flip = true, phase_ = 3;
} else {
phase_ = 0, trials_++;
const int took = short_ ? kTakeShort : kTake;
s.ok = cnt_[0] >= kTake * 3 / 4 && cnt_[1] >= took * 3 / 4 && cnt_[2] >= kTake * 3 / 4;
s.a = cnt_[0] ? sum_[0] / cnt_[0] : 0, s.b = cnt_[1] ? sum_[1] / cnt_[1] : 0, s.c = cnt_[2] ? sum_[2] / cnt_[2] : 0;
s.better = s.ok && s.b < std::min(s.a, s.c) * 0.96;
s.short_trial = short_;
if (s.better && short_) s.better = false, s.recheck = true; // (a full trial decides: at once, below)
if (s.better) {
s.flip = true, changes_++;
if (on) hold_until_ = frames_ + kHold; // (the trial took them off: the rule's "on" waits)
}
if (!backoff) {
next_trial_ = frames_ + (s.better ? kEvery : kEvery * 2);
} else {
// (a scene that changed under the trial: its first and last parts, the same way, describe two scenes. The
// windows after it are the new scene's reference, so the change is not seen again: the next trial kEvery on,
// as after a change, not put off)
// (and only a trial whose other way was longer than both of the way's parts holds it: one in between, too close
// to tell, puts the next off 2 x kEvery as before, so that a wrong "as it was" right after a change is not
// put off ever longer; such a trial costs little, its two ways' frames being about as long. The first trial
// after a change of scene or way that is too close to tell: kEvery, a second look soon. The trials' means
// drift with the scene, and right after a change the first is the one most often wrong)
s.moved = cnt_[0] && cnt_[2] && scene_moved(part_[0].mean(), part_[2].mean());
const double top = std::max(s.a, s.c);
s.sure = s.ok && !s.better && !s.recheck && !s.moved && s.b > top;
const bool again = !s.sure && fresh_;
fresh_ = s.better || s.moved || s.recheck;
if (s.better || s.recheck || s.moved) held_ = 0;
else if (s.sure) held_ = std::min(held_ + (s.b >= top * kClear ? 2 : 1), 30);
next_trial_ = frames_ + (s.recheck ? 1 : s.better || s.moved || again ? kEvery : s.sure ? put_off(held_) : 2 * kEvery);
last_end_ = frames_;
scene_restart();
}
s.next = next_trial_;
s.trial = true, s.parts = part_;
}
s.trials = trials_, s.changes = changes_;
return s;
}
int64_t frames() const { return frames_; }
/// (the back-off's interval after trials that kept the way, held_ steps of them: 2 x kEvery for none or one, doubling with
/// each step, up to kBackoffMax)
static int64_t put_off(int held) {
const int64_t base = 2 * (int64_t)kEvery;
return held <= 1 ? base : std::min<int64_t>(base << std::min(held - 1, 16), kBackoffMax);
}
/// (whether the scene a window's means describe moved from the reference's: the back-off's early trial)
static bool scene_moved(const Frame& ref, const Frame& now) {
const auto off = [](double was, double is, double part, double least) { return std::fabs(is - was) > std::max(part * was, least); };
return off(ref.draws, now.draws, kSceneDraws, kSceneDrawsMin) || off(ref.replay, now.replay, kScenePart, kScenePartMs) ||
off(ref.gpu, now.gpu, kScenePart, kScenePartMs);
}
private:
void scene_restart() { scene_win_ = Part{}, scene_frames_ = 0, ref_set_ = false, strikes_ = 0; }
/// (between trials: the scene's windows against the reference, the next trial brought forward when two in a row moved)
void scene_look(const Frame& f, Step& s) {
if (f.frame <= 60.0) scene_win_.add(f);
if (++scene_frames_ < kWindow) return;
const bool counted = scene_win_.n >= kWindow * 3 / 4;
const Frame m = scene_win_.mean();
scene_win_ = Part{}, scene_frames_ = 0;
if (!counted) return; // (a load: neither a reference nor a change)
if (!ref_set_) {
ref_ = m, ref_set_ = true, strikes_ = 0;
return;
}
if (!scene_moved(ref_, m)) {
strikes_ = 0;
return;
}
if (++strikes_ < 2) return;
strikes_ = 0, ref_ = m, held_ = 0, fresh_ = true, s.scene = true;
const int64_t at = std::max({frames_ + kWindow, last_end_ + kEvery, no_trial_until_});
if (at < next_trial_) next_trial_ = at;
s.next = next_trial_;
}
int phase_ = 0, n_ = 0, cnt_[3] = {0, 0, 0};
double sum_[3] = {0, 0, 0};
Part part_[3];
Part win_;
int win_frames_ = 0;
int64_t frames_ = 0, next_trial_ = 600, trials_ = 0, changes_ = 0, hold_until_ = 0;
Verdict::Kind last_ = Verdict::kTrial;
// (the back-off's: trials that kept the way since it was last set, in held_ steps; the trial in progress a short one; no
// trial since the scene or the way changed (fresh_); the last trial's end; the rule's "no trial" until; the scene's window,
// its reference and the windows in a row that moved)
int held_ = 0;
bool short_ = false, fresh_ = true;
int64_t last_end_ = 0, no_trial_until_ = 0;
Part scene_win_;
int scene_frames_ = 0, strikes_ = 0;
bool ref_set_ = false;
Frame ref_;
};
} // namespace replay_threads

109
gpu/vk/reupload_early.h Normal file
View File

@@ -0,0 +1,109 @@
// (Triforce, 11-16) The re-uploads made before a parallel replay's segments (vk_texture.cpp texture_reupload_early, called by
// vk_replay.cpp replay_handle after the dry pass), apart from Vulkan so that tests/test_reupload_early.py can run it.
//
// After some tours of the lab's scenario S2 the game kept rewriting, from the CPU, the storage of textures the Market samples
// (4 to 12 re-uploads of 28 to 84 MB in ~13% of the frames: frames of 18-26 ms against 12.5). Each re-upload ran in
// texture_touch, which a segment's draw calls under the parallel replay's lock (vk_draw.cpp texture_view_of): the other
// recording threads queued behind it, and the frame's descriptor time grew from 3 to 24 ms over the threads ("the recording
// lock waited for 3.9 ms" a frame there, against ~0.1 in the steady Market).
//
// So the textures the CPU keeps rewriting are looked at once more before the segments begin, on the replay thread, the lock
// not taken by anyone yet: those that were uploaded again from the CPU's writes twice or more within kWindow presents, the
// last time within kWindow, and sampled in this frame or the one before (Hot below). What the look uploads goes into the
// uploads of the recording before the segments, which the queue runs before every segment, as a serial replay's go before
// its recording's draws; the segments' own looks find them looked at in this submission and uploaded by an earlier part of
// it, and go on without the lock (touch_quick.h). The look is texture_touch itself, so what it decides is what a segment's
// first look at the texture would decide, with two differences only:
// - it comes before the segments' recording. What the dry pass found this command buffer writes into a texture (a draw's
// targets, a dispatch's storage images, copies and clears) is marked as the GPU's before (texture_written), and such a
// texture is not looked at here (wanted: gpu_written). What the dry pass cannot see (a storage image a draw's program
// writes, a storage buffer over texture storage) is as in the parallel replay now, whose segments look in any order;
// - a texture not sampled in this command buffer after all is looked at (and uploaded) once in this submission, as its next
// sampling would have done with the bytes of this submission's capture, the newest there are.
// RX_VK_REUPLOAD_EARLY=0: not, the segments' looks alone, as before.
#pragma once
#include <algorithm>
#include <cstddef>
#include <cstdint>
#include <vector>
namespace reupload_early {
inline constexpr uint64_t kMinBytes = 1ull << 20; // (the bursts' textures were 4 to 34 MB; a small one holds the lock briefly)
inline constexpr uint64_t kWindow = 120; // (presents: two re-uploads within it make a texture hot)
inline constexpr uint32_t kHotAfter = 2; // (re-uploads in a row, each within kWindow of the one before)
inline constexpr size_t kMost = 64; // (textures kept: the bursts named 5 to 12)
/// What the list knows of a texture: its creation number (Texture::id) and the textures-gone count when it was last known to
/// be there (texture_gone.h: its pointer is good while its number is not among those gone since), its re-uploads in a row
struct Entry {
void* tex = nullptr;
uint32_t id = 0;
uint64_t since = 0;
uint64_t last = 0; // (the present of its last re-upload)
uint32_t n = 0; // (re-uploads in a row, each within kWindow presents of the one before)
};
/// A texture as the early look asks about it (vk_texture.cpp fills this from Texture)
struct Seen {
bool uploaded = false, gpu_written = false, released = false, sparse = false, only_cleared = false;
uint64_t bytes = 0; // (its storage)
uint64_t last_frame = 0; // (the present a draw last looked at it)
uint32_t owed_later = 0; // (frames its upload was left for the next one: owed first)
};
/// Whether a hot texture is looked at before the segments at present `frame`: one uploaded before that the GPU did not draw
/// into (nor marked as drawn into by the dry pass of this command buffer), not finalized, not sparse, not one the GPU only
/// cleared (its letters are the CPU's over the GPU's: texture_touch's own branch), large enough, sampled this frame or the
/// one before
constexpr bool wanted(const Seen& t, uint64_t frame) {
return t.uploaded && !t.gpu_written && !t.released && !t.sparse && !t.only_cleared && t.bytes >= kMinBytes &&
t.last_frame + 1 >= frame;
}
class List {
public:
/// A re-upload of a texture from the CPU's writes at present `frame` (since: the textures-gone count now)
void note(void* tex, uint32_t id, uint64_t frame, uint64_t since) {
for (Entry& e : e_)
if (e.tex == tex && e.id == id) {
e.n = frame - e.last <= kWindow ? e.n + 1 : 1;
e.last = frame, e.since = since;
return;
}
if (e_.size() >= kMost) { // (the one re-uploaded longest ago makes room)
auto old = std::min_element(e_.begin(), e_.end(), [](const Entry& a, const Entry& b) { return a.last < b.last; });
*old = Entry{tex, id, since, frame, 1};
return;
}
e_.push_back(Entry{tex, id, since, frame, 1});
}
/// The entries hot at present `frame`, in the order they are looked at; those cold for kWindow presents dropped
std::vector<Entry*> hot(uint64_t frame) {
std::erase_if(e_, [frame](const Entry& e) { return frame > e.last + kWindow; });
std::vector<Entry*> out;
for (Entry& e : e_)
if (e.n >= kHotAfter) out.push_back(&e);
return out;
}
/// A texture gone (or not one any more): out of the list (the entries hot() handed out are not good after this)
void forget(const void* tex, uint32_t id) {
std::erase_if(e_, [tex, id](const Entry& x) { return x.tex == tex && x.id == id; });
}
size_t size() const { return e_.size(); }
private:
std::vector<Entry> e_;
};
/// The order of the early looks: uploads left for this frame by the budget first (they are a frame late already), then the
/// textures re-uploaded most recently
inline void order(std::vector<std::pair<Entry*, Seen>>& v) {
std::stable_sort(v.begin(), v.end(), [](const auto& a, const auto& b) {
if ((a.second.owed_later > 0) != (b.second.owed_later > 0)) return a.second.owed_later > 0;
return a.first->last > b.first->last;
});
}
} // namespace reupload_early

780
gpu/vk/rtss_decide.h Normal file
View File

@@ -0,0 +1,780 @@
// (Triforce) RivaTuner Statistics Server as the game's frame limiter (gpu/vk/rtss_limit.cpp): the parts that make no
// system call, so that tests/test_rtss_decide.py builds them on any system (plain C: the test compiles it as C and as
// C++). After NX2navi's runtime/rtss_decide.h, whose rules these are: rtss_parse reads RTSS's shared memory,
// rtss_decide decides each poll of the worker, rtss_poll carries that decision out through the calls rtss_limit.cpp
// gives it (RtssOps), and the state file's lines keep what a profile held before the game changed it.
//
// What Triforce does differently, by the owner's word that RTSS is the limiter whenever it can hold the game:
// - A give-back that RTSS's own behaviour caused (its frame count past the limit, 30 s without showing its hold, or
// frames that came sooner than the game's period) is not for the rest of the run: RTSS is tried again after
// RTSS_RETRY_NS, RTSS_RETRIES times, and at once when RTSS was closed and runs again or another limit is wanted.
// - Frame generation does not keep RTSS out where every game frame is presented a fixed number of times: the limit
// wanted is then the one on presented frames, the game's cap times that number (pace_rule.h). Only a mode whose
// presents do not follow the game's frames keeps it out (RTSS_OUT_*).
// - Where the profile cannot be changed (no administrator rights: RTSS keeps its profiles under Program Files), RTSS
// may still hold the game by a limit of the player's own: rtss_own_decide, and then the game's own wait steps
// aside for that limit.
// - The game's own wait keeps no ceiling beside RTSS: where the frame starts it is one or the other
// (rtss_frame_holds), and the frames RTSS holds are checked by their own times (rtss_watch_step).
// - The period an entry of the shared memory holds when it is first seen is where the watch starts, not a sign of
// life: an entry a closed program of the same process id left never counts as RTSS measuring the game.
#pragma once
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
// The want's argument: a limit on the frames RTSS counts, in millihertz (>0), 0 for none, or why RTSS is kept out.
// RTSS_UNSET is the want before the first one.
enum {
RTSS_UNSET = -2,
RTSS_OUT_SETTING = -1, // HOS_RTSS=0, for the whole run
RTSS_OUT_AUTO = -3, // frame generation on Auto: a frame for each refresh of the screen, whatever the game's rate
RTSS_OUT_DLSSG = -4, // DLSS frame generation: Streamline shows its frames on a pace of its own
};
// What keeps RTSS out, for the log.
static inline const char *rtss_out_text(int64_t want) {
return want == RTSS_OUT_AUTO ? "frame generation on Auto shows a frame for each refresh of the screen, so holding those does not hold the game"
: want == RTSS_OUT_DLSSG ? "DLSS frame generation shows its frames on a pace of its own"
: "HOS_RTSS=0";
}
// The limiter that holds the frames while RTSS is kept out, for the log.
static inline const char *rtss_out_limiter(int64_t want) {
return want == RTSS_OUT_DLSSG ? "the game's own wait or Reflex" : "the game's own wait";
}
// ---- RTSS's shared memory, "RTSSSharedMemoryV2". The header: signature, version, application entry size, offset of
// the application array, its length. An entry: the process id, the executable path (260 bytes), flags, then RTSS's
// last frame rate period for the process: its start and end in ms and the frames in it. The flags' low 16 bits hold
// the API RTSS saw the process present with, a value rather than bits (Vulkan is 0xA), and RTSS_UPDATE_ASKED stays set
// from UpdateProfiles until the process has read its profile again.
#define RTSS_SIGNATURE 0x52545353u // 'RTSS' as RTSS writes it: the memory holds valid data
#define RTSS_SIGNATURE_DEAD 0xDEADu // a closed RTSS's block, while a hooked program still keeps it open
#define RTSS_API_MASK 0xFFFFu
#define RTSS_API_VULKAN 0xAu
#define RTSS_UPDATE_ASKED 0x10000000u
enum { RTSS_FLAGS_AT = 4 + 260, RTSS_PERIOD_AT = RTSS_FLAGS_AT + 4, RTSS_ENTRY_MIN = RTSS_PERIOD_AT + 12,
RTSS_APPS_MAX = 4096 };
typedef struct {
int running; // the signature and a version of 2.0 or later
int dead; // the block of an RTSS that closed
int listed; // the application list holds this process: RTSS has hooked it
uint32_t flags; // the entry's flags
uint32_t t0, t1, frames; // the period of this process's entry
} RtssApp;
// Reads the memory of size bytes at p for the process pid; never reads past size. p NULL: no block at all.
static inline void rtss_parse(const uint8_t *p, size_t size, uint32_t pid, RtssApp *a) {
memset(a, 0, sizeof *a);
if (!p || size < 20) return;
uint32_t h[5];
memcpy(h, p, sizeof h);
a->running = h[0] == RTSS_SIGNATURE && h[1] >= 0x20000;
a->dead = h[0] == RTSS_SIGNATURE_DEAD;
for (uint32_t i = 0; a->running && h[2] >= RTSS_ENTRY_MIN && i < h[4] && i < RTSS_APPS_MAX; i++) {
uint64_t at = (uint64_t)h[3] + (uint64_t)i * h[2];
if (at + RTSS_ENTRY_MIN > size) break;
uint32_t id, t[3];
memcpy(&id, p + at, 4);
if (id != pid) continue;
a->listed = 1;
memcpy(&a->flags, p + at + RTSS_FLAGS_AT, 4);
memcpy(t, p + at + RTSS_PERIOD_AT, sizeof t);
a->t0 = t[0], a->t1 = t[1], a->frames = t[2];
break;
}
}
// The frame rate RTSS measured for the process over its period; 0 unknown.
static inline double rtss_fps(const RtssApp *a) {
return a->listed && a->t1 > a->t0 ? 1000.0 * a->frames / (a->t1 - a->t0) : 0;
}
// ---- the worker's decisions
#define RTSS_OVER_PERIODS 3 // fresh periods in a row RTSS's frame rate of the game may stay above the limit
#define RTSS_OVER_SLACK 1.05 // above limit * this, RTSS is taken not to hold the game to it
#define RTSS_FRESH_NS 2500000000ull // a period is stale after this, or 2.5 times its own length if longer
#define RTSS_HOLD_IDLE_NS 200000000ull // no present for this long: RTSS holds nothing (its limiter sits in the present)
#define RTSS_SETTLE_POLLS 2 // polls in a row a want must stay before it acts on the profile
#define RTSS_SETTLE_DOWN_POLLS 10 // (Triforce) and a limit under the one the profile holds (rtss_decide says why)
#define RTSS_UNPROVEN_POLLS 60 // polls (30 s) RTSS may hook the game with the limit and not show it holds it
#define RTSS_RETRY_NS 60000000000ull // (Triforce) after a give-back RTSS itself caused, it is tried again after this
#define RTSS_RETRIES 3 // (Triforce) and so many times; then only when RTSS restarts or the limit changes
#define RTSS_WATCH_WINDOW 60 // (Triforce) frames over which RTSS's hold is checked by their own times
enum { // out of it: for the run, or until it is tried again (rtss_fail_retried)
RTSS_FAIL_NONE,
RTSS_FAIL_LIBRARY, // its hooks library did not load
RTSS_FAIL_READONLY, // its profile cannot be written
RTSS_FAIL_NOT_LIMITING, // its frame count showed the game past the limit
RTSS_FAIL_UNPROVEN, // it hooked the game with the limit for RTSS_UNPROVEN_POLLS and never showed it holds it
RTSS_FAIL_FAST, // (Triforce) the frames' own times showed that nothing held the game (rtss_watch_step)
};
// The failures RTSS's own behaviour caused, which a later try may find gone. A library that does not load and a
// profile that cannot be written do not change while the game runs.
static inline int rtss_fail_retried(int fail) {
return fail == RTSS_FAIL_NOT_LIMITING || fail == RTSS_FAIL_UNPROVEN || fail == RTSS_FAIL_FAST;
}
enum { RTSS_DO_NOTHING, RTSS_DO_APPLY, RTSS_DO_PUT_BACK };
enum {
RTSS_WHY_HOLDS, // RTSS holds the game to the want
RTSS_WHY_NOT_LISTED, // RTSS has the limit but has not hooked the game yet
RTSS_WHY_OUT, // kept out (the want says why)
RTSS_WHY_FAILED, // RtssState.fail
RTSS_WHY_NOT_RUNNING,
RTSS_WHY_REFUSED, // RTSS refused this limit
RTSS_WHY_NEWER, // the want changed while the poll ran: nothing decided, the next poll takes the new one
RTSS_WHY_UNSET, // no want yet
RTSS_WHY_MOVIE, // a movie plays: the game's own wait holds it at its rate
RTSS_WHY_NOT_VULKAN, // RTSS lists the game but has not seen its Vulkan presents
RTSS_WHY_UPDATE, // RTSS has not read the profile the game wrote yet
RTSS_WHY_NOT_COUNTING, // RTSS has measured no period of the game's frames lately
RTSS_WHY_IDLE, // no picture went out lately (a minimised window): RTSS has none to hold
RTSS_WHY_SETTLING, // a new want: it reaches the profile once it stayed RTSS_SETTLE_POLLS polls
RTSS_WHY_NO_PRESENT, // no picture went out yet: nothing is written before the first
// (Triforce)
RTSS_WHY_NO_LIMIT, // no limit is wanted, or RTSS has no frame rate limit of its own for the game (rtss_own_decide)
RTSS_WHY_OTHER, // a frame time limit or scanline sync of RTSS's own: not acted on (rtss_own_decide)
RTSS_WHY_ABOVE, // RTSS's own limit is above the one the game wants (rtss_own_decide)
};
// What the worker keeps between polls.
typedef struct {
int64_t applied; // the millihertz the profile holds from this run; -1 untouched
int64_t refused; // a limit RTSS refused (out of its range): the game's own limiter keeps that one
int changed; // the profile holds a value from this run: put the old one back on exit
int over; // fresh periods in a row RTSS's frame rate of the game stayed above the limit
int fail; // RTSS_FAIL_*
int held; // the poll before stood aside: a fresh period now shows how RTSS holds the game
int listed; // (Triforce) the poll before saw the game in RTSS's list: its period's end is known from then
uint32_t t1, span; // the end and length (ms) of the newest period seen
uint64_t fresh_ns; // when a new period was last seen; 0 never
int64_t seen; // the want of the poll before
int polls; // polls in a row with that want
int unproven; // polls in a row RTSS hooked the game with a limit in the profile and did not show its hold
// (Triforce) the retry after a give-back
uint64_t fail_ns; // when a failure that is tried again came
int64_t fail_want; // the want it came at
int retries; // tries made since by the clock
int gone; // RTSS was seen not running since it
} RtssState;
#define RTSS_STATE_INIT {-1, -1, 0, 0, RTSS_FAIL_NONE, 0, 0, 0, 0, 0, RTSS_UNSET, 0, 0, 0, 0, 0, 0}
// One poll's view.
typedef struct {
int64_t want; // the want the poll read
int current; // the want is still that one when the poll decides
int hooks; // RTSS's hooks library: 1 loaded, 0 failed to load, -1 not tried (rtss_wants_hooks)
int64_t movie_mhz; // the rate of a movie that plays, 0 none
uint64_t now_ns; // the clock at the poll
uint64_t hosted_ns; // the clock after the newest present, 0 none yet
int fast; // (Triforce) the frames RTSS was to hold came too soon since the poll before (rtss_watch_step)
RtssApp app;
} RtssIn;
typedef struct {
int act; // RTSS_DO_*
int64_t mhz; // the value decided for: the profile's (rtss_target) for a limit, the want for a kept-out one
int holds; // RTSS holds the game to it: the game's own wait stands aside
int why; // RTSS_WHY_*
double fps; // RTSS's frame rate of the game, when the frame count check gave up on it
int retry; // (Triforce) this poll tries RTSS again after a give-back: 1 by the clock, 2 RTSS runs again, 3 another limit
} RtssOut;
// What the profile gets for a want w >= 0 while a movie plays at movie_mhz: w, so that a movie does not rewrite the
// profile at its start and end; but no limit (0) while the movie runs faster than w, where RTSS would hold the movie
// below its rate. The game's own wait holds every movie (rtss_decide RTSS_WHY_MOVIE). (Triforce paces a movie at the
// cap as it paces play, so the movie's rate given here is the want itself and the profile keeps it.)
static inline int64_t rtss_target(int64_t w, int64_t movie_mhz) { return w > 0 && movie_mhz > w ? 0 : w; }
// The profile's FramerateLimit and FramerateLimitDenominator for mhz (>= 0): a whole rate over 1, a fraction over 1000
// where RTSS knows the denominator (has_den), else rounded down to whole frames, never above what was asked.
static inline void rtss_limit_value(int64_t mhz, int has_den, uint32_t *limit, uint32_t *den) {
*den = 1;
if (mhz % 1000 == 0) *limit = (uint32_t)(mhz / 1000);
else if (has_den) *limit = (uint32_t)mhz, *den = 1000;
else *limit = (uint32_t)(mhz / 1000);
}
// The hooks library is loaded only on a poll that may write the profile: a limit is wanted, RTSS is not out of it,
// and it runs. A poll of a kept-out want never loads it; a later limit does.
static inline int rtss_wants_hooks(const RtssState *s, const RtssIn *in) {
return in->want >= 0 && !s->fail && in->app.running;
}
// (Triforce) A failure that is tried again is put down: when and at which want.
static inline void rtss_failed(RtssState *s, int fail, const RtssIn *in) {
s->fail = fail, s->fail_ns = in->now_ns, s->fail_want = in->want, s->gone = 0;
}
// The decision of one poll, in this order: no want does nothing; a want acts on the profile only once it stayed
// RTSS_SETTLE_POLLS polls in a row, so a want that flips does not write and remove the profile at each flip, while the
// game's own wait takes a new want at once. A settled kept-out want puts back what this run wrote (once the hooks are
// loaded); a failure lasts until it is tried again (below); RTSS not running leaves the profile for its return; a
// library that did not load fails the run; nothing is written before the first picture went out; a refused limit
// stays with the game; a settled new limit (rtss_target) goes into the profile. The game's own wait stands aside only
// once RTSS is seen to hold the game: the profile held the limit at the poll before (RTSS reads it after the write),
// RTSS lists the game with its Vulkan presents, has read the profile (RTSS_UPDATE_ASKED clear) and measured a period
// of the game's frames lately (a fresh one: its end moved between two polls that both saw the game listed, within
// RTSS_FRESH_NS, or within 2.5 times the period's length if that is longer), no movie plays and a picture went out
// within RTSS_HOLD_IDLE_NS. An RTSS that lists the game with a limit in its profile while pictures go out, but does
// not show its hold, may hold the frames beside the game's own wait: after RTSS_UNPROVEN_POLLS such polls in a row the
// profile goes back.
// Each fresh period after a poll that stood aside is checked against the limit: more than RTSS_OVER_SLACK times it
// for RTSS_OVER_PERIODS periods in a row puts the profile back; a stale period counts neither way. So does a poll
// that is told the frames came too soon (in->fast).
// (Triforce) After such a give-back (rtss_fail_retried) RTSS is tried again: once RTSS_RETRY_NS have passed,
// RTSS_RETRIES times; and at once, with the count starting again, when RTSS was seen closed and runs again, or when a
// limit other than the one it failed at is wanted.
// (Triforce) The hysteresis on a limit that changes in play (the frames shown for each of the game's: pace_rule.h): a
// limit under the one the profile holds reaches it only once it stayed RTSS_SETTLE_DOWN_POLLS polls (5 s), a higher one
// after RTSS_SETTLE_POLLS. Meanwhile the game's own wait holds the game (the hold is for the limit the profile holds
// alone), and the profile's higher limit sits above every frame the game presents, so it holds none of them; a lower
// one would hold the game under its cap at once, which is why a higher one goes in sooner. A limit that flickers
// between two settles on the higher one and is not written again at each flicker.
static inline RtssOut rtss_decide(RtssState *s, const RtssIn *in) {
int64_t w = in->want;
RtssOut o = {RTSS_DO_NOTHING, w, 0, RTSS_WHY_NOT_LISTED, 0, 0};
if (w == RTSS_UNSET) return o.why = RTSS_WHY_UNSET, o;
const RtssApp *a = &in->app;
if (rtss_fail_retried(s->fail)) {
if (!a->running) s->gone = 1;
int again = a->running && s->gone ? 2
: w >= 0 && w != s->fail_want ? 3
: w >= 0 && s->retries < RTSS_RETRIES && in->now_ns - s->fail_ns >= RTSS_RETRY_NS ? 1
: 0;
if (again) {
s->retries = again == 1 ? s->retries + 1 : 0;
s->fail = RTSS_FAIL_NONE, s->gone = 0, s->over = s->unproven = s->held = 0;
o.retry = again;
}
}
int listed = a->running && a->listed;
int fresh = listed && s->listed && a->t1 > a->t0 && a->frames && a->t1 != s->t1; // a period RTSS measured since
if (fresh) s->span = a->t1 - a->t0, s->fresh_ns = in->now_ns;
if (listed) s->t1 = a->t1;
s->listed = listed;
int held = s->held, unproven = s->unproven;
s->held = s->unproven = 0; // until this poll stands aside again, or counts as unproven again
s->polls = w == s->seen ? s->polls + 1 : 1;
s->seen = w;
int settled = s->polls >= RTSS_SETTLE_POLLS;
if (w < 0) {
if (settled && in->hooks == 1 && s->changed) o.act = RTSS_DO_PUT_BACK;
s->over = 0;
return o.why = RTSS_WHY_OUT, o;
}
if (s->fail) return o.why = RTSS_WHY_FAILED, o;
if (!a->running) return o.why = RTSS_WHY_NOT_RUNNING, o;
if (in->hooks == 0) return s->fail = RTSS_FAIL_LIBRARY, o.why = RTSS_WHY_FAILED, o;
if (!in->hosted_ns) return o.why = RTSS_WHY_NO_PRESENT, o;
int64_t t = o.mhz = rtss_target(w, in->movie_mhz);
if (t == s->refused) return o.why = RTSS_WHY_REFUSED, o;
if (t != s->applied && s->polls < (t > 0 && t < s->applied ? RTSS_SETTLE_DOWN_POLLS : RTSS_SETTLE_POLLS))
return o.why = RTSS_WHY_SETTLING, o;
if (t != s->applied) { // RTSS reads it after the write: a later poll sees whether it holds
o.act = RTSS_DO_APPLY;
o.why = in->movie_mhz ? RTSS_WHY_MOVIE : a->listed ? RTSS_WHY_UPDATE : RTSS_WHY_NOT_LISTED;
s->over = 0;
return o;
}
if (!in->current) return o.why = RTSS_WHY_NEWER, o; // these frames are not the new want's
if (in->fast && held && t > 0) { // (Triforce) the frames it was to hold came sooner than the game's period
rtss_failed(s, RTSS_FAIL_FAST, in);
o.act = RTSS_DO_PUT_BACK, o.why = RTSS_WHY_FAILED;
return o;
}
uint64_t idle = in->now_ns > in->hosted_ns ? in->now_ns - in->hosted_ns : 0;
int shown = idle < RTSS_HOLD_IDLE_NS; // a picture went out lately (one went out: RTSS_WHY_NO_PRESENT above)
uint64_t stale = (uint64_t)s->span * 2500000ull > RTSS_FRESH_NS ? (uint64_t)s->span * 2500000ull : RTSS_FRESH_NS;
int unseen = !a->listed ? RTSS_WHY_NOT_LISTED
: (a->flags & RTSS_API_MASK) != RTSS_API_VULKAN ? RTSS_WHY_NOT_VULKAN
: a->flags & RTSS_UPDATE_ASKED ? RTSS_WHY_UPDATE
: !s->fresh_ns || in->now_ns - s->fresh_ns > stale ? RTSS_WHY_NOT_COUNTING
: RTSS_WHY_HOLDS;
if (unseen != RTSS_WHY_HOLDS) { // hooked with a limit while pictures go out, RTSS holds them beside the game's wait
s->unproven = a->listed && t > 0 && shown ? unproven + 1 : 0;
if (s->unproven < RTSS_UNPROVEN_POLLS) return o.why = unseen, o;
rtss_failed(s, RTSS_FAIL_UNPROVEN, in);
o.act = RTSS_DO_PUT_BACK, o.why = RTSS_WHY_FAILED;
return o;
}
if (fresh && held && t > 0) {
double fps = rtss_fps(a);
s->over = fps > t / 1000.0 * RTSS_OVER_SLACK ? s->over + 1 : 0;
if (s->over == RTSS_OVER_PERIODS) {
rtss_failed(s, RTSS_FAIL_NOT_LIMITING, in);
o.act = RTSS_DO_PUT_BACK, o.why = RTSS_WHY_FAILED, o.fps = fps;
return o;
}
}
if (in->movie_mhz) return o.why = RTSS_WHY_MOVIE, o;
if (!shown) return o.why = RTSS_WHY_IDLE, o;
if (t <= 0) return o.why = RTSS_WHY_NO_LIMIT, o; // (Triforce) no limit is wanted: nothing for RTSS to hold
s->held = o.holds = 1;
return o.why = RTSS_WHY_HOLDS, o;
}
// What a write of the limit did (RtssOps.apply).
enum {
RTSS_WROTE_READONLY = -1, // the save did not reach the file
RTSS_WROTE_REFUSED = 0, // RTSS refused the value; the profile keeps what it held
RTSS_WROTE_HELD = 1, // the profile holds the limit
RTSS_WROTE_REFUSED_CLEARED = 2, // refused, and the earlier limit of this run was cleared to none
};
static inline void rtss_written(RtssState *s, RtssOut *o, int r) {
if (r == RTSS_WROTE_READONLY) {
s->fail = RTSS_FAIL_READONLY, o->holds = 0, o->why = RTSS_WHY_FAILED;
return;
}
s->changed = 1;
if (r == RTSS_WROTE_HELD) {
s->applied = o->mhz;
return;
}
if (r == RTSS_WROTE_REFUSED_CLEARED) s->applied = 0;
s->refused = o->mhz, o->holds = 0, o->why = RTSS_WHY_REFUSED;
}
// The profile was put back: nothing of this run is in it.
static inline void rtss_put_back_done(RtssState *s) { s->changed = 0, s->applied = -1; }
// The profile calls rtss_poll makes. apply writes mhz (first: the first write of the run, which records what to put
// back; clear: on a refusal, clear this run's earlier limit to none) and returns RTSS_WROTE_*; put_back puts the profile
// back as it was before the run (o says why: kept out, or a check gave up on RTSS).
typedef struct {
void *ctx;
int (*apply)(void *ctx, int64_t mhz, int first, int clear);
void (*put_back)(void *ctx, const RtssOut *o);
} RtssOps;
static inline RtssOut rtss_poll(RtssState *s, const RtssIn *in, const RtssOps *ops) {
RtssOut o = rtss_decide(s, in);
if (o.act == RTSS_DO_APPLY) {
rtss_written(s, &o, ops->apply(ops->ctx, o.mhz, !s->changed, s->applied != 0));
} else if (o.act == RTSS_DO_PUT_BACK) {
ops->put_back(ops->ctx, &o);
rtss_put_back_done(s);
}
return o;
}
static inline const char *rtss_fail_text(int fail) {
return fail == RTSS_FAIL_READONLY ? "RivaTuner Statistics Server's profile for the game cannot be changed"
: fail == RTSS_FAIL_NOT_LIMITING ? "RivaTuner Statistics Server did not hold the game to the limit"
: fail == RTSS_FAIL_UNPROVEN ? "RivaTuner Statistics Server never showed it holds the game"
: fail == RTSS_FAIL_FAST ? "the frames came sooner than the limit while RivaTuner Statistics Server was to hold them"
: "RivaTuner Statistics Server's hooks library did not load";
}
// Why the game's own wait paces, for the log line "the game's own wait (<text>)".
static inline void rtss_why_text(char *out, size_t n, const RtssOut *o, int fail) {
if (o->why == RTSS_WHY_OUT) snprintf(out, n, "RivaTuner Statistics Server kept out: %s", rtss_out_text(o->mhz));
else if (o->why == RTSS_WHY_FAILED) snprintf(out, n, "%s", rtss_fail_text(fail));
else if (o->why == RTSS_WHY_NOT_RUNNING) snprintf(out, n, "RivaTuner Statistics Server is not running");
else if (o->why == RTSS_WHY_REFUSED) snprintf(out, n, "RivaTuner Statistics Server refused this limit");
else if (o->why == RTSS_WHY_MOVIE) snprintf(out, n, "a movie plays: the game keeps its pictures on time");
else if (o->why == RTSS_WHY_NOT_VULKAN)
snprintf(out, n, "RivaTuner Statistics Server lists the game but has not seen its Vulkan presents yet");
else if (o->why == RTSS_WHY_UPDATE)
snprintf(out, n, "RivaTuner Statistics Server has not read the game's limit from its profile yet");
else if (o->why == RTSS_WHY_NOT_COUNTING)
snprintf(out, n, "RivaTuner Statistics Server has measured no frames of the game lately");
else if (o->why == RTSS_WHY_IDLE) snprintf(out, n, "no picture went out lately: RivaTuner has none to hold");
else if (o->why == RTSS_WHY_SETTLING) snprintf(out, n, "a new limit: RivaTuner Statistics Server gets it once it stays");
else if (o->why == RTSS_WHY_NO_PRESENT) snprintf(out, n, "no picture has gone out yet");
else if (o->why == RTSS_WHY_NO_LIMIT) snprintf(out, n, "no limit is wanted");
else snprintf(out, n, "RivaTuner Statistics Server has not hooked the game yet");
}
// ---- where the game's frame starts (the game's presenting thread, each frame)
// The worker says RTSS holds the game (holds) and a present went out within RTSS_HOLD_IDLE_NS (hosted_ns: the clock
// after the newest present, 0 before the first): this frame is RTSS's, the game's own wait steps aside. A minimised
// window shows nothing, RTSS then has no present to hold, and the game's own wait is back with the first frame after.
// (Triforce) And a present went out since the start of the frame two frames before this one (before2_ns, 0 while there
// is none): the game's own wait keeps no ceiling beside RTSS, so the frames of a window that stopped showing would run
// free until the 200 ms were up, a hundred of them and more, and look to rtss_watch_step like an RTSS that holds
// nothing. While RTSS holds the game a present goes out every frame (it is what holds the game), so two frames' time
// without one leaves nothing for RTSS to hold.
static inline int rtss_frame_holds(int holds, uint64_t now_ns, uint64_t hosted_ns, uint64_t before2_ns) {
if (!holds || !hosted_ns || hosted_ns < before2_ns) return 0;
return hosted_ns >= now_ns || now_ns - hosted_ns < RTSS_HOLD_IDLE_NS;
}
// The frames RTSS held, checked by their own times, as dlssg_pace.h checks Reflex's: RTSS's own count comes once a
// second and counts presented frames, and a game nothing holds runs free meanwhile. now_ns: where the game's frame
// starts; holds: this frame is RTSS's; period_ns: the game's own period (its cap, whatever is made of the frames
// after). 1 when, of RTSS_WATCH_WINDOW frames in a row that RTSS held to one period, over half came sooner than three
// quarters of it: the caller gives the frames back to the game's own wait at once and tells the worker (RtssIn.fast).
typedef struct {
uint64_t last_ns; // the newest frame start RTSS held, 0 when the game's own wait held it
uint64_t last_period; // its period
int window, fast; // frames counted in the window, those too soon
} RtssWatch;
static inline int rtss_watch_step(RtssWatch *w, uint64_t now_ns, int holds, uint64_t period_ns) {
int bad = 0;
if (holds && w->last_ns && now_ns >= w->last_ns && w->last_period == period_ns) {
if (now_ns - w->last_ns < period_ns / 4 * 3) w->fast++;
if (++w->window >= RTSS_WATCH_WINDOW) {
bad = w->fast > RTSS_WATCH_WINDOW / 2;
w->window = w->fast = 0;
}
} else {
w->window = w->fast = 0;
}
w->last_ns = holds ? now_ns : 0;
w->last_period = holds ? period_ns : 0;
return bad;
}
// The cap of a period in millihertz, rounded (16666666 ns is 60000); 0 for no period.
static inline int64_t rtss_cap_mhz(uint64_t period_ns) {
return period_ns ? (int64_t)((1000000000000ull + period_ns / 2) / period_ns) : 0;
}
// ---- the state file (rtss_limit.cpp, triforce_rtss_profile.txt in the game folder): each RTSS profile a run changed,
// as it was before, so that a start after a run that did not close (a kill, a hang, a crash) puts it back. One line
// per profile, so executables that share the folder keep their own:
// rtss1 <existed 0|1> <old limit> <old denominator> <written limit> <written denominator> <previous limit>
// <previous denominator> <profile name>
// on one line. The name comes last and may hold spaces; lines that start with # are comments. The line takes a new
// value before the profile does, and the value written before it stays as the previous one until the profile holds the
// new one: a run that ends between the two leaves either value in the profile, and both are the game's.
#define RTSS_SAVED_HEADER \
"# RivaTuner Statistics Server's profiles as they were before the game changed them (gpu/vk/rtss_limit.cpp): existed,\n" \
"# old limit and denominator, the game's limit and denominator, the game's previous ones, profile. The game puts a\n" \
"# profile back and removes its line when it closes; a start that finds a line puts that profile back first.\n"
enum { RTSS_NAME_MAX = 260 };
typedef struct {
int existed; // the profile existed before the run changed it
uint32_t old_limit, old_den; // what it held then
uint32_t limit, den; // what the run wrote last
uint32_t prev_limit, prev_den; // what it wrote before, while the profile may still hold it; else limit and den
char name[RTSS_NAME_MAX]; // the profile's name, the executable's (Triforce.exe)
} RtssSaved;
// The line of r, with its newline: its length, 0 when it does not fit n.
static inline size_t rtss_saved_format(char *out, size_t n, const RtssSaved *r) {
int k = snprintf(out, n, "rtss1 %d %u %u %u %u %u %u %s\n", r->existed ? 1 : 0, r->old_limit, r->old_den,
r->limit, r->den, r->prev_limit, r->prev_den, r->name);
return k > 0 && (size_t)k < n ? (size_t)k : 0;
}
// One line (len bytes, no newline) into r: 1 when it is a well-formed line of this format, 0 otherwise.
static inline int rtss_saved_parse(const char *line, size_t len, RtssSaved *r) {
char buf[RTSS_NAME_MAX + 96];
while (len && (line[len - 1] == '\r' || line[len - 1] == ' ')) len--;
if (len >= sizeof buf) return 0;
memcpy(buf, line, len);
buf[len] = 0;
int existed = -1, at = 0;
unsigned v[6];
if (sscanf(buf, "rtss1 %d %u %u %u %u %u %u %n", &existed, &v[0], &v[1], &v[2], &v[3], &v[4], &v[5], &at) != 7 ||
!at)
return 0;
size_t name = len - (size_t)at;
if ((existed != 0 && existed != 1) || !v[1] || !v[3] || !v[5] || !name || name >= RTSS_NAME_MAX) return 0;
r->existed = existed, r->old_limit = v[0], r->old_den = v[1], r->limit = v[2], r->den = v[3];
r->prev_limit = v[4], r->prev_den = v[5];
memcpy(r->name, buf + at, name + 1);
return 1;
}
// Profile names compare as Windows compares file names: ASCII letters in either case.
static inline int rtss_name_eq(const char *a, const char *b) {
for (;; a++, b++) {
int x = *a >= 'A' && *a <= 'Z' ? *a + 32 : *a, y = *b >= 'A' && *b <= 'Z' ? *b + 32 : *b;
if (x != y) return 0;
if (!x) return 1;
}
}
// The line of profile name in the file's text (NUL-terminated): 1 and r, or 0. A line that is not well formed is
// passed over, and the lines after it are still read.
static inline int rtss_saved_find(const char *text, const char *name, RtssSaved *r) {
for (const char *p = text; p && *p;) {
const char *end = strchr(p, '\n');
size_t len = end ? (size_t)(end - p) : strlen(p);
if (*p != '#' && rtss_saved_parse(p, len, r) && rtss_name_eq(r->name, name)) return 1;
p = end ? end + 1 : NULL;
}
return 0;
}
// The file's text with profile name's line replaced by r's (r NULL: removed): the header, every other profile's
// well-formed line as it was, then r's. Comments, blank and malformed lines are dropped. Writes the text into out and
// returns its length (0 when it does not fit cap); *records is the number of profile lines in it, 0 when the file is
// to go.
static inline size_t rtss_saved_update(const char *text, const char *name, const RtssSaved *r, char *out, size_t cap,
int *records) {
int head = snprintf(out, cap, "%s", RTSS_SAVED_HEADER);
size_t at = head > 0 ? (size_t)head : cap;
*records = 0;
if (at >= cap) return 0;
for (const char *p = text; p && *p;) {
const char *end = strchr(p, '\n');
size_t len = end ? (size_t)(end - p) : strlen(p);
RtssSaved other;
if (*p != '#' && rtss_saved_parse(p, len, &other) && !rtss_name_eq(other.name, name)) {
size_t k = rtss_saved_format(out + at, cap - at, &other);
if (!k) return 0;
at += k, ++*records;
}
p = end ? end + 1 : NULL;
}
if (r) {
size_t k = rtss_saved_format(out + at, cap - at, r);
if (!k) return 0;
at += k, ++*records;
}
return at;
}
// What a start does with a line a run that did not close left (r), given the profile as it is now: whether it exists
// and the limit and denominator it holds. Either value of the line is the game's (RtssSaved.prev_limit).
enum {
RTSS_PLAN_NONE, // no profile, and there was none before that run: as it was
RTSS_PLAN_LEAVE, // the player changed or removed the profile since: theirs as it is
RTSS_PLAN_WRITE, // it still holds what the run wrote: the old limit and denominator back
RTSS_PLAN_DELETE, // the same, for a profile that run made: removed
};
static inline int rtss_restore_plan(const RtssSaved *r, int exists, uint32_t limit, uint32_t den) {
if (!exists) return r->existed ? RTSS_PLAN_LEAVE : RTSS_PLAN_NONE;
den = den ? den : 1;
if ((limit != r->limit || den != r->den) && (limit != r->prev_limit || den != r->prev_den)) return RTSS_PLAN_LEAVE;
return r->existed ? RTSS_PLAN_WRITE : RTSS_PLAN_DELETE;
}
// ---- RTSS's own settings
// RTSS's profile files are ini texts, of which only the [Framerate] section's limits are read (rtss_limit.cpp
// read_settings, as files: no call into RTSS's hooks library): a frame rate limit (Limit over LimitDenominator), a
// frame time limit (LimitTime) and scanline sync (SyncScanline0 and SyncScanline1), 0 off for each.
typedef struct {
int has_limit, has_den, has_time, has_scan0, has_scan1; // the file has the key
long limit, den, time, scan0, scan1;
} RtssProfile;
// n bytes at s are the word lit, in either case
static inline int rtss_word(const char *s, size_t n, const char *lit) {
size_t i = 0;
for (; i < n && lit[i]; i++) {
int x = s[i] >= 'A' && s[i] <= 'Z' ? s[i] + 32 : s[i], y = lit[i] >= 'A' && lit[i] <= 'Z' ? lit[i] + 32 : lit[i];
if (x != y) return 0;
}
return i == n && !lit[i];
}
// A whole number at s (n bytes, a sign allowed): 1 and *v, or 0 for anything else. Held to +-1e9: no limit is larger.
static inline int rtss_number(const char *s, size_t n, long *v) {
size_t i = 0;
int neg = 0;
int64_t x = 0; // (a long is 32 bits on Windows: the digits are gathered in 64)
if (i < n && (s[i] == '-' || s[i] == '+')) neg = s[i++] == '-';
if (i >= n) return 0;
for (; i < n; i++) {
if (s[i] < '0' || s[i] > '9') return 0;
if (x <= 1000000000) x = x * 10 + (s[i] - '0');
}
if (x > 1000000000) x = 1000000000;
*v = (long)(neg ? -x : x);
return 1;
}
// The profile text of n bytes into p. A key that is there twice keeps its first value, as the system's ini reader does;
// a value that is not a whole number counts as a key that is not there.
static inline void rtss_profile_parse(const char *text, size_t n, RtssProfile *p) {
memset(p, 0, sizeof *p);
int in = 0;
for (size_t i = 0; text && i < n;) {
size_t e = i;
while (e < n && text[e] != '\n') e++;
size_t a = i, b = e;
while (a < b && (text[a] == ' ' || text[a] == '\t')) a++;
while (b > a && (text[b - 1] == ' ' || text[b - 1] == '\t' || text[b - 1] == '\r')) b--;
if (a < b && text[a] == '[') {
in = rtss_word(text + a, b - a, "[Framerate]");
} else if (in) {
size_t eq = a;
while (eq < b && text[eq] != '=') eq++;
size_t ke = eq, vs = eq < b ? eq + 1 : b;
while (ke > a && (text[ke - 1] == ' ' || text[ke - 1] == '\t')) ke--;
while (vs < b && (text[vs] == ' ' || text[vs] == '\t')) vs++;
long v = 0;
if (eq < b && rtss_number(text + vs, b - vs, &v)) {
const char *k = text + a;
size_t kn = ke - a;
if (rtss_word(k, kn, "Limit") && !p->has_limit) p->has_limit = 1, p->limit = v;
else if (rtss_word(k, kn, "LimitDenominator") && !p->has_den) p->has_den = 1, p->den = v;
else if (rtss_word(k, kn, "LimitTime") && !p->has_time) p->has_time = 1, p->time = v;
else if (rtss_word(k, kn, "SyncScanline0") && !p->has_scan0) p->has_scan0 = 1, p->scan0 = v;
else if (rtss_word(k, kn, "SyncScanline1") && !p->has_scan1) p->has_scan1 = 1, p->scan1 = v;
}
}
i = e + 1;
}
}
// RTSS's own [Framerate] settings for the game: the game's own profile (NULL: it has none), else the global one, and
// a key the game's profile lacks from the global one, as RTSS reads them.
typedef struct {
long limit, limit_den, time, scan0, scan1;
} RtssSettings;
static inline RtssSettings rtss_settings_of(const RtssProfile *own, const RtssProfile *global) {
static const RtssProfile none = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
const RtssProfile *o = own ? own : &none, *g = global ? global : &none;
RtssSettings s;
s.limit = o->has_limit ? o->limit : g->has_limit ? g->limit : 0;
s.limit_den = o->has_den ? o->den : g->has_den ? g->den : 1;
s.time = o->has_time ? o->time : g->has_time ? g->time : 0;
s.scan0 = o->has_scan0 ? o->scan0 : g->has_scan0 ? g->scan0 : 0;
s.scan1 = o->has_scan1 ? o->scan1 : g->has_scan1 ? g->scan1 : 0;
return s;
}
// RTSS holds the game's frames by a setting of its own.
static inline int rtss_settings_limit(const RtssSettings *s) { return s->limit > 0 || s->time || s->scan0 || s->scan1; }
// Its frame rate limit in millihertz, 0 none; and whether it limits another way, which is not acted on.
static inline int64_t rtss_settings_mhz(const RtssSettings *s) {
return s->limit > 0 ? (int64_t)s->limit * 1000 / (s->limit_den > 0 ? s->limit_den : 1) : 0;
}
static inline int rtss_settings_other(const RtssSettings *s) { return s->time != 0 || s->scan0 != 0 || s->scan1 != 0; }
// "limit L/D, frame time limit T, scanline sync S0/S1": the values as the files hold them.
static inline void rtss_settings_text(char *out, size_t n, const RtssSettings *s) {
snprintf(out, n, "limit %ld/%ld, frame time limit %ld, scanline sync %ld/%ld", s->limit, s->limit_den, s->time, s->scan0,
s->scan1);
}
// RTSS has no part in the game's limit although it may list the game: the want keeps it out, or its profile cannot be
// written, its hooks library did not load or it never showed it holds the game (its profile put back, a setting of its
// own may still hold the game). A run whose frame checks gave up on RTSS does not count: RTSS was seen not to hold
// the game.
static inline int rtss_out_of_limit(int64_t want, int fail) {
return want != RTSS_UNSET &&
(want < 0 || fail == RTSS_FAIL_READONLY || fail == RTSS_FAIL_LIBRARY || fail == RTSS_FAIL_UNPROVEN);
}
// Two limiters: RTSS is out of the game's limit (rtss_out_of_limit), and while it lists the game a setting of its own
// still holds the frames (rtss_settings_limit), beside the game's own wait or Reflex.
static inline int rtss_two(int64_t want, int fail, int listed, int own_limit) {
return rtss_out_of_limit(want, fail) && listed && own_limit;
}
// ---- (Triforce) the player's own limit, where the game cannot change the profile
// With RTSS_FAIL_READONLY or RTSS_FAIL_LIBRARY the game's limit is not in RTSS's profile. RTSS may still hold the
// game, by a frame rate limit the player set in RTSS: the game's own wait then steps aside for it, so that there is
// one limiter for a player without administrator rights too. Only while all of this is seen: RTSS runs and lists the
// game with Vulkan presents and its profile read; its own frame rate limit for the game is above 0 and at or under
// the limit the game wants (above it the game's own wait is the one that holds, and stepping aside would let the game
// past its cap); no frame time limit or scanline sync (another way of limiting, not acted on); that limit stayed
// RTSS_SETTLE_POLLS polls; RTSS measured a period of the game's frames between two polls that both saw the game
// listed, lately; a picture went out within RTSS_HOLD_IDLE_NS. A movie is held by that limit too (below). Each fresh
// period after a poll that stood aside is checked as rtss_decide checks its own, and a failure is tried again by the
// same rule.
typedef struct {
int fail; // RTSS_FAIL_NONE, RTSS_FAIL_NOT_LIMITING or RTSS_FAIL_FAST
int over, held, listed;
uint32_t t1, span;
uint64_t fresh_ns;
int64_t seen_mhz; // RTSS's own limit at the poll before
int polls; // polls in a row with it
uint64_t fail_ns;
int64_t fail_want;
int retries, gone;
} RtssOwnState;
#define RTSS_OWN_INIT {RTSS_FAIL_NONE, 0, 0, 0, 0, 0, 0, -1, 0, 0, 0, 0, 0}
typedef struct {
int64_t want; // the limit the game wants RTSS to hold (> 0), in millihertz of presented frames
int64_t limit_mhz; // RTSS's own frame rate limit for the game (rtss_settings_mhz), 0 none or not read
int other; // rtss_settings_other
int64_t movie_mhz;
uint64_t now_ns, hosted_ns;
int fast;
RtssApp app;
} RtssOwnIn;
static inline RtssOut rtss_own_decide(RtssOwnState *s, const RtssOwnIn *in) {
RtssOut o = {RTSS_DO_NOTHING, in->limit_mhz, 0, RTSS_WHY_NOT_RUNNING, 0, 0};
const RtssApp *a = &in->app;
if (s->fail) {
if (!a->running) s->gone = 1;
int again = a->running && s->gone ? 2
: in->want != s->fail_want ? 3
: s->retries < RTSS_RETRIES && in->now_ns - s->fail_ns >= RTSS_RETRY_NS ? 1
: 0;
if (again) {
s->retries = again == 1 ? s->retries + 1 : 0;
s->fail = RTSS_FAIL_NONE, s->gone = 0, s->over = s->held = 0;
o.retry = again;
}
}
int listed = a->running && a->listed;
int fresh = listed && s->listed && a->t1 > a->t0 && a->frames && a->t1 != s->t1;
if (fresh) s->span = a->t1 - a->t0, s->fresh_ns = in->now_ns;
if (listed) s->t1 = a->t1;
s->listed = listed;
int held = s->held, over = s->over;
s->held = s->over = 0;
s->polls = in->limit_mhz == s->seen_mhz ? s->polls + 1 : 1;
s->seen_mhz = in->limit_mhz;
if (s->fail) return o.why = RTSS_WHY_FAILED, o;
if (!a->running) return o;
if (!a->listed) return o.why = RTSS_WHY_NOT_LISTED, o;
if ((a->flags & RTSS_API_MASK) != RTSS_API_VULKAN) return o.why = RTSS_WHY_NOT_VULKAN, o;
if (a->flags & RTSS_UPDATE_ASKED) return o.why = RTSS_WHY_UPDATE, o;
if (in->other) return o.why = RTSS_WHY_OTHER, o;
if (in->limit_mhz <= 0) return o.why = RTSS_WHY_NO_LIMIT, o;
if (in->want <= 0 || in->limit_mhz > in->want) return o.why = RTSS_WHY_ABOVE, o;
if (s->polls < RTSS_SETTLE_POLLS) return o.why = RTSS_WHY_SETTLING, o;
uint64_t stale = (uint64_t)s->span * 2500000ull > RTSS_FRESH_NS ? (uint64_t)s->span * 2500000ull : RTSS_FRESH_NS;
if (!s->fresh_ns || in->now_ns - s->fresh_ns > stale) return o.why = RTSS_WHY_NOT_COUNTING, o;
if (in->fast && held) {
s->fail = RTSS_FAIL_FAST, s->fail_ns = in->now_ns, s->fail_want = in->want, s->gone = 0;
return o.why = RTSS_WHY_FAILED, o;
}
if (fresh && held) {
double fps = rtss_fps(a);
over = fps > (double)in->limit_mhz / 1000.0 * RTSS_OVER_SLACK ? over + 1 : 0;
if (over >= RTSS_OVER_PERIODS) {
s->fail = RTSS_FAIL_NOT_LIMITING, s->fail_ns = in->now_ns, s->fail_want = in->want, s->gone = 0;
o.why = RTSS_WHY_FAILED, o.fps = fps;
return o;
}
}
s->over = over;
// (a movie stays RTSS's here: the player's limit holds it whatever the game does, and the game's wait beside it
// would be a second limiter. movie_mhz is not looked at)
if (!in->hosted_ns) return o.why = RTSS_WHY_NO_PRESENT, o;
if ((in->now_ns > in->hosted_ns ? in->now_ns - in->hosted_ns : 0) >= RTSS_HOLD_IDLE_NS) return o.why = RTSS_WHY_IDLE, o;
s->held = o.holds = 1;
return o.why = RTSS_WHY_HOLDS, o;
}
// Why the player's own limit does not hold the game, for the log.
static inline const char *rtss_own_why_text(int why, int fail) {
switch (why) {
case RTSS_WHY_FAILED:
return fail == RTSS_FAIL_FAST ? "the frames came sooner than that limit" : "RivaTuner Statistics Server counted the game past that limit";
case RTSS_WHY_NOT_RUNNING: return "RivaTuner Statistics Server is not running";
case RTSS_WHY_NOT_LISTED: return "RivaTuner Statistics Server has not hooked the game";
case RTSS_WHY_NOT_VULKAN: return "RivaTuner Statistics Server has not seen the game's Vulkan presents";
case RTSS_WHY_UPDATE: return "RivaTuner Statistics Server has not read its changed profile yet";
case RTSS_WHY_OTHER: return "RivaTuner Statistics Server limits by frame time or scanline sync, which is left beside the game's own wait";
case RTSS_WHY_NO_LIMIT: return "RivaTuner Statistics Server has no frame rate limit of its own for the game";
case RTSS_WHY_ABOVE: return "RivaTuner Statistics Server's own limit is above the game's";
case RTSS_WHY_SETTLING: return "a new limit in RivaTuner Statistics Server: taken once it stays";
case RTSS_WHY_NOT_COUNTING: return "RivaTuner Statistics Server has counted no frames of the game lately";
case RTSS_WHY_MOVIE: return "a movie plays";
case RTSS_WHY_NO_PRESENT: return "no picture has gone out yet";
case RTSS_WHY_IDLE: return "no picture went out lately";
default: return "RivaTuner Statistics Server holds the frames";
}
}

620
gpu/vk/rtss_limit.cpp Normal file
View File

@@ -0,0 +1,620 @@
// (Triforce) RivaTuner Statistics Server as the game's frame limiter, after NX2navi's runtime/rtss_limit.c, whose
// worker this is: where RTSS is installed and running, the game's frame limit goes into RTSS's profile for this
// executable and, while RTSS is seen holding the game to it, the game's own wait steps aside (vk_pace.cpp), so there
// is one limiter. Anywhere else, or when any step fails, the game's own wait paces the frames as it always did.
//
// RTSS's hooks library (RTSSHooks64.dll in the folder its installer records) has the profile calls, loaded at run
// time: LoadProfile(name) reads a profile (the global one's values for a name without its own), Get/SetProfileProperty
// read and change the loaded copy, SaveProfile(name) writes it to Profiles\<name>.cfg, UpdateProfiles makes every
// hooked process read its profile again, DeleteProfile removes one. "FramerateLimit" is the limit in frames per
// second (0: none); "FramerateLimitDenominator" divides it. SaveProfile reports nothing: RTSS's Profiles folder sits
// under Program Files, which a process without administrator rights cannot write, and the save then fails silently.
// So every change is read back from the saved profile, and a limit that did not stick leaves the game's own wait in
// charge (and RTSS too, where the player set a limit of their own in it: rtss_own_decide).
//
// What is written, and it is all that is: the profile of this game's executable, by name, through those calls; of it
// the two properties above and nothing else; and it is put back. On every orderly exit (restore) the profile gets back
// what it held before the game changed it: the old limit, or no profile of its own when there was none. What it held
// is also kept in triforce_rtss_profile.txt in the game folder (rtss_decide.h RtssSaved), written before the run's
// first change of the profile and removed once the profile is back: an exit that skips the restore (a kill, a crash, a
// lost device) leaves the line, and the next start puts the profile back from it before it changes anything
// (left_start, restore_left), unless the player changed the profile since. No other profile is ever saved, not the
// global one; RTSS is never started or stopped; nothing else of it is changed.
//
// The game's own wait stands aside only once RTSS is seen to hold the game (rtss_decide.h rtss_decide), and takes the
// frames back whenever one of the signs ends. Everything here runs on the worker's thread, every half second, so a
// present never waits on the registry, a file or RTSS's library; want(), movie(), fast() and active() are the
// presenting thread's and only touch atomics.
//
// Everything outside this file is reached through rtss_limit.h's Sys: the tests run this file against a stand-in.
#include "rtss_limit.h"
#include <atomic>
#include <chrono>
#include <cstdarg>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <mutex>
#include <thread>
namespace mkwvk::rtss {
namespace {
// The two properties of a profile this file ever sets, and the only ones it reads through the library.
constexpr const char* kLimit = "FramerateLimit";
constexpr const char* kDenominator = "FramerateLimitDenominator";
constexpr int kLimiterLines = 100;
struct St {
const Sys* sys = nullptr;
std::atomic<int64_t> want{RTSS_UNSET}; // millihertz from the pacing: >0 a limit, 0 none; RTSS_OUT_* kept out
std::atomic<int64_t> active_for{RTSS_UNSET}; // the want RTSS holds the game to now; RTSS_UNSET none
std::atomic<int> active_own{0}; // and it holds it by a limit of the player's own (rtss_own_decide)
std::atomic<int64_t> movie_mhz{0}; // a movie's rate while one plays, 0 none
std::atomic<int> fast{0}; // the frames came too soon since the poll before (fast())
std::atomic<int> closed{0}; // restore ran: nothing reaches the profile any more
std::atomic<int> ready{0}; // the worker found RTSS: dll_path, rtss_dir and profile hold their values
char dll_path[560] = "", profile[RTSS_NAME_MAX] = "", rtss_dir[520] = "", state_path[560] = "";
int found = -1; // the worker's look for RTSS: -1 not yet, 0 not installed, 1 found
bool hooks = false; // RTSS's hooks library is loaded
RtssState S = RTSS_STATE_INIT; // the worker's decisions (rtss_decide.h), under lock
RtssOwnState own = RTSS_OWN_INIT; // and those about the player's own limit
int has_denominator = 0; // this RTSS knows FramerateLimitDenominator
int had_profile = 0; // the profile existed before this run changed it
uint32_t old_limit = 0, old_denominator = 1;
int owed = 0; // this run changed the profile and has not put it back: the three above stay as found
int creations = 0, removals = 0; // the profile's "created" and "removed" lines this run, numbered
RtssSaved rec{}; // this run's line in the state file, while the profile holds a value of this run
std::atomic<int> recorded{0}; // the file holds rec
int state_off = 0; // a line this run could not act on stays as it is: no writes of the file
int unrecorded_said = 0;
// the worker's memory between polls
int was_running = -1, was_listed = 0, left_done = 0, left_tried = 0;
int said_not_running = 0, said_readonly = 0;
// set_active's
int logged = -1, lines = 0;
int64_t logged_mhz = 0; // (the limit of the hold logged last: a hold at another limit is logged too)
char said[200] = "";
// read_settings' and watch's
RtssSettings settings{};
uint64_t stamps[2] = {~0ull, ~0ull};
char settings_file[RTSS_NAME_MAX + 24] = "";
int two_said = 0, two_lines = 0, settings_lines = 0;
char two_by[64] = "", two_text[160] = "", settings_said[RTSS_NAME_MAX + 200] = "";
// what is in force, for view()
std::atomic<int> v_limiter{GAME}, v_why{RTSS_WHY_UNSET}, v_fail{RTSS_FAIL_NONE}, v_looked{0}, v_installed{0}, v_running{0}, v_listed{0};
std::atomic<int64_t> v_mhz{0};
};
St* st = new St;
std::mutex lock; // the hooks calls and the worker's state
void say(const char* fmt, ...) {
char line[900];
const int head = snprintf(line, sizeof line, "vk: pace: ");
va_list ap;
va_start(ap, fmt);
vsnprintf(line + head, sizeof line - (size_t)head, fmt, ap);
va_end(ap);
if (st->sys && st->sys->log) st->sys->log(line);
}
bool load_hooks() {
St& s = *st;
if (s.hooks) return true;
if (!s.sys->hooks_load(s.dll_path)) {
say("RivaTuner Statistics Server's %s did not load, or lacks the profile calls", s.dll_path);
return false;
}
return s.hooks = true;
}
// The profile exists on its own: RTSS keeps it as Profiles\<name>.cfg in its folder. (RTSS's EnumProfiles would say
// the same, but NX2navi found it to crash a process whose heap lies above 4 GB: it is never called, nor looked up.)
bool profile_exists() {
St& s = *st;
char path[1100];
snprintf(path, sizeof path, "%s\\Profiles\\%s.cfg", s.rtss_dir, s.profile);
return s.sys->file_stamp(path) != 0;
}
// Writes limit/denominator into the profile and reads them back from the saved file: 1 when they stuck, 0 when RTSS
// refused a value, -1 when the save did not reach the file. RTSS checks each value against the other one already in
// the profile (the limit over the denominator must be a rate it accepts), so the limit goes to 0 (none) before the
// denominator changes.
int write_limit(uint32_t limit, uint32_t denominator) {
St& s = *st;
s.sys->load_profile(s.profile);
if (!s.sys->set_property(kLimit, 0)) return 0;
if (s.has_denominator && !s.sys->set_property(kDenominator, denominator)) return 0;
if (limit && !s.sys->set_property(kLimit, limit)) return 0;
s.sys->save_profile(s.profile);
s.sys->load_profile(s.profile);
uint32_t l = ~0u, d = 1;
if (!s.sys->get_property(kLimit, &l)) return -1;
if (s.has_denominator && !s.sys->get_property(kDenominator, &d)) return -1;
return l == limit && (!s.has_denominator || d == denominator) ? 1 : -1;
}
// The state file with this profile's line set to r, or removed for r null; the file goes once it holds no line. Under
// lock: the worker, or restore's thread at the exit. True when the file holds what was asked.
bool state_put(const RtssSaved* r) {
St& s = *st;
if (s.state_off || !*s.state_path) return false;
size_t n = 0;
char* text = s.sys->file_read(s.state_path, &n);
// a file that is there but did not read (another program holds it, a scan) keeps the lines of the other executables
// in the folder: rewritten from nothing, they would go
if (!text && s.sys->file_size(s.state_path) > 0) return false;
const size_t cap = (text ? n : 0) + sizeof RTSS_SAVED_HEADER + 2 * (RTSS_NAME_MAX + 96);
char* out = (char*)malloc(cap);
int records = 0;
bool ok = false;
const size_t len = out ? rtss_saved_update(text ? text : "", s.profile, r, out, cap, &records) : 0;
if (len && !records) {
s.sys->file_delete(s.state_path);
ok = s.sys->file_size(s.state_path) < 0;
} else if (len) {
ok = s.sys->file_write(s.state_path, out, len);
}
free(out);
free(text);
return ok;
}
// The state file did not take this run's line: a kill would then leave the game's limit in the profile for good.
// Logged once a run.
void unrecorded() {
St& s = *st;
s.recorded.store(0);
if (!s.state_off && !s.unrecorded_said++)
say("%s could not be written; a crash would leave the game's limit in RivaTuner Statistics Server's profile %s",
*s.state_path ? s.state_path : "triforce_rtss_profile.txt", s.profile);
}
// Before a write of limit/denominator into the profile: rec's written value is that one, and the one written before it
// stays as the previous one (rtss_decide.h RtssSaved), so that a run that ends before the profile changed, or after,
// leaves a profile the next start tells from a value the player set since.
void record_before(uint32_t limit, uint32_t denominator) {
St& s = *st;
if (!s.recorded.load() || (s.rec.limit == limit && s.rec.den == denominator)) return;
s.rec.prev_limit = s.rec.limit, s.rec.prev_den = s.rec.den;
s.rec.limit = limit, s.rec.den = denominator;
if (!state_put(&s.rec)) unrecorded();
}
// After that write: the profile holds the written value alone (landed), or kept the previous one.
void record_after(bool landed) {
St& s = *st;
if (!s.recorded.load() || (s.rec.limit == s.rec.prev_limit && s.rec.den == s.rec.prev_den)) return;
if (landed) s.rec.prev_limit = s.rec.limit, s.rec.prev_den = s.rec.den;
else s.rec.limit = s.rec.prev_limit, s.rec.den = s.rec.prev_den;
if (!state_put(&s.rec)) unrecorded();
}
// Writes mhz into the profile (rtss_poll's RtssOps.apply): RTSS_WROTE_*. first: the run's first write, which records
// what to put back on exit, in the state file too before anything changes; clear: on a refusal, clear the limit of this
// run the profile holds.
int apply(void*, int64_t mhz, int first, int clear) {
St& s = *st;
if (first && !s.owed) { // what to put back on exit; after a put back that did not land, the profile still owes it
s.had_profile = profile_exists();
s.sys->load_profile(s.profile);
s.old_limit = 0, s.old_denominator = 1;
s.sys->get_property(kLimit, &s.old_limit);
s.has_denominator = s.sys->get_property(kDenominator, &s.old_denominator);
if (!s.old_denominator) s.old_denominator = 1;
}
uint32_t limit, denominator;
rtss_limit_value(mhz, s.has_denominator, &limit, &denominator);
if (first) { // the line goes in before the profile changes: a kill between the two finds it as it was
const RtssSaved was = s.rec; // still in the file after a put back that did not land: the profile holds its value
const bool kept = s.recorded.load() != 0;
s.rec = RtssSaved{s.had_profile, s.old_limit, s.old_denominator, limit, denominator, kept ? was.limit : limit,
kept ? was.den : denominator, ""};
snprintf(s.rec.name, sizeof s.rec.name, "%s", s.profile);
if (state_put(&s.rec)) s.recorded.store(1);
else unrecorded();
} else {
record_before(limit, denominator);
}
const int r = write_limit(limit, denominator);
if (r < 0) { // the line keeps both values: the profile may hold either
if (first && !s.owed && s.recorded.exchange(0)) state_put(nullptr); // the profile is as it was
return RTSS_WROTE_READONLY;
}
s.owed = 1;
if (first) {
if (!s.had_profile) // numbered: a want that flips between a limit and kept out creates it again at each flip
say("RivaTuner Statistics Server profile %s created; it goes away when the game closes (creation %d this run)",
s.profile, ++s.creations);
else
say("RivaTuner Statistics Server profile %s held a limit of %u/%u; it gets that back when the game closes", s.profile,
s.old_limit, s.old_denominator);
}
if (r == 0) { // RTSS must not keep an earlier limit of this run under the game's own one
say("RivaTuner Statistics Server refused a limit of %.3f fps", mhz / 1000.0);
record_after(false);
if (!clear) return RTSS_WROTE_REFUSED;
record_before(0, 1);
const bool cleared = write_limit(0, 1) > 0;
record_after(cleared);
if (!cleared) return RTSS_WROTE_REFUSED;
s.sys->update_profiles();
return RTSS_WROTE_REFUSED_CLEARED;
}
record_after(true);
s.sys->update_profiles();
if (!mhz) say("RivaTuner Statistics Server's profile %s has no limit", s.profile);
else if ((int64_t)limit * 1000 == mhz * denominator)
say("RivaTuner Statistics Server's profile %s limits to %.3f fps", s.profile, mhz / 1000.0);
else // rounded to whole frames by an RTSS without FramerateLimitDenominator: the log says what RTSS holds
say("RivaTuner Statistics Server's profile %s limits to %u/%u (%.3f fps; the game asked %.3f)", s.profile, limit,
denominator, (double)limit / denominator, mhz / 1000.0);
return RTSS_WROTE_HELD;
}
// The profile as it was before this run changed it (the caller checked S.changed or owed, and marks S done). Its line
// in the state file goes once it is back; one that did not go back keeps the line, and owed, for the exit and the next
// start.
void put_back() {
St& s = *st;
bool back = true;
if (!s.had_profile) {
s.sys->delete_profile(s.profile);
back = !profile_exists();
if (back)
say("RivaTuner Statistics Server profile %s removed, as before the game started (removal %d this run)", s.profile,
++s.removals);
else say("RivaTuner Statistics Server profile %s did not go away", s.profile);
} else if (write_limit(s.old_limit, s.old_denominator) > 0) {
say("RivaTuner Statistics Server profile %s has its limit of %u/%u back", s.profile, s.old_limit, s.old_denominator);
} else {
say("RivaTuner Statistics Server profile %s did not take its old limit back", s.profile);
back = false;
}
s.sys->update_profiles();
if (back) s.owed = 0;
if (back && s.recorded.exchange(0)) state_put(nullptr);
}
// This profile's line in the state file into r: true when there is one.
bool left_line(RtssSaved* r) {
St& s = *st;
size_t n = 0;
char* text = *s.state_path ? s.sys->file_read(s.state_path, &n) : nullptr;
const bool found = text && rtss_saved_find(text, s.profile, r);
free(text);
return found;
}
// The line r of this profile, left in the state file by a run that did not close (left_start): before this run changes
// anything, the profile gets back what it held before that run (rtss_restore_plan), unless the player changed it since,
// and the line goes. A profile that does not go back keeps the line, and this run leaves the file alone.
void restore_left(const RtssSaved* r) {
St& s = *st;
const bool exists = profile_exists();
s.sys->load_profile(s.profile);
uint32_t limit = 0, den = 1;
s.sys->get_property(kLimit, &limit);
s.has_denominator = s.sys->get_property(kDenominator, &den);
if (!den) den = 1;
const int plan = rtss_restore_plan(r, exists, limit, den);
if (plan == RTSS_PLAN_DELETE) s.sys->delete_profile(s.profile);
if ((plan == RTSS_PLAN_WRITE && write_limit(r->old_limit, r->old_den) <= 0) || (plan == RTSS_PLAN_DELETE && profile_exists())) {
say("RivaTuner Statistics Server profile %s still has the game's limit of %u/%u from a run that did not close, and did "
"not go back as it was; %s keeps its line", s.profile, limit, den, s.state_path);
s.state_off = 1;
return;
}
if (plan == RTSS_PLAN_WRITE || plan == RTSS_PLAN_DELETE) s.sys->update_profiles();
if (plan == RTSS_PLAN_WRITE)
say("RivaTuner Statistics Server profile %s had the game's limit of %u/%u from a run that did not close (%s); its limit "
"of %u/%u is back", s.profile, limit, den, s.state_path, r->old_limit, r->old_den);
else if (plan == RTSS_PLAN_DELETE)
say("RivaTuner Statistics Server profile %s had the game's limit of %u/%u from a run that did not close (%s); removed, "
"as before that run", s.profile, limit, den, s.state_path);
else if (plan == RTSS_PLAN_LEAVE)
say("RivaTuner Statistics Server profile %s changed since a run that did not close (%s); left as it is", s.profile,
s.state_path);
else
say("RivaTuner Statistics Server profile %s is as it was before a run that did not close (%s)", s.profile, s.state_path);
if (!state_put(nullptr)) {
say("%s could not be written; its line for %s stays", s.state_path, s.profile);
s.state_off = 1;
}
}
// The state file's line for this profile, on the worker's first poll that finds RTSS running (w the want it read,
// hooks as RtssIn.hooks): true once the line is dealt with or there is none, false when a restore is still owed, for
// the first poll with the hooks library. A limit puts the profile back with the library it loads for its own writes. A
// kept-out want (frame generation on Auto, DLSS frame generation) writes nothing of its own, but a profile left with an
// old limit would hold the game beside the game's own wait: the library is loaded for that one restore.
bool left_start(int64_t w, int hooks) {
RtssSaved r;
if (!left_line(&r)) return true;
if (hooks != 1 && (w >= 0 || !load_hooks())) return false;
restore_left(&r);
return true;
}
void put_back_op(void*, const RtssOut* o) { // rtss_poll's RtssOps.put_back
St& s = *st;
if (o->why == RTSS_WHY_FAILED && s.S.fail == RTSS_FAIL_NOT_LIMITING) // the frame count check gave up on RTSS
say("RivaTuner Statistics Server measures %.1f fps with its limit at %.3f fps", o->fps, o->mhz / 1000.0);
else if (o->why == RTSS_WHY_FAILED && s.S.fail == RTSS_FAIL_UNPROVEN) // the lines before say which sign never came
say("RivaTuner Statistics Server listed the game with its limit at %.3f fps for %d s and never showed it holds it",
o->mhz / 1000.0, RTSS_UNPROVEN_POLLS / 2);
else if (o->why == RTSS_WHY_FAILED)
say("the game's frames came sooner than its limit while RivaTuner Statistics Server was to hold them at %.3f fps",
o->mhz / 1000.0);
put_back();
}
// Whether RTSS holds the game to the want w this poll read, and the log of which of the three is in force: RTSS with
// the game's limit, RTSS with the player's own limit, or the game's own wait and why. The first decision, then each
// change, up to kLimiterLines lines. The want may have changed since the poll read it: active() compares the two, so a
// stale store never lets the game's own wait stand aside for a newer want. The store is made either way, but a hold is
// logged only while its want is still the current one.
void set_active(int64_t w, int limiter, int64_t mhz, const char* why) {
St& s = *st;
const bool on = limiter != GAME;
s.active_own.store(limiter == RTSS_OWN_LIMIT);
s.active_for.store(on ? w : (int64_t)RTSS_UNSET);
if ((on && s.want.load() != w) || (s.logged == limiter && (on ? mhz == s.logged_mhz : !strcmp(why, s.said)))) return;
s.logged = limiter, s.logged_mhz = mhz;
snprintf(s.said, sizeof s.said, "%s", on ? "" : why);
if (++s.lines > kLimiterLines) return;
if (limiter == RTSS_GAME_LIMIT)
say("RivaTuner Statistics Server limits the game, with the game's limit of %.3f fps; the game's own wait stands aside",
mhz / 1000.0);
else if (limiter == RTSS_OWN_LIMIT)
say("RivaTuner Statistics Server limits the game, with your own limit of %.3f fps in RivaTuner (the game could not put "
"its own into RivaTuner's profile); the game's own wait stands aside", mhz / 1000.0);
else say("the game's own wait limits the game (%s)", why);
if (s.lines == kLimiterLines) say("later changes of the limiter in charge are not logged");
}
// RTSS's own settings for the game, read from its profile files as RTSS keeps them: Profiles\<exe>.cfg, else
// Profiles\Global, and a key the game's profile lacks from Global. Read as files, so no call goes into RTSS's hooks
// library; read again only when the time either was written changes. Worker thread only, while RTSS lists the game.
void read_settings() {
St& s = *st;
char own[1100], global[1100];
snprintf(own, sizeof own, "%s\\Profiles\\%s.cfg", s.rtss_dir, s.profile);
snprintf(global, sizeof global, "%s\\Profiles\\Global", s.rtss_dir);
const uint64_t a = s.sys->file_stamp(own), b = s.sys->file_stamp(global);
if (a == s.stamps[0] && b == s.stamps[1]) return;
RtssProfile po{}, pg{};
bool have_own = false, have_global = false;
size_t n = 0;
if (char* t = a ? s.sys->file_read(own, &n) : nullptr) rtss_profile_parse(t, n, &po), have_own = true, free(t);
if (char* t = b ? s.sys->file_read(global, &n) : nullptr) rtss_profile_parse(t, n, &pg), have_global = true, free(t);
if ((a && !have_own) || (b && !have_global)) return; // (in use this moment: read again at the next poll)
s.stamps[0] = a, s.stamps[1] = b;
s.settings = rtss_settings_of(have_own ? &po : nullptr, have_global ? &pg : nullptr);
if (a) snprintf(s.settings_file, sizeof s.settings_file, "Profiles\\%s.cfg", s.profile);
else snprintf(s.settings_file, sizeof s.settings_file, "Profiles\\Global");
}
// While RTSS lists the game, the settings it holds for it are logged once per change. While RTSS has no part in the
// game's limit (rtss_out_of_limit) the game's own wait or Reflex holds the frames, and a limit the player set in RTSS
// itself for the game (its global profile or the game's own, a frame time limit or scanline sync) still holds them a
// second time, unless it is the one limiter in force (own_holds: rtss_own_decide). Logged once per change.
void watch(int64_t w, int fail, bool listed, bool own_holds) {
St& s = *st;
const bool out = rtss_out_of_limit(w, fail);
bool on = false;
const char* by = rtss_out_limiter(w);
char text[160] = "";
if (listed) {
read_settings();
char all[RTSS_NAME_MAX + 200], values[160];
rtss_settings_text(values, sizeof values, &s.settings);
snprintf(all, sizeof all, "(%s): %s", s.settings_file, values);
if (strcmp(all, s.settings_said) && s.settings_lines++ < 40) say("RivaTuner Statistics Server's settings for the game %s", all);
snprintf(s.settings_said, sizeof s.settings_said, "%s", all);
}
if (out && listed) {
on = rtss_two(w, fail, listed, rtss_settings_limit(&s.settings)) && !own_holds;
rtss_settings_text(text, sizeof text, &s.settings);
}
if (!out) s.two_said = 0; // RTSS takes the game's limit: a later want out of it logs afresh
else if ((int)on != s.two_said || (on && (strcmp(text, s.two_text) || strcmp(by, s.two_by)))) {
if (s.two_lines++ < 40) {
if (on)
say("two limiters: RivaTuner Statistics Server's own setting for the game (%s: %s) limits it beside %s",
s.settings_file, text, by);
else if (s.two_said) say("RivaTuner Statistics Server no longer limits the game beside %s", by);
}
s.two_said = on;
snprintf(s.two_by, sizeof s.two_by, "%s", by);
snprintf(s.two_text, sizeof s.two_text, "%s", text);
}
}
// RTSS's folder, its hooks library and this executable's profile name, on the worker before its first poll. False
// when RTSS is not installed: the game's own wait for the run.
bool find_rtss() {
St& s = *st;
char dir[500];
if (!s.sys->install_dir(dir, sizeof dir) || !*dir) {
say("RivaTuner Statistics Server is not installed: the game's own wait limits the game");
return false;
}
snprintf(s.dll_path, sizeof s.dll_path, "%s\\RTSSHooks64.dll", dir);
snprintf(s.rtss_dir, sizeof s.rtss_dir, "%s", dir);
if (!s.sys->file_stamp(s.dll_path)) {
say("RivaTuner Statistics Server's folder %s has no RTSSHooks64.dll: the game's own wait limits the game", dir);
return false;
}
// (RTSS names a profile after the executable, as Triforce.exe. Without a name nothing is ever loaded or saved: an
// empty name is the global profile's)
if (!s.sys->exe_name(s.profile, sizeof s.profile) || !*s.profile) return false;
if (!s.sys->state_path(s.state_path, sizeof s.state_path)) s.state_path[0] = 0;
const int64_t first = s.want.load();
if (first >= 0 || first == RTSS_UNSET)
say("RivaTuner Statistics Server found in %s; it takes the game's frame limit while it runs", dir);
else
say("RivaTuner Statistics Server found in %s; kept out (%s), and while it is kept out its thread only reads RTSS's own "
"settings for the game, to tell when RTSS limits it a second time", dir, rtss_out_text(first));
return true;
}
} // namespace
void start(const Sys* sys) { st->sys = sys; }
void want(int64_t mhz) {
St& s = *st;
if (s.closed.load(std::memory_order_relaxed)) return; // the exit put the profile back
// a new limit: the game's own wait holds it until the worker had RTSS apply it, up to half a second later:
// active_for no longer matches
s.want.store(mhz);
}
void movie(int64_t mhz) { st->movie_mhz.store(mhz, std::memory_order_relaxed); }
void fast() {
St& s = *st;
s.fast.store(1);
s.active_for.store(RTSS_UNSET); // the game's own wait from this frame on, until the worker says otherwise
}
bool active() {
St& s = *st;
if (s.closed.load(std::memory_order_relaxed)) return false;
const int64_t a = s.active_for.load(std::memory_order_relaxed);
return a > 0 && a == s.want.load(std::memory_order_relaxed);
}
bool own() { return st->active_own.load(std::memory_order_relaxed) != 0; }
bool poll() {
St& s = *st;
if (s.found < 0) {
s.found = find_rtss() ? 1 : 0;
s.v_installed.store(s.found), s.v_looked.store(1);
if (s.found) s.ready.store(1, std::memory_order_release); // restore may read the names now
}
if (!s.found) return false;
const int64_t w = s.want.load();
if (w == RTSS_UNSET) return true;
std::lock_guard lk(lock);
if (s.closed.load()) return true; // a want read before restore must not reach the profile after it
RtssIn in{};
in.want = w, in.movie_mhz = s.movie_mhz.load(), in.now_ns = s.sys->now_ns(), in.hosted_ns = s.sys->hosted_ns();
in.fast = s.fast.exchange(0);
s.sys->shared(&in.app);
if (in.app.running != s.was_running || in.app.listed != s.was_listed) {
if (!in.app.running && !s.said_not_running++)
say("RivaTuner Statistics Server is installed but not running: start it and it limits the game's frames; until then "
"the game limits itself");
else if (!in.app.running) say("RivaTuner Statistics Server is not running");
else if (!in.app.listed) say("RivaTuner Statistics Server runs; it has not hooked the game yet");
else say("RivaTuner Statistics Server has hooked the game");
s.was_running = in.app.running, s.was_listed = in.app.listed;
}
in.hooks = s.hooks ? 1 : rtss_wants_hooks(&s.S, &in) ? (load_hooks() ? 1 : 0) : -1;
if (!s.left_done && in.app.running && (in.hooks == 1 || !s.left_tried)) { // before this run writes anything
s.left_tried = 1;
s.left_done = left_start(w, in.hooks);
if (s.hooks) in.hooks = 1; // (loaded for the restore alone)
}
in.current = s.want.load() == w;
const RtssOps ops = {nullptr, apply, put_back_op};
const int fail_before = s.S.fail;
RtssOut o = rtss_poll(&s.S, &in, &ops);
if (o.retry)
say("RivaTuner Statistics Server is tried again as the game's limiter (%s)",
o.retry == 2 ? "it was closed and runs again" : o.retry == 3 ? "another limit is wanted" : "a minute after it was given up");
// (each run: the exit takes the game's limit out of the profile again, and a file under Program Files that an
// administrator's run made is not a user's to change either)
if (s.S.fail == RTSS_FAIL_READONLY && fail_before != RTSS_FAIL_READONLY && !s.said_readonly++)
say("RivaTuner Statistics Server's profile %s did not keep the game's limit: its Profiles folder needs administrator "
"rights. Run the game as administrator, or set the limit in RivaTuner yourself; until then the game limits "
"itself, unless a limit of your own in RivaTuner already holds it", s.profile);
if (rtss_fail_retried(s.S.fail) && !rtss_fail_retried(fail_before))
say(s.S.retries < RTSS_RETRIES
? "the game's own wait limits the game for now; RivaTuner Statistics Server is tried again in a minute"
: "the game's own wait limits the game; RivaTuner Statistics Server is tried again when it starts again or "
"another limit is wanted");
// (Triforce) the game's limit cannot go into the profile (it cannot be written, RTSS's library did not load, or RTSS
// refused this limit): RTSS may still hold the game by a limit of the player's own
int limiter = o.holds ? RTSS_GAME_LIMIT : GAME, why = o.why;
int64_t mhz = o.holds ? o.mhz : 0;
char text[200];
rtss_why_text(text, sizeof text, &o, s.S.fail);
if (!o.holds && w > 0 && in.current &&
(s.S.fail == RTSS_FAIL_READONLY || s.S.fail == RTSS_FAIL_LIBRARY || o.why == RTSS_WHY_REFUSED)) {
if (in.app.listed) read_settings();
RtssOwnIn oi{};
oi.want = w, oi.limit_mhz = in.app.listed ? rtss_settings_mhz(&s.settings) : 0;
oi.other = in.app.listed && rtss_settings_other(&s.settings);
oi.movie_mhz = in.movie_mhz, oi.now_ns = in.now_ns, oi.hosted_ns = in.hosted_ns, oi.fast = in.fast, oi.app = in.app;
const RtssOut oo = rtss_own_decide(&s.own, &oi);
why = oo.why;
if (oo.holds) limiter = RTSS_OWN_LIMIT, mhz = oi.limit_mhz;
else {
char base[200];
snprintf(base, sizeof base, "%s", text);
snprintf(text, sizeof text, "%s; %s", base, rtss_own_why_text(oo.why, s.own.fail));
}
}
if (o.why != RTSS_WHY_NEWER) set_active(w, limiter, mhz, text); // a newer want waits for the next poll
watch(w, s.S.fail, in.app.listed != 0, limiter == RTSS_OWN_LIMIT);
s.v_limiter.store(limiter), s.v_why.store(why), s.v_fail.store(s.S.fail), s.v_mhz.store(mhz);
s.v_running.store(in.app.running), s.v_listed.store(in.app.listed);
return true;
}
void restore() {
St& s = *st;
// The game presents on after this call (the rest of the exit runs first), so from here the game's own wait holds
// the frames left and no want reaches the profile again: the worker would otherwise write the game's limit back
// into the profile just put back, and the next run would take it for the player's own. Set before the check
// below, so that an exit before the first limit also stops a later want from reaching the worker.
s.closed.store(1);
s.active_for.store(RTSS_UNSET);
if (!s.ready.load(std::memory_order_acquire)) return;
// the exit may come while the worker is inside RTSS's library: give up after a second rather than wait on it
for (int i = 0; i < 100; i++) {
if (lock.try_lock()) {
if (s.hooks && (s.S.changed || s.owed)) put_back(), rtss_put_back_done(&s.S);
lock.unlock();
return;
}
std::this_thread::sleep_for(std::chrono::milliseconds(10));
}
}
void report_left() {
St& s = *st;
char name[RTSS_NAME_MAX], path[560];
if (!s.sys || !s.sys->exe_name(name, sizeof name) || !*name || !s.sys->state_path(path, sizeof path)) return;
size_t n = 0;
char* text = s.sys->file_read(path, &n);
RtssSaved r;
if (text && rtss_saved_find(text, name, &r))
say("%s holds the game's limit of %u/%u for RivaTuner Statistics Server's profile %s from a run that did not close; "
"HOS_RTSS=0 leaves the profile as it is, and a start without it puts it back", path, r.limit, r.den, name);
free(text);
}
View view() {
St& s = *st;
View v;
v.limiter = s.closed.load() ? (int)GAME : s.v_limiter.load(), v.why = s.v_why.load(), v.fail = s.v_fail.load();
v.want = s.want.load(), v.mhz = s.v_mhz.load();
v.looked = s.v_looked.load() != 0, v.installed = s.v_installed.load() != 0, v.running = s.v_running.load() != 0;
v.listed = s.v_listed.load() != 0;
return v;
}
void reset_for_test() {
delete st;
st = new St;
}
} // namespace mkwvk::rtss

82
gpu/vk/rtss_limit.h Normal file
View File

@@ -0,0 +1,82 @@
// (Triforce) RivaTuner Statistics Server as the game's frame limiter (rtss_limit.cpp), after NX2navi's
// runtime/rtss_limit.c: what the pacing asks of it (vk_pace.cpp), and the one interface through which anything of
// RTSS's, or of the system, is reached (Sys). The game's build gives the real one (rtss_sys_win.cpp); the tests give a
// stand-in with a fake hooks library, a fake shared memory and a fake clock, and never the real one.
#pragma once
#include <cstddef>
#include <cstdint>
#include "rtss_decide.h"
namespace mkwvk::rtss {
/// Everything rtss_limit.cpp asks for outside itself. Three kinds, and the third is the only one that changes
/// anything of RTSS's: its profile calls, which rtss_limit.cpp makes for the game's own profile alone.
struct Sys {
// ---- the game's own: the clock, the log, the executable's name, and the state file beside the game
uint64_t (*now_ns)();
uint64_t (*hosted_ns)(); // the clock after the newest present of the window, 0 none yet
void (*log)(const char* line);
bool (*exe_name)(char* out, size_t n); // the executable's file name: what RTSS names its profile by
bool (*state_path)(char* out, size_t n); // triforce_rtss_profile.txt in the game folder
char* (*file_read)(const char* path, size_t* n); // a whole file, malloc'd and NUL-terminated; null: none, or it does not read
int64_t (*file_size)(const char* path); // -1: no such file
bool (*file_write)(const char* path, const void* data, size_t n); // the whole file, replaced in one step
void (*file_delete)(const char* path);
// ---- RTSS, read only
bool (*install_dir)(char* out, size_t n); // its folder, as its installer recorded it; false: not installed
uint64_t (*file_stamp)(const char* path); // a file's write time, 0: no such file (its library, its profile files)
void (*shared)(RtssApp* a); // its shared memory as rtss_parse reads it for this process
// ---- RTSS's hooks library: the profile calls, loaded at run time. A name is a profile's, never empty
bool (*hooks_load)(const char* dll_path);
void (*load_profile)(const char* name); // reads a profile into the loaded copy (the global one's values for a name without its own)
void (*save_profile)(const char* name); // writes the loaded copy as Profiles\<name>.cfg; reports nothing
void (*delete_profile)(const char* name);
bool (*get_property)(const char* property, uint32_t* value);
bool (*set_property)(const char* property, uint32_t value);
void (*update_profiles)(); // every hooked process reads its profile again
};
/// What is in force, for the log and the performance overlay's line.
enum Limiter { GAME = 0, RTSS_GAME_LIMIT = 1, RTSS_OWN_LIMIT = 2 };
struct View {
int limiter = GAME; // a Limiter
int why = RTSS_WHY_UNSET; // while it is the game's own wait: RTSS_WHY_*, of the game's limit or of the player's own
int fail = RTSS_FAIL_NONE;
int64_t want = RTSS_UNSET; // what RTSS was asked to hold
int64_t mhz = 0; // the limit RTSS holds the game to, while it does
bool looked = false; // the worker has looked for RTSS
bool installed = false, running = false, listed = false;
};
/// The interface, once, before anything else here is called.
void start(const Sys* sys);
/// Where the game's frame starts (the game's presenting thread): what RTSS is to hold, in millihertz of presented
/// frames, or RTSS_OUT_*. A new want is the game's own wait's until the worker has RTSS hold it. Makes no system call.
void want(int64_t mhz);
/// The rate of a movie that plays, 0 none: the game's own wait holds a movie.
void movie(int64_t mhz);
/// The frames RTSS was to hold came sooner than the game's period (rtss_watch_step): the game's own wait from now, and
/// the worker gives up on RTSS until it is tried again.
void fast();
/// RTSS holds the game to the want now: the game's own wait stands aside (with rtss_frame_holds).
bool active();
/// That hold is by a limit the player set in RTSS (the game's own could not be written): it holds a movie too, which
/// the game's own wait holds while RTSS has the game's limit.
bool own();
/// One pass of the worker (its thread calls it every half second; the first one looks for RTSS). False: there is
/// nothing more to do in this run (RTSS is not installed).
bool poll();
/// An orderly exit: the profile gets back what it held before the game changed it; nothing reaches it after.
void restore();
/// HOS_RTSS=0: RTSS is out entirely, and nothing of it is read or loaded. A line a run that did not close left in the
/// state file is only reported: the game's own file, read once.
void report_left();
View view();
/// The real interface (rtss_sys_win.cpp); null where there is no RTSS (not Windows).
const Sys* real_sys();
/// (tests) Everything as at the start of a run: the next start() begins a new one.
void reset_for_test();
} // namespace mkwvk::rtss

171
gpu/vk/rtss_sys_win.cpp Normal file
View File

@@ -0,0 +1,171 @@
// (Triforce) The real interface of rtss_limit.h's Sys: Windows, and RivaTuner Statistics Server as it is installed.
// Every call that reaches RTSS, its files or the registry is in this file and nowhere else; rtss_limit.cpp decides
// when, and the tests run rtss_limit.cpp against a stand-in instead, so nothing here is ever run by a test.
//
// What it does to RTSS, all of it: reads its install folder from the registry key its installer wrote; reads its
// shared memory; reads its profile files; loads its hooks library (RTSSHooks64.dll) and calls six of its exports:
// LoadProfile, SaveProfile, DeleteProfile, GetProfileProperty, SetProfileProperty and UpdateProfiles. It writes no
// file of RTSS's itself, and never starts or stops RTSS.
#include "rtss_limit.h"
#ifdef _WIN32
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <string>
#include "vk_internal.h"
#include "vk_pace.h"
#include "vk_paths.h"
namespace mkwvk::rtss {
namespace {
typedef void (*ProfileFn)(LPCSTR name);
typedef BOOL (*PropertyFn)(LPCSTR property, LPBYTE data, DWORD size);
typedef void (*UpdateFn)(void);
HMODULE dll;
ProfileFn fn_load, fn_save, fn_delete;
PropertyFn fn_get, fn_set;
UpdateFn fn_update;
uint64_t now_ns() { return (uint64_t)host->now_ns(); }
uint64_t hosted_ns() { return pace::presented_ns(); }
void log_line(const char* line) { logf("%s", line); }
bool exe_name(char* out, size_t n) {
char path[MAX_PATH];
const DWORD len = GetModuleFileNameA(nullptr, path, sizeof path);
if (!len || len >= sizeof path) return false;
const char* base = path;
for (const char* p = path; *p; p++)
if (*p == '\\' || *p == '/') base = p + 1;
return snprintf(out, n, "%s", base) < (int)n && *out;
}
bool state_path(char* out, size_t n) { // (the game folder, where the options file is: vk_paths.h)
char exe[MAX_PATH];
const DWORD len = game_module_path(exe, MAX_PATH);
if (!len || len >= MAX_PATH) return false;
std::string dir(exe, len);
dir = dir.substr(0, dir.find_last_of("\\/") + 1) + "triforce_rtss_profile.txt";
return snprintf(out, n, "%s", dir.c_str()) < (int)n;
}
char* file_read(const char* path, size_t* n) {
const HANDLE f = CreateFileA(path, GENERIC_READ, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, nullptr, OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL, nullptr);
if (f == INVALID_HANDLE_VALUE) return nullptr;
LARGE_INTEGER size;
char* text = nullptr;
if (GetFileSizeEx(f, &size) && size.QuadPart >= 0 && size.QuadPart < (1 << 20) && (text = (char*)malloc((size_t)size.QuadPart + 1))) {
DWORD got = 0;
if (ReadFile(f, text, (DWORD)size.QuadPart, &got, nullptr) && got == (DWORD)size.QuadPart) text[got] = 0, *n = got;
else free(text), text = nullptr;
}
CloseHandle(f);
return text;
}
int64_t file_size(const char* path) {
WIN32_FILE_ATTRIBUTE_DATA a;
if (!GetFileAttributesExA(path, GetFileExInfoStandard, &a)) return -1;
return (int64_t)((uint64_t)a.nFileSizeHigh << 32 | a.nFileSizeLow);
}
/// The game's own state file, replaced in one step: written beside it, then moved over it
bool file_write(const char* path, const void* data, size_t n) {
const std::string tmp = std::string(path) + ".tmp";
FILE* f = fopen(tmp.c_str(), "wb");
if (!f) return false;
const bool wrote = fwrite(data, 1, n, f) == n;
if (fclose(f) || !wrote) return remove(tmp.c_str()), false;
if (MoveFileExA(tmp.c_str(), path, MOVEFILE_REPLACE_EXISTING)) return true;
remove(tmp.c_str());
return false;
}
void file_delete(const char* path) { remove(path); }
bool install_dir(char* out, size_t n) {
DWORD size = (DWORD)n;
return RegGetValueA(HKEY_LOCAL_MACHINE, "SOFTWARE\\WOW6432Node\\Unwinder\\RTSS", "InstallDir", RRF_RT_REG_SZ, nullptr, out, &size) ==
ERROR_SUCCESS && *out;
}
uint64_t file_stamp(const char* path) {
WIN32_FILE_ATTRIBUTE_DATA a;
if (!GetFileAttributesExA(path, GetFileExInfoStandard, &a)) return 0;
return ((uint64_t)a.ftLastWriteTime.dwHighDateTime << 32 | a.ftLastWriteTime.dwLowDateTime) | 1;
}
/// RTSS's shared memory, opened for reading and closed again at each look
void shared(RtssApp* a) {
memset(a, 0, sizeof *a);
const HANDLE m = OpenFileMappingA(FILE_MAP_READ, FALSE, "RTSSSharedMemoryV2");
if (!m) return;
const uint8_t* p = (const uint8_t*)MapViewOfFile(m, FILE_MAP_READ, 0, 0, 0);
MEMORY_BASIC_INFORMATION mi;
if (p && VirtualQuery(p, &mi, sizeof mi)) rtss_parse(p, mi.RegionSize, GetCurrentProcessId(), a);
if (p) UnmapViewOfFile(p);
CloseHandle(m);
}
bool hooks_load(const char* dll_path) {
if (dll) return true;
dll = LoadLibraryA(dll_path);
if (!dll) return false;
fn_load = (ProfileFn)(void*)GetProcAddress(dll, "LoadProfile");
fn_save = (ProfileFn)(void*)GetProcAddress(dll, "SaveProfile");
fn_delete = (ProfileFn)(void*)GetProcAddress(dll, "DeleteProfile");
fn_get = (PropertyFn)(void*)GetProcAddress(dll, "GetProfileProperty");
fn_set = (PropertyFn)(void*)GetProcAddress(dll, "SetProfileProperty");
fn_update = (UpdateFn)(void*)GetProcAddress(dll, "UpdateProfiles");
if (fn_load && fn_save && fn_delete && fn_get && fn_set && fn_update) return true;
FreeLibrary(dll);
dll = nullptr;
return false;
}
// (a name is never empty here: to RTSS an empty name is its global profile, which the game never loads, saves or removes)
void load_profile(const char* name) {
if (name && *name) fn_load(name);
}
void save_profile(const char* name) {
if (name && *name) fn_save(name);
}
void delete_profile(const char* name) {
if (name && *name) fn_delete(name);
}
bool get_property(const char* property, uint32_t* value) {
DWORD v = *value;
const bool ok = fn_get(property, (LPBYTE)&v, sizeof v) != 0;
if (ok) *value = v;
return ok;
}
bool set_property(const char* property, uint32_t value) {
DWORD v = value;
return fn_set(property, (LPBYTE)&v, sizeof v) != 0;
}
void update_profiles() { fn_update(); }
const Sys real = {now_ns, hosted_ns, log_line, exe_name, state_path, file_read, file_size, file_write, file_delete,
install_dir, file_stamp, shared, hooks_load, load_profile, save_profile, delete_profile, get_property, set_property,
update_profiles};
} // namespace
const Sys* real_sys() { return &real; }
} // namespace mkwvk::rtss
#else
namespace mkwvk::rtss {
const Sys* real_sys() { return nullptr; } // (RivaTuner Statistics Server is a Windows program)
} // namespace mkwvk::rtss
#endif

View File

@@ -0,0 +1,32 @@
// (Triforce) Shared by the passes that stand in for the game's SMAA composition (gpu/vk/vk_aa.cpp; aa_post.comp, smaa_luma.comp,
// smaa_blend.comp). After NX2navi's shaders/aa_common.glsl.
//
// The game's composition stores the post-processed picture into its B10G11R11 target as raw words: each channel is the square
// root of its linear value, the three packed together as RGB9E5 (red in bits 0-8, green 9-17, blue 18-26, the shared exponent
// in 27-31: Vulkan's E5B9G9R9 layout), which the gameplay upscale then samples through an RGB9E5 view. The passes here write the
// same words through an R32_UINT view of that target.
//
// Written in what GLSL and C++ share (scalar steps, no swizzles, no arrays): tests/test_aa_shaders.py compiles this file as
// C++ and runs it on authored pixels.
const float AA_MAX_9E5 = 65408.0; // ((2^9 - 1) / 2^9 * 2^16: the largest value the format holds)
// (a channel as the format can hold it: not negative, not past its largest; a NaN is 0)
float aa_channel_9e5(float v) { return v > 0.0 ? min(v, AA_MAX_9E5) : 0.0; }
// RGB9E5 of a colour: the shared exponent of the largest channel, each channel rounded to 9 bits of it
uint aa_pack_rgb9e5(vec3 c) {
float r = aa_channel_9e5(c.r), g = aa_channel_9e5(c.g), b = aa_channel_9e5(c.b);
float m = max(max(r, g), b);
float e = max(-16.0, floor(log2(max(m, 1e-30)))) + 16.0; // (biased by 15, plus one)
float scale = exp2(e - 24.0); // (2^(e - 15 - 9))
if (floor(m / scale + 0.5) >= 512.0) { // (the largest channel rounds up to the next power of two)
e += 1.0;
scale *= 2.0;
}
return uint(floor(r / scale + 0.5)) | (uint(floor(g / scale + 0.5)) << 9) | (uint(floor(b / scale + 0.5)) << 18) |
(uint(e) << 27);
}
// The stored form of a linear colour: its square root, which is also the space the edges are looked for in
vec3 aa_encode(vec3 c) { return sqrt(max(c, vec3(0.0))); }

View File

@@ -0,0 +1,93 @@
// (Triforce) The Anti-aliasing row's FXAA (gpu/vk/vk_aa.cpp; aa_post.comp): a fast approximate smoothing of the tone mapped
// picture's edges, in place of the game's SMAA. After NX2navi's shaders/aa_post.comp.
//
// It works on the square-rooted luma (aa_common.glsl aa_encode: the space the picture is stored in). Where the local contrast
// is high enough, it finds whether the edge runs across or along the pixel, walks along the edge both ways until the luma
// pair it follows changes, and moves the sample point toward the neighbour across the edge by how far the pixel sits from
// the nearer end of the edge. A blur by the 3 x 3 neighbourhood's contrast handles features thinner than a pixel. The sample
// at the moved point is bilinear, so the move blends the two sides.
//
// The file that includes it says what the picture is: vec3 aa_at(vec2 uv), the picture's encoded colour at uv (0 to 1,
// bilinear between the texels' centres, the edge's texels past the picture's edge). Written in what GLSL and C++ share:
// tests/test_aa_shaders.py compiles this file as C++ and runs it on authored pixels.
const float AA_EDGE_MIN = 0.0312; // (no smoothing below this contrast: dark areas)
const float AA_EDGE_REL = 0.125; // (nor below this share of the brightest neighbour)
const float AA_SUBPIXEL = 0.75; // (how much a thin feature is blurred)
const int AA_STEPS = 12;
// (how far each step of the walk along the edge goes, in texels: 1 x 5, 1.5, 2 x 4, 4, 8)
float aa_step_len(int i) { return i < 5 ? 1.0 : i == 5 ? 1.5 : i < 10 ? 2.0 : i == 10 ? 4.0 : 8.0; }
float aa_luma(vec3 c) { return dot(c, vec3(0.299, 0.587, 0.114)); }
float aa_luma_at(vec2 uv) { return aa_luma(aa_at(uv)); }
// The picture at uv, smoothed. d: one texel (1 / the picture's size)
vec3 aa_fxaa(vec2 uv, vec2 d) {
vec3 cM = aa_at(uv);
float lM = aa_luma(cM);
float lN = aa_luma_at(uv + vec2(0.0, -d.y)), lS = aa_luma_at(uv + vec2(0.0, d.y));
float lW = aa_luma_at(uv + vec2(-d.x, 0.0)), lE = aa_luma_at(uv + vec2(d.x, 0.0));
float lo = min(lM, min(min(lN, lS), min(lW, lE)));
float hi = max(lM, max(max(lN, lS), max(lW, lE)));
float range = hi - lo;
if (range < max(AA_EDGE_MIN, hi * AA_EDGE_REL)) return cM;
float lNW = aa_luma_at(uv + vec2(-d.x, -d.y)), lNE = aa_luma_at(uv + vec2(d.x, -d.y));
float lSW = aa_luma_at(uv + vec2(-d.x, d.y)), lSE = aa_luma_at(uv + vec2(d.x, d.y));
// thin features: how far the centre stands out from the neighbourhood's weighted mean
float mean = (2.0 * (lN + lS + lW + lE) + lNW + lNE + lSW + lSE) / 12.0;
float sub = clamp(abs(mean - lM) / range, 0.0, 1.0);
sub = sub * sub * (3.0 - 2.0 * sub); // (smoothstep(0, 1, sub))
sub = sub * sub * AA_SUBPIXEL;
// the edge's direction: horizontal when the luma changes more from row to row than from column to column
float across_rows = abs(0.25 * lNW - 0.5 * lN + 0.25 * lNE) + abs(0.5 * lW - lM + 0.5 * lE) +
abs(0.25 * lSW - 0.5 * lS + 0.25 * lSE);
float across_cols = abs(0.25 * lNW - 0.5 * lW + 0.25 * lSW) + abs(0.5 * lN - lM + 0.5 * lS) +
abs(0.25 * lNE - 0.5 * lE + 0.25 * lSE);
bool horizontal = across_cols >= across_rows;
// which side of the pixel the edge is on: the neighbour across it with the larger step
float l1 = horizontal ? lN : lW, l2 = horizontal ? lS : lE;
float g1 = abs(l1 - lM), g2 = abs(l2 - lM);
bool first = g1 >= g2;
float grad = 0.25 * max(g1, g2);
float step_len = horizontal ? d.y : d.x;
float pair = 0.5 * (l2 + lM); // (the mean luma of the pixel and its neighbour across the edge, which the walk follows)
if (first) {
step_len = -step_len;
pair = 0.5 * (l1 + lM);
}
vec2 edge = uv;
if (horizontal) edge.y += 0.5 * step_len;
else edge.x += 0.5 * step_len;
// walk along the edge both ways until the pair's luma changes by more than a quarter of the step across it
vec2 dir = horizontal ? vec2(d.x, 0.0) : vec2(0.0, d.y);
vec2 uv1 = edge - dir * aa_step_len(0), uv2 = edge + dir * aa_step_len(0);
float e1 = aa_luma_at(uv1) - pair, e2 = aa_luma_at(uv2) - pair;
bool done1 = abs(e1) >= grad, done2 = abs(e2) >= grad;
for (int i = 1; i < AA_STEPS && !(done1 && done2); i++) {
if (!done1) {
uv1 = uv1 - dir * aa_step_len(i);
e1 = aa_luma_at(uv1) - pair;
done1 = abs(e1) >= grad;
}
if (!done2) {
uv2 = uv2 + dir * aa_step_len(i);
e2 = aa_luma_at(uv2) - pair;
done2 = abs(e2) >= grad;
}
}
float dist1 = horizontal ? uv.x - uv1.x : uv.y - uv1.y;
float dist2 = horizontal ? uv2.x - uv.x : uv2.y - uv.y;
bool nearer1 = dist1 < dist2;
float offset = 0.5 - min(dist1, dist2) / (dist1 + dist2);
// only when the luma at the nearer end moves the way the centre's does: else the pixel is outside the edge's span
if (((nearer1 ? e1 : e2) < 0.0) == (lM < pair)) offset = 0.0;
offset = max(offset, sub);
vec2 moved = uv;
if (horizontal) moved.y += offset * step_len;
else moved.x += offset * step_len;
return aa_at(moved);
}

View File

@@ -0,0 +1,27 @@
#version 450
// (Triforce) The game's SMAA composition replaced (gpu/vk/vk_aa.cpp, the Insert menu's Anti-aliasing row): mode 0 stores the
// tone mapped picture as it is (Off), mode 1 smooths its edges first (FXAA: aa_fxaa.glsl). Input: the picture the game's pass
// samples, linear. Output: the target's raw RGB9E5 words of the square root of each channel (aa_common.glsl), through an
// R32_UINT view. After NX2navi's shaders/aa_post.comp.
#extension GL_GOOGLE_include_directive : require
layout(local_size_x = 8, local_size_y = 8) in;
layout(binding = 0) uniform sampler2D src;
layout(binding = 1, r32ui) uniform writeonly uimage2D dst;
layout(push_constant) uniform Push {
vec2 inv; // (1 / size)
ivec2 size; // (the target's size)
int mode; // (0 the picture as it is, 1 FXAA)
} pc;
#include "aa_common.glsl"
vec3 aa_at(vec2 uv) { return aa_encode(textureLod(src, uv, 0.0).rgb); }
#include "aa_fxaa.glsl"
void main() {
ivec2 p = ivec2(gl_GlobalInvocationID.xy);
if (p.x >= pc.size.x || p.y >= pc.size.y) return;
vec2 uv = (vec2(p) + 0.5) * pc.inv;
vec3 c = pc.mode == 1 ? aa_fxaa(uv, pc.inv) : aa_encode(texelFetch(src, p, 0).rgb);
imageStore(dst, p, uvec4(aa_pack_rgb9e5(c), 0u, 0u, 0u));
}

View File

@@ -0,0 +1,551 @@
// aa_post.comp + aa_common.glsl + aa_fxaa.glsl sha256 738724b74ed7f024352fd8cce5cf5eb6ef34c2ba838a88ef0944219ac4df84b5 (tests/test_aa_shaders.py --write: glslangValidator -V --target-env vulkan1.2 -I third_party/smaa --vn AA_POST_COMP_SPV)
// 1116.6.0
#pragma once
const uint32_t AA_POST_COMP_SPV[] = {
0x07230203,0x00010500,0x0008000b,0x0000030c,0x00000000,0x00020011,0x00000001,0x0006000b,
0x00000001,0x4c534c47,0x6474732e,0x3035342e,0x00000000,0x0003000e,0x00000000,0x00000001,
0x0009000f,0x00000005,0x00000004,0x6e69616d,0x00000000,0x0000009d,0x000002c6,0x000002cf,
0x00000302,0x00060010,0x00000004,0x00000011,0x00000008,0x00000008,0x00000001,0x00030003,
0x00000002,0x000001c2,0x000a0004,0x475f4c47,0x4c474f4f,0x70635f45,0x74735f70,0x5f656c79,
0x656e696c,0x7269645f,0x69746365,0x00006576,0x00080004,0x475f4c47,0x4c474f4f,0x6e695f45,
0x64756c63,0x69645f65,0x74636572,0x00657669,0x00040005,0x00000004,0x6e69616d,0x00000000,
0x00070005,0x0000000a,0x635f6161,0x6e6e6168,0x395f6c65,0x66283565,0x00003b31,0x00030005,
0x00000009,0x00000076,0x00070005,0x00000011,0x705f6161,0x5f6b6361,0x39626772,0x76283565,
0x003b3366,0x00030005,0x00000010,0x00000063,0x00060005,0x00000015,0x655f6161,0x646f636e,
0x66762865,0x00003b33,0x00030005,0x00000014,0x00000063,0x00050005,0x0000001b,0x615f6161,
0x66762874,0x00003b32,0x00030005,0x0000001a,0x00007675,0x00060005,0x00000021,0x735f6161,
0x5f706574,0x286e656c,0x003b3169,0x00030005,0x00000020,0x00000069,0x00060005,0x00000025,
0x6c5f6161,0x28616d75,0x3b336676,0x00000000,0x00030005,0x00000024,0x00000063,0x00060005,
0x00000029,0x6c5f6161,0x5f616d75,0x76287461,0x003b3266,0x00030005,0x00000028,0x00007675,
0x00070005,0x0000002e,0x665f6161,0x28616178,0x3b326676,0x3b326676,0x00000000,0x00030005,
0x0000002c,0x00007675,0x00030005,0x0000002d,0x00000064,0x00030005,0x0000003e,0x00000072,
0x00040005,0x0000003f,0x61726170,0x0000006d,0x00030005,0x00000044,0x00000067,0x00040005,
0x00000045,0x61726170,0x0000006d,0x00030005,0x0000004a,0x00000062,0x00040005,0x0000004b,
0x61726170,0x0000006d,0x00030005,0x00000050,0x0000006d,0x00030005,0x00000056,0x00000065,
0x00040005,0x00000060,0x6c616373,0x00000065,0x00030005,0x0000009d,0x00637273,0x00040005,
0x000000a2,0x61726170,0x0000006d,0x00040005,0x000000ce,0x61726170,0x0000006d,0x00040005,
0x000000d1,0x61726170,0x0000006d,0x00030005,0x000000d5,0x00004d63,0x00040005,0x000000d6,
0x61726170,0x0000006d,0x00030005,0x000000d9,0x00004d6c,0x00040005,0x000000da,0x61726170,
0x0000006d,0x00030005,0x000000dd,0x00004e6c,0x00040005,0x000000e4,0x61726170,0x0000006d,
0x00030005,0x000000e6,0x0000536c,0x00040005,0x000000ec,0x61726170,0x0000006d,0x00030005,
0x000000ee,0x0000576c,0x00040005,0x000000f5,0x61726170,0x0000006d,0x00030005,0x000000f7,
0x0000456c,0x00040005,0x000000fd,0x61726170,0x0000006d,0x00030005,0x000000ff,0x00006f6c,
0x00030005,0x00000109,0x00006968,0x00040005,0x00000113,0x676e6172,0x00000065,0x00030005,
0x00000122,0x00574e6c,0x00040005,0x0000012c,0x61726170,0x0000006d,0x00030005,0x0000012e,
0x00454e6c,0x00040005,0x00000137,0x61726170,0x0000006d,0x00030005,0x00000139,0x0057536c,
0x00040005,0x00000142,0x61726170,0x0000006d,0x00030005,0x00000144,0x0045536c,0x00040005,
0x0000014c,0x61726170,0x0000006d,0x00040005,0x0000014e,0x6e61656d,0x00000000,0x00030005,
0x00000161,0x00627573,0x00050005,0x00000176,0x6f726361,0x725f7373,0x0073776f,0x00050005,
0x00000194,0x6f726361,0x635f7373,0x00736c6f,0x00050005,0x000001b2,0x69726f68,0x746e6f7a,
0x00006c61,0x00030005,0x000001b6,0x0000316c,0x00030005,0x000001bb,0x0000326c,0x00030005,
0x000001c0,0x00003167,0x00030005,0x000001c5,0x00003267,0x00040005,0x000001ca,0x73726966,
0x00000074,0x00040005,0x000001ce,0x64617267,0x00000000,0x00050005,0x000001d3,0x70657473,
0x6e656c5f,0x00000000,0x00040005,0x000001de,0x72696170,0x00000000,0x00040005,0x000001ec,
0x65676465,0x00000000,0x00030005,0x000001fe,0x00726964,0x00030005,0x0000020b,0x00317675,
0x00040005,0x0000020f,0x61726170,0x0000006d,0x00030005,0x00000213,0x00327675,0x00040005,
0x00000216,0x61726170,0x0000006d,0x00030005,0x0000021a,0x00003165,0x00040005,0x0000021b,
0x61726170,0x0000006d,0x00030005,0x00000220,0x00003265,0x00040005,0x00000221,0x61726170,
0x0000006d,0x00040005,0x00000226,0x656e6f64,0x00000031,0x00040005,0x0000022b,0x656e6f64,
0x00000032,0x00030005,0x00000230,0x00000069,0x00040005,0x00000247,0x61726170,0x0000006d,
0x00040005,0x0000024c,0x61726170,0x0000006d,0x00040005,0x0000025b,0x61726170,0x0000006d,
0x00040005,0x00000260,0x61726170,0x0000006d,0x00040005,0x0000026b,0x74736964,0x00000031,
0x00040005,0x0000027c,0x74736964,0x00000032,0x00040005,0x0000028d,0x7261656e,0x00317265,
0x00040005,0x00000291,0x7366666f,0x00007465,0x00040005,0x000002a8,0x65766f6d,0x00000064,
0x00040005,0x000002bc,0x61726170,0x0000006d,0x00030005,0x000002c3,0x00000070,0x00080005,
0x000002c6,0x475f6c67,0x61626f6c,0x766e496c,0x7461636f,0x496e6f69,0x00000044,0x00040005,
0x000002cd,0x68737550,0x00000000,0x00040006,0x000002cd,0x00000000,0x00766e69,0x00050006,
0x000002cd,0x00000001,0x657a6973,0x00000000,0x00050006,0x000002cd,0x00000002,0x65646f6d,
0x00000000,0x00030005,0x000002cf,0x00006370,0x00030005,0x000002e0,0x00007675,0x00030005,
0x000002e9,0x00000063,0x00040005,0x000002f1,0x61726170,0x0000006d,0x00040005,0x000002f3,
0x61726170,0x0000006d,0x00040005,0x000002fc,0x61726170,0x0000006d,0x00030005,0x00000302,
0x00747364,0x00040005,0x00000305,0x61726170,0x0000006d,0x00040047,0x0000009d,0x00000021,
0x00000000,0x00040047,0x0000009d,0x00000022,0x00000000,0x00040047,0x000002c6,0x0000000b,
0x0000001c,0x00030047,0x000002cd,0x00000002,0x00050048,0x000002cd,0x00000000,0x00000023,
0x00000000,0x00050048,0x000002cd,0x00000001,0x00000023,0x00000008,0x00050048,0x000002cd,
0x00000002,0x00000023,0x00000010,0x00030047,0x00000302,0x00000019,0x00040047,0x00000302,
0x00000021,0x00000001,0x00040047,0x00000302,0x00000022,0x00000000,0x00040047,0x0000030b,
0x0000000b,0x00000019,0x00020013,0x00000002,0x00030021,0x00000003,0x00000002,0x00030016,
0x00000006,0x00000020,0x00040020,0x00000007,0x00000007,0x00000006,0x00040021,0x00000008,
0x00000006,0x00000007,0x00040017,0x0000000c,0x00000006,0x00000003,0x00040020,0x0000000d,
0x00000007,0x0000000c,0x00040015,0x0000000e,0x00000020,0x00000000,0x00040021,0x0000000f,
0x0000000e,0x0000000d,0x00040021,0x00000013,0x0000000c,0x0000000d,0x00040017,0x00000017,
0x00000006,0x00000002,0x00040020,0x00000018,0x00000007,0x00000017,0x00040021,0x00000019,
0x0000000c,0x00000018,0x00040015,0x0000001d,0x00000020,0x00000001,0x00040020,0x0000001e,
0x00000007,0x0000001d,0x00040021,0x0000001f,0x00000006,0x0000001e,0x00040021,0x00000023,
0x00000006,0x0000000d,0x00040021,0x00000027,0x00000006,0x00000018,0x00050021,0x0000002b,
0x0000000c,0x00000018,0x00000018,0x0004002b,0x00000006,0x00000031,0x00000000,0x00020014,
0x00000032,0x0004002b,0x00000006,0x00000038,0x477f8000,0x0004002b,0x0000000e,0x00000040,
0x00000000,0x0004002b,0x0000000e,0x00000046,0x00000001,0x0004002b,0x0000000e,0x0000004c,
0x00000002,0x0004002b,0x00000006,0x00000057,0xc1800000,0x0004002b,0x00000006,0x00000059,
0x0da24260,0x0004002b,0x00000006,0x0000005e,0x41800000,0x0004002b,0x00000006,0x00000062,
0x41c00000,0x0004002b,0x00000006,0x00000068,0x3f000000,0x0004002b,0x00000006,0x0000006b,
0x44000000,0x0004002b,0x00000006,0x0000006f,0x3f800000,0x0004002b,0x00000006,0x00000072,
0x40000000,0x0004002b,0x0000001d,0x00000081,0x00000009,0x0004002b,0x0000001d,0x0000008a,
0x00000012,0x0004002b,0x0000001d,0x0000008f,0x0000001b,0x0006002c,0x0000000c,0x00000095,
0x00000031,0x00000031,0x00000031,0x00090019,0x0000009a,0x00000006,0x00000001,0x00000000,
0x00000000,0x00000000,0x00000001,0x00000000,0x0003001b,0x0000009b,0x0000009a,0x00040020,
0x0000009c,0x00000000,0x0000009b,0x0004003b,0x0000009c,0x0000009d,0x00000000,0x00040017,
0x000000a0,0x00000006,0x00000004,0x0004002b,0x0000001d,0x000000a8,0x00000005,0x0004002b,
0x00000006,0x000000b3,0x3fc00000,0x0004002b,0x0000001d,0x000000b6,0x0000000a,0x0004002b,
0x00000006,0x000000be,0x40800000,0x0004002b,0x00000006,0x000000bf,0x41000000,0x0004002b,
0x00000006,0x000000c7,0x3e991687,0x0004002b,0x00000006,0x000000c8,0x3f1645a2,0x0004002b,
0x00000006,0x000000c9,0x3de978d5,0x0006002c,0x0000000c,0x000000ca,0x000000c7,0x000000c8,
0x000000c9,0x0004002b,0x00000006,0x00000118,0x3cff9724,0x0004002b,0x00000006,0x0000011a,
0x3e000000,0x0004002b,0x00000006,0x0000015f,0x41400000,0x0004002b,0x00000006,0x0000016c,
0x40400000,0x0004002b,0x00000006,0x00000174,0x3f400000,0x0004002b,0x00000006,0x00000177,
0x3e800000,0x00040020,0x000001b1,0x00000007,0x00000032,0x0004002b,0x0000001d,0x0000020e,
0x00000000,0x0004002b,0x0000001d,0x00000231,0x00000001,0x0004002b,0x0000001d,0x00000238,
0x0000000c,0x00040017,0x000002c1,0x0000001d,0x00000002,0x00040020,0x000002c2,0x00000007,
0x000002c1,0x00040017,0x000002c4,0x0000000e,0x00000003,0x00040020,0x000002c5,0x00000001,
0x000002c4,0x0004003b,0x000002c5,0x000002c6,0x00000001,0x00040017,0x000002c7,0x0000000e,
0x00000002,0x0005001e,0x000002cd,0x00000017,0x000002c1,0x0000001d,0x00040020,0x000002ce,
0x00000009,0x000002cd,0x0004003b,0x000002ce,0x000002cf,0x00000009,0x00040020,0x000002d0,
0x00000009,0x0000001d,0x00040020,0x000002e5,0x00000009,0x00000017,0x0004002b,0x0000001d,
0x000002ea,0x00000002,0x00090019,0x00000300,0x0000000e,0x00000001,0x00000000,0x00000000,
0x00000000,0x00000002,0x00000021,0x00040020,0x00000301,0x00000000,0x00000300,0x0004003b,
0x00000301,0x00000302,0x00000000,0x00040017,0x00000308,0x0000000e,0x00000004,0x0004002b,
0x0000000e,0x0000030a,0x00000008,0x0006002c,0x000002c4,0x0000030b,0x0000030a,0x0000030a,
0x00000046,0x00050036,0x00000002,0x00000004,0x00000000,0x00000003,0x000200f8,0x00000005,
0x0004003b,0x000002c2,0x000002c3,0x00000007,0x0004003b,0x00000018,0x000002e0,0x00000007,
0x0004003b,0x0000000d,0x000002e9,0x00000007,0x0004003b,0x0000000d,0x000002ee,0x00000007,
0x0004003b,0x00000018,0x000002f1,0x00000007,0x0004003b,0x00000018,0x000002f3,0x00000007,
0x0004003b,0x0000000d,0x000002fc,0x00000007,0x0004003b,0x0000000d,0x00000305,0x00000007,
0x0004003d,0x000002c4,0x000002c8,0x000002c6,0x0007004f,0x000002c7,0x000002c9,0x000002c8,
0x000002c8,0x00000000,0x00000001,0x0004007c,0x000002c1,0x000002ca,0x000002c9,0x0003003e,
0x000002c3,0x000002ca,0x00050041,0x0000001e,0x000002cb,0x000002c3,0x00000040,0x0004003d,
0x0000001d,0x000002cc,0x000002cb,0x00060041,0x000002d0,0x000002d1,0x000002cf,0x00000231,
0x00000040,0x0004003d,0x0000001d,0x000002d2,0x000002d1,0x000500af,0x00000032,0x000002d3,
0x000002cc,0x000002d2,0x000400a8,0x00000032,0x000002d4,0x000002d3,0x000300f7,0x000002d6,
0x00000000,0x000400fa,0x000002d4,0x000002d5,0x000002d6,0x000200f8,0x000002d5,0x00050041,
0x0000001e,0x000002d7,0x000002c3,0x00000046,0x0004003d,0x0000001d,0x000002d8,0x000002d7,
0x00060041,0x000002d0,0x000002d9,0x000002cf,0x00000231,0x00000046,0x0004003d,0x0000001d,
0x000002da,0x000002d9,0x000500af,0x00000032,0x000002db,0x000002d8,0x000002da,0x000200f9,
0x000002d6,0x000200f8,0x000002d6,0x000700f5,0x00000032,0x000002dc,0x000002d3,0x00000005,
0x000002db,0x000002d5,0x000300f7,0x000002de,0x00000000,0x000400fa,0x000002dc,0x000002dd,
0x000002de,0x000200f8,0x000002dd,0x000100fd,0x000200f8,0x000002de,0x0004003d,0x000002c1,
0x000002e1,0x000002c3,0x0004006f,0x00000017,0x000002e2,0x000002e1,0x00050050,0x00000017,
0x000002e3,0x00000068,0x00000068,0x00050081,0x00000017,0x000002e4,0x000002e2,0x000002e3,
0x00050041,0x000002e5,0x000002e6,0x000002cf,0x0000020e,0x0004003d,0x00000017,0x000002e7,
0x000002e6,0x00050085,0x00000017,0x000002e8,0x000002e4,0x000002e7,0x0003003e,0x000002e0,
0x000002e8,0x00050041,0x000002d0,0x000002eb,0x000002cf,0x000002ea,0x0004003d,0x0000001d,
0x000002ec,0x000002eb,0x000500aa,0x00000032,0x000002ed,0x000002ec,0x00000231,0x000300f7,
0x000002f0,0x00000000,0x000400fa,0x000002ed,0x000002ef,0x000002f7,0x000200f8,0x000002ef,
0x0004003d,0x00000017,0x000002f2,0x000002e0,0x0003003e,0x000002f1,0x000002f2,0x00050041,
0x000002e5,0x000002f4,0x000002cf,0x0000020e,0x0004003d,0x00000017,0x000002f5,0x000002f4,
0x0003003e,0x000002f3,0x000002f5,0x00060039,0x0000000c,0x000002f6,0x0000002e,0x000002f1,
0x000002f3,0x0003003e,0x000002ee,0x000002f6,0x000200f9,0x000002f0,0x000200f8,0x000002f7,
0x0004003d,0x0000009b,0x000002f8,0x0000009d,0x0004003d,0x000002c1,0x000002f9,0x000002c3,
0x00040064,0x0000009a,0x000002fa,0x000002f8,0x0007005f,0x000000a0,0x000002fb,0x000002fa,
0x000002f9,0x00000002,0x0000020e,0x0008004f,0x0000000c,0x000002fd,0x000002fb,0x000002fb,
0x00000000,0x00000001,0x00000002,0x0003003e,0x000002fc,0x000002fd,0x00050039,0x0000000c,
0x000002fe,0x00000015,0x000002fc,0x0003003e,0x000002ee,0x000002fe,0x000200f9,0x000002f0,
0x000200f8,0x000002f0,0x0004003d,0x0000000c,0x000002ff,0x000002ee,0x0003003e,0x000002e9,
0x000002ff,0x0004003d,0x00000300,0x00000303,0x00000302,0x0004003d,0x000002c1,0x00000304,
0x000002c3,0x0004003d,0x0000000c,0x00000306,0x000002e9,0x0003003e,0x00000305,0x00000306,
0x00050039,0x0000000e,0x00000307,0x00000011,0x00000305,0x00070050,0x00000308,0x00000309,
0x00000307,0x00000040,0x00000040,0x00000040,0x00050063,0x00000303,0x00000304,0x00000309,
0x00002000,0x000100fd,0x00010038,0x00050036,0x00000006,0x0000000a,0x00000000,0x00000008,
0x00030037,0x00000007,0x00000009,0x000200f8,0x0000000b,0x0004003b,0x00000007,0x00000034,
0x00000007,0x0004003d,0x00000006,0x00000030,0x00000009,0x000500ba,0x00000032,0x00000033,
0x00000030,0x00000031,0x000300f7,0x00000036,0x00000000,0x000400fa,0x00000033,0x00000035,
0x0000003a,0x000200f8,0x00000035,0x0004003d,0x00000006,0x00000037,0x00000009,0x0007000c,
0x00000006,0x00000039,0x00000001,0x00000025,0x00000037,0x00000038,0x0003003e,0x00000034,
0x00000039,0x000200f9,0x00000036,0x000200f8,0x0000003a,0x0003003e,0x00000034,0x00000031,
0x000200f9,0x00000036,0x000200f8,0x00000036,0x0004003d,0x00000006,0x0000003b,0x00000034,
0x000200fe,0x0000003b,0x00010038,0x00050036,0x0000000e,0x00000011,0x00000000,0x0000000f,
0x00030037,0x0000000d,0x00000010,0x000200f8,0x00000012,0x0004003b,0x00000007,0x0000003e,
0x00000007,0x0004003b,0x00000007,0x0000003f,0x00000007,0x0004003b,0x00000007,0x00000044,
0x00000007,0x0004003b,0x00000007,0x00000045,0x00000007,0x0004003b,0x00000007,0x0000004a,
0x00000007,0x0004003b,0x00000007,0x0000004b,0x00000007,0x0004003b,0x00000007,0x00000050,
0x00000007,0x0004003b,0x00000007,0x00000056,0x00000007,0x0004003b,0x00000007,0x00000060,
0x00000007,0x00050041,0x00000007,0x00000041,0x00000010,0x00000040,0x0004003d,0x00000006,
0x00000042,0x00000041,0x0003003e,0x0000003f,0x00000042,0x00050039,0x00000006,0x00000043,
0x0000000a,0x0000003f,0x0003003e,0x0000003e,0x00000043,0x00050041,0x00000007,0x00000047,
0x00000010,0x00000046,0x0004003d,0x00000006,0x00000048,0x00000047,0x0003003e,0x00000045,
0x00000048,0x00050039,0x00000006,0x00000049,0x0000000a,0x00000045,0x0003003e,0x00000044,
0x00000049,0x00050041,0x00000007,0x0000004d,0x00000010,0x0000004c,0x0004003d,0x00000006,
0x0000004e,0x0000004d,0x0003003e,0x0000004b,0x0000004e,0x00050039,0x00000006,0x0000004f,
0x0000000a,0x0000004b,0x0003003e,0x0000004a,0x0000004f,0x0004003d,0x00000006,0x00000051,
0x0000003e,0x0004003d,0x00000006,0x00000052,0x00000044,0x0007000c,0x00000006,0x00000053,
0x00000001,0x00000028,0x00000051,0x00000052,0x0004003d,0x00000006,0x00000054,0x0000004a,
0x0007000c,0x00000006,0x00000055,0x00000001,0x00000028,0x00000053,0x00000054,0x0003003e,
0x00000050,0x00000055,0x0004003d,0x00000006,0x00000058,0x00000050,0x0007000c,0x00000006,
0x0000005a,0x00000001,0x00000028,0x00000058,0x00000059,0x0006000c,0x00000006,0x0000005b,
0x00000001,0x0000001e,0x0000005a,0x0006000c,0x00000006,0x0000005c,0x00000001,0x00000008,
0x0000005b,0x0007000c,0x00000006,0x0000005d,0x00000001,0x00000028,0x00000057,0x0000005c,
0x00050081,0x00000006,0x0000005f,0x0000005d,0x0000005e,0x0003003e,0x00000056,0x0000005f,
0x0004003d,0x00000006,0x00000061,0x00000056,0x00050083,0x00000006,0x00000063,0x00000061,
0x00000062,0x0006000c,0x00000006,0x00000064,0x00000001,0x0000001d,0x00000063,0x0003003e,
0x00000060,0x00000064,0x0004003d,0x00000006,0x00000065,0x00000050,0x0004003d,0x00000006,
0x00000066,0x00000060,0x00050088,0x00000006,0x00000067,0x00000065,0x00000066,0x00050081,
0x00000006,0x00000069,0x00000067,0x00000068,0x0006000c,0x00000006,0x0000006a,0x00000001,
0x00000008,0x00000069,0x000500be,0x00000032,0x0000006c,0x0000006a,0x0000006b,0x000300f7,
0x0000006e,0x00000000,0x000400fa,0x0000006c,0x0000006d,0x0000006e,0x000200f8,0x0000006d,
0x0004003d,0x00000006,0x00000070,0x00000056,0x00050081,0x00000006,0x00000071,0x00000070,
0x0000006f,0x0003003e,0x00000056,0x00000071,0x0004003d,0x00000006,0x00000073,0x00000060,
0x00050085,0x00000006,0x00000074,0x00000073,0x00000072,0x0003003e,0x00000060,0x00000074,
0x000200f9,0x0000006e,0x000200f8,0x0000006e,0x0004003d,0x00000006,0x00000075,0x0000003e,
0x0004003d,0x00000006,0x00000076,0x00000060,0x00050088,0x00000006,0x00000077,0x00000075,
0x00000076,0x00050081,0x00000006,0x00000078,0x00000077,0x00000068,0x0006000c,0x00000006,
0x00000079,0x00000001,0x00000008,0x00000078,0x0004006d,0x0000000e,0x0000007a,0x00000079,
0x0004003d,0x00000006,0x0000007b,0x00000044,0x0004003d,0x00000006,0x0000007c,0x00000060,
0x00050088,0x00000006,0x0000007d,0x0000007b,0x0000007c,0x00050081,0x00000006,0x0000007e,
0x0000007d,0x00000068,0x0006000c,0x00000006,0x0000007f,0x00000001,0x00000008,0x0000007e,
0x0004006d,0x0000000e,0x00000080,0x0000007f,0x000500c4,0x0000000e,0x00000082,0x00000080,
0x00000081,0x000500c5,0x0000000e,0x00000083,0x0000007a,0x00000082,0x0004003d,0x00000006,
0x00000084,0x0000004a,0x0004003d,0x00000006,0x00000085,0x00000060,0x00050088,0x00000006,
0x00000086,0x00000084,0x00000085,0x00050081,0x00000006,0x00000087,0x00000086,0x00000068,
0x0006000c,0x00000006,0x00000088,0x00000001,0x00000008,0x00000087,0x0004006d,0x0000000e,
0x00000089,0x00000088,0x000500c4,0x0000000e,0x0000008b,0x00000089,0x0000008a,0x000500c5,
0x0000000e,0x0000008c,0x00000083,0x0000008b,0x0004003d,0x00000006,0x0000008d,0x00000056,
0x0004006d,0x0000000e,0x0000008e,0x0000008d,0x000500c4,0x0000000e,0x00000090,0x0000008e,
0x0000008f,0x000500c5,0x0000000e,0x00000091,0x0000008c,0x00000090,0x000200fe,0x00000091,
0x00010038,0x00050036,0x0000000c,0x00000015,0x00000000,0x00000013,0x00030037,0x0000000d,
0x00000014,0x000200f8,0x00000016,0x0004003d,0x0000000c,0x00000094,0x00000014,0x0007000c,
0x0000000c,0x00000096,0x00000001,0x00000028,0x00000094,0x00000095,0x0006000c,0x0000000c,
0x00000097,0x00000001,0x0000001f,0x00000096,0x000200fe,0x00000097,0x00010038,0x00050036,
0x0000000c,0x0000001b,0x00000000,0x00000019,0x00030037,0x00000018,0x0000001a,0x000200f8,
0x0000001c,0x0004003b,0x0000000d,0x000000a2,0x00000007,0x0004003d,0x0000009b,0x0000009e,
0x0000009d,0x0004003d,0x00000017,0x0000009f,0x0000001a,0x00070058,0x000000a0,0x000000a1,
0x0000009e,0x0000009f,0x00000002,0x00000031,0x0008004f,0x0000000c,0x000000a3,0x000000a1,
0x000000a1,0x00000000,0x00000001,0x00000002,0x0003003e,0x000000a2,0x000000a3,0x00050039,
0x0000000c,0x000000a4,0x00000015,0x000000a2,0x000200fe,0x000000a4,0x00010038,0x00050036,
0x00000006,0x00000021,0x00000000,0x0000001f,0x00030037,0x0000001e,0x00000020,0x000200f8,
0x00000022,0x0004003b,0x00000007,0x000000aa,0x00000007,0x0004003b,0x00000007,0x000000b0,
0x00000007,0x0004003b,0x00000007,0x000000b8,0x00000007,0x0004003d,0x0000001d,0x000000a7,
0x00000020,0x000500b1,0x00000032,0x000000a9,0x000000a7,0x000000a8,0x000300f7,0x000000ac,
0x00000000,0x000400fa,0x000000a9,0x000000ab,0x000000ad,0x000200f8,0x000000ab,0x0003003e,
0x000000aa,0x0000006f,0x000200f9,0x000000ac,0x000200f8,0x000000ad,0x0004003d,0x0000001d,
0x000000ae,0x00000020,0x000500aa,0x00000032,0x000000af,0x000000ae,0x000000a8,0x000300f7,
0x000000b2,0x00000000,0x000400fa,0x000000af,0x000000b1,0x000000b4,0x000200f8,0x000000b1,
0x0003003e,0x000000b0,0x000000b3,0x000200f9,0x000000b2,0x000200f8,0x000000b4,0x0004003d,
0x0000001d,0x000000b5,0x00000020,0x000500b1,0x00000032,0x000000b7,0x000000b5,0x000000b6,
0x000300f7,0x000000ba,0x00000000,0x000400fa,0x000000b7,0x000000b9,0x000000bb,0x000200f8,
0x000000b9,0x0003003e,0x000000b8,0x00000072,0x000200f9,0x000000ba,0x000200f8,0x000000bb,
0x0004003d,0x0000001d,0x000000bc,0x00000020,0x000500aa,0x00000032,0x000000bd,0x000000bc,
0x000000b6,0x000600a9,0x00000006,0x000000c0,0x000000bd,0x000000be,0x000000bf,0x0003003e,
0x000000b8,0x000000c0,0x000200f9,0x000000ba,0x000200f8,0x000000ba,0x0004003d,0x00000006,
0x000000c1,0x000000b8,0x0003003e,0x000000b0,0x000000c1,0x000200f9,0x000000b2,0x000200f8,
0x000000b2,0x0004003d,0x00000006,0x000000c2,0x000000b0,0x0003003e,0x000000aa,0x000000c2,
0x000200f9,0x000000ac,0x000200f8,0x000000ac,0x0004003d,0x00000006,0x000000c3,0x000000aa,
0x000200fe,0x000000c3,0x00010038,0x00050036,0x00000006,0x00000025,0x00000000,0x00000023,
0x00030037,0x0000000d,0x00000024,0x000200f8,0x00000026,0x0004003d,0x0000000c,0x000000c6,
0x00000024,0x00050094,0x00000006,0x000000cb,0x000000c6,0x000000ca,0x000200fe,0x000000cb,
0x00010038,0x00050036,0x00000006,0x00000029,0x00000000,0x00000027,0x00030037,0x00000018,
0x00000028,0x000200f8,0x0000002a,0x0004003b,0x00000018,0x000000ce,0x00000007,0x0004003b,
0x0000000d,0x000000d1,0x00000007,0x0004003d,0x00000017,0x000000cf,0x00000028,0x0003003e,
0x000000ce,0x000000cf,0x00050039,0x0000000c,0x000000d0,0x0000001b,0x000000ce,0x0003003e,
0x000000d1,0x000000d0,0x00050039,0x00000006,0x000000d2,0x00000025,0x000000d1,0x000200fe,
0x000000d2,0x00010038,0x00050036,0x0000000c,0x0000002e,0x00000000,0x0000002b,0x00030037,
0x00000018,0x0000002c,0x00030037,0x00000018,0x0000002d,0x000200f8,0x0000002f,0x0004003b,
0x0000000d,0x000000d5,0x00000007,0x0004003b,0x00000018,0x000000d6,0x00000007,0x0004003b,
0x00000007,0x000000d9,0x00000007,0x0004003b,0x0000000d,0x000000da,0x00000007,0x0004003b,
0x00000007,0x000000dd,0x00000007,0x0004003b,0x00000018,0x000000e4,0x00000007,0x0004003b,
0x00000007,0x000000e6,0x00000007,0x0004003b,0x00000018,0x000000ec,0x00000007,0x0004003b,
0x00000007,0x000000ee,0x00000007,0x0004003b,0x00000018,0x000000f5,0x00000007,0x0004003b,
0x00000007,0x000000f7,0x00000007,0x0004003b,0x00000018,0x000000fd,0x00000007,0x0004003b,
0x00000007,0x000000ff,0x00000007,0x0004003b,0x00000007,0x00000109,0x00000007,0x0004003b,
0x00000007,0x00000113,0x00000007,0x0004003b,0x00000007,0x00000122,0x00000007,0x0004003b,
0x00000018,0x0000012c,0x00000007,0x0004003b,0x00000007,0x0000012e,0x00000007,0x0004003b,
0x00000018,0x00000137,0x00000007,0x0004003b,0x00000007,0x00000139,0x00000007,0x0004003b,
0x00000018,0x00000142,0x00000007,0x0004003b,0x00000007,0x00000144,0x00000007,0x0004003b,
0x00000018,0x0000014c,0x00000007,0x0004003b,0x00000007,0x0000014e,0x00000007,0x0004003b,
0x00000007,0x00000161,0x00000007,0x0004003b,0x00000007,0x00000176,0x00000007,0x0004003b,
0x00000007,0x00000194,0x00000007,0x0004003b,0x000001b1,0x000001b2,0x00000007,0x0004003b,
0x00000007,0x000001b6,0x00000007,0x0004003b,0x00000007,0x000001bb,0x00000007,0x0004003b,
0x00000007,0x000001c0,0x00000007,0x0004003b,0x00000007,0x000001c5,0x00000007,0x0004003b,
0x000001b1,0x000001ca,0x00000007,0x0004003b,0x00000007,0x000001ce,0x00000007,0x0004003b,
0x00000007,0x000001d3,0x00000007,0x0004003b,0x00000007,0x000001d5,0x00000007,0x0004003b,
0x00000007,0x000001de,0x00000007,0x0004003b,0x00000018,0x000001ec,0x00000007,0x0004003b,
0x00000018,0x000001fe,0x00000007,0x0004003b,0x00000018,0x00000200,0x00000007,0x0004003b,
0x00000018,0x0000020b,0x00000007,0x0004003b,0x0000001e,0x0000020f,0x00000007,0x0004003b,
0x00000018,0x00000213,0x00000007,0x0004003b,0x0000001e,0x00000216,0x00000007,0x0004003b,
0x00000007,0x0000021a,0x00000007,0x0004003b,0x00000018,0x0000021b,0x00000007,0x0004003b,
0x00000007,0x00000220,0x00000007,0x0004003b,0x00000018,0x00000221,0x00000007,0x0004003b,
0x000001b1,0x00000226,0x00000007,0x0004003b,0x000001b1,0x0000022b,0x00000007,0x0004003b,
0x0000001e,0x00000230,0x00000007,0x0004003b,0x0000001e,0x00000247,0x00000007,0x0004003b,
0x00000018,0x0000024c,0x00000007,0x0004003b,0x0000001e,0x0000025b,0x00000007,0x0004003b,
0x00000018,0x00000260,0x00000007,0x0004003b,0x00000007,0x0000026b,0x00000007,0x0004003b,
0x00000007,0x0000026d,0x00000007,0x0004003b,0x00000007,0x0000027c,0x00000007,0x0004003b,
0x00000007,0x0000027e,0x00000007,0x0004003b,0x000001b1,0x0000028d,0x00000007,0x0004003b,
0x00000007,0x00000291,0x00000007,0x0004003b,0x00000018,0x000002a8,0x00000007,0x0004003b,
0x00000018,0x000002bc,0x00000007,0x0004003d,0x00000017,0x000000d7,0x0000002c,0x0003003e,
0x000000d6,0x000000d7,0x00050039,0x0000000c,0x000000d8,0x0000001b,0x000000d6,0x0003003e,
0x000000d5,0x000000d8,0x0004003d,0x0000000c,0x000000db,0x000000d5,0x0003003e,0x000000da,
0x000000db,0x00050039,0x00000006,0x000000dc,0x00000025,0x000000da,0x0003003e,0x000000d9,
0x000000dc,0x0004003d,0x00000017,0x000000de,0x0000002c,0x00050041,0x00000007,0x000000df,
0x0000002d,0x00000046,0x0004003d,0x00000006,0x000000e0,0x000000df,0x0004007f,0x00000006,
0x000000e1,0x000000e0,0x00050050,0x00000017,0x000000e2,0x00000031,0x000000e1,0x00050081,
0x00000017,0x000000e3,0x000000de,0x000000e2,0x0003003e,0x000000e4,0x000000e3,0x00050039,
0x00000006,0x000000e5,0x00000029,0x000000e4,0x0003003e,0x000000dd,0x000000e5,0x0004003d,
0x00000017,0x000000e7,0x0000002c,0x00050041,0x00000007,0x000000e8,0x0000002d,0x00000046,
0x0004003d,0x00000006,0x000000e9,0x000000e8,0x00050050,0x00000017,0x000000ea,0x00000031,
0x000000e9,0x00050081,0x00000017,0x000000eb,0x000000e7,0x000000ea,0x0003003e,0x000000ec,
0x000000eb,0x00050039,0x00000006,0x000000ed,0x00000029,0x000000ec,0x0003003e,0x000000e6,
0x000000ed,0x0004003d,0x00000017,0x000000ef,0x0000002c,0x00050041,0x00000007,0x000000f0,
0x0000002d,0x00000040,0x0004003d,0x00000006,0x000000f1,0x000000f0,0x0004007f,0x00000006,
0x000000f2,0x000000f1,0x00050050,0x00000017,0x000000f3,0x000000f2,0x00000031,0x00050081,
0x00000017,0x000000f4,0x000000ef,0x000000f3,0x0003003e,0x000000f5,0x000000f4,0x00050039,
0x00000006,0x000000f6,0x00000029,0x000000f5,0x0003003e,0x000000ee,0x000000f6,0x0004003d,
0x00000017,0x000000f8,0x0000002c,0x00050041,0x00000007,0x000000f9,0x0000002d,0x00000040,
0x0004003d,0x00000006,0x000000fa,0x000000f9,0x00050050,0x00000017,0x000000fb,0x000000fa,
0x00000031,0x00050081,0x00000017,0x000000fc,0x000000f8,0x000000fb,0x0003003e,0x000000fd,
0x000000fc,0x00050039,0x00000006,0x000000fe,0x00000029,0x000000fd,0x0003003e,0x000000f7,
0x000000fe,0x0004003d,0x00000006,0x00000100,0x000000d9,0x0004003d,0x00000006,0x00000101,
0x000000dd,0x0004003d,0x00000006,0x00000102,0x000000e6,0x0007000c,0x00000006,0x00000103,
0x00000001,0x00000025,0x00000101,0x00000102,0x0004003d,0x00000006,0x00000104,0x000000ee,
0x0004003d,0x00000006,0x00000105,0x000000f7,0x0007000c,0x00000006,0x00000106,0x00000001,
0x00000025,0x00000104,0x00000105,0x0007000c,0x00000006,0x00000107,0x00000001,0x00000025,
0x00000103,0x00000106,0x0007000c,0x00000006,0x00000108,0x00000001,0x00000025,0x00000100,
0x00000107,0x0003003e,0x000000ff,0x00000108,0x0004003d,0x00000006,0x0000010a,0x000000d9,
0x0004003d,0x00000006,0x0000010b,0x000000dd,0x0004003d,0x00000006,0x0000010c,0x000000e6,
0x0007000c,0x00000006,0x0000010d,0x00000001,0x00000028,0x0000010b,0x0000010c,0x0004003d,
0x00000006,0x0000010e,0x000000ee,0x0004003d,0x00000006,0x0000010f,0x000000f7,0x0007000c,
0x00000006,0x00000110,0x00000001,0x00000028,0x0000010e,0x0000010f,0x0007000c,0x00000006,
0x00000111,0x00000001,0x00000028,0x0000010d,0x00000110,0x0007000c,0x00000006,0x00000112,
0x00000001,0x00000028,0x0000010a,0x00000111,0x0003003e,0x00000109,0x00000112,0x0004003d,
0x00000006,0x00000114,0x00000109,0x0004003d,0x00000006,0x00000115,0x000000ff,0x00050083,
0x00000006,0x00000116,0x00000114,0x00000115,0x0003003e,0x00000113,0x00000116,0x0004003d,
0x00000006,0x00000117,0x00000113,0x0004003d,0x00000006,0x00000119,0x00000109,0x00050085,
0x00000006,0x0000011b,0x00000119,0x0000011a,0x0007000c,0x00000006,0x0000011c,0x00000001,
0x00000028,0x00000118,0x0000011b,0x000500b8,0x00000032,0x0000011d,0x00000117,0x0000011c,
0x000300f7,0x0000011f,0x00000000,0x000400fa,0x0000011d,0x0000011e,0x0000011f,0x000200f8,
0x0000011e,0x0004003d,0x0000000c,0x00000120,0x000000d5,0x000200fe,0x00000120,0x000200f8,
0x0000011f,0x0004003d,0x00000017,0x00000123,0x0000002c,0x00050041,0x00000007,0x00000124,
0x0000002d,0x00000040,0x0004003d,0x00000006,0x00000125,0x00000124,0x0004007f,0x00000006,
0x00000126,0x00000125,0x00050041,0x00000007,0x00000127,0x0000002d,0x00000046,0x0004003d,
0x00000006,0x00000128,0x00000127,0x0004007f,0x00000006,0x00000129,0x00000128,0x00050050,
0x00000017,0x0000012a,0x00000126,0x00000129,0x00050081,0x00000017,0x0000012b,0x00000123,
0x0000012a,0x0003003e,0x0000012c,0x0000012b,0x00050039,0x00000006,0x0000012d,0x00000029,
0x0000012c,0x0003003e,0x00000122,0x0000012d,0x0004003d,0x00000017,0x0000012f,0x0000002c,
0x00050041,0x00000007,0x00000130,0x0000002d,0x00000040,0x0004003d,0x00000006,0x00000131,
0x00000130,0x00050041,0x00000007,0x00000132,0x0000002d,0x00000046,0x0004003d,0x00000006,
0x00000133,0x00000132,0x0004007f,0x00000006,0x00000134,0x00000133,0x00050050,0x00000017,
0x00000135,0x00000131,0x00000134,0x00050081,0x00000017,0x00000136,0x0000012f,0x00000135,
0x0003003e,0x00000137,0x00000136,0x00050039,0x00000006,0x00000138,0x00000029,0x00000137,
0x0003003e,0x0000012e,0x00000138,0x0004003d,0x00000017,0x0000013a,0x0000002c,0x00050041,
0x00000007,0x0000013b,0x0000002d,0x00000040,0x0004003d,0x00000006,0x0000013c,0x0000013b,
0x0004007f,0x00000006,0x0000013d,0x0000013c,0x00050041,0x00000007,0x0000013e,0x0000002d,
0x00000046,0x0004003d,0x00000006,0x0000013f,0x0000013e,0x00050050,0x00000017,0x00000140,
0x0000013d,0x0000013f,0x00050081,0x00000017,0x00000141,0x0000013a,0x00000140,0x0003003e,
0x00000142,0x00000141,0x00050039,0x00000006,0x00000143,0x00000029,0x00000142,0x0003003e,
0x00000139,0x00000143,0x0004003d,0x00000017,0x00000145,0x0000002c,0x00050041,0x00000007,
0x00000146,0x0000002d,0x00000040,0x0004003d,0x00000006,0x00000147,0x00000146,0x00050041,
0x00000007,0x00000148,0x0000002d,0x00000046,0x0004003d,0x00000006,0x00000149,0x00000148,
0x00050050,0x00000017,0x0000014a,0x00000147,0x00000149,0x00050081,0x00000017,0x0000014b,
0x00000145,0x0000014a,0x0003003e,0x0000014c,0x0000014b,0x00050039,0x00000006,0x0000014d,
0x00000029,0x0000014c,0x0003003e,0x00000144,0x0000014d,0x0004003d,0x00000006,0x0000014f,
0x000000dd,0x0004003d,0x00000006,0x00000150,0x000000e6,0x00050081,0x00000006,0x00000151,
0x0000014f,0x00000150,0x0004003d,0x00000006,0x00000152,0x000000ee,0x00050081,0x00000006,
0x00000153,0x00000151,0x00000152,0x0004003d,0x00000006,0x00000154,0x000000f7,0x00050081,
0x00000006,0x00000155,0x00000153,0x00000154,0x00050085,0x00000006,0x00000156,0x00000072,
0x00000155,0x0004003d,0x00000006,0x00000157,0x00000122,0x00050081,0x00000006,0x00000158,
0x00000156,0x00000157,0x0004003d,0x00000006,0x00000159,0x0000012e,0x00050081,0x00000006,
0x0000015a,0x00000158,0x00000159,0x0004003d,0x00000006,0x0000015b,0x00000139,0x00050081,
0x00000006,0x0000015c,0x0000015a,0x0000015b,0x0004003d,0x00000006,0x0000015d,0x00000144,
0x00050081,0x00000006,0x0000015e,0x0000015c,0x0000015d,0x00050088,0x00000006,0x00000160,
0x0000015e,0x0000015f,0x0003003e,0x0000014e,0x00000160,0x0004003d,0x00000006,0x00000162,
0x0000014e,0x0004003d,0x00000006,0x00000163,0x000000d9,0x00050083,0x00000006,0x00000164,
0x00000162,0x00000163,0x0006000c,0x00000006,0x00000165,0x00000001,0x00000004,0x00000164,
0x0004003d,0x00000006,0x00000166,0x00000113,0x00050088,0x00000006,0x00000167,0x00000165,
0x00000166,0x0008000c,0x00000006,0x00000168,0x00000001,0x0000002b,0x00000167,0x00000031,
0x0000006f,0x0003003e,0x00000161,0x00000168,0x0004003d,0x00000006,0x00000169,0x00000161,
0x0004003d,0x00000006,0x0000016a,0x00000161,0x00050085,0x00000006,0x0000016b,0x00000169,
0x0000016a,0x0004003d,0x00000006,0x0000016d,0x00000161,0x00050085,0x00000006,0x0000016e,
0x00000072,0x0000016d,0x00050083,0x00000006,0x0000016f,0x0000016c,0x0000016e,0x00050085,
0x00000006,0x00000170,0x0000016b,0x0000016f,0x0003003e,0x00000161,0x00000170,0x0004003d,
0x00000006,0x00000171,0x00000161,0x0004003d,0x00000006,0x00000172,0x00000161,0x00050085,
0x00000006,0x00000173,0x00000171,0x00000172,0x00050085,0x00000006,0x00000175,0x00000173,
0x00000174,0x0003003e,0x00000161,0x00000175,0x0004003d,0x00000006,0x00000178,0x00000122,
0x00050085,0x00000006,0x00000179,0x00000177,0x00000178,0x0004003d,0x00000006,0x0000017a,
0x000000dd,0x00050085,0x00000006,0x0000017b,0x00000068,0x0000017a,0x00050083,0x00000006,
0x0000017c,0x00000179,0x0000017b,0x0004003d,0x00000006,0x0000017d,0x0000012e,0x00050085,
0x00000006,0x0000017e,0x00000177,0x0000017d,0x00050081,0x00000006,0x0000017f,0x0000017c,
0x0000017e,0x0006000c,0x00000006,0x00000180,0x00000001,0x00000004,0x0000017f,0x0004003d,
0x00000006,0x00000181,0x000000ee,0x00050085,0x00000006,0x00000182,0x00000068,0x00000181,
0x0004003d,0x00000006,0x00000183,0x000000d9,0x00050083,0x00000006,0x00000184,0x00000182,
0x00000183,0x0004003d,0x00000006,0x00000185,0x000000f7,0x00050085,0x00000006,0x00000186,
0x00000068,0x00000185,0x00050081,0x00000006,0x00000187,0x00000184,0x00000186,0x0006000c,
0x00000006,0x00000188,0x00000001,0x00000004,0x00000187,0x00050081,0x00000006,0x00000189,
0x00000180,0x00000188,0x0004003d,0x00000006,0x0000018a,0x00000139,0x00050085,0x00000006,
0x0000018b,0x00000177,0x0000018a,0x0004003d,0x00000006,0x0000018c,0x000000e6,0x00050085,
0x00000006,0x0000018d,0x00000068,0x0000018c,0x00050083,0x00000006,0x0000018e,0x0000018b,
0x0000018d,0x0004003d,0x00000006,0x0000018f,0x00000144,0x00050085,0x00000006,0x00000190,
0x00000177,0x0000018f,0x00050081,0x00000006,0x00000191,0x0000018e,0x00000190,0x0006000c,
0x00000006,0x00000192,0x00000001,0x00000004,0x00000191,0x00050081,0x00000006,0x00000193,
0x00000189,0x00000192,0x0003003e,0x00000176,0x00000193,0x0004003d,0x00000006,0x00000195,
0x00000122,0x00050085,0x00000006,0x00000196,0x00000177,0x00000195,0x0004003d,0x00000006,
0x00000197,0x000000ee,0x00050085,0x00000006,0x00000198,0x00000068,0x00000197,0x00050083,
0x00000006,0x00000199,0x00000196,0x00000198,0x0004003d,0x00000006,0x0000019a,0x00000139,
0x00050085,0x00000006,0x0000019b,0x00000177,0x0000019a,0x00050081,0x00000006,0x0000019c,
0x00000199,0x0000019b,0x0006000c,0x00000006,0x0000019d,0x00000001,0x00000004,0x0000019c,
0x0004003d,0x00000006,0x0000019e,0x000000dd,0x00050085,0x00000006,0x0000019f,0x00000068,
0x0000019e,0x0004003d,0x00000006,0x000001a0,0x000000d9,0x00050083,0x00000006,0x000001a1,
0x0000019f,0x000001a0,0x0004003d,0x00000006,0x000001a2,0x000000e6,0x00050085,0x00000006,
0x000001a3,0x00000068,0x000001a2,0x00050081,0x00000006,0x000001a4,0x000001a1,0x000001a3,
0x0006000c,0x00000006,0x000001a5,0x00000001,0x00000004,0x000001a4,0x00050081,0x00000006,
0x000001a6,0x0000019d,0x000001a5,0x0004003d,0x00000006,0x000001a7,0x0000012e,0x00050085,
0x00000006,0x000001a8,0x00000177,0x000001a7,0x0004003d,0x00000006,0x000001a9,0x000000f7,
0x00050085,0x00000006,0x000001aa,0x00000068,0x000001a9,0x00050083,0x00000006,0x000001ab,
0x000001a8,0x000001aa,0x0004003d,0x00000006,0x000001ac,0x00000144,0x00050085,0x00000006,
0x000001ad,0x00000177,0x000001ac,0x00050081,0x00000006,0x000001ae,0x000001ab,0x000001ad,
0x0006000c,0x00000006,0x000001af,0x00000001,0x00000004,0x000001ae,0x00050081,0x00000006,
0x000001b0,0x000001a6,0x000001af,0x0003003e,0x00000194,0x000001b0,0x0004003d,0x00000006,
0x000001b3,0x00000194,0x0004003d,0x00000006,0x000001b4,0x00000176,0x000500be,0x00000032,
0x000001b5,0x000001b3,0x000001b4,0x0003003e,0x000001b2,0x000001b5,0x0004003d,0x00000032,
0x000001b7,0x000001b2,0x0004003d,0x00000006,0x000001b8,0x000000dd,0x0004003d,0x00000006,
0x000001b9,0x000000ee,0x000600a9,0x00000006,0x000001ba,0x000001b7,0x000001b8,0x000001b9,
0x0003003e,0x000001b6,0x000001ba,0x0004003d,0x00000032,0x000001bc,0x000001b2,0x0004003d,
0x00000006,0x000001bd,0x000000e6,0x0004003d,0x00000006,0x000001be,0x000000f7,0x000600a9,
0x00000006,0x000001bf,0x000001bc,0x000001bd,0x000001be,0x0003003e,0x000001bb,0x000001bf,
0x0004003d,0x00000006,0x000001c1,0x000001b6,0x0004003d,0x00000006,0x000001c2,0x000000d9,
0x00050083,0x00000006,0x000001c3,0x000001c1,0x000001c2,0x0006000c,0x00000006,0x000001c4,
0x00000001,0x00000004,0x000001c3,0x0003003e,0x000001c0,0x000001c4,0x0004003d,0x00000006,
0x000001c6,0x000001bb,0x0004003d,0x00000006,0x000001c7,0x000000d9,0x00050083,0x00000006,
0x000001c8,0x000001c6,0x000001c7,0x0006000c,0x00000006,0x000001c9,0x00000001,0x00000004,
0x000001c8,0x0003003e,0x000001c5,0x000001c9,0x0004003d,0x00000006,0x000001cb,0x000001c0,
0x0004003d,0x00000006,0x000001cc,0x000001c5,0x000500be,0x00000032,0x000001cd,0x000001cb,
0x000001cc,0x0003003e,0x000001ca,0x000001cd,0x0004003d,0x00000006,0x000001cf,0x000001c0,
0x0004003d,0x00000006,0x000001d0,0x000001c5,0x0007000c,0x00000006,0x000001d1,0x00000001,
0x00000028,0x000001cf,0x000001d0,0x00050085,0x00000006,0x000001d2,0x00000177,0x000001d1,
0x0003003e,0x000001ce,0x000001d2,0x0004003d,0x00000032,0x000001d4,0x000001b2,0x000300f7,
0x000001d7,0x00000000,0x000400fa,0x000001d4,0x000001d6,0x000001da,0x000200f8,0x000001d6,
0x00050041,0x00000007,0x000001d8,0x0000002d,0x00000046,0x0004003d,0x00000006,0x000001d9,
0x000001d8,0x0003003e,0x000001d5,0x000001d9,0x000200f9,0x000001d7,0x000200f8,0x000001da,
0x00050041,0x00000007,0x000001db,0x0000002d,0x00000040,0x0004003d,0x00000006,0x000001dc,
0x000001db,0x0003003e,0x000001d5,0x000001dc,0x000200f9,0x000001d7,0x000200f8,0x000001d7,
0x0004003d,0x00000006,0x000001dd,0x000001d5,0x0003003e,0x000001d3,0x000001dd,0x0004003d,
0x00000006,0x000001df,0x000001bb,0x0004003d,0x00000006,0x000001e0,0x000000d9,0x00050081,
0x00000006,0x000001e1,0x000001df,0x000001e0,0x00050085,0x00000006,0x000001e2,0x00000068,
0x000001e1,0x0003003e,0x000001de,0x000001e2,0x0004003d,0x00000032,0x000001e3,0x000001ca,
0x000300f7,0x000001e5,0x00000000,0x000400fa,0x000001e3,0x000001e4,0x000001e5,0x000200f8,
0x000001e4,0x0004003d,0x00000006,0x000001e6,0x000001d3,0x0004007f,0x00000006,0x000001e7,
0x000001e6,0x0003003e,0x000001d3,0x000001e7,0x0004003d,0x00000006,0x000001e8,0x000001b6,
0x0004003d,0x00000006,0x000001e9,0x000000d9,0x00050081,0x00000006,0x000001ea,0x000001e8,
0x000001e9,0x00050085,0x00000006,0x000001eb,0x00000068,0x000001ea,0x0003003e,0x000001de,
0x000001eb,0x000200f9,0x000001e5,0x000200f8,0x000001e5,0x0004003d,0x00000017,0x000001ed,
0x0000002c,0x0003003e,0x000001ec,0x000001ed,0x0004003d,0x00000032,0x000001ee,0x000001b2,
0x000300f7,0x000001f0,0x00000000,0x000400fa,0x000001ee,0x000001ef,0x000001f7,0x000200f8,
0x000001ef,0x0004003d,0x00000006,0x000001f1,0x000001d3,0x00050085,0x00000006,0x000001f2,
0x00000068,0x000001f1,0x00050041,0x00000007,0x000001f3,0x000001ec,0x00000046,0x0004003d,
0x00000006,0x000001f4,0x000001f3,0x00050081,0x00000006,0x000001f5,0x000001f4,0x000001f2,
0x00050041,0x00000007,0x000001f6,0x000001ec,0x00000046,0x0003003e,0x000001f6,0x000001f5,
0x000200f9,0x000001f0,0x000200f8,0x000001f7,0x0004003d,0x00000006,0x000001f8,0x000001d3,
0x00050085,0x00000006,0x000001f9,0x00000068,0x000001f8,0x00050041,0x00000007,0x000001fa,
0x000001ec,0x00000040,0x0004003d,0x00000006,0x000001fb,0x000001fa,0x00050081,0x00000006,
0x000001fc,0x000001fb,0x000001f9,0x00050041,0x00000007,0x000001fd,0x000001ec,0x00000040,
0x0003003e,0x000001fd,0x000001fc,0x000200f9,0x000001f0,0x000200f8,0x000001f0,0x0004003d,
0x00000032,0x000001ff,0x000001b2,0x000300f7,0x00000202,0x00000000,0x000400fa,0x000001ff,
0x00000201,0x00000206,0x000200f8,0x00000201,0x00050041,0x00000007,0x00000203,0x0000002d,
0x00000040,0x0004003d,0x00000006,0x00000204,0x00000203,0x00050050,0x00000017,0x00000205,
0x00000204,0x00000031,0x0003003e,0x00000200,0x00000205,0x000200f9,0x00000202,0x000200f8,
0x00000206,0x00050041,0x00000007,0x00000207,0x0000002d,0x00000046,0x0004003d,0x00000006,
0x00000208,0x00000207,0x00050050,0x00000017,0x00000209,0x00000031,0x00000208,0x0003003e,
0x00000200,0x00000209,0x000200f9,0x00000202,0x000200f8,0x00000202,0x0004003d,0x00000017,
0x0000020a,0x00000200,0x0003003e,0x000001fe,0x0000020a,0x0004003d,0x00000017,0x0000020c,
0x000001ec,0x0004003d,0x00000017,0x0000020d,0x000001fe,0x0003003e,0x0000020f,0x0000020e,
0x00050039,0x00000006,0x00000210,0x00000021,0x0000020f,0x0005008e,0x00000017,0x00000211,
0x0000020d,0x00000210,0x00050083,0x00000017,0x00000212,0x0000020c,0x00000211,0x0003003e,
0x0000020b,0x00000212,0x0004003d,0x00000017,0x00000214,0x000001ec,0x0004003d,0x00000017,
0x00000215,0x000001fe,0x0003003e,0x00000216,0x0000020e,0x00050039,0x00000006,0x00000217,
0x00000021,0x00000216,0x0005008e,0x00000017,0x00000218,0x00000215,0x00000217,0x00050081,
0x00000017,0x00000219,0x00000214,0x00000218,0x0003003e,0x00000213,0x00000219,0x0004003d,
0x00000017,0x0000021c,0x0000020b,0x0003003e,0x0000021b,0x0000021c,0x00050039,0x00000006,
0x0000021d,0x00000029,0x0000021b,0x0004003d,0x00000006,0x0000021e,0x000001de,0x00050083,
0x00000006,0x0000021f,0x0000021d,0x0000021e,0x0003003e,0x0000021a,0x0000021f,0x0004003d,
0x00000017,0x00000222,0x00000213,0x0003003e,0x00000221,0x00000222,0x00050039,0x00000006,
0x00000223,0x00000029,0x00000221,0x0004003d,0x00000006,0x00000224,0x000001de,0x00050083,
0x00000006,0x00000225,0x00000223,0x00000224,0x0003003e,0x00000220,0x00000225,0x0004003d,
0x00000006,0x00000227,0x0000021a,0x0006000c,0x00000006,0x00000228,0x00000001,0x00000004,
0x00000227,0x0004003d,0x00000006,0x00000229,0x000001ce,0x000500be,0x00000032,0x0000022a,
0x00000228,0x00000229,0x0003003e,0x00000226,0x0000022a,0x0004003d,0x00000006,0x0000022c,
0x00000220,0x0006000c,0x00000006,0x0000022d,0x00000001,0x00000004,0x0000022c,0x0004003d,
0x00000006,0x0000022e,0x000001ce,0x000500be,0x00000032,0x0000022f,0x0000022d,0x0000022e,
0x0003003e,0x0000022b,0x0000022f,0x0003003e,0x00000230,0x00000231,0x000200f9,0x00000232,
0x000200f8,0x00000232,0x000400f6,0x00000234,0x00000235,0x00000000,0x000200f9,0x00000236,
0x000200f8,0x00000236,0x0004003d,0x0000001d,0x00000237,0x00000230,0x000500b1,0x00000032,
0x00000239,0x00000237,0x00000238,0x000300f7,0x0000023b,0x00000000,0x000400fa,0x00000239,
0x0000023a,0x0000023b,0x000200f8,0x0000023a,0x0004003d,0x00000032,0x0000023c,0x00000226,
0x0004003d,0x00000032,0x0000023d,0x0000022b,0x000500a7,0x00000032,0x0000023e,0x0000023c,
0x0000023d,0x000400a8,0x00000032,0x0000023f,0x0000023e,0x000200f9,0x0000023b,0x000200f8,
0x0000023b,0x000700f5,0x00000032,0x00000240,0x00000239,0x00000236,0x0000023f,0x0000023a,
0x000400fa,0x00000240,0x00000233,0x00000234,0x000200f8,0x00000233,0x0004003d,0x00000032,
0x00000241,0x00000226,0x000400a8,0x00000032,0x00000242,0x00000241,0x000300f7,0x00000244,
0x00000000,0x000400fa,0x00000242,0x00000243,0x00000244,0x000200f8,0x00000243,0x0004003d,
0x00000017,0x00000245,0x0000020b,0x0004003d,0x00000017,0x00000246,0x000001fe,0x0004003d,
0x0000001d,0x00000248,0x00000230,0x0003003e,0x00000247,0x00000248,0x00050039,0x00000006,
0x00000249,0x00000021,0x00000247,0x0005008e,0x00000017,0x0000024a,0x00000246,0x00000249,
0x00050083,0x00000017,0x0000024b,0x00000245,0x0000024a,0x0003003e,0x0000020b,0x0000024b,
0x0004003d,0x00000017,0x0000024d,0x0000020b,0x0003003e,0x0000024c,0x0000024d,0x00050039,
0x00000006,0x0000024e,0x00000029,0x0000024c,0x0004003d,0x00000006,0x0000024f,0x000001de,
0x00050083,0x00000006,0x00000250,0x0000024e,0x0000024f,0x0003003e,0x0000021a,0x00000250,
0x0004003d,0x00000006,0x00000251,0x0000021a,0x0006000c,0x00000006,0x00000252,0x00000001,
0x00000004,0x00000251,0x0004003d,0x00000006,0x00000253,0x000001ce,0x000500be,0x00000032,
0x00000254,0x00000252,0x00000253,0x0003003e,0x00000226,0x00000254,0x000200f9,0x00000244,
0x000200f8,0x00000244,0x0004003d,0x00000032,0x00000255,0x0000022b,0x000400a8,0x00000032,
0x00000256,0x00000255,0x000300f7,0x00000258,0x00000000,0x000400fa,0x00000256,0x00000257,
0x00000258,0x000200f8,0x00000257,0x0004003d,0x00000017,0x00000259,0x00000213,0x0004003d,
0x00000017,0x0000025a,0x000001fe,0x0004003d,0x0000001d,0x0000025c,0x00000230,0x0003003e,
0x0000025b,0x0000025c,0x00050039,0x00000006,0x0000025d,0x00000021,0x0000025b,0x0005008e,
0x00000017,0x0000025e,0x0000025a,0x0000025d,0x00050081,0x00000017,0x0000025f,0x00000259,
0x0000025e,0x0003003e,0x00000213,0x0000025f,0x0004003d,0x00000017,0x00000261,0x00000213,
0x0003003e,0x00000260,0x00000261,0x00050039,0x00000006,0x00000262,0x00000029,0x00000260,
0x0004003d,0x00000006,0x00000263,0x000001de,0x00050083,0x00000006,0x00000264,0x00000262,
0x00000263,0x0003003e,0x00000220,0x00000264,0x0004003d,0x00000006,0x00000265,0x00000220,
0x0006000c,0x00000006,0x00000266,0x00000001,0x00000004,0x00000265,0x0004003d,0x00000006,
0x00000267,0x000001ce,0x000500be,0x00000032,0x00000268,0x00000266,0x00000267,0x0003003e,
0x0000022b,0x00000268,0x000200f9,0x00000258,0x000200f8,0x00000258,0x000200f9,0x00000235,
0x000200f8,0x00000235,0x0004003d,0x0000001d,0x00000269,0x00000230,0x00050080,0x0000001d,
0x0000026a,0x00000269,0x00000231,0x0003003e,0x00000230,0x0000026a,0x000200f9,0x00000232,
0x000200f8,0x00000234,0x0004003d,0x00000032,0x0000026c,0x000001b2,0x000300f7,0x0000026f,
0x00000000,0x000400fa,0x0000026c,0x0000026e,0x00000275,0x000200f8,0x0000026e,0x00050041,
0x00000007,0x00000270,0x0000002c,0x00000040,0x0004003d,0x00000006,0x00000271,0x00000270,
0x00050041,0x00000007,0x00000272,0x0000020b,0x00000040,0x0004003d,0x00000006,0x00000273,
0x00000272,0x00050083,0x00000006,0x00000274,0x00000271,0x00000273,0x0003003e,0x0000026d,
0x00000274,0x000200f9,0x0000026f,0x000200f8,0x00000275,0x00050041,0x00000007,0x00000276,
0x0000002c,0x00000046,0x0004003d,0x00000006,0x00000277,0x00000276,0x00050041,0x00000007,
0x00000278,0x0000020b,0x00000046,0x0004003d,0x00000006,0x00000279,0x00000278,0x00050083,
0x00000006,0x0000027a,0x00000277,0x00000279,0x0003003e,0x0000026d,0x0000027a,0x000200f9,
0x0000026f,0x000200f8,0x0000026f,0x0004003d,0x00000006,0x0000027b,0x0000026d,0x0003003e,
0x0000026b,0x0000027b,0x0004003d,0x00000032,0x0000027d,0x000001b2,0x000300f7,0x00000280,
0x00000000,0x000400fa,0x0000027d,0x0000027f,0x00000286,0x000200f8,0x0000027f,0x00050041,
0x00000007,0x00000281,0x00000213,0x00000040,0x0004003d,0x00000006,0x00000282,0x00000281,
0x00050041,0x00000007,0x00000283,0x0000002c,0x00000040,0x0004003d,0x00000006,0x00000284,
0x00000283,0x00050083,0x00000006,0x00000285,0x00000282,0x00000284,0x0003003e,0x0000027e,
0x00000285,0x000200f9,0x00000280,0x000200f8,0x00000286,0x00050041,0x00000007,0x00000287,
0x00000213,0x00000046,0x0004003d,0x00000006,0x00000288,0x00000287,0x00050041,0x00000007,
0x00000289,0x0000002c,0x00000046,0x0004003d,0x00000006,0x0000028a,0x00000289,0x00050083,
0x00000006,0x0000028b,0x00000288,0x0000028a,0x0003003e,0x0000027e,0x0000028b,0x000200f9,
0x00000280,0x000200f8,0x00000280,0x0004003d,0x00000006,0x0000028c,0x0000027e,0x0003003e,
0x0000027c,0x0000028c,0x0004003d,0x00000006,0x0000028e,0x0000026b,0x0004003d,0x00000006,
0x0000028f,0x0000027c,0x000500b8,0x00000032,0x00000290,0x0000028e,0x0000028f,0x0003003e,
0x0000028d,0x00000290,0x0004003d,0x00000006,0x00000292,0x0000026b,0x0004003d,0x00000006,
0x00000293,0x0000027c,0x0007000c,0x00000006,0x00000294,0x00000001,0x00000025,0x00000292,
0x00000293,0x0004003d,0x00000006,0x00000295,0x0000026b,0x0004003d,0x00000006,0x00000296,
0x0000027c,0x00050081,0x00000006,0x00000297,0x00000295,0x00000296,0x00050088,0x00000006,
0x00000298,0x00000294,0x00000297,0x00050083,0x00000006,0x00000299,0x00000068,0x00000298,
0x0003003e,0x00000291,0x00000299,0x0004003d,0x00000032,0x0000029a,0x0000028d,0x0004003d,
0x00000006,0x0000029b,0x0000021a,0x0004003d,0x00000006,0x0000029c,0x00000220,0x000600a9,
0x00000006,0x0000029d,0x0000029a,0x0000029b,0x0000029c,0x000500b8,0x00000032,0x0000029e,
0x0000029d,0x00000031,0x0004003d,0x00000006,0x0000029f,0x000000d9,0x0004003d,0x00000006,
0x000002a0,0x000001de,0x000500b8,0x00000032,0x000002a1,0x0000029f,0x000002a0,0x000500a4,
0x00000032,0x000002a2,0x0000029e,0x000002a1,0x000300f7,0x000002a4,0x00000000,0x000400fa,
0x000002a2,0x000002a3,0x000002a4,0x000200f8,0x000002a3,0x0003003e,0x00000291,0x00000031,
0x000200f9,0x000002a4,0x000200f8,0x000002a4,0x0004003d,0x00000006,0x000002a5,0x00000291,
0x0004003d,0x00000006,0x000002a6,0x00000161,0x0007000c,0x00000006,0x000002a7,0x00000001,
0x00000028,0x000002a5,0x000002a6,0x0003003e,0x00000291,0x000002a7,0x0004003d,0x00000017,
0x000002a9,0x0000002c,0x0003003e,0x000002a8,0x000002a9,0x0004003d,0x00000032,0x000002aa,
0x000001b2,0x000300f7,0x000002ac,0x00000000,0x000400fa,0x000002aa,0x000002ab,0x000002b4,
0x000200f8,0x000002ab,0x0004003d,0x00000006,0x000002ad,0x00000291,0x0004003d,0x00000006,
0x000002ae,0x000001d3,0x00050085,0x00000006,0x000002af,0x000002ad,0x000002ae,0x00050041,
0x00000007,0x000002b0,0x000002a8,0x00000046,0x0004003d,0x00000006,0x000002b1,0x000002b0,
0x00050081,0x00000006,0x000002b2,0x000002b1,0x000002af,0x00050041,0x00000007,0x000002b3,
0x000002a8,0x00000046,0x0003003e,0x000002b3,0x000002b2,0x000200f9,0x000002ac,0x000200f8,
0x000002b4,0x0004003d,0x00000006,0x000002b5,0x00000291,0x0004003d,0x00000006,0x000002b6,
0x000001d3,0x00050085,0x00000006,0x000002b7,0x000002b5,0x000002b6,0x00050041,0x00000007,
0x000002b8,0x000002a8,0x00000040,0x0004003d,0x00000006,0x000002b9,0x000002b8,0x00050081,
0x00000006,0x000002ba,0x000002b9,0x000002b7,0x00050041,0x00000007,0x000002bb,0x000002a8,
0x00000040,0x0003003e,0x000002bb,0x000002ba,0x000200f9,0x000002ac,0x000200f8,0x000002ac,
0x0004003d,0x00000017,0x000002bd,0x000002a8,0x0003003e,0x000002bc,0x000002bd,0x00050039,
0x0000000c,0x000002be,0x0000001b,0x000002bc,0x000200fe,0x000002be,0x00010038
};

View File

@@ -0,0 +1,103 @@
#version 450
// (Triforce) The fine vectors (vk_framegen.cpp, RX_VK_FRAMEGEN_FINE, on by default): one per 8x8 pixels, to a fraction of a
// pixel. The old way's vectors (fg_flow.comp, fg_median.comp) are one per 16x16 pixels in whole steps of 4 pixels, and
// fg_synth.comp moved each pixel by a blend of the four around it: a straight line moving 6 pixels was moved 4 by some
// blocks and 8 by others and came out bent and broken, where the two frames did not agree it took a frame unmoved, and a
// thin line was not drawn at all (tests/test_framegen_lines.py measures it). For each block of 8x8 pixels:
// 1. the coarse vectors (fg_flow.comp, here asked to be less sure of no motion) of the 3x3 coarse blocks around it, its
// own before the median, and none are tried on its window, 16x16 pixels of half-size luma (fg_half.comp: 8x8 texels,
// so the windows of neighbouring blocks overlap); the best is kept (a block on the edge of a moving object takes the
// motion its own pixels follow), a little cost for each texel away from its coarse block's vector keeping a window of
// no detail (sky, a flat wall), which matches any vector as well, on the motion around it;
// 2. it is moved a texel at a time while that matches better;
// 3. placed between texels at the minimum of a parabola through the costs either side;
// 4. and refined on the frames themselves, at full size: one step of Lucas-Kanade over 12x12 pixels (the block and 2
// around), kept only when it matches better. A line alone in a window (no detail along it) cannot tell its motion
// along itself: the step is held back there (the gradients' sum regularized), and fg_fine_pick.comp settles it.
// Output: xy the vector in pixels (where the frame before's pixel at the block went), z the window's mean difference.
layout(local_size_x = 8, local_size_y = 8) in;
layout(binding = 0) uniform sampler2D la; // (half-size luma, the frame before)
layout(binding = 1) uniform sampler2D lb; // (this frame's)
layout(binding = 2) uniform sampler2D coarse; // (fg_median.comp's vectors, pixels)
layout(binding = 3) uniform sampler2D raw; // (fg_flow.comp's, before the median)
layout(binding = 4, rgba16f) uniform writeonly image2D fine;
layout(binding = 5) uniform sampler2D fa; // (the frame before, the linear sampler)
layout(binding = 6) uniform sampler2D fb; // (this frame)
float pa[64]; // (the window in the frame before)
ivec2 c0; // (its first texel)
ivec2 lm;
ivec2 vc; // (the coarse vector of the block, texels)
ivec2 fmax;
vec2 finv;
float sad(ivec2 v) {
float s = 0.0;
for (int i = 0; i < 64; i++)
s += abs(pa[i] - texelFetch(lb, clamp(c0 + ivec2(i % 8, i / 8) + v, ivec2(0), lm), 0).r);
return s / 64.0;
}
float cost(ivec2 v) { return sad(v) + 0.002 * float(abs(v.x - vc.x) + abs(v.y - vc.y)); }
ivec2 texels(vec2 px) { return ivec2(round(px * 0.5)); }
float fit(float l, float c, float r) { // (the parabola's minimum, -0.5 to 0.5 texels from the middle one)
const float d = l + r - 2.0 * c;
return d > 1e-6 ? clamp(0.5 * (l - r) / d, -0.5, 0.5) : 0.0;
}
float luma_a(ivec2 q) { return dot(texelFetch(fa, clamp(q, ivec2(0), fmax), 0).rgb, vec3(0.299, 0.587, 0.114)); }
float luma_b(vec2 p) { return dot(textureLod(fb, p * finv, 0.0).rgb, vec3(0.299, 0.587, 0.114)); }
void main() {
const ivec2 g = ivec2(gl_GlobalInvocationID.xy);
if (any(greaterThanEqual(g, imageSize(fine)))) return;
lm = textureSize(la, 0) - 1;
c0 = g * 4 - 2;
for (int i = 0; i < 64; i++) pa[i] = texelFetch(la, clamp(c0 + ivec2(i % 8, i / 8), ivec2(0), lm), 0).r;
const ivec2 cm = textureSize(coarse, 0) - 1;
const ivec2 cb = (g * 8 + 4) / 16; // (the coarse block holding this block's middle)
vc = texels(texelFetch(coarse, clamp(cb, ivec2(0), cm), 0).xy);
ivec2 bv = ivec2(0);
float best = cost(bv);
for (int i = 0; i < 10; i++) {
const ivec2 v = i < 9 ? texels(texelFetch(coarse, clamp(cb + ivec2(i % 3 - 1, i / 3 - 1), ivec2(0), cm), 0).xy)
: texels(texelFetch(raw, clamp(cb, ivec2(0), cm), 0).xy);
const float c = cost(v);
if (c < best) best = c, bv = v;
}
for (int it = 0; it < 4; it++) {
ivec2 to = bv;
float tb = best;
for (int i = 0; i < 9; i++) {
if (i == 4) continue;
const ivec2 v = bv + ivec2(i % 3 - 1, i / 3 - 1);
const float c = cost(v);
if (c < tb) tb = c, to = v;
}
if (to == bv) break;
bv = to, best = tb;
}
const float s0 = sad(bv);
const vec2 start = (vec2(bv) + vec2(fit(sad(bv - ivec2(1, 0)), s0, sad(bv + ivec2(1, 0))),
fit(sad(bv - ivec2(0, 1)), s0, sad(bv + ivec2(0, 1))))) * 2.0;
// (4. above: the squared difference at the start and the step's sums, then the difference after the step)
fmax = textureSize(fa, 0) - 1;
finv = 1.0 / vec2(textureSize(fb, 0));
const ivec2 q0 = g * 8 - 2;
float h11 = 0.0, h12 = 0.0, h22 = 0.0, b1 = 0.0, b2 = 0.0, e0 = 0.0;
for (int i = 0; i < 144; i++) {
const ivec2 q = q0 + ivec2(i % 12, i / 12);
const float r = luma_b(vec2(q) + 0.5 + start) - luma_a(q);
const float gx = 0.5 * (luma_a(q + ivec2(1, 0)) - luma_a(q - ivec2(1, 0)));
const float gy = 0.5 * (luma_a(q + ivec2(0, 1)) - luma_a(q - ivec2(0, 1)));
e0 += r * r, h11 += gx * gx, h12 += gx * gy, h22 += gy * gy, b1 += gx * r, b2 += gy * r;
}
const float eps = 1e-3 + 0.02 * (h11 + h22);
const float d11 = h11 + eps, d22 = h22 + eps;
const vec2 v = start - clamp(vec2(d22 * b1 - h12 * b2, d11 * b2 - h12 * b1) / (d11 * d22 - h12 * h12), -1.0, 1.0);
float e1 = 0.0;
for (int i = 0; i < 144; i++) {
const ivec2 q = q0 + ivec2(i % 12, i / 12);
const float r = luma_b(vec2(q) + 0.5 + v) - luma_a(q);
e1 += r * r;
}
imageStore(fine, g, vec4(e1 < e0 ? v : start, s0, 1.0));
}

View File

@@ -0,0 +1,453 @@
// 1116.6.0
#pragma once
const uint32_t FG_FINE_FLOW_COMP_SPV[] = {
0x07230203,0x00010500,0x0008000b,0x0000027c,0x00000000,0x00020011,0x00000001,0x00020011,
0x00000032,0x0006000b,0x00000001,0x4c534c47,0x6474732e,0x3035342e,0x00000000,0x0003000e,
0x00000000,0x00000001,0x0013000f,0x00000005,0x00000004,0x6e69616d,0x00000000,0x00000037,
0x0000003f,0x00000042,0x0000004e,0x00000068,0x0000009c,0x0000009f,0x000000ad,0x000000b1,
0x000000bc,0x000000c4,0x000000cd,0x000000f4,0x00000134,0x00060010,0x00000004,0x00000011,
0x00000008,0x00000008,0x00000001,0x00030003,0x00000002,0x000001c2,0x00040005,0x00000004,
0x6e69616d,0x00000000,0x00050005,0x0000000c,0x28646173,0x3b326976,0x00000000,0x00030005,
0x0000000b,0x00000076,0x00050005,0x0000000f,0x74736f63,0x32697628,0x0000003b,0x00030005,
0x0000000e,0x00000076,0x00050005,0x00000015,0x65786574,0x7628736c,0x003b3266,0x00030005,
0x00000014,0x00007870,0x00060005,0x0000001c,0x28746966,0x663b3166,0x31663b31,0x0000003b,
0x00030005,0x00000019,0x0000006c,0x00030005,0x0000001a,0x00000063,0x00030005,0x0000001b,
0x00000072,0x00050005,0x0000001f,0x616d756c,0x7628615f,0x003b3269,0x00030005,0x0000001e,
0x00000071,0x00050005,0x00000023,0x616d756c,0x7628625f,0x003b3266,0x00030005,0x00000022,
0x00000070,0x00030005,0x00000025,0x00000073,0x00030005,0x00000028,0x00000069,0x00030005,
0x00000037,0x00006170,0x00030005,0x0000003f,0x0000626c,0x00030005,0x00000042,0x00003063,
0x00030005,0x0000004e,0x00006d6c,0x00040005,0x00000062,0x61726170,0x0000006d,0x00030005,
0x00000068,0x00006376,0x00030005,0x00000082,0x00000064,0x00030005,0x0000009c,0x00006166,
0x00040005,0x0000009f,0x78616d66,0x00000000,0x00030005,0x000000ad,0x00006266,0x00040005,
0x000000b1,0x766e6966,0x00000000,0x00030005,0x000000b9,0x00000067,0x00080005,0x000000bc,
0x475f6c67,0x61626f6c,0x766e496c,0x7461636f,0x496e6f69,0x00000044,0x00040005,0x000000c4,
0x656e6966,0x00000000,0x00030005,0x000000cd,0x0000616c,0x00030005,0x000000da,0x00000069,
0x00030005,0x000000f3,0x00006d63,0x00040005,0x000000f4,0x72616f63,0x00006573,0x00030005,
0x000000fa,0x00006263,0x00040005,0x00000109,0x61726170,0x0000006d,0x00030005,0x0000010c,
0x00007662,0x00040005,0x0000010d,0x74736562,0x00000000,0x00040005,0x0000010e,0x61726170,
0x0000006d,0x00030005,0x00000111,0x00000069,0x00030005,0x0000011a,0x00000076,0x00040005,
0x00000130,0x61726170,0x0000006d,0x00030005,0x00000134,0x00776172,0x00040005,0x0000013b,
0x61726170,0x0000006d,0x00030005,0x0000013f,0x00000063,0x00040005,0x00000140,0x61726170,
0x0000006d,0x00030005,0x0000014c,0x00007469,0x00030005,0x00000154,0x00006f74,0x00030005,
0x00000156,0x00006274,0x00030005,0x00000158,0x00000069,0x00030005,0x00000165,0x00000076,
0x00030005,0x0000016f,0x00000063,0x00040005,0x00000170,0x61726170,0x0000006d,0x00030005,
0x00000187,0x00003073,0x00040005,0x00000188,0x61726170,0x0000006d,0x00040005,0x0000018b,
0x72617473,0x00000074,0x00040005,0x00000191,0x61726170,0x0000006d,0x00040005,0x00000195,
0x61726170,0x0000006d,0x00040005,0x00000197,0x61726170,0x0000006d,0x00040005,0x00000198,
0x61726170,0x0000006d,0x00040005,0x0000019a,0x61726170,0x0000006d,0x00040005,0x0000019f,
0x61726170,0x0000006d,0x00040005,0x000001a3,0x61726170,0x0000006d,0x00040005,0x000001a5,
0x61726170,0x0000006d,0x00040005,0x000001a6,0x61726170,0x0000006d,0x00040005,0x000001a8,
0x61726170,0x0000006d,0x00030005,0x000001b9,0x00003071,0x00030005,0x000001bf,0x00313168,
0x00030005,0x000001c0,0x00323168,0x00030005,0x000001c1,0x00323268,0x00030005,0x000001c2,
0x00003162,0x00030005,0x000001c3,0x00003262,0x00030005,0x000001c4,0x00003065,0x00030005,
0x000001c5,0x00000069,0x00030005,0x000001ce,0x00000071,0x00030005,0x000001d7,0x00000072,
0x00040005,0x000001de,0x61726170,0x0000006d,0x00040005,0x000001e0,0x61726170,0x0000006d,
0x00030005,0x000001e4,0x00007867,0x00040005,0x000001e7,0x61726170,0x0000006d,0x00040005,
0x000001eb,0x61726170,0x0000006d,0x00030005,0x000001ef,0x00007967,0x00040005,0x000001f2,
0x61726170,0x0000006d,0x00040005,0x000001f6,0x61726170,0x0000006d,0x00030005,0x0000021a,
0x00737065,0x00030005,0x00000222,0x00313164,0x00030005,0x00000226,0x00323264,0x00030005,
0x0000022a,0x00000076,0x00030005,0x00000249,0x00003165,0x00030005,0x0000024a,0x00000069,
0x00030005,0x00000252,0x00000071,0x00030005,0x0000025a,0x00000072,0x00040005,0x00000261,
0x61726170,0x0000006d,0x00040005,0x00000263,0x61726170,0x0000006d,0x00040047,0x0000003f,
0x00000021,0x00000001,0x00040047,0x0000003f,0x00000022,0x00000000,0x00040047,0x0000009c,
0x00000021,0x00000005,0x00040047,0x0000009c,0x00000022,0x00000000,0x00040047,0x000000ad,
0x00000021,0x00000006,0x00040047,0x000000ad,0x00000022,0x00000000,0x00040047,0x000000bc,
0x0000000b,0x0000001c,0x00030047,0x000000c4,0x00000019,0x00040047,0x000000c4,0x00000021,
0x00000004,0x00040047,0x000000c4,0x00000022,0x00000000,0x00040047,0x000000cd,0x00000021,
0x00000000,0x00040047,0x000000cd,0x00000022,0x00000000,0x00040047,0x000000f4,0x00000021,
0x00000002,0x00040047,0x000000f4,0x00000022,0x00000000,0x00040047,0x00000134,0x00000021,
0x00000003,0x00040047,0x00000134,0x00000022,0x00000000,0x00040047,0x0000027b,0x0000000b,
0x00000019,0x00020013,0x00000002,0x00030021,0x00000003,0x00000002,0x00040015,0x00000006,
0x00000020,0x00000001,0x00040017,0x00000007,0x00000006,0x00000002,0x00040020,0x00000008,
0x00000007,0x00000007,0x00030016,0x00000009,0x00000020,0x00040021,0x0000000a,0x00000009,
0x00000008,0x00040017,0x00000011,0x00000009,0x00000002,0x00040020,0x00000012,0x00000007,
0x00000011,0x00040021,0x00000013,0x00000007,0x00000012,0x00040020,0x00000017,0x00000007,
0x00000009,0x00060021,0x00000018,0x00000009,0x00000017,0x00000017,0x00000017,0x00040021,
0x00000021,0x00000009,0x00000012,0x0004002b,0x00000009,0x00000026,0x00000000,0x00040020,
0x00000027,0x00000007,0x00000006,0x0004002b,0x00000006,0x00000029,0x00000000,0x0004002b,
0x00000006,0x00000030,0x00000040,0x00020014,0x00000031,0x00040015,0x00000033,0x00000020,
0x00000000,0x0004002b,0x00000033,0x00000034,0x00000040,0x0004001c,0x00000035,0x00000009,
0x00000034,0x00040020,0x00000036,0x00000006,0x00000035,0x0004003b,0x00000036,0x00000037,
0x00000006,0x00040020,0x00000039,0x00000006,0x00000009,0x00090019,0x0000003c,0x00000009,
0x00000001,0x00000000,0x00000000,0x00000000,0x00000001,0x00000000,0x0003001b,0x0000003d,
0x0000003c,0x00040020,0x0000003e,0x00000000,0x0000003d,0x0004003b,0x0000003e,0x0000003f,
0x00000000,0x00040020,0x00000041,0x00000006,0x00000007,0x0004003b,0x00000041,0x00000042,
0x00000006,0x0004002b,0x00000006,0x00000045,0x00000008,0x0005002c,0x00000007,0x0000004d,
0x00000029,0x00000029,0x0004003b,0x00000041,0x0000004e,0x00000006,0x00040017,0x00000052,
0x00000009,0x00000004,0x0004002b,0x00000033,0x00000054,0x00000000,0x0004002b,0x00000006,
0x0000005b,0x00000001,0x0004002b,0x00000009,0x0000005e,0x42800000,0x0004002b,0x00000009,
0x00000065,0x3b03126f,0x0004003b,0x00000041,0x00000068,0x00000006,0x00040020,0x00000069,
0x00000006,0x00000006,0x0004002b,0x00000033,0x0000006e,0x00000001,0x0004002b,0x00000009,
0x0000007c,0x3f000000,0x0004002b,0x00000009,0x00000086,0x40000000,0x0004002b,0x00000009,
0x0000008b,0x358637bd,0x0004002b,0x00000009,0x00000096,0xbf000000,0x0004003b,0x0000003e,
0x0000009c,0x00000000,0x0004003b,0x00000041,0x0000009f,0x00000006,0x00040017,0x000000a4,
0x00000009,0x00000003,0x0004002b,0x00000009,0x000000a6,0x3e991687,0x0004002b,0x00000009,
0x000000a7,0x3f1645a2,0x0004002b,0x00000009,0x000000a8,0x3de978d5,0x0006002c,0x000000a4,
0x000000a9,0x000000a6,0x000000a7,0x000000a8,0x0004003b,0x0000003e,0x000000ad,0x00000000,
0x00040020,0x000000b0,0x00000006,0x00000011,0x0004003b,0x000000b0,0x000000b1,0x00000006,
0x00040017,0x000000ba,0x00000033,0x00000003,0x00040020,0x000000bb,0x00000001,0x000000ba,
0x0004003b,0x000000bb,0x000000bc,0x00000001,0x00040017,0x000000bd,0x00000033,0x00000002,
0x00090019,0x000000c2,0x00000009,0x00000001,0x00000000,0x00000000,0x00000000,0x00000002,
0x00000002,0x00040020,0x000000c3,0x00000000,0x000000c2,0x0004003b,0x000000c3,0x000000c4,
0x00000000,0x00040017,0x000000c7,0x00000031,0x00000002,0x0004003b,0x0000003e,0x000000cd,
0x00000000,0x0004002b,0x00000006,0x000000d4,0x00000004,0x0004002b,0x00000006,0x000000d7,
0x00000002,0x0004003b,0x0000003e,0x000000f4,0x00000000,0x0004002b,0x00000006,0x00000100,
0x00000010,0x0004002b,0x00000006,0x00000118,0x0000000a,0x0004002b,0x00000006,0x0000011c,
0x00000009,0x0004002b,0x00000006,0x00000124,0x00000003,0x0004003b,0x0000003e,0x00000134,
0x00000000,0x0005002c,0x00000007,0x0000018f,0x0000005b,0x00000029,0x0005002c,0x00000007,
0x0000019d,0x00000029,0x0000005b,0x0004002b,0x00000009,0x000001b2,0x3f800000,0x0004002b,
0x00000006,0x000001cc,0x00000090,0x0004002b,0x00000006,0x000001d1,0x0000000c,0x0004002b,
0x00000009,0x0000021b,0x3a83126f,0x0004002b,0x00000009,0x0000021c,0x3ca3d70a,0x0004002b,
0x00000009,0x00000244,0xbf800000,0x0004002b,0x00000033,0x0000027a,0x00000008,0x0006002c,
0x000000ba,0x0000027b,0x0000027a,0x0000027a,0x0000006e,0x00050036,0x00000002,0x00000004,
0x00000000,0x00000003,0x000200f8,0x00000005,0x0004003b,0x00000008,0x000000b9,0x00000007,
0x0004003b,0x00000027,0x000000da,0x00000007,0x0004003b,0x00000008,0x000000f3,0x00000007,
0x0004003b,0x00000008,0x000000fa,0x00000007,0x0004003b,0x00000012,0x00000109,0x00000007,
0x0004003b,0x00000008,0x0000010c,0x00000007,0x0004003b,0x00000017,0x0000010d,0x00000007,
0x0004003b,0x00000008,0x0000010e,0x00000007,0x0004003b,0x00000027,0x00000111,0x00000007,
0x0004003b,0x00000008,0x0000011a,0x00000007,0x0004003b,0x00000008,0x0000011e,0x00000007,
0x0004003b,0x00000012,0x00000130,0x00000007,0x0004003b,0x00000012,0x0000013b,0x00000007,
0x0004003b,0x00000017,0x0000013f,0x00000007,0x0004003b,0x00000008,0x00000140,0x00000007,
0x0004003b,0x00000027,0x0000014c,0x00000007,0x0004003b,0x00000008,0x00000154,0x00000007,
0x0004003b,0x00000017,0x00000156,0x00000007,0x0004003b,0x00000027,0x00000158,0x00000007,
0x0004003b,0x00000008,0x00000165,0x00000007,0x0004003b,0x00000017,0x0000016f,0x00000007,
0x0004003b,0x00000008,0x00000170,0x00000007,0x0004003b,0x00000017,0x00000187,0x00000007,
0x0004003b,0x00000008,0x00000188,0x00000007,0x0004003b,0x00000012,0x0000018b,0x00000007,
0x0004003b,0x00000008,0x00000191,0x00000007,0x0004003b,0x00000008,0x00000195,0x00000007,
0x0004003b,0x00000017,0x00000197,0x00000007,0x0004003b,0x00000017,0x00000198,0x00000007,
0x0004003b,0x00000017,0x0000019a,0x00000007,0x0004003b,0x00000008,0x0000019f,0x00000007,
0x0004003b,0x00000008,0x000001a3,0x00000007,0x0004003b,0x00000017,0x000001a5,0x00000007,
0x0004003b,0x00000017,0x000001a6,0x00000007,0x0004003b,0x00000017,0x000001a8,0x00000007,
0x0004003b,0x00000008,0x000001b9,0x00000007,0x0004003b,0x00000017,0x000001bf,0x00000007,
0x0004003b,0x00000017,0x000001c0,0x00000007,0x0004003b,0x00000017,0x000001c1,0x00000007,
0x0004003b,0x00000017,0x000001c2,0x00000007,0x0004003b,0x00000017,0x000001c3,0x00000007,
0x0004003b,0x00000017,0x000001c4,0x00000007,0x0004003b,0x00000027,0x000001c5,0x00000007,
0x0004003b,0x00000008,0x000001ce,0x00000007,0x0004003b,0x00000017,0x000001d7,0x00000007,
0x0004003b,0x00000012,0x000001de,0x00000007,0x0004003b,0x00000008,0x000001e0,0x00000007,
0x0004003b,0x00000017,0x000001e4,0x00000007,0x0004003b,0x00000008,0x000001e7,0x00000007,
0x0004003b,0x00000008,0x000001eb,0x00000007,0x0004003b,0x00000017,0x000001ef,0x00000007,
0x0004003b,0x00000008,0x000001f2,0x00000007,0x0004003b,0x00000008,0x000001f6,0x00000007,
0x0004003b,0x00000017,0x0000021a,0x00000007,0x0004003b,0x00000017,0x00000222,0x00000007,
0x0004003b,0x00000017,0x00000226,0x00000007,0x0004003b,0x00000012,0x0000022a,0x00000007,
0x0004003b,0x00000017,0x00000249,0x00000007,0x0004003b,0x00000027,0x0000024a,0x00000007,
0x0004003b,0x00000008,0x00000252,0x00000007,0x0004003b,0x00000017,0x0000025a,0x00000007,
0x0004003b,0x00000012,0x00000261,0x00000007,0x0004003b,0x00000008,0x00000263,0x00000007,
0x0004003d,0x000000ba,0x000000be,0x000000bc,0x0007004f,0x000000bd,0x000000bf,0x000000be,
0x000000be,0x00000000,0x00000001,0x0004007c,0x00000007,0x000000c0,0x000000bf,0x0003003e,
0x000000b9,0x000000c0,0x0004003d,0x00000007,0x000000c1,0x000000b9,0x0004003d,0x000000c2,
0x000000c5,0x000000c4,0x00040068,0x00000007,0x000000c6,0x000000c5,0x000500af,0x000000c7,
0x000000c8,0x000000c1,0x000000c6,0x0004009a,0x00000031,0x000000c9,0x000000c8,0x000300f7,
0x000000cb,0x00000000,0x000400fa,0x000000c9,0x000000ca,0x000000cb,0x000200f8,0x000000ca,
0x000100fd,0x000200f8,0x000000cb,0x0004003d,0x0000003d,0x000000ce,0x000000cd,0x00040064,
0x0000003c,0x000000cf,0x000000ce,0x00050067,0x00000007,0x000000d0,0x000000cf,0x00000029,
0x00050050,0x00000007,0x000000d1,0x0000005b,0x0000005b,0x00050082,0x00000007,0x000000d2,
0x000000d0,0x000000d1,0x0003003e,0x0000004e,0x000000d2,0x0004003d,0x00000007,0x000000d3,
0x000000b9,0x00050050,0x00000007,0x000000d5,0x000000d4,0x000000d4,0x00050084,0x00000007,
0x000000d6,0x000000d3,0x000000d5,0x00050050,0x00000007,0x000000d8,0x000000d7,0x000000d7,
0x00050082,0x00000007,0x000000d9,0x000000d6,0x000000d8,0x0003003e,0x00000042,0x000000d9,
0x0003003e,0x000000da,0x00000029,0x000200f9,0x000000db,0x000200f8,0x000000db,0x000400f6,
0x000000dd,0x000000de,0x00000000,0x000200f9,0x000000df,0x000200f8,0x000000df,0x0004003d,
0x00000006,0x000000e0,0x000000da,0x000500b1,0x00000031,0x000000e1,0x000000e0,0x00000030,
0x000400fa,0x000000e1,0x000000dc,0x000000dd,0x000200f8,0x000000dc,0x0004003d,0x00000006,
0x000000e2,0x000000da,0x0004003d,0x0000003d,0x000000e3,0x000000cd,0x0004003d,0x00000007,
0x000000e4,0x00000042,0x0004003d,0x00000006,0x000000e5,0x000000da,0x0005008b,0x00000006,
0x000000e6,0x000000e5,0x00000045,0x0004003d,0x00000006,0x000000e7,0x000000da,0x00050087,
0x00000006,0x000000e8,0x000000e7,0x00000045,0x00050050,0x00000007,0x000000e9,0x000000e6,
0x000000e8,0x00050080,0x00000007,0x000000ea,0x000000e4,0x000000e9,0x0004003d,0x00000007,
0x000000eb,0x0000004e,0x0008000c,0x00000007,0x000000ec,0x00000001,0x0000002d,0x000000ea,
0x0000004d,0x000000eb,0x00040064,0x0000003c,0x000000ed,0x000000e3,0x0007005f,0x00000052,
0x000000ee,0x000000ed,0x000000ec,0x00000002,0x00000029,0x00050051,0x00000009,0x000000ef,
0x000000ee,0x00000000,0x00050041,0x00000039,0x000000f0,0x00000037,0x000000e2,0x0003003e,
0x000000f0,0x000000ef,0x000200f9,0x000000de,0x000200f8,0x000000de,0x0004003d,0x00000006,
0x000000f1,0x000000da,0x00050080,0x00000006,0x000000f2,0x000000f1,0x0000005b,0x0003003e,
0x000000da,0x000000f2,0x000200f9,0x000000db,0x000200f8,0x000000dd,0x0004003d,0x0000003d,
0x000000f5,0x000000f4,0x00040064,0x0000003c,0x000000f6,0x000000f5,0x00050067,0x00000007,
0x000000f7,0x000000f6,0x00000029,0x00050050,0x00000007,0x000000f8,0x0000005b,0x0000005b,
0x00050082,0x00000007,0x000000f9,0x000000f7,0x000000f8,0x0003003e,0x000000f3,0x000000f9,
0x0004003d,0x00000007,0x000000fb,0x000000b9,0x00050050,0x00000007,0x000000fc,0x00000045,
0x00000045,0x00050084,0x00000007,0x000000fd,0x000000fb,0x000000fc,0x00050050,0x00000007,
0x000000fe,0x000000d4,0x000000d4,0x00050080,0x00000007,0x000000ff,0x000000fd,0x000000fe,
0x00050050,0x00000007,0x00000101,0x00000100,0x00000100,0x00050087,0x00000007,0x00000102,
0x000000ff,0x00000101,0x0003003e,0x000000fa,0x00000102,0x0004003d,0x0000003d,0x00000103,
0x000000f4,0x0004003d,0x00000007,0x00000104,0x000000fa,0x0004003d,0x00000007,0x00000105,
0x000000f3,0x0008000c,0x00000007,0x00000106,0x00000001,0x0000002d,0x00000104,0x0000004d,
0x00000105,0x00040064,0x0000003c,0x00000107,0x00000103,0x0007005f,0x00000052,0x00000108,
0x00000107,0x00000106,0x00000002,0x00000029,0x0007004f,0x00000011,0x0000010a,0x00000108,
0x00000108,0x00000000,0x00000001,0x0003003e,0x00000109,0x0000010a,0x00050039,0x00000007,
0x0000010b,0x00000015,0x00000109,0x0003003e,0x00000068,0x0000010b,0x0003003e,0x0000010c,
0x0000004d,0x0004003d,0x00000007,0x0000010f,0x0000010c,0x0003003e,0x0000010e,0x0000010f,
0x00050039,0x00000009,0x00000110,0x0000000f,0x0000010e,0x0003003e,0x0000010d,0x00000110,
0x0003003e,0x00000111,0x00000029,0x000200f9,0x00000112,0x000200f8,0x00000112,0x000400f6,
0x00000114,0x00000115,0x00000000,0x000200f9,0x00000116,0x000200f8,0x00000116,0x0004003d,
0x00000006,0x00000117,0x00000111,0x000500b1,0x00000031,0x00000119,0x00000117,0x00000118,
0x000400fa,0x00000119,0x00000113,0x00000114,0x000200f8,0x00000113,0x0004003d,0x00000006,
0x0000011b,0x00000111,0x000500b1,0x00000031,0x0000011d,0x0000011b,0x0000011c,0x000300f7,
0x00000120,0x00000000,0x000400fa,0x0000011d,0x0000011f,0x00000133,0x000200f8,0x0000011f,
0x0004003d,0x0000003d,0x00000121,0x000000f4,0x0004003d,0x00000007,0x00000122,0x000000fa,
0x0004003d,0x00000006,0x00000123,0x00000111,0x0005008b,0x00000006,0x00000125,0x00000123,
0x00000124,0x00050082,0x00000006,0x00000126,0x00000125,0x0000005b,0x0004003d,0x00000006,
0x00000127,0x00000111,0x00050087,0x00000006,0x00000128,0x00000127,0x00000124,0x00050082,
0x00000006,0x00000129,0x00000128,0x0000005b,0x00050050,0x00000007,0x0000012a,0x00000126,
0x00000129,0x00050080,0x00000007,0x0000012b,0x00000122,0x0000012a,0x0004003d,0x00000007,
0x0000012c,0x000000f3,0x0008000c,0x00000007,0x0000012d,0x00000001,0x0000002d,0x0000012b,
0x0000004d,0x0000012c,0x00040064,0x0000003c,0x0000012e,0x00000121,0x0007005f,0x00000052,
0x0000012f,0x0000012e,0x0000012d,0x00000002,0x00000029,0x0007004f,0x00000011,0x00000131,
0x0000012f,0x0000012f,0x00000000,0x00000001,0x0003003e,0x00000130,0x00000131,0x00050039,
0x00000007,0x00000132,0x00000015,0x00000130,0x0003003e,0x0000011e,0x00000132,0x000200f9,
0x00000120,0x000200f8,0x00000133,0x0004003d,0x0000003d,0x00000135,0x00000134,0x0004003d,
0x00000007,0x00000136,0x000000fa,0x0004003d,0x00000007,0x00000137,0x000000f3,0x0008000c,
0x00000007,0x00000138,0x00000001,0x0000002d,0x00000136,0x0000004d,0x00000137,0x00040064,
0x0000003c,0x00000139,0x00000135,0x0007005f,0x00000052,0x0000013a,0x00000139,0x00000138,
0x00000002,0x00000029,0x0007004f,0x00000011,0x0000013c,0x0000013a,0x0000013a,0x00000000,
0x00000001,0x0003003e,0x0000013b,0x0000013c,0x00050039,0x00000007,0x0000013d,0x00000015,
0x0000013b,0x0003003e,0x0000011e,0x0000013d,0x000200f9,0x00000120,0x000200f8,0x00000120,
0x0004003d,0x00000007,0x0000013e,0x0000011e,0x0003003e,0x0000011a,0x0000013e,0x0004003d,
0x00000007,0x00000141,0x0000011a,0x0003003e,0x00000140,0x00000141,0x00050039,0x00000009,
0x00000142,0x0000000f,0x00000140,0x0003003e,0x0000013f,0x00000142,0x0004003d,0x00000009,
0x00000143,0x0000013f,0x0004003d,0x00000009,0x00000144,0x0000010d,0x000500b8,0x00000031,
0x00000145,0x00000143,0x00000144,0x000300f7,0x00000147,0x00000000,0x000400fa,0x00000145,
0x00000146,0x00000147,0x000200f8,0x00000146,0x0004003d,0x00000009,0x00000148,0x0000013f,
0x0003003e,0x0000010d,0x00000148,0x0004003d,0x00000007,0x00000149,0x0000011a,0x0003003e,
0x0000010c,0x00000149,0x000200f9,0x00000147,0x000200f8,0x00000147,0x000200f9,0x00000115,
0x000200f8,0x00000115,0x0004003d,0x00000006,0x0000014a,0x00000111,0x00050080,0x00000006,
0x0000014b,0x0000014a,0x0000005b,0x0003003e,0x00000111,0x0000014b,0x000200f9,0x00000112,
0x000200f8,0x00000114,0x0003003e,0x0000014c,0x00000029,0x000200f9,0x0000014d,0x000200f8,
0x0000014d,0x000400f6,0x0000014f,0x00000150,0x00000000,0x000200f9,0x00000151,0x000200f8,
0x00000151,0x0004003d,0x00000006,0x00000152,0x0000014c,0x000500b1,0x00000031,0x00000153,
0x00000152,0x000000d4,0x000400fa,0x00000153,0x0000014e,0x0000014f,0x000200f8,0x0000014e,
0x0004003d,0x00000007,0x00000155,0x0000010c,0x0003003e,0x00000154,0x00000155,0x0004003d,
0x00000009,0x00000157,0x0000010d,0x0003003e,0x00000156,0x00000157,0x0003003e,0x00000158,
0x00000029,0x000200f9,0x00000159,0x000200f8,0x00000159,0x000400f6,0x0000015b,0x0000015c,
0x00000000,0x000200f9,0x0000015d,0x000200f8,0x0000015d,0x0004003d,0x00000006,0x0000015e,
0x00000158,0x000500b1,0x00000031,0x0000015f,0x0000015e,0x0000011c,0x000400fa,0x0000015f,
0x0000015a,0x0000015b,0x000200f8,0x0000015a,0x0004003d,0x00000006,0x00000160,0x00000158,
0x000500aa,0x00000031,0x00000161,0x00000160,0x000000d4,0x000300f7,0x00000163,0x00000000,
0x000400fa,0x00000161,0x00000162,0x00000163,0x000200f8,0x00000162,0x000200f9,0x0000015c,
0x000200f8,0x00000163,0x0004003d,0x00000007,0x00000166,0x0000010c,0x0004003d,0x00000006,
0x00000167,0x00000158,0x0005008b,0x00000006,0x00000168,0x00000167,0x00000124,0x00050082,
0x00000006,0x00000169,0x00000168,0x0000005b,0x0004003d,0x00000006,0x0000016a,0x00000158,
0x00050087,0x00000006,0x0000016b,0x0000016a,0x00000124,0x00050082,0x00000006,0x0000016c,
0x0000016b,0x0000005b,0x00050050,0x00000007,0x0000016d,0x00000169,0x0000016c,0x00050080,
0x00000007,0x0000016e,0x00000166,0x0000016d,0x0003003e,0x00000165,0x0000016e,0x0004003d,
0x00000007,0x00000171,0x00000165,0x0003003e,0x00000170,0x00000171,0x00050039,0x00000009,
0x00000172,0x0000000f,0x00000170,0x0003003e,0x0000016f,0x00000172,0x0004003d,0x00000009,
0x00000173,0x0000016f,0x0004003d,0x00000009,0x00000174,0x00000156,0x000500b8,0x00000031,
0x00000175,0x00000173,0x00000174,0x000300f7,0x00000177,0x00000000,0x000400fa,0x00000175,
0x00000176,0x00000177,0x000200f8,0x00000176,0x0004003d,0x00000009,0x00000178,0x0000016f,
0x0003003e,0x00000156,0x00000178,0x0004003d,0x00000007,0x00000179,0x00000165,0x0003003e,
0x00000154,0x00000179,0x000200f9,0x00000177,0x000200f8,0x00000177,0x000200f9,0x0000015c,
0x000200f8,0x0000015c,0x0004003d,0x00000006,0x0000017a,0x00000158,0x00050080,0x00000006,
0x0000017b,0x0000017a,0x0000005b,0x0003003e,0x00000158,0x0000017b,0x000200f9,0x00000159,
0x000200f8,0x0000015b,0x0004003d,0x00000007,0x0000017c,0x00000154,0x0004003d,0x00000007,
0x0000017d,0x0000010c,0x000500aa,0x000000c7,0x0000017e,0x0000017c,0x0000017d,0x0004009b,
0x00000031,0x0000017f,0x0000017e,0x000300f7,0x00000181,0x00000000,0x000400fa,0x0000017f,
0x00000180,0x00000181,0x000200f8,0x00000180,0x000200f9,0x0000014f,0x000200f8,0x00000181,
0x0004003d,0x00000007,0x00000183,0x00000154,0x0003003e,0x0000010c,0x00000183,0x0004003d,
0x00000009,0x00000184,0x00000156,0x0003003e,0x0000010d,0x00000184,0x000200f9,0x00000150,
0x000200f8,0x00000150,0x0004003d,0x00000006,0x00000185,0x0000014c,0x00050080,0x00000006,
0x00000186,0x00000185,0x0000005b,0x0003003e,0x0000014c,0x00000186,0x000200f9,0x0000014d,
0x000200f8,0x0000014f,0x0004003d,0x00000007,0x00000189,0x0000010c,0x0003003e,0x00000188,
0x00000189,0x00050039,0x00000009,0x0000018a,0x0000000c,0x00000188,0x0003003e,0x00000187,
0x0000018a,0x0004003d,0x00000007,0x0000018c,0x0000010c,0x0004006f,0x00000011,0x0000018d,
0x0000018c,0x0004003d,0x00000007,0x0000018e,0x0000010c,0x00050082,0x00000007,0x00000190,
0x0000018e,0x0000018f,0x0003003e,0x00000191,0x00000190,0x00050039,0x00000009,0x00000192,
0x0000000c,0x00000191,0x0004003d,0x00000007,0x00000193,0x0000010c,0x00050080,0x00000007,
0x00000194,0x00000193,0x0000018f,0x0003003e,0x00000195,0x00000194,0x00050039,0x00000009,
0x00000196,0x0000000c,0x00000195,0x0003003e,0x00000197,0x00000192,0x0004003d,0x00000009,
0x00000199,0x00000187,0x0003003e,0x00000198,0x00000199,0x0003003e,0x0000019a,0x00000196,
0x00070039,0x00000009,0x0000019b,0x0000001c,0x00000197,0x00000198,0x0000019a,0x0004003d,
0x00000007,0x0000019c,0x0000010c,0x00050082,0x00000007,0x0000019e,0x0000019c,0x0000019d,
0x0003003e,0x0000019f,0x0000019e,0x00050039,0x00000009,0x000001a0,0x0000000c,0x0000019f,
0x0004003d,0x00000007,0x000001a1,0x0000010c,0x00050080,0x00000007,0x000001a2,0x000001a1,
0x0000019d,0x0003003e,0x000001a3,0x000001a2,0x00050039,0x00000009,0x000001a4,0x0000000c,
0x000001a3,0x0003003e,0x000001a5,0x000001a0,0x0004003d,0x00000009,0x000001a7,0x00000187,
0x0003003e,0x000001a6,0x000001a7,0x0003003e,0x000001a8,0x000001a4,0x00070039,0x00000009,
0x000001a9,0x0000001c,0x000001a5,0x000001a6,0x000001a8,0x00050050,0x00000011,0x000001aa,
0x0000019b,0x000001a9,0x00050081,0x00000011,0x000001ab,0x0000018d,0x000001aa,0x0005008e,
0x00000011,0x000001ac,0x000001ab,0x00000086,0x0003003e,0x0000018b,0x000001ac,0x0004003d,
0x0000003d,0x000001ad,0x0000009c,0x00040064,0x0000003c,0x000001ae,0x000001ad,0x00050067,
0x00000007,0x000001af,0x000001ae,0x00000029,0x00050050,0x00000007,0x000001b0,0x0000005b,
0x0000005b,0x00050082,0x00000007,0x000001b1,0x000001af,0x000001b0,0x0003003e,0x0000009f,
0x000001b1,0x0004003d,0x0000003d,0x000001b3,0x000000ad,0x00040064,0x0000003c,0x000001b4,
0x000001b3,0x00050067,0x00000007,0x000001b5,0x000001b4,0x00000029,0x0004006f,0x00000011,
0x000001b6,0x000001b5,0x00050050,0x00000011,0x000001b7,0x000001b2,0x000001b2,0x00050088,
0x00000011,0x000001b8,0x000001b7,0x000001b6,0x0003003e,0x000000b1,0x000001b8,0x0004003d,
0x00000007,0x000001ba,0x000000b9,0x00050050,0x00000007,0x000001bb,0x00000045,0x00000045,
0x00050084,0x00000007,0x000001bc,0x000001ba,0x000001bb,0x00050050,0x00000007,0x000001bd,
0x000000d7,0x000000d7,0x00050082,0x00000007,0x000001be,0x000001bc,0x000001bd,0x0003003e,
0x000001b9,0x000001be,0x0003003e,0x000001bf,0x00000026,0x0003003e,0x000001c0,0x00000026,
0x0003003e,0x000001c1,0x00000026,0x0003003e,0x000001c2,0x00000026,0x0003003e,0x000001c3,
0x00000026,0x0003003e,0x000001c4,0x00000026,0x0003003e,0x000001c5,0x00000029,0x000200f9,
0x000001c6,0x000200f8,0x000001c6,0x000400f6,0x000001c8,0x000001c9,0x00000000,0x000200f9,
0x000001ca,0x000200f8,0x000001ca,0x0004003d,0x00000006,0x000001cb,0x000001c5,0x000500b1,
0x00000031,0x000001cd,0x000001cb,0x000001cc,0x000400fa,0x000001cd,0x000001c7,0x000001c8,
0x000200f8,0x000001c7,0x0004003d,0x00000007,0x000001cf,0x000001b9,0x0004003d,0x00000006,
0x000001d0,0x000001c5,0x0005008b,0x00000006,0x000001d2,0x000001d0,0x000001d1,0x0004003d,
0x00000006,0x000001d3,0x000001c5,0x00050087,0x00000006,0x000001d4,0x000001d3,0x000001d1,
0x00050050,0x00000007,0x000001d5,0x000001d2,0x000001d4,0x00050080,0x00000007,0x000001d6,
0x000001cf,0x000001d5,0x0003003e,0x000001ce,0x000001d6,0x0004003d,0x00000007,0x000001d8,
0x000001ce,0x0004006f,0x00000011,0x000001d9,0x000001d8,0x00050050,0x00000011,0x000001da,
0x0000007c,0x0000007c,0x00050081,0x00000011,0x000001db,0x000001d9,0x000001da,0x0004003d,
0x00000011,0x000001dc,0x0000018b,0x00050081,0x00000011,0x000001dd,0x000001db,0x000001dc,
0x0003003e,0x000001de,0x000001dd,0x00050039,0x00000009,0x000001df,0x00000023,0x000001de,
0x0004003d,0x00000007,0x000001e1,0x000001ce,0x0003003e,0x000001e0,0x000001e1,0x00050039,
0x00000009,0x000001e2,0x0000001f,0x000001e0,0x00050083,0x00000009,0x000001e3,0x000001df,
0x000001e2,0x0003003e,0x000001d7,0x000001e3,0x0004003d,0x00000007,0x000001e5,0x000001ce,
0x00050080,0x00000007,0x000001e6,0x000001e5,0x0000018f,0x0003003e,0x000001e7,0x000001e6,
0x00050039,0x00000009,0x000001e8,0x0000001f,0x000001e7,0x0004003d,0x00000007,0x000001e9,
0x000001ce,0x00050082,0x00000007,0x000001ea,0x000001e9,0x0000018f,0x0003003e,0x000001eb,
0x000001ea,0x00050039,0x00000009,0x000001ec,0x0000001f,0x000001eb,0x00050083,0x00000009,
0x000001ed,0x000001e8,0x000001ec,0x00050085,0x00000009,0x000001ee,0x0000007c,0x000001ed,
0x0003003e,0x000001e4,0x000001ee,0x0004003d,0x00000007,0x000001f0,0x000001ce,0x00050080,
0x00000007,0x000001f1,0x000001f0,0x0000019d,0x0003003e,0x000001f2,0x000001f1,0x00050039,
0x00000009,0x000001f3,0x0000001f,0x000001f2,0x0004003d,0x00000007,0x000001f4,0x000001ce,
0x00050082,0x00000007,0x000001f5,0x000001f4,0x0000019d,0x0003003e,0x000001f6,0x000001f5,
0x00050039,0x00000009,0x000001f7,0x0000001f,0x000001f6,0x00050083,0x00000009,0x000001f8,
0x000001f3,0x000001f7,0x00050085,0x00000009,0x000001f9,0x0000007c,0x000001f8,0x0003003e,
0x000001ef,0x000001f9,0x0004003d,0x00000009,0x000001fa,0x000001d7,0x0004003d,0x00000009,
0x000001fb,0x000001d7,0x00050085,0x00000009,0x000001fc,0x000001fa,0x000001fb,0x0004003d,
0x00000009,0x000001fd,0x000001c4,0x00050081,0x00000009,0x000001fe,0x000001fd,0x000001fc,
0x0003003e,0x000001c4,0x000001fe,0x0004003d,0x00000009,0x000001ff,0x000001e4,0x0004003d,
0x00000009,0x00000200,0x000001e4,0x00050085,0x00000009,0x00000201,0x000001ff,0x00000200,
0x0004003d,0x00000009,0x00000202,0x000001bf,0x00050081,0x00000009,0x00000203,0x00000202,
0x00000201,0x0003003e,0x000001bf,0x00000203,0x0004003d,0x00000009,0x00000204,0x000001e4,
0x0004003d,0x00000009,0x00000205,0x000001ef,0x00050085,0x00000009,0x00000206,0x00000204,
0x00000205,0x0004003d,0x00000009,0x00000207,0x000001c0,0x00050081,0x00000009,0x00000208,
0x00000207,0x00000206,0x0003003e,0x000001c0,0x00000208,0x0004003d,0x00000009,0x00000209,
0x000001ef,0x0004003d,0x00000009,0x0000020a,0x000001ef,0x00050085,0x00000009,0x0000020b,
0x00000209,0x0000020a,0x0004003d,0x00000009,0x0000020c,0x000001c1,0x00050081,0x00000009,
0x0000020d,0x0000020c,0x0000020b,0x0003003e,0x000001c1,0x0000020d,0x0004003d,0x00000009,
0x0000020e,0x000001e4,0x0004003d,0x00000009,0x0000020f,0x000001d7,0x00050085,0x00000009,
0x00000210,0x0000020e,0x0000020f,0x0004003d,0x00000009,0x00000211,0x000001c2,0x00050081,
0x00000009,0x00000212,0x00000211,0x00000210,0x0003003e,0x000001c2,0x00000212,0x0004003d,
0x00000009,0x00000213,0x000001ef,0x0004003d,0x00000009,0x00000214,0x000001d7,0x00050085,
0x00000009,0x00000215,0x00000213,0x00000214,0x0004003d,0x00000009,0x00000216,0x000001c3,
0x00050081,0x00000009,0x00000217,0x00000216,0x00000215,0x0003003e,0x000001c3,0x00000217,
0x000200f9,0x000001c9,0x000200f8,0x000001c9,0x0004003d,0x00000006,0x00000218,0x000001c5,
0x00050080,0x00000006,0x00000219,0x00000218,0x0000005b,0x0003003e,0x000001c5,0x00000219,
0x000200f9,0x000001c6,0x000200f8,0x000001c8,0x0004003d,0x00000009,0x0000021d,0x000001bf,
0x0004003d,0x00000009,0x0000021e,0x000001c1,0x00050081,0x00000009,0x0000021f,0x0000021d,
0x0000021e,0x00050085,0x00000009,0x00000220,0x0000021c,0x0000021f,0x00050081,0x00000009,
0x00000221,0x0000021b,0x00000220,0x0003003e,0x0000021a,0x00000221,0x0004003d,0x00000009,
0x00000223,0x000001bf,0x0004003d,0x00000009,0x00000224,0x0000021a,0x00050081,0x00000009,
0x00000225,0x00000223,0x00000224,0x0003003e,0x00000222,0x00000225,0x0004003d,0x00000009,
0x00000227,0x000001c1,0x0004003d,0x00000009,0x00000228,0x0000021a,0x00050081,0x00000009,
0x00000229,0x00000227,0x00000228,0x0003003e,0x00000226,0x00000229,0x0004003d,0x00000011,
0x0000022b,0x0000018b,0x0004003d,0x00000009,0x0000022c,0x00000226,0x0004003d,0x00000009,
0x0000022d,0x000001c2,0x00050085,0x00000009,0x0000022e,0x0000022c,0x0000022d,0x0004003d,
0x00000009,0x0000022f,0x000001c0,0x0004003d,0x00000009,0x00000230,0x000001c3,0x00050085,
0x00000009,0x00000231,0x0000022f,0x00000230,0x00050083,0x00000009,0x00000232,0x0000022e,
0x00000231,0x0004003d,0x00000009,0x00000233,0x00000222,0x0004003d,0x00000009,0x00000234,
0x000001c3,0x00050085,0x00000009,0x00000235,0x00000233,0x00000234,0x0004003d,0x00000009,
0x00000236,0x000001c0,0x0004003d,0x00000009,0x00000237,0x000001c2,0x00050085,0x00000009,
0x00000238,0x00000236,0x00000237,0x00050083,0x00000009,0x00000239,0x00000235,0x00000238,
0x00050050,0x00000011,0x0000023a,0x00000232,0x00000239,0x0004003d,0x00000009,0x0000023b,
0x00000222,0x0004003d,0x00000009,0x0000023c,0x00000226,0x00050085,0x00000009,0x0000023d,
0x0000023b,0x0000023c,0x0004003d,0x00000009,0x0000023e,0x000001c0,0x0004003d,0x00000009,
0x0000023f,0x000001c0,0x00050085,0x00000009,0x00000240,0x0000023e,0x0000023f,0x00050083,
0x00000009,0x00000241,0x0000023d,0x00000240,0x00050050,0x00000011,0x00000242,0x00000241,
0x00000241,0x00050088,0x00000011,0x00000243,0x0000023a,0x00000242,0x00050050,0x00000011,
0x00000245,0x00000244,0x00000244,0x00050050,0x00000011,0x00000246,0x000001b2,0x000001b2,
0x0008000c,0x00000011,0x00000247,0x00000001,0x0000002b,0x00000243,0x00000245,0x00000246,
0x00050083,0x00000011,0x00000248,0x0000022b,0x00000247,0x0003003e,0x0000022a,0x00000248,
0x0003003e,0x00000249,0x00000026,0x0003003e,0x0000024a,0x00000029,0x000200f9,0x0000024b,
0x000200f8,0x0000024b,0x000400f6,0x0000024d,0x0000024e,0x00000000,0x000200f9,0x0000024f,
0x000200f8,0x0000024f,0x0004003d,0x00000006,0x00000250,0x0000024a,0x000500b1,0x00000031,
0x00000251,0x00000250,0x000001cc,0x000400fa,0x00000251,0x0000024c,0x0000024d,0x000200f8,
0x0000024c,0x0004003d,0x00000007,0x00000253,0x000001b9,0x0004003d,0x00000006,0x00000254,
0x0000024a,0x0005008b,0x00000006,0x00000255,0x00000254,0x000001d1,0x0004003d,0x00000006,
0x00000256,0x0000024a,0x00050087,0x00000006,0x00000257,0x00000256,0x000001d1,0x00050050,
0x00000007,0x00000258,0x00000255,0x00000257,0x00050080,0x00000007,0x00000259,0x00000253,
0x00000258,0x0003003e,0x00000252,0x00000259,0x0004003d,0x00000007,0x0000025b,0x00000252,
0x0004006f,0x00000011,0x0000025c,0x0000025b,0x00050050,0x00000011,0x0000025d,0x0000007c,
0x0000007c,0x00050081,0x00000011,0x0000025e,0x0000025c,0x0000025d,0x0004003d,0x00000011,
0x0000025f,0x0000022a,0x00050081,0x00000011,0x00000260,0x0000025e,0x0000025f,0x0003003e,
0x00000261,0x00000260,0x00050039,0x00000009,0x00000262,0x00000023,0x00000261,0x0004003d,
0x00000007,0x00000264,0x00000252,0x0003003e,0x00000263,0x00000264,0x00050039,0x00000009,
0x00000265,0x0000001f,0x00000263,0x00050083,0x00000009,0x00000266,0x00000262,0x00000265,
0x0003003e,0x0000025a,0x00000266,0x0004003d,0x00000009,0x00000267,0x0000025a,0x0004003d,
0x00000009,0x00000268,0x0000025a,0x00050085,0x00000009,0x00000269,0x00000267,0x00000268,
0x0004003d,0x00000009,0x0000026a,0x00000249,0x00050081,0x00000009,0x0000026b,0x0000026a,
0x00000269,0x0003003e,0x00000249,0x0000026b,0x000200f9,0x0000024e,0x000200f8,0x0000024e,
0x0004003d,0x00000006,0x0000026c,0x0000024a,0x00050080,0x00000006,0x0000026d,0x0000026c,
0x0000005b,0x0003003e,0x0000024a,0x0000026d,0x000200f9,0x0000024b,0x000200f8,0x0000024d,
0x0004003d,0x000000c2,0x0000026e,0x000000c4,0x0004003d,0x00000007,0x0000026f,0x000000b9,
0x0004003d,0x00000009,0x00000270,0x00000249,0x0004003d,0x00000009,0x00000271,0x000001c4,
0x000500b8,0x00000031,0x00000272,0x00000270,0x00000271,0x0004003d,0x00000011,0x00000273,
0x0000022a,0x0004003d,0x00000011,0x00000274,0x0000018b,0x000600a9,0x00000011,0x00000275,
0x00000272,0x00000273,0x00000274,0x0004003d,0x00000009,0x00000276,0x00000187,0x00050051,
0x00000009,0x00000277,0x00000275,0x00000000,0x00050051,0x00000009,0x00000278,0x00000275,
0x00000001,0x00070050,0x00000052,0x00000279,0x00000277,0x00000278,0x00000276,0x000001b2,
0x00040063,0x0000026e,0x0000026f,0x00000279,0x000100fd,0x00010038,0x00050036,0x00000009,
0x0000000c,0x00000000,0x0000000a,0x00030037,0x00000008,0x0000000b,0x000200f8,0x0000000d,
0x0004003b,0x00000017,0x00000025,0x00000007,0x0004003b,0x00000027,0x00000028,0x00000007,
0x0003003e,0x00000025,0x00000026,0x0003003e,0x00000028,0x00000029,0x000200f9,0x0000002a,
0x000200f8,0x0000002a,0x000400f6,0x0000002c,0x0000002d,0x00000000,0x000200f9,0x0000002e,
0x000200f8,0x0000002e,0x0004003d,0x00000006,0x0000002f,0x00000028,0x000500b1,0x00000031,
0x00000032,0x0000002f,0x00000030,0x000400fa,0x00000032,0x0000002b,0x0000002c,0x000200f8,
0x0000002b,0x0004003d,0x00000006,0x00000038,0x00000028,0x00050041,0x00000039,0x0000003a,
0x00000037,0x00000038,0x0004003d,0x00000009,0x0000003b,0x0000003a,0x0004003d,0x0000003d,
0x00000040,0x0000003f,0x0004003d,0x00000007,0x00000043,0x00000042,0x0004003d,0x00000006,
0x00000044,0x00000028,0x0005008b,0x00000006,0x00000046,0x00000044,0x00000045,0x0004003d,
0x00000006,0x00000047,0x00000028,0x00050087,0x00000006,0x00000048,0x00000047,0x00000045,
0x00050050,0x00000007,0x00000049,0x00000046,0x00000048,0x00050080,0x00000007,0x0000004a,
0x00000043,0x00000049,0x0004003d,0x00000007,0x0000004b,0x0000000b,0x00050080,0x00000007,
0x0000004c,0x0000004a,0x0000004b,0x0004003d,0x00000007,0x0000004f,0x0000004e,0x0008000c,
0x00000007,0x00000050,0x00000001,0x0000002d,0x0000004c,0x0000004d,0x0000004f,0x00040064,
0x0000003c,0x00000051,0x00000040,0x0007005f,0x00000052,0x00000053,0x00000051,0x00000050,
0x00000002,0x00000029,0x00050051,0x00000009,0x00000055,0x00000053,0x00000000,0x00050083,
0x00000009,0x00000056,0x0000003b,0x00000055,0x0006000c,0x00000009,0x00000057,0x00000001,
0x00000004,0x00000056,0x0004003d,0x00000009,0x00000058,0x00000025,0x00050081,0x00000009,
0x00000059,0x00000058,0x00000057,0x0003003e,0x00000025,0x00000059,0x000200f9,0x0000002d,
0x000200f8,0x0000002d,0x0004003d,0x00000006,0x0000005a,0x00000028,0x00050080,0x00000006,
0x0000005c,0x0000005a,0x0000005b,0x0003003e,0x00000028,0x0000005c,0x000200f9,0x0000002a,
0x000200f8,0x0000002c,0x0004003d,0x00000009,0x0000005d,0x00000025,0x00050088,0x00000009,
0x0000005f,0x0000005d,0x0000005e,0x000200fe,0x0000005f,0x00010038,0x00050036,0x00000009,
0x0000000f,0x00000000,0x0000000a,0x00030037,0x00000008,0x0000000e,0x000200f8,0x00000010,
0x0004003b,0x00000008,0x00000062,0x00000007,0x0004003d,0x00000007,0x00000063,0x0000000e,
0x0003003e,0x00000062,0x00000063,0x00050039,0x00000009,0x00000064,0x0000000c,0x00000062,
0x00050041,0x00000027,0x00000066,0x0000000e,0x00000054,0x0004003d,0x00000006,0x00000067,
0x00000066,0x00050041,0x00000069,0x0000006a,0x00000068,0x00000054,0x0004003d,0x00000006,
0x0000006b,0x0000006a,0x00050082,0x00000006,0x0000006c,0x00000067,0x0000006b,0x0006000c,
0x00000006,0x0000006d,0x00000001,0x00000005,0x0000006c,0x00050041,0x00000027,0x0000006f,
0x0000000e,0x0000006e,0x0004003d,0x00000006,0x00000070,0x0000006f,0x00050041,0x00000069,
0x00000071,0x00000068,0x0000006e,0x0004003d,0x00000006,0x00000072,0x00000071,0x00050082,
0x00000006,0x00000073,0x00000070,0x00000072,0x0006000c,0x00000006,0x00000074,0x00000001,
0x00000005,0x00000073,0x00050080,0x00000006,0x00000075,0x0000006d,0x00000074,0x0004006f,
0x00000009,0x00000076,0x00000075,0x00050085,0x00000009,0x00000077,0x00000065,0x00000076,
0x00050081,0x00000009,0x00000078,0x00000064,0x00000077,0x000200fe,0x00000078,0x00010038,
0x00050036,0x00000007,0x00000015,0x00000000,0x00000013,0x00030037,0x00000012,0x00000014,
0x000200f8,0x00000016,0x0004003d,0x00000011,0x0000007b,0x00000014,0x0005008e,0x00000011,
0x0000007d,0x0000007b,0x0000007c,0x0006000c,0x00000011,0x0000007e,0x00000001,0x00000001,
0x0000007d,0x0004006e,0x00000007,0x0000007f,0x0000007e,0x000200fe,0x0000007f,0x00010038,
0x00050036,0x00000009,0x0000001c,0x00000000,0x00000018,0x00030037,0x00000017,0x00000019,
0x00030037,0x00000017,0x0000001a,0x00030037,0x00000017,0x0000001b,0x000200f8,0x0000001d,
0x0004003b,0x00000017,0x00000082,0x00000007,0x0004003b,0x00000017,0x0000008d,0x00000007,
0x0004003d,0x00000009,0x00000083,0x00000019,0x0004003d,0x00000009,0x00000084,0x0000001b,
0x00050081,0x00000009,0x00000085,0x00000083,0x00000084,0x0004003d,0x00000009,0x00000087,
0x0000001a,0x00050085,0x00000009,0x00000088,0x00000086,0x00000087,0x00050083,0x00000009,
0x00000089,0x00000085,0x00000088,0x0003003e,0x00000082,0x00000089,0x0004003d,0x00000009,
0x0000008a,0x00000082,0x000500ba,0x00000031,0x0000008c,0x0000008a,0x0000008b,0x000300f7,
0x0000008f,0x00000000,0x000400fa,0x0000008c,0x0000008e,0x00000098,0x000200f8,0x0000008e,
0x0004003d,0x00000009,0x00000090,0x00000019,0x0004003d,0x00000009,0x00000091,0x0000001b,
0x00050083,0x00000009,0x00000092,0x00000090,0x00000091,0x00050085,0x00000009,0x00000093,
0x0000007c,0x00000092,0x0004003d,0x00000009,0x00000094,0x00000082,0x00050088,0x00000009,
0x00000095,0x00000093,0x00000094,0x0008000c,0x00000009,0x00000097,0x00000001,0x0000002b,
0x00000095,0x00000096,0x0000007c,0x0003003e,0x0000008d,0x00000097,0x000200f9,0x0000008f,
0x000200f8,0x00000098,0x0003003e,0x0000008d,0x00000026,0x000200f9,0x0000008f,0x000200f8,
0x0000008f,0x0004003d,0x00000009,0x00000099,0x0000008d,0x000200fe,0x00000099,0x00010038,
0x00050036,0x00000009,0x0000001f,0x00000000,0x0000000a,0x00030037,0x00000008,0x0000001e,
0x000200f8,0x00000020,0x0004003d,0x0000003d,0x0000009d,0x0000009c,0x0004003d,0x00000007,
0x0000009e,0x0000001e,0x0004003d,0x00000007,0x000000a0,0x0000009f,0x0008000c,0x00000007,
0x000000a1,0x00000001,0x0000002d,0x0000009e,0x0000004d,0x000000a0,0x00040064,0x0000003c,
0x000000a2,0x0000009d,0x0007005f,0x00000052,0x000000a3,0x000000a2,0x000000a1,0x00000002,
0x00000029,0x0008004f,0x000000a4,0x000000a5,0x000000a3,0x000000a3,0x00000000,0x00000001,
0x00000002,0x00050094,0x00000009,0x000000aa,0x000000a5,0x000000a9,0x000200fe,0x000000aa,
0x00010038,0x00050036,0x00000009,0x00000023,0x00000000,0x00000021,0x00030037,0x00000012,
0x00000022,0x000200f8,0x00000024,0x0004003d,0x0000003d,0x000000ae,0x000000ad,0x0004003d,
0x00000011,0x000000af,0x00000022,0x0004003d,0x00000011,0x000000b2,0x000000b1,0x00050085,
0x00000011,0x000000b3,0x000000af,0x000000b2,0x00070058,0x00000052,0x000000b4,0x000000ae,
0x000000b3,0x00000002,0x00000026,0x0008004f,0x000000a4,0x000000b5,0x000000b4,0x000000b4,
0x00000000,0x00000001,0x00000002,0x00050094,0x00000009,0x000000b6,0x000000b5,0x000000a9,
0x000200fe,0x000000b6,0x00010038
};

View File

@@ -0,0 +1,55 @@
#version 450
// (Triforce) The fine vectors (fg_fine_flow.comp) settled among their neighbours, run twice: each block keeps whichever of
// its own vector and its 8 neighbours' matches its own pixels best (the squared difference of the frames' luma at 8x8
// points 1.5 pixels apart over the block and 2 pixels around it), a vector far from the neighbours' median paying a
// little more. A block that could not tell its motion -- a line alone in it, which matches as well moved along itself, a
// fence whose bars match one bar over -- takes its neighbours' where it fits as well, so that the vectors along a line or
// across a fence are one motion, not each block's own guess (a line moved by vectors that differ comes out bent).
layout(local_size_x = 8, local_size_y = 8) in;
layout(binding = 0) uniform sampler2D fa; // (the frame before, the linear sampler)
layout(binding = 1) uniform sampler2D fb; // (this frame)
layout(binding = 2) uniform sampler2D src; // (the vectors)
layout(binding = 3, rgba16f) uniform writeonly image2D dst;
float pa[64];
vec2 q0;
vec2 inv;
float luma(vec3 c) { return dot(c, vec3(0.299, 0.587, 0.114)); }
float ssd(vec2 v) {
float s = 0.0;
for (int i = 0; i < 64; i++) {
const float d = luma(textureLod(fb, (q0 + vec2(float(i % 8), float(i / 8)) * 1.5 + v) * inv, 0.0).rgb) - pa[i];
s += d * d;
}
return s / 64.0;
}
float med9(float v[9]) { // (as fg_median.comp's)
for (int i = 0; i < 5; i++)
for (int j = i + 1; j < 9; j++)
if (v[j] < v[i]) { const float t = v[i]; v[i] = v[j]; v[j] = t; }
return v[4];
}
void main() {
const ivec2 g = ivec2(gl_GlobalInvocationID.xy);
const ivec2 d = imageSize(dst);
if (any(greaterThanEqual(g, d))) return;
inv = 1.0 / vec2(textureSize(fa, 0));
q0 = vec2(g * 8) - 1.25;
for (int i = 0; i < 64; i++) pa[i] = luma(textureLod(fa, (q0 + vec2(float(i % 8), float(i / 8)) * 1.5) * inv, 0.0).rgb);
vec2 c[9];
float xs[9], ys[9];
for (int i = 0; i < 9; i++) {
c[i] = texelFetch(src, clamp(g + ivec2(i % 3 - 1, i / 3 - 1), ivec2(0), d - 1), 0).xy;
xs[i] = c[i].x, ys[i] = c[i].y;
}
const vec2 m = vec2(med9(xs), med9(ys));
vec2 best = c[4];
float least = 1e9;
for (int i = 0; i < 9; i++) {
const float s = ssd(c[i]) + 0.00002 * length(c[i] - m);
if (s < least) least = s, best = c[i];
}
imageStore(dst, g, vec4(best, texelFetch(src, g, 0).z, 1.0));
}

View File

@@ -0,0 +1,229 @@
// 1116.6.0
#pragma once
const uint32_t FG_FINE_PICK_COMP_SPV[] = {
0x07230203,0x00010500,0x0008000b,0x0000014b,0x00000000,0x00020011,0x00000001,0x00020011,
0x00000032,0x0006000b,0x00000001,0x4c534c47,0x6474732e,0x3035342e,0x00000000,0x0003000e,
0x00000000,0x00000001,0x000d000f,0x00000005,0x00000004,0x6e69616d,0x00000000,0x00000037,
0x0000003a,0x00000049,0x00000054,0x0000009e,0x000000a6,0x000000b2,0x000000e9,0x00060010,
0x00000004,0x00000011,0x00000008,0x00000008,0x00000001,0x00030003,0x00000002,0x000001c2,
0x00040005,0x00000004,0x6e69616d,0x00000000,0x00050005,0x0000000b,0x616d756c,0x33667628,
0x0000003b,0x00030005,0x0000000a,0x00000063,0x00050005,0x00000011,0x28647373,0x3b326676,
0x00000000,0x00030005,0x00000010,0x00000076,0x00050005,0x00000019,0x3964656d,0x5b316628,
0x003b5d39,0x00030005,0x00000018,0x00000076,0x00030005,0x00000024,0x00000073,0x00030005,
0x00000028,0x00000069,0x00030005,0x00000033,0x00000064,0x00030005,0x00000037,0x00006266,
0x00030005,0x0000003a,0x00003071,0x00030005,0x00000049,0x00766e69,0x00040005,0x0000004e,
0x61726170,0x0000006d,0x00030005,0x00000054,0x00006170,0x00030005,0x00000067,0x00000069,
0x00030005,0x00000070,0x0000006a,0x00030005,0x00000084,0x00000074,0x00030005,0x0000009b,
0x00000067,0x00080005,0x0000009e,0x475f6c67,0x61626f6c,0x766e496c,0x7461636f,0x496e6f69,
0x00000044,0x00030005,0x000000a3,0x00000064,0x00030005,0x000000a6,0x00747364,0x00030005,
0x000000b2,0x00006166,0x00030005,0x000000c0,0x00000069,0x00040005,0x000000d7,0x61726170,
0x0000006d,0x00030005,0x000000dd,0x00000069,0x00030005,0x000000e7,0x00000063,0x00030005,
0x000000e9,0x00637273,0x00030005,0x000000fe,0x00007378,0x00030005,0x00000105,0x00007379,
0x00030005,0x0000010e,0x0000006d,0x00040005,0x0000010f,0x61726170,0x0000006d,0x00040005,
0x00000112,0x61726170,0x0000006d,0x00040005,0x00000116,0x74736562,0x00000000,0x00040005,
0x00000119,0x7361656c,0x00000074,0x00030005,0x0000011b,0x00000069,0x00030005,0x00000123,
0x00000073,0x00040005,0x00000125,0x61726170,0x0000006d,0x00040047,0x00000037,0x00000021,
0x00000001,0x00040047,0x00000037,0x00000022,0x00000000,0x00040047,0x0000009e,0x0000000b,
0x0000001c,0x00030047,0x000000a6,0x00000019,0x00040047,0x000000a6,0x00000021,0x00000003,
0x00040047,0x000000a6,0x00000022,0x00000000,0x00040047,0x000000b2,0x00000021,0x00000000,
0x00040047,0x000000b2,0x00000022,0x00000000,0x00040047,0x000000e9,0x00000021,0x00000002,
0x00040047,0x000000e9,0x00000022,0x00000000,0x00040047,0x0000014a,0x0000000b,0x00000019,
0x00020013,0x00000002,0x00030021,0x00000003,0x00000002,0x00030016,0x00000006,0x00000020,
0x00040017,0x00000007,0x00000006,0x00000003,0x00040020,0x00000008,0x00000007,0x00000007,
0x00040021,0x00000009,0x00000006,0x00000008,0x00040017,0x0000000d,0x00000006,0x00000002,
0x00040020,0x0000000e,0x00000007,0x0000000d,0x00040021,0x0000000f,0x00000006,0x0000000e,
0x00040015,0x00000013,0x00000020,0x00000000,0x0004002b,0x00000013,0x00000014,0x00000009,
0x0004001c,0x00000015,0x00000006,0x00000014,0x00040020,0x00000016,0x00000007,0x00000015,
0x00040021,0x00000017,0x00000006,0x00000016,0x0004002b,0x00000006,0x0000001c,0x3e991687,
0x0004002b,0x00000006,0x0000001d,0x3f1645a2,0x0004002b,0x00000006,0x0000001e,0x3de978d5,
0x0006002c,0x00000007,0x0000001f,0x0000001c,0x0000001d,0x0000001e,0x00040020,0x00000023,
0x00000007,0x00000006,0x0004002b,0x00000006,0x00000025,0x00000000,0x00040015,0x00000026,
0x00000020,0x00000001,0x00040020,0x00000027,0x00000007,0x00000026,0x0004002b,0x00000026,
0x00000029,0x00000000,0x0004002b,0x00000026,0x00000030,0x00000040,0x00020014,0x00000031,
0x00090019,0x00000034,0x00000006,0x00000001,0x00000000,0x00000000,0x00000000,0x00000001,
0x00000000,0x0003001b,0x00000035,0x00000034,0x00040020,0x00000036,0x00000000,0x00000035,
0x0004003b,0x00000036,0x00000037,0x00000000,0x00040020,0x00000039,0x00000006,0x0000000d,
0x0004003b,0x00000039,0x0000003a,0x00000006,0x0004002b,0x00000026,0x0000003d,0x00000008,
0x0004002b,0x00000006,0x00000044,0x3fc00000,0x0004003b,0x00000039,0x00000049,0x00000006,
0x00040017,0x0000004c,0x00000006,0x00000004,0x0004002b,0x00000013,0x00000051,0x00000040,
0x0004001c,0x00000052,0x00000006,0x00000051,0x00040020,0x00000053,0x00000006,0x00000052,
0x0004003b,0x00000053,0x00000054,0x00000006,0x00040020,0x00000056,0x00000006,0x00000006,
0x0004002b,0x00000026,0x00000060,0x00000001,0x0004002b,0x00000006,0x00000063,0x42800000,
0x0004002b,0x00000026,0x0000006e,0x00000005,0x0004002b,0x00000026,0x00000079,0x00000009,
0x0004002b,0x00000026,0x00000094,0x00000004,0x00040017,0x00000099,0x00000026,0x00000002,
0x00040020,0x0000009a,0x00000007,0x00000099,0x00040017,0x0000009c,0x00000013,0x00000003,
0x00040020,0x0000009d,0x00000001,0x0000009c,0x0004003b,0x0000009d,0x0000009e,0x00000001,
0x00040017,0x0000009f,0x00000013,0x00000002,0x00090019,0x000000a4,0x00000006,0x00000001,
0x00000000,0x00000000,0x00000000,0x00000002,0x00000002,0x00040020,0x000000a5,0x00000000,
0x000000a4,0x0004003b,0x000000a5,0x000000a6,0x00000000,0x00040017,0x000000ab,0x00000031,
0x00000002,0x0004002b,0x00000006,0x000000b1,0x3f800000,0x0004003b,0x00000036,0x000000b2,
0x00000000,0x0004002b,0x00000006,0x000000bd,0x3fa00000,0x0004001c,0x000000e5,0x0000000d,
0x00000014,0x00040020,0x000000e6,0x00000007,0x000000e5,0x0004003b,0x00000036,0x000000e9,
0x00000000,0x0004002b,0x00000026,0x000000ed,0x00000003,0x0005002c,0x00000099,0x000000f5,
0x00000029,0x00000029,0x0004002b,0x00000013,0x00000101,0x00000000,0x0004002b,0x00000013,
0x00000108,0x00000001,0x0004002b,0x00000006,0x0000011a,0x4e6e6b28,0x0004002b,0x00000006,
0x00000129,0x37a7c5ac,0x0004002b,0x00000013,0x00000144,0x00000002,0x0004002b,0x00000013,
0x00000149,0x00000008,0x0006002c,0x0000009c,0x0000014a,0x00000149,0x00000149,0x00000108,
0x00050036,0x00000002,0x00000004,0x00000000,0x00000003,0x000200f8,0x00000005,0x0004003b,
0x0000009a,0x0000009b,0x00000007,0x0004003b,0x0000009a,0x000000a3,0x00000007,0x0004003b,
0x00000027,0x000000c0,0x00000007,0x0004003b,0x00000008,0x000000d7,0x00000007,0x0004003b,
0x00000027,0x000000dd,0x00000007,0x0004003b,0x000000e6,0x000000e7,0x00000007,0x0004003b,
0x00000016,0x000000fe,0x00000007,0x0004003b,0x00000016,0x00000105,0x00000007,0x0004003b,
0x0000000e,0x0000010e,0x00000007,0x0004003b,0x00000016,0x0000010f,0x00000007,0x0004003b,
0x00000016,0x00000112,0x00000007,0x0004003b,0x0000000e,0x00000116,0x00000007,0x0004003b,
0x00000023,0x00000119,0x00000007,0x0004003b,0x00000027,0x0000011b,0x00000007,0x0004003b,
0x00000023,0x00000123,0x00000007,0x0004003b,0x0000000e,0x00000125,0x00000007,0x0004003d,
0x0000009c,0x000000a0,0x0000009e,0x0007004f,0x0000009f,0x000000a1,0x000000a0,0x000000a0,
0x00000000,0x00000001,0x0004007c,0x00000099,0x000000a2,0x000000a1,0x0003003e,0x0000009b,
0x000000a2,0x0004003d,0x000000a4,0x000000a7,0x000000a6,0x00040068,0x00000099,0x000000a8,
0x000000a7,0x0003003e,0x000000a3,0x000000a8,0x0004003d,0x00000099,0x000000a9,0x0000009b,
0x0004003d,0x00000099,0x000000aa,0x000000a3,0x000500af,0x000000ab,0x000000ac,0x000000a9,
0x000000aa,0x0004009a,0x00000031,0x000000ad,0x000000ac,0x000300f7,0x000000af,0x00000000,
0x000400fa,0x000000ad,0x000000ae,0x000000af,0x000200f8,0x000000ae,0x000100fd,0x000200f8,
0x000000af,0x0004003d,0x00000035,0x000000b3,0x000000b2,0x00040064,0x00000034,0x000000b4,
0x000000b3,0x00050067,0x00000099,0x000000b5,0x000000b4,0x00000029,0x0004006f,0x0000000d,
0x000000b6,0x000000b5,0x00050050,0x0000000d,0x000000b7,0x000000b1,0x000000b1,0x00050088,
0x0000000d,0x000000b8,0x000000b7,0x000000b6,0x0003003e,0x00000049,0x000000b8,0x0004003d,
0x00000099,0x000000b9,0x0000009b,0x00050050,0x00000099,0x000000ba,0x0000003d,0x0000003d,
0x00050084,0x00000099,0x000000bb,0x000000b9,0x000000ba,0x0004006f,0x0000000d,0x000000bc,
0x000000bb,0x00050050,0x0000000d,0x000000be,0x000000bd,0x000000bd,0x00050083,0x0000000d,
0x000000bf,0x000000bc,0x000000be,0x0003003e,0x0000003a,0x000000bf,0x0003003e,0x000000c0,
0x00000029,0x000200f9,0x000000c1,0x000200f8,0x000000c1,0x000400f6,0x000000c3,0x000000c4,
0x00000000,0x000200f9,0x000000c5,0x000200f8,0x000000c5,0x0004003d,0x00000026,0x000000c6,
0x000000c0,0x000500b1,0x00000031,0x000000c7,0x000000c6,0x00000030,0x000400fa,0x000000c7,
0x000000c2,0x000000c3,0x000200f8,0x000000c2,0x0004003d,0x00000026,0x000000c8,0x000000c0,
0x0004003d,0x00000035,0x000000c9,0x000000b2,0x0004003d,0x0000000d,0x000000ca,0x0000003a,
0x0004003d,0x00000026,0x000000cb,0x000000c0,0x0005008b,0x00000026,0x000000cc,0x000000cb,
0x0000003d,0x0004006f,0x00000006,0x000000cd,0x000000cc,0x0004003d,0x00000026,0x000000ce,
0x000000c0,0x00050087,0x00000026,0x000000cf,0x000000ce,0x0000003d,0x0004006f,0x00000006,
0x000000d0,0x000000cf,0x00050050,0x0000000d,0x000000d1,0x000000cd,0x000000d0,0x0005008e,
0x0000000d,0x000000d2,0x000000d1,0x00000044,0x00050081,0x0000000d,0x000000d3,0x000000ca,
0x000000d2,0x0004003d,0x0000000d,0x000000d4,0x00000049,0x00050085,0x0000000d,0x000000d5,
0x000000d3,0x000000d4,0x00070058,0x0000004c,0x000000d6,0x000000c9,0x000000d5,0x00000002,
0x00000025,0x0008004f,0x00000007,0x000000d8,0x000000d6,0x000000d6,0x00000000,0x00000001,
0x00000002,0x0003003e,0x000000d7,0x000000d8,0x00050039,0x00000006,0x000000d9,0x0000000b,
0x000000d7,0x00050041,0x00000056,0x000000da,0x00000054,0x000000c8,0x0003003e,0x000000da,
0x000000d9,0x000200f9,0x000000c4,0x000200f8,0x000000c4,0x0004003d,0x00000026,0x000000db,
0x000000c0,0x00050080,0x00000026,0x000000dc,0x000000db,0x00000060,0x0003003e,0x000000c0,
0x000000dc,0x000200f9,0x000000c1,0x000200f8,0x000000c3,0x0003003e,0x000000dd,0x00000029,
0x000200f9,0x000000de,0x000200f8,0x000000de,0x000400f6,0x000000e0,0x000000e1,0x00000000,
0x000200f9,0x000000e2,0x000200f8,0x000000e2,0x0004003d,0x00000026,0x000000e3,0x000000dd,
0x000500b1,0x00000031,0x000000e4,0x000000e3,0x00000079,0x000400fa,0x000000e4,0x000000df,
0x000000e0,0x000200f8,0x000000df,0x0004003d,0x00000026,0x000000e8,0x000000dd,0x0004003d,
0x00000035,0x000000ea,0x000000e9,0x0004003d,0x00000099,0x000000eb,0x0000009b,0x0004003d,
0x00000026,0x000000ec,0x000000dd,0x0005008b,0x00000026,0x000000ee,0x000000ec,0x000000ed,
0x00050082,0x00000026,0x000000ef,0x000000ee,0x00000060,0x0004003d,0x00000026,0x000000f0,
0x000000dd,0x00050087,0x00000026,0x000000f1,0x000000f0,0x000000ed,0x00050082,0x00000026,
0x000000f2,0x000000f1,0x00000060,0x00050050,0x00000099,0x000000f3,0x000000ef,0x000000f2,
0x00050080,0x00000099,0x000000f4,0x000000eb,0x000000f3,0x0004003d,0x00000099,0x000000f6,
0x000000a3,0x00050050,0x00000099,0x000000f7,0x00000060,0x00000060,0x00050082,0x00000099,
0x000000f8,0x000000f6,0x000000f7,0x0008000c,0x00000099,0x000000f9,0x00000001,0x0000002d,
0x000000f4,0x000000f5,0x000000f8,0x00040064,0x00000034,0x000000fa,0x000000ea,0x0007005f,
0x0000004c,0x000000fb,0x000000fa,0x000000f9,0x00000002,0x00000029,0x0007004f,0x0000000d,
0x000000fc,0x000000fb,0x000000fb,0x00000000,0x00000001,0x00050041,0x0000000e,0x000000fd,
0x000000e7,0x000000e8,0x0003003e,0x000000fd,0x000000fc,0x0004003d,0x00000026,0x000000ff,
0x000000dd,0x0004003d,0x00000026,0x00000100,0x000000dd,0x00060041,0x00000023,0x00000102,
0x000000e7,0x00000100,0x00000101,0x0004003d,0x00000006,0x00000103,0x00000102,0x00050041,
0x00000023,0x00000104,0x000000fe,0x000000ff,0x0003003e,0x00000104,0x00000103,0x0004003d,
0x00000026,0x00000106,0x000000dd,0x0004003d,0x00000026,0x00000107,0x000000dd,0x00060041,
0x00000023,0x00000109,0x000000e7,0x00000107,0x00000108,0x0004003d,0x00000006,0x0000010a,
0x00000109,0x00050041,0x00000023,0x0000010b,0x00000105,0x00000106,0x0003003e,0x0000010b,
0x0000010a,0x000200f9,0x000000e1,0x000200f8,0x000000e1,0x0004003d,0x00000026,0x0000010c,
0x000000dd,0x00050080,0x00000026,0x0000010d,0x0000010c,0x00000060,0x0003003e,0x000000dd,
0x0000010d,0x000200f9,0x000000de,0x000200f8,0x000000e0,0x0004003d,0x00000015,0x00000110,
0x000000fe,0x0003003e,0x0000010f,0x00000110,0x00050039,0x00000006,0x00000111,0x00000019,
0x0000010f,0x0004003d,0x00000015,0x00000113,0x00000105,0x0003003e,0x00000112,0x00000113,
0x00050039,0x00000006,0x00000114,0x00000019,0x00000112,0x00050050,0x0000000d,0x00000115,
0x00000111,0x00000114,0x0003003e,0x0000010e,0x00000115,0x00050041,0x0000000e,0x00000117,
0x000000e7,0x00000094,0x0004003d,0x0000000d,0x00000118,0x00000117,0x0003003e,0x00000116,
0x00000118,0x0003003e,0x00000119,0x0000011a,0x0003003e,0x0000011b,0x00000029,0x000200f9,
0x0000011c,0x000200f8,0x0000011c,0x000400f6,0x0000011e,0x0000011f,0x00000000,0x000200f9,
0x00000120,0x000200f8,0x00000120,0x0004003d,0x00000026,0x00000121,0x0000011b,0x000500b1,
0x00000031,0x00000122,0x00000121,0x00000079,0x000400fa,0x00000122,0x0000011d,0x0000011e,
0x000200f8,0x0000011d,0x0004003d,0x00000026,0x00000124,0x0000011b,0x00050041,0x0000000e,
0x00000126,0x000000e7,0x00000124,0x0004003d,0x0000000d,0x00000127,0x00000126,0x0003003e,
0x00000125,0x00000127,0x00050039,0x00000006,0x00000128,0x00000011,0x00000125,0x0004003d,
0x00000026,0x0000012a,0x0000011b,0x00050041,0x0000000e,0x0000012b,0x000000e7,0x0000012a,
0x0004003d,0x0000000d,0x0000012c,0x0000012b,0x0004003d,0x0000000d,0x0000012d,0x0000010e,
0x00050083,0x0000000d,0x0000012e,0x0000012c,0x0000012d,0x0006000c,0x00000006,0x0000012f,
0x00000001,0x00000042,0x0000012e,0x00050085,0x00000006,0x00000130,0x00000129,0x0000012f,
0x00050081,0x00000006,0x00000131,0x00000128,0x00000130,0x0003003e,0x00000123,0x00000131,
0x0004003d,0x00000006,0x00000132,0x00000123,0x0004003d,0x00000006,0x00000133,0x00000119,
0x000500b8,0x00000031,0x00000134,0x00000132,0x00000133,0x000300f7,0x00000136,0x00000000,
0x000400fa,0x00000134,0x00000135,0x00000136,0x000200f8,0x00000135,0x0004003d,0x00000006,
0x00000137,0x00000123,0x0003003e,0x00000119,0x00000137,0x0004003d,0x00000026,0x00000138,
0x0000011b,0x00050041,0x0000000e,0x00000139,0x000000e7,0x00000138,0x0004003d,0x0000000d,
0x0000013a,0x00000139,0x0003003e,0x00000116,0x0000013a,0x000200f9,0x00000136,0x000200f8,
0x00000136,0x000200f9,0x0000011f,0x000200f8,0x0000011f,0x0004003d,0x00000026,0x0000013b,
0x0000011b,0x00050080,0x00000026,0x0000013c,0x0000013b,0x00000060,0x0003003e,0x0000011b,
0x0000013c,0x000200f9,0x0000011c,0x000200f8,0x0000011e,0x0004003d,0x000000a4,0x0000013d,
0x000000a6,0x0004003d,0x00000099,0x0000013e,0x0000009b,0x0004003d,0x0000000d,0x0000013f,
0x00000116,0x0004003d,0x00000035,0x00000140,0x000000e9,0x0004003d,0x00000099,0x00000141,
0x0000009b,0x00040064,0x00000034,0x00000142,0x00000140,0x0007005f,0x0000004c,0x00000143,
0x00000142,0x00000141,0x00000002,0x00000029,0x00050051,0x00000006,0x00000145,0x00000143,
0x00000002,0x00050051,0x00000006,0x00000146,0x0000013f,0x00000000,0x00050051,0x00000006,
0x00000147,0x0000013f,0x00000001,0x00070050,0x0000004c,0x00000148,0x00000146,0x00000147,
0x00000145,0x000000b1,0x00040063,0x0000013d,0x0000013e,0x00000148,0x000100fd,0x00010038,
0x00050036,0x00000006,0x0000000b,0x00000000,0x00000009,0x00030037,0x00000008,0x0000000a,
0x000200f8,0x0000000c,0x0004003d,0x00000007,0x0000001b,0x0000000a,0x00050094,0x00000006,
0x00000020,0x0000001b,0x0000001f,0x000200fe,0x00000020,0x00010038,0x00050036,0x00000006,
0x00000011,0x00000000,0x0000000f,0x00030037,0x0000000e,0x00000010,0x000200f8,0x00000012,
0x0004003b,0x00000023,0x00000024,0x00000007,0x0004003b,0x00000027,0x00000028,0x00000007,
0x0004003b,0x00000023,0x00000033,0x00000007,0x0004003b,0x00000008,0x0000004e,0x00000007,
0x0003003e,0x00000024,0x00000025,0x0003003e,0x00000028,0x00000029,0x000200f9,0x0000002a,
0x000200f8,0x0000002a,0x000400f6,0x0000002c,0x0000002d,0x00000000,0x000200f9,0x0000002e,
0x000200f8,0x0000002e,0x0004003d,0x00000026,0x0000002f,0x00000028,0x000500b1,0x00000031,
0x00000032,0x0000002f,0x00000030,0x000400fa,0x00000032,0x0000002b,0x0000002c,0x000200f8,
0x0000002b,0x0004003d,0x00000035,0x00000038,0x00000037,0x0004003d,0x0000000d,0x0000003b,
0x0000003a,0x0004003d,0x00000026,0x0000003c,0x00000028,0x0005008b,0x00000026,0x0000003e,
0x0000003c,0x0000003d,0x0004006f,0x00000006,0x0000003f,0x0000003e,0x0004003d,0x00000026,
0x00000040,0x00000028,0x00050087,0x00000026,0x00000041,0x00000040,0x0000003d,0x0004006f,
0x00000006,0x00000042,0x00000041,0x00050050,0x0000000d,0x00000043,0x0000003f,0x00000042,
0x0005008e,0x0000000d,0x00000045,0x00000043,0x00000044,0x00050081,0x0000000d,0x00000046,
0x0000003b,0x00000045,0x0004003d,0x0000000d,0x00000047,0x00000010,0x00050081,0x0000000d,
0x00000048,0x00000046,0x00000047,0x0004003d,0x0000000d,0x0000004a,0x00000049,0x00050085,
0x0000000d,0x0000004b,0x00000048,0x0000004a,0x00070058,0x0000004c,0x0000004d,0x00000038,
0x0000004b,0x00000002,0x00000025,0x0008004f,0x00000007,0x0000004f,0x0000004d,0x0000004d,
0x00000000,0x00000001,0x00000002,0x0003003e,0x0000004e,0x0000004f,0x00050039,0x00000006,
0x00000050,0x0000000b,0x0000004e,0x0004003d,0x00000026,0x00000055,0x00000028,0x00050041,
0x00000056,0x00000057,0x00000054,0x00000055,0x0004003d,0x00000006,0x00000058,0x00000057,
0x00050083,0x00000006,0x00000059,0x00000050,0x00000058,0x0003003e,0x00000033,0x00000059,
0x0004003d,0x00000006,0x0000005a,0x00000033,0x0004003d,0x00000006,0x0000005b,0x00000033,
0x00050085,0x00000006,0x0000005c,0x0000005a,0x0000005b,0x0004003d,0x00000006,0x0000005d,
0x00000024,0x00050081,0x00000006,0x0000005e,0x0000005d,0x0000005c,0x0003003e,0x00000024,
0x0000005e,0x000200f9,0x0000002d,0x000200f8,0x0000002d,0x0004003d,0x00000026,0x0000005f,
0x00000028,0x00050080,0x00000026,0x00000061,0x0000005f,0x00000060,0x0003003e,0x00000028,
0x00000061,0x000200f9,0x0000002a,0x000200f8,0x0000002c,0x0004003d,0x00000006,0x00000062,
0x00000024,0x00050088,0x00000006,0x00000064,0x00000062,0x00000063,0x000200fe,0x00000064,
0x00010038,0x00050036,0x00000006,0x00000019,0x00000000,0x00000017,0x00030037,0x00000016,
0x00000018,0x000200f8,0x0000001a,0x0004003b,0x00000027,0x00000067,0x00000007,0x0004003b,
0x00000027,0x00000070,0x00000007,0x0004003b,0x00000023,0x00000084,0x00000007,0x0003003e,
0x00000067,0x00000029,0x000200f9,0x00000068,0x000200f8,0x00000068,0x000400f6,0x0000006a,
0x0000006b,0x00000000,0x000200f9,0x0000006c,0x000200f8,0x0000006c,0x0004003d,0x00000026,
0x0000006d,0x00000067,0x000500b1,0x00000031,0x0000006f,0x0000006d,0x0000006e,0x000400fa,
0x0000006f,0x00000069,0x0000006a,0x000200f8,0x00000069,0x0004003d,0x00000026,0x00000071,
0x00000067,0x00050080,0x00000026,0x00000072,0x00000071,0x00000060,0x0003003e,0x00000070,
0x00000072,0x000200f9,0x00000073,0x000200f8,0x00000073,0x000400f6,0x00000075,0x00000076,
0x00000000,0x000200f9,0x00000077,0x000200f8,0x00000077,0x0004003d,0x00000026,0x00000078,
0x00000070,0x000500b1,0x00000031,0x0000007a,0x00000078,0x00000079,0x000400fa,0x0000007a,
0x00000074,0x00000075,0x000200f8,0x00000074,0x0004003d,0x00000026,0x0000007b,0x00000070,
0x00050041,0x00000023,0x0000007c,0x00000018,0x0000007b,0x0004003d,0x00000006,0x0000007d,
0x0000007c,0x0004003d,0x00000026,0x0000007e,0x00000067,0x00050041,0x00000023,0x0000007f,
0x00000018,0x0000007e,0x0004003d,0x00000006,0x00000080,0x0000007f,0x000500b8,0x00000031,
0x00000081,0x0000007d,0x00000080,0x000300f7,0x00000083,0x00000000,0x000400fa,0x00000081,
0x00000082,0x00000083,0x000200f8,0x00000082,0x0004003d,0x00000026,0x00000085,0x00000067,
0x00050041,0x00000023,0x00000086,0x00000018,0x00000085,0x0004003d,0x00000006,0x00000087,
0x00000086,0x0003003e,0x00000084,0x00000087,0x0004003d,0x00000026,0x00000088,0x00000067,
0x0004003d,0x00000026,0x00000089,0x00000070,0x00050041,0x00000023,0x0000008a,0x00000018,
0x00000089,0x0004003d,0x00000006,0x0000008b,0x0000008a,0x00050041,0x00000023,0x0000008c,
0x00000018,0x00000088,0x0003003e,0x0000008c,0x0000008b,0x0004003d,0x00000026,0x0000008d,
0x00000070,0x0004003d,0x00000006,0x0000008e,0x00000084,0x00050041,0x00000023,0x0000008f,
0x00000018,0x0000008d,0x0003003e,0x0000008f,0x0000008e,0x000200f9,0x00000083,0x000200f8,
0x00000083,0x000200f9,0x00000076,0x000200f8,0x00000076,0x0004003d,0x00000026,0x00000090,
0x00000070,0x00050080,0x00000026,0x00000091,0x00000090,0x00000060,0x0003003e,0x00000070,
0x00000091,0x000200f9,0x00000073,0x000200f8,0x00000075,0x000200f9,0x0000006b,0x000200f8,
0x0000006b,0x0004003d,0x00000026,0x00000092,0x00000067,0x00050080,0x00000026,0x00000093,
0x00000092,0x00000060,0x0003003e,0x00000067,0x00000093,0x000200f9,0x00000068,0x000200f8,
0x0000006a,0x00050041,0x00000023,0x00000095,0x00000018,0x00000094,0x0004003d,0x00000006,
0x00000096,0x00000095,0x000200fe,0x00000096,0x00010038
};

View File

@@ -0,0 +1,82 @@
#version 450
// (Triforce) An in-between frame at time t from the fine vectors (fg_fine_flow.comp, fg_fine_pick.comp; vk_framegen.cpp,
// RX_VK_FRAMEGEN_FINE, on by default). Each pixel takes the frame before back along t of a motion and this frame forward
// along 1 - t, blended by t, the motion chosen per pixel by how well those two agree on it and its four neighbours:
// - the vectors' own, between the four blocks around (a vector is where the frame before's pixel went, so it is looked
// up again where the pixel came from: a moving edge's vectors end where the edge is at t, not where it was);
// - where the four blocks' vectors differ (an object's edge, a block that is wrong), each of the four as it is, taken
// only when it agrees clearly better: one motion a pixel, never a blend of two, which bent a straight edge into a ramp;
// - none (the HUD, anything still on the screen): only when it agrees clearly better too, and the pixel is the same in
// both frames or a block around holds still. fg_synth.comp took a pixel alike in both frames as still: a thin line
// moving over a plain surface is not at its place of t in either frame, so it was not drawn at all between the game's.
// Where no motion agrees (what is uncovered or covered, a cut, a flash), the nearer of the two frames as it is, as
// fg_synth.comp does: a picture that is one of the game's, not one made of two that do not belong together. Where the
// vectors around all hold still, the plain blend, which is either frame.
layout(local_size_x = 8, local_size_y = 8) in;
layout(binding = 0) uniform sampler2D fa; // (the frame before, the linear sampler)
layout(binding = 1) uniform sampler2D fb; // (this frame)
layout(binding = 2) uniform sampler2D fine; // (the vectors, pixels, one per 8x8)
layout(binding = 3, rgba8) uniform writeonly image2D dst;
layout(push_constant) uniform P { float t; } pc;
ivec2 fm;
vec2 inv;
float t;
vec2 vat(ivec2 q) { return texelFetch(fine, clamp(q, ivec2(0), fm), 0).xy; }
float gap(vec3 x, vec3 y) { const vec3 d = abs(x - y); return max(d.r, max(d.g, d.b)); }
vec3 at_a(vec2 p) { return textureLod(fa, p * inv, 0.0).rgb; }
vec3 at_b(vec2 p) { return textureLod(fb, p * inv, 0.0).rgb; }
float miss(vec2 p, vec2 v) { // (how far the two frames moved by v disagree at p and its four neighbours)
const vec2 pa = p - t * v, pb = p + (1.0 - t) * v;
float e = gap(at_a(pa), at_b(pb));
e += gap(at_a(pa + vec2(1.0, 0.0)), at_b(pb + vec2(1.0, 0.0)));
e += gap(at_a(pa - vec2(1.0, 0.0)), at_b(pb - vec2(1.0, 0.0)));
e += gap(at_a(pa + vec2(0.0, 1.0)), at_b(pb + vec2(0.0, 1.0)));
e += gap(at_a(pa - vec2(0.0, 1.0)), at_b(pb - vec2(0.0, 1.0)));
return e / 5.0;
}
float far(vec2 a, vec2 b) { return max(abs(a.x - b.x), abs(a.y - b.y)); }
void main() {
const ivec2 g = ivec2(gl_GlobalInvocationID.xy);
const ivec2 d = imageSize(dst);
if (any(greaterThanEqual(g, d))) return;
t = pc.t;
fm = textureSize(fine, 0) - 1;
inv = 1.0 / vec2(d);
const vec2 p = vec2(g) + 0.5;
vec2 fp = p / 8.0 - 0.5;
ivec2 i = ivec2(floor(fp));
vec2 w = fp - floor(fp);
const vec2 v1 = mix(mix(vat(i), vat(i + ivec2(1, 0)), w.x), mix(vat(i + ivec2(0, 1)), vat(i + ivec2(1, 1)), w.x), w.y);
fp = (p - t * v1) / 8.0 - 0.5;
i = ivec2(floor(fp));
w = fp - floor(fp);
const vec2 c00 = vat(i), c10 = vat(i + ivec2(1, 0)), c01 = vat(i + ivec2(0, 1)), c11 = vat(i + ivec2(1, 1));
const vec2 v = mix(mix(c00, c10, w.x), mix(c01, c11, w.x), w.y);
const vec3 a0 = texelFetch(fa, g, 0).rgb, b0 = texelFetch(fb, g, 0).rgb;
const vec2 zero = vec2(0.0);
if (max(max(far(c00, zero), far(c10, zero)), max(far(c01, zero), far(c11, zero))) < 0.125) {
imageStore(dst, g, vec4(mix(a0, b0, t), 1.0));
return;
}
vec2 best = v;
float eb = miss(p, v);
if (max(max(far(c00, v), far(c10, v)), max(far(c01, v), far(c11, v))) > 0.5) {
float e = miss(p, c00);
if (e < eb - 0.01) best = c00, eb = e;
e = miss(p, c10);
if (e < eb - 0.01) best = c10, eb = e;
e = miss(p, c01);
if (e < eb - 0.01) best = c01, eb = e;
e = miss(p, c11);
if (e < eb - 0.01) best = c11, eb = e;
}
const float e0 = miss(p, zero);
const float still = min(min(far(c00, zero), far(c10, zero)), min(far(c01, zero), far(c11, zero)));
if (e0 < eb - 0.01 && (e0 < 0.004 || still < 1.0)) best = zero, eb = e0;
const vec3 a = at_a(p - t * best), b = at_b(p + (1.0 - t) * best);
const float lost = smoothstep(0.06, 0.18, eb);
imageStore(dst, g, vec4(mix(mix(a, b, t), t <= 0.5 ? a0 : b0, lost), 1.0));
}

View File

@@ -0,0 +1,435 @@
// 1116.6.0
#pragma once
const uint32_t FG_FINE_SYNTH_COMP_SPV[] = {
0x07230203,0x00010500,0x0008000b,0x00000232,0x00000000,0x00020011,0x00000001,0x00020011,
0x00000032,0x0006000b,0x00000001,0x4c534c47,0x6474732e,0x3035342e,0x00000000,0x0003000e,
0x00000000,0x00000001,0x000e000f,0x00000005,0x00000004,0x6e69616d,0x00000000,0x0000002a,
0x00000030,0x0000004d,0x00000051,0x00000059,0x00000065,0x000000c9,0x000000d1,0x000000df,
0x00060010,0x00000004,0x00000011,0x00000008,0x00000008,0x00000001,0x00030003,0x00000002,
0x000001c2,0x00040005,0x00000004,0x6e69616d,0x00000000,0x00050005,0x0000000d,0x28746176,
0x3b326976,0x00000000,0x00030005,0x0000000c,0x00000071,0x00060005,0x00000014,0x28706167,
0x3b336676,0x3b336676,0x00000000,0x00030005,0x00000012,0x00000078,0x00030005,0x00000013,
0x00000079,0x00050005,0x00000019,0x615f7461,0x32667628,0x0000003b,0x00030005,0x00000018,
0x00000070,0x00050005,0x0000001c,0x625f7461,0x32667628,0x0000003b,0x00030005,0x0000001b,
0x00000070,0x00060005,0x00000021,0x7373696d,0x32667628,0x3266763b,0x0000003b,0x00030005,
0x0000001f,0x00000070,0x00030005,0x00000020,0x00000076,0x00060005,0x00000025,0x28726166,
0x3b326676,0x3b326676,0x00000000,0x00030005,0x00000023,0x00000061,0x00030005,0x00000024,
0x00000062,0x00040005,0x0000002a,0x656e6966,0x00000000,0x00030005,0x00000030,0x00006d66,
0x00030005,0x00000039,0x00000064,0x00030005,0x0000004d,0x00006166,0x00030005,0x00000051,
0x00766e69,0x00030005,0x00000059,0x00006266,0x00030005,0x00000062,0x00006170,0x00030005,
0x00000065,0x00000074,0x00030005,0x0000006a,0x00006270,0x00030005,0x00000072,0x00000065,
0x00040005,0x00000073,0x61726170,0x0000006d,0x00040005,0x00000076,0x61726170,0x0000006d,
0x00040005,0x00000079,0x61726170,0x0000006d,0x00040005,0x0000007a,0x61726170,0x0000006d,
0x00040005,0x0000007f,0x61726170,0x0000006d,0x00040005,0x00000083,0x61726170,0x0000006d,
0x00040005,0x00000085,0x61726170,0x0000006d,0x00040005,0x00000086,0x61726170,0x0000006d,
0x00040005,0x0000008c,0x61726170,0x0000006d,0x00040005,0x00000090,0x61726170,0x0000006d,
0x00040005,0x00000092,0x61726170,0x0000006d,0x00040005,0x00000093,0x61726170,0x0000006d,
0x00040005,0x0000009a,0x61726170,0x0000006d,0x00040005,0x0000009e,0x61726170,0x0000006d,
0x00040005,0x000000a0,0x61726170,0x0000006d,0x00040005,0x000000a1,0x61726170,0x0000006d,
0x00040005,0x000000a7,0x61726170,0x0000006d,0x00040005,0x000000ab,0x61726170,0x0000006d,
0x00040005,0x000000ad,0x61726170,0x0000006d,0x00040005,0x000000ae,0x61726170,0x0000006d,
0x00030005,0x000000c6,0x00000067,0x00080005,0x000000c9,0x475f6c67,0x61626f6c,0x766e496c,
0x7461636f,0x496e6f69,0x00000044,0x00030005,0x000000ce,0x00000064,0x00030005,0x000000d1,
0x00747364,0x00030005,0x000000dd,0x00000050,0x00040006,0x000000dd,0x00000000,0x00000074,
0x00030005,0x000000df,0x00006370,0x00030005,0x000000ed,0x00000070,0x00030005,0x000000f3,
0x00007066,0x00030005,0x000000fa,0x00000069,0x00030005,0x000000fe,0x00000077,0x00030005,
0x00000103,0x00003176,0x00040005,0x00000104,0x61726170,0x0000006d,0x00040005,0x0000010a,
0x61726170,0x0000006d,0x00040005,0x00000113,0x61726170,0x0000006d,0x00040005,0x00000118,
0x61726170,0x0000006d,0x00030005,0x00000132,0x00303063,0x00040005,0x00000133,0x61726170,
0x0000006d,0x00030005,0x00000136,0x00303163,0x00040005,0x00000139,0x61726170,0x0000006d,
0x00030005,0x0000013b,0x00313063,0x00040005,0x0000013e,0x61726170,0x0000006d,0x00030005,
0x00000140,0x00313163,0x00040005,0x00000143,0x61726170,0x0000006d,0x00030005,0x00000145,
0x00000076,0x00030005,0x00000156,0x00003061,0x00030005,0x0000015c,0x00003062,0x00040005,
0x00000163,0x61726170,0x0000006d,0x00040005,0x00000165,0x61726170,0x0000006d,0x00040005,
0x00000167,0x61726170,0x0000006d,0x00040005,0x00000169,0x61726170,0x0000006d,0x00040005,
0x0000016c,0x61726170,0x0000006d,0x00040005,0x0000016e,0x61726170,0x0000006d,0x00040005,
0x00000170,0x61726170,0x0000006d,0x00040005,0x00000172,0x61726170,0x0000006d,0x00040005,
0x00000186,0x74736562,0x00000000,0x00030005,0x00000188,0x00006265,0x00040005,0x00000189,
0x61726170,0x0000006d,0x00040005,0x0000018b,0x61726170,0x0000006d,0x00040005,0x0000018e,
0x61726170,0x0000006d,0x00040005,0x00000190,0x61726170,0x0000006d,0x00040005,0x00000193,
0x61726170,0x0000006d,0x00040005,0x00000195,0x61726170,0x0000006d,0x00040005,0x00000199,
0x61726170,0x0000006d,0x00040005,0x0000019b,0x61726170,0x0000006d,0x00040005,0x0000019e,
0x61726170,0x0000006d,0x00040005,0x000001a0,0x61726170,0x0000006d,0x00030005,0x000001a8,
0x00000065,0x00040005,0x000001a9,0x61726170,0x0000006d,0x00040005,0x000001ab,0x61726170,
0x0000006d,0x00040005,0x000001b7,0x61726170,0x0000006d,0x00040005,0x000001b9,0x61726170,
0x0000006d,0x00040005,0x000001c4,0x61726170,0x0000006d,0x00040005,0x000001c6,0x61726170,
0x0000006d,0x00040005,0x000001d1,0x61726170,0x0000006d,0x00040005,0x000001d3,0x61726170,
0x0000006d,0x00030005,0x000001de,0x00003065,0x00040005,0x000001df,0x61726170,0x0000006d,
0x00040005,0x000001e1,0x61726170,0x0000006d,0x00040005,0x000001e3,0x6c697473,0x0000006c,
0x00040005,0x000001e4,0x61726170,0x0000006d,0x00040005,0x000001e6,0x61726170,0x0000006d,
0x00040005,0x000001e8,0x61726170,0x0000006d,0x00040005,0x000001ea,0x61726170,0x0000006d,
0x00040005,0x000001ed,0x61726170,0x0000006d,0x00040005,0x000001ef,0x61726170,0x0000006d,
0x00040005,0x000001f1,0x61726170,0x0000006d,0x00040005,0x000001f3,0x61726170,0x0000006d,
0x00030005,0x00000207,0x00000061,0x00040005,0x0000020d,0x61726170,0x0000006d,0x00030005,
0x0000020f,0x00000062,0x00040005,0x00000216,0x61726170,0x0000006d,0x00040005,0x00000218,
0x74736f6c,0x00000000,0x00040047,0x0000002a,0x00000021,0x00000002,0x00040047,0x0000002a,
0x00000022,0x00000000,0x00040047,0x0000004d,0x00000021,0x00000000,0x00040047,0x0000004d,
0x00000022,0x00000000,0x00040047,0x00000059,0x00000021,0x00000001,0x00040047,0x00000059,
0x00000022,0x00000000,0x00040047,0x000000c9,0x0000000b,0x0000001c,0x00030047,0x000000d1,
0x00000019,0x00040047,0x000000d1,0x00000021,0x00000003,0x00040047,0x000000d1,0x00000022,
0x00000000,0x00030047,0x000000dd,0x00000002,0x00050048,0x000000dd,0x00000000,0x00000023,
0x00000000,0x00040047,0x00000231,0x0000000b,0x00000019,0x00020013,0x00000002,0x00030021,
0x00000003,0x00000002,0x00040015,0x00000006,0x00000020,0x00000001,0x00040017,0x00000007,
0x00000006,0x00000002,0x00040020,0x00000008,0x00000007,0x00000007,0x00030016,0x00000009,
0x00000020,0x00040017,0x0000000a,0x00000009,0x00000002,0x00040021,0x0000000b,0x0000000a,
0x00000008,0x00040017,0x0000000f,0x00000009,0x00000003,0x00040020,0x00000010,0x00000007,
0x0000000f,0x00050021,0x00000011,0x00000009,0x00000010,0x00000010,0x00040020,0x00000016,
0x00000007,0x0000000a,0x00040021,0x00000017,0x0000000f,0x00000016,0x00050021,0x0000001e,
0x00000009,0x00000016,0x00000016,0x00090019,0x00000027,0x00000009,0x00000001,0x00000000,
0x00000000,0x00000000,0x00000001,0x00000000,0x0003001b,0x00000028,0x00000027,0x00040020,
0x00000029,0x00000000,0x00000028,0x0004003b,0x00000029,0x0000002a,0x00000000,0x0004002b,
0x00000006,0x0000002d,0x00000000,0x0005002c,0x00000007,0x0000002e,0x0000002d,0x0000002d,
0x00040020,0x0000002f,0x00000006,0x00000007,0x0004003b,0x0000002f,0x00000030,0x00000006,
0x00040017,0x00000034,0x00000009,0x00000004,0x00040015,0x0000003e,0x00000020,0x00000000,
0x0004002b,0x0000003e,0x0000003f,0x00000000,0x00040020,0x00000040,0x00000007,0x00000009,
0x0004002b,0x0000003e,0x00000043,0x00000001,0x0004002b,0x0000003e,0x00000046,0x00000002,
0x0004003b,0x00000029,0x0000004d,0x00000000,0x00040020,0x00000050,0x00000006,0x0000000a,
0x0004003b,0x00000050,0x00000051,0x00000006,0x0004002b,0x00000009,0x00000054,0x00000000,
0x0004003b,0x00000029,0x00000059,0x00000000,0x00040020,0x00000064,0x00000006,0x00000009,
0x0004003b,0x00000064,0x00000065,0x00000006,0x0004002b,0x00000009,0x0000006c,0x3f800000,
0x0005002c,0x0000000a,0x0000007d,0x0000006c,0x00000054,0x0005002c,0x0000000a,0x00000098,
0x00000054,0x0000006c,0x0004002b,0x00000009,0x000000b3,0x40a00000,0x00040017,0x000000c7,
0x0000003e,0x00000003,0x00040020,0x000000c8,0x00000001,0x000000c7,0x0004003b,0x000000c8,
0x000000c9,0x00000001,0x00040017,0x000000ca,0x0000003e,0x00000002,0x00090019,0x000000cf,
0x00000009,0x00000001,0x00000000,0x00000000,0x00000000,0x00000002,0x00000004,0x00040020,
0x000000d0,0x00000000,0x000000cf,0x0004003b,0x000000d0,0x000000d1,0x00000000,0x00020014,
0x000000d6,0x00040017,0x000000d7,0x000000d6,0x00000002,0x0003001e,0x000000dd,0x00000009,
0x00040020,0x000000de,0x00000009,0x000000dd,0x0004003b,0x000000de,0x000000df,0x00000009,
0x00040020,0x000000e0,0x00000009,0x00000009,0x0004002b,0x00000006,0x000000e6,0x00000001,
0x0004002b,0x00000009,0x000000f0,0x3f000000,0x0004002b,0x00000009,0x000000f5,0x41000000,
0x0005002c,0x00000007,0x00000108,0x000000e6,0x0000002d,0x0005002c,0x00000007,0x00000111,
0x0000002d,0x000000e6,0x0005002c,0x00000007,0x00000116,0x000000e6,0x000000e6,0x0005002c,
0x0000000a,0x00000162,0x00000054,0x00000054,0x0004002b,0x00000009,0x00000176,0x3e000000,
0x0004002b,0x00000009,0x000001b0,0x3c23d70a,0x0004002b,0x00000009,0x000001fe,0x3b83126f,
0x0004002b,0x00000009,0x00000219,0x3d75c28f,0x0004002b,0x00000009,0x0000021a,0x3e3851ec,
0x0004002b,0x0000003e,0x00000230,0x00000008,0x0006002c,0x000000c7,0x00000231,0x00000230,
0x00000230,0x00000043,0x00050036,0x00000002,0x00000004,0x00000000,0x00000003,0x000200f8,
0x00000005,0x0004003b,0x00000008,0x000000c6,0x00000007,0x0004003b,0x00000008,0x000000ce,
0x00000007,0x0004003b,0x00000016,0x000000ed,0x00000007,0x0004003b,0x00000016,0x000000f3,
0x00000007,0x0004003b,0x00000008,0x000000fa,0x00000007,0x0004003b,0x00000016,0x000000fe,
0x00000007,0x0004003b,0x00000016,0x00000103,0x00000007,0x0004003b,0x00000008,0x00000104,
0x00000007,0x0004003b,0x00000008,0x0000010a,0x00000007,0x0004003b,0x00000008,0x00000113,
0x00000007,0x0004003b,0x00000008,0x00000118,0x00000007,0x0004003b,0x00000016,0x00000132,
0x00000007,0x0004003b,0x00000008,0x00000133,0x00000007,0x0004003b,0x00000016,0x00000136,
0x00000007,0x0004003b,0x00000008,0x00000139,0x00000007,0x0004003b,0x00000016,0x0000013b,
0x00000007,0x0004003b,0x00000008,0x0000013e,0x00000007,0x0004003b,0x00000016,0x00000140,
0x00000007,0x0004003b,0x00000008,0x00000143,0x00000007,0x0004003b,0x00000016,0x00000145,
0x00000007,0x0004003b,0x00000010,0x00000156,0x00000007,0x0004003b,0x00000010,0x0000015c,
0x00000007,0x0004003b,0x00000016,0x00000163,0x00000007,0x0004003b,0x00000016,0x00000165,
0x00000007,0x0004003b,0x00000016,0x00000167,0x00000007,0x0004003b,0x00000016,0x00000169,
0x00000007,0x0004003b,0x00000016,0x0000016c,0x00000007,0x0004003b,0x00000016,0x0000016e,
0x00000007,0x0004003b,0x00000016,0x00000170,0x00000007,0x0004003b,0x00000016,0x00000172,
0x00000007,0x0004003b,0x00000016,0x00000186,0x00000007,0x0004003b,0x00000040,0x00000188,
0x00000007,0x0004003b,0x00000016,0x00000189,0x00000007,0x0004003b,0x00000016,0x0000018b,
0x00000007,0x0004003b,0x00000016,0x0000018e,0x00000007,0x0004003b,0x00000016,0x00000190,
0x00000007,0x0004003b,0x00000016,0x00000193,0x00000007,0x0004003b,0x00000016,0x00000195,
0x00000007,0x0004003b,0x00000016,0x00000199,0x00000007,0x0004003b,0x00000016,0x0000019b,
0x00000007,0x0004003b,0x00000016,0x0000019e,0x00000007,0x0004003b,0x00000016,0x000001a0,
0x00000007,0x0004003b,0x00000040,0x000001a8,0x00000007,0x0004003b,0x00000016,0x000001a9,
0x00000007,0x0004003b,0x00000016,0x000001ab,0x00000007,0x0004003b,0x00000016,0x000001b7,
0x00000007,0x0004003b,0x00000016,0x000001b9,0x00000007,0x0004003b,0x00000016,0x000001c4,
0x00000007,0x0004003b,0x00000016,0x000001c6,0x00000007,0x0004003b,0x00000016,0x000001d1,
0x00000007,0x0004003b,0x00000016,0x000001d3,0x00000007,0x0004003b,0x00000040,0x000001de,
0x00000007,0x0004003b,0x00000016,0x000001df,0x00000007,0x0004003b,0x00000016,0x000001e1,
0x00000007,0x0004003b,0x00000040,0x000001e3,0x00000007,0x0004003b,0x00000016,0x000001e4,
0x00000007,0x0004003b,0x00000016,0x000001e6,0x00000007,0x0004003b,0x00000016,0x000001e8,
0x00000007,0x0004003b,0x00000016,0x000001ea,0x00000007,0x0004003b,0x00000016,0x000001ed,
0x00000007,0x0004003b,0x00000016,0x000001ef,0x00000007,0x0004003b,0x00000016,0x000001f1,
0x00000007,0x0004003b,0x00000016,0x000001f3,0x00000007,0x0004003b,0x00000010,0x00000207,
0x00000007,0x0004003b,0x00000016,0x0000020d,0x00000007,0x0004003b,0x00000010,0x0000020f,
0x00000007,0x0004003b,0x00000016,0x00000216,0x00000007,0x0004003b,0x00000040,0x00000218,
0x00000007,0x0004003d,0x000000c7,0x000000cb,0x000000c9,0x0007004f,0x000000ca,0x000000cc,
0x000000cb,0x000000cb,0x00000000,0x00000001,0x0004007c,0x00000007,0x000000cd,0x000000cc,
0x0003003e,0x000000c6,0x000000cd,0x0004003d,0x000000cf,0x000000d2,0x000000d1,0x00040068,
0x00000007,0x000000d3,0x000000d2,0x0003003e,0x000000ce,0x000000d3,0x0004003d,0x00000007,
0x000000d4,0x000000c6,0x0004003d,0x00000007,0x000000d5,0x000000ce,0x000500af,0x000000d7,
0x000000d8,0x000000d4,0x000000d5,0x0004009a,0x000000d6,0x000000d9,0x000000d8,0x000300f7,
0x000000db,0x00000000,0x000400fa,0x000000d9,0x000000da,0x000000db,0x000200f8,0x000000da,
0x000100fd,0x000200f8,0x000000db,0x00050041,0x000000e0,0x000000e1,0x000000df,0x0000002d,
0x0004003d,0x00000009,0x000000e2,0x000000e1,0x0003003e,0x00000065,0x000000e2,0x0004003d,
0x00000028,0x000000e3,0x0000002a,0x00040064,0x00000027,0x000000e4,0x000000e3,0x00050067,
0x00000007,0x000000e5,0x000000e4,0x0000002d,0x00050050,0x00000007,0x000000e7,0x000000e6,
0x000000e6,0x00050082,0x00000007,0x000000e8,0x000000e5,0x000000e7,0x0003003e,0x00000030,
0x000000e8,0x0004003d,0x00000007,0x000000e9,0x000000ce,0x0004006f,0x0000000a,0x000000ea,
0x000000e9,0x00050050,0x0000000a,0x000000eb,0x0000006c,0x0000006c,0x00050088,0x0000000a,
0x000000ec,0x000000eb,0x000000ea,0x0003003e,0x00000051,0x000000ec,0x0004003d,0x00000007,
0x000000ee,0x000000c6,0x0004006f,0x0000000a,0x000000ef,0x000000ee,0x00050050,0x0000000a,
0x000000f1,0x000000f0,0x000000f0,0x00050081,0x0000000a,0x000000f2,0x000000ef,0x000000f1,
0x0003003e,0x000000ed,0x000000f2,0x0004003d,0x0000000a,0x000000f4,0x000000ed,0x00050050,
0x0000000a,0x000000f6,0x000000f5,0x000000f5,0x00050088,0x0000000a,0x000000f7,0x000000f4,
0x000000f6,0x00050050,0x0000000a,0x000000f8,0x000000f0,0x000000f0,0x00050083,0x0000000a,
0x000000f9,0x000000f7,0x000000f8,0x0003003e,0x000000f3,0x000000f9,0x0004003d,0x0000000a,
0x000000fb,0x000000f3,0x0006000c,0x0000000a,0x000000fc,0x00000001,0x00000008,0x000000fb,
0x0004006e,0x00000007,0x000000fd,0x000000fc,0x0003003e,0x000000fa,0x000000fd,0x0004003d,
0x0000000a,0x000000ff,0x000000f3,0x0004003d,0x0000000a,0x00000100,0x000000f3,0x0006000c,
0x0000000a,0x00000101,0x00000001,0x00000008,0x00000100,0x00050083,0x0000000a,0x00000102,
0x000000ff,0x00000101,0x0003003e,0x000000fe,0x00000102,0x0004003d,0x00000007,0x00000105,
0x000000fa,0x0003003e,0x00000104,0x00000105,0x00050039,0x0000000a,0x00000106,0x0000000d,
0x00000104,0x0004003d,0x00000007,0x00000107,0x000000fa,0x00050080,0x00000007,0x00000109,
0x00000107,0x00000108,0x0003003e,0x0000010a,0x00000109,0x00050039,0x0000000a,0x0000010b,
0x0000000d,0x0000010a,0x00050041,0x00000040,0x0000010c,0x000000fe,0x0000003f,0x0004003d,
0x00000009,0x0000010d,0x0000010c,0x00050050,0x0000000a,0x0000010e,0x0000010d,0x0000010d,
0x0008000c,0x0000000a,0x0000010f,0x00000001,0x0000002e,0x00000106,0x0000010b,0x0000010e,
0x0004003d,0x00000007,0x00000110,0x000000fa,0x00050080,0x00000007,0x00000112,0x00000110,
0x00000111,0x0003003e,0x00000113,0x00000112,0x00050039,0x0000000a,0x00000114,0x0000000d,
0x00000113,0x0004003d,0x00000007,0x00000115,0x000000fa,0x00050080,0x00000007,0x00000117,
0x00000115,0x00000116,0x0003003e,0x00000118,0x00000117,0x00050039,0x0000000a,0x00000119,
0x0000000d,0x00000118,0x00050041,0x00000040,0x0000011a,0x000000fe,0x0000003f,0x0004003d,
0x00000009,0x0000011b,0x0000011a,0x00050050,0x0000000a,0x0000011c,0x0000011b,0x0000011b,
0x0008000c,0x0000000a,0x0000011d,0x00000001,0x0000002e,0x00000114,0x00000119,0x0000011c,
0x00050041,0x00000040,0x0000011e,0x000000fe,0x00000043,0x0004003d,0x00000009,0x0000011f,
0x0000011e,0x00050050,0x0000000a,0x00000120,0x0000011f,0x0000011f,0x0008000c,0x0000000a,
0x00000121,0x00000001,0x0000002e,0x0000010f,0x0000011d,0x00000120,0x0003003e,0x00000103,
0x00000121,0x0004003d,0x0000000a,0x00000122,0x000000ed,0x0004003d,0x00000009,0x00000123,
0x00000065,0x0004003d,0x0000000a,0x00000124,0x00000103,0x0005008e,0x0000000a,0x00000125,
0x00000124,0x00000123,0x00050083,0x0000000a,0x00000126,0x00000122,0x00000125,0x00050050,
0x0000000a,0x00000127,0x000000f5,0x000000f5,0x00050088,0x0000000a,0x00000128,0x00000126,
0x00000127,0x00050050,0x0000000a,0x00000129,0x000000f0,0x000000f0,0x00050083,0x0000000a,
0x0000012a,0x00000128,0x00000129,0x0003003e,0x000000f3,0x0000012a,0x0004003d,0x0000000a,
0x0000012b,0x000000f3,0x0006000c,0x0000000a,0x0000012c,0x00000001,0x00000008,0x0000012b,
0x0004006e,0x00000007,0x0000012d,0x0000012c,0x0003003e,0x000000fa,0x0000012d,0x0004003d,
0x0000000a,0x0000012e,0x000000f3,0x0004003d,0x0000000a,0x0000012f,0x000000f3,0x0006000c,
0x0000000a,0x00000130,0x00000001,0x00000008,0x0000012f,0x00050083,0x0000000a,0x00000131,
0x0000012e,0x00000130,0x0003003e,0x000000fe,0x00000131,0x0004003d,0x00000007,0x00000134,
0x000000fa,0x0003003e,0x00000133,0x00000134,0x00050039,0x0000000a,0x00000135,0x0000000d,
0x00000133,0x0003003e,0x00000132,0x00000135,0x0004003d,0x00000007,0x00000137,0x000000fa,
0x00050080,0x00000007,0x00000138,0x00000137,0x00000108,0x0003003e,0x00000139,0x00000138,
0x00050039,0x0000000a,0x0000013a,0x0000000d,0x00000139,0x0003003e,0x00000136,0x0000013a,
0x0004003d,0x00000007,0x0000013c,0x000000fa,0x00050080,0x00000007,0x0000013d,0x0000013c,
0x00000111,0x0003003e,0x0000013e,0x0000013d,0x00050039,0x0000000a,0x0000013f,0x0000000d,
0x0000013e,0x0003003e,0x0000013b,0x0000013f,0x0004003d,0x00000007,0x00000141,0x000000fa,
0x00050080,0x00000007,0x00000142,0x00000141,0x00000116,0x0003003e,0x00000143,0x00000142,
0x00050039,0x0000000a,0x00000144,0x0000000d,0x00000143,0x0003003e,0x00000140,0x00000144,
0x0004003d,0x0000000a,0x00000146,0x00000132,0x0004003d,0x0000000a,0x00000147,0x00000136,
0x00050041,0x00000040,0x00000148,0x000000fe,0x0000003f,0x0004003d,0x00000009,0x00000149,
0x00000148,0x00050050,0x0000000a,0x0000014a,0x00000149,0x00000149,0x0008000c,0x0000000a,
0x0000014b,0x00000001,0x0000002e,0x00000146,0x00000147,0x0000014a,0x0004003d,0x0000000a,
0x0000014c,0x0000013b,0x0004003d,0x0000000a,0x0000014d,0x00000140,0x00050041,0x00000040,
0x0000014e,0x000000fe,0x0000003f,0x0004003d,0x00000009,0x0000014f,0x0000014e,0x00050050,
0x0000000a,0x00000150,0x0000014f,0x0000014f,0x0008000c,0x0000000a,0x00000151,0x00000001,
0x0000002e,0x0000014c,0x0000014d,0x00000150,0x00050041,0x00000040,0x00000152,0x000000fe,
0x00000043,0x0004003d,0x00000009,0x00000153,0x00000152,0x00050050,0x0000000a,0x00000154,
0x00000153,0x00000153,0x0008000c,0x0000000a,0x00000155,0x00000001,0x0000002e,0x0000014b,
0x00000151,0x00000154,0x0003003e,0x00000145,0x00000155,0x0004003d,0x00000028,0x00000157,
0x0000004d,0x0004003d,0x00000007,0x00000158,0x000000c6,0x00040064,0x00000027,0x00000159,
0x00000157,0x0007005f,0x00000034,0x0000015a,0x00000159,0x00000158,0x00000002,0x0000002d,
0x0008004f,0x0000000f,0x0000015b,0x0000015a,0x0000015a,0x00000000,0x00000001,0x00000002,
0x0003003e,0x00000156,0x0000015b,0x0004003d,0x00000028,0x0000015d,0x00000059,0x0004003d,
0x00000007,0x0000015e,0x000000c6,0x00040064,0x00000027,0x0000015f,0x0000015d,0x0007005f,
0x00000034,0x00000160,0x0000015f,0x0000015e,0x00000002,0x0000002d,0x0008004f,0x0000000f,
0x00000161,0x00000160,0x00000160,0x00000000,0x00000001,0x00000002,0x0003003e,0x0000015c,
0x00000161,0x0004003d,0x0000000a,0x00000164,0x00000132,0x0003003e,0x00000163,0x00000164,
0x0003003e,0x00000165,0x00000162,0x00060039,0x00000009,0x00000166,0x00000025,0x00000163,
0x00000165,0x0004003d,0x0000000a,0x00000168,0x00000136,0x0003003e,0x00000167,0x00000168,
0x0003003e,0x00000169,0x00000162,0x00060039,0x00000009,0x0000016a,0x00000025,0x00000167,
0x00000169,0x0007000c,0x00000009,0x0000016b,0x00000001,0x00000028,0x00000166,0x0000016a,
0x0004003d,0x0000000a,0x0000016d,0x0000013b,0x0003003e,0x0000016c,0x0000016d,0x0003003e,
0x0000016e,0x00000162,0x00060039,0x00000009,0x0000016f,0x00000025,0x0000016c,0x0000016e,
0x0004003d,0x0000000a,0x00000171,0x00000140,0x0003003e,0x00000170,0x00000171,0x0003003e,
0x00000172,0x00000162,0x00060039,0x00000009,0x00000173,0x00000025,0x00000170,0x00000172,
0x0007000c,0x00000009,0x00000174,0x00000001,0x00000028,0x0000016f,0x00000173,0x0007000c,
0x00000009,0x00000175,0x00000001,0x00000028,0x0000016b,0x00000174,0x000500b8,0x000000d6,
0x00000177,0x00000175,0x00000176,0x000300f7,0x00000179,0x00000000,0x000400fa,0x00000177,
0x00000178,0x00000179,0x000200f8,0x00000178,0x0004003d,0x000000cf,0x0000017a,0x000000d1,
0x0004003d,0x00000007,0x0000017b,0x000000c6,0x0004003d,0x0000000f,0x0000017c,0x00000156,
0x0004003d,0x0000000f,0x0000017d,0x0000015c,0x0004003d,0x00000009,0x0000017e,0x00000065,
0x00060050,0x0000000f,0x0000017f,0x0000017e,0x0000017e,0x0000017e,0x0008000c,0x0000000f,
0x00000180,0x00000001,0x0000002e,0x0000017c,0x0000017d,0x0000017f,0x00050051,0x00000009,
0x00000181,0x00000180,0x00000000,0x00050051,0x00000009,0x00000182,0x00000180,0x00000001,
0x00050051,0x00000009,0x00000183,0x00000180,0x00000002,0x00070050,0x00000034,0x00000184,
0x00000181,0x00000182,0x00000183,0x0000006c,0x00040063,0x0000017a,0x0000017b,0x00000184,
0x000100fd,0x000200f8,0x00000179,0x0004003d,0x0000000a,0x00000187,0x00000145,0x0003003e,
0x00000186,0x00000187,0x0004003d,0x0000000a,0x0000018a,0x000000ed,0x0003003e,0x00000189,
0x0000018a,0x0004003d,0x0000000a,0x0000018c,0x00000145,0x0003003e,0x0000018b,0x0000018c,
0x00060039,0x00000009,0x0000018d,0x00000021,0x00000189,0x0000018b,0x0003003e,0x00000188,
0x0000018d,0x0004003d,0x0000000a,0x0000018f,0x00000132,0x0003003e,0x0000018e,0x0000018f,
0x0004003d,0x0000000a,0x00000191,0x00000145,0x0003003e,0x00000190,0x00000191,0x00060039,
0x00000009,0x00000192,0x00000025,0x0000018e,0x00000190,0x0004003d,0x0000000a,0x00000194,
0x00000136,0x0003003e,0x00000193,0x00000194,0x0004003d,0x0000000a,0x00000196,0x00000145,
0x0003003e,0x00000195,0x00000196,0x00060039,0x00000009,0x00000197,0x00000025,0x00000193,
0x00000195,0x0007000c,0x00000009,0x00000198,0x00000001,0x00000028,0x00000192,0x00000197,
0x0004003d,0x0000000a,0x0000019a,0x0000013b,0x0003003e,0x00000199,0x0000019a,0x0004003d,
0x0000000a,0x0000019c,0x00000145,0x0003003e,0x0000019b,0x0000019c,0x00060039,0x00000009,
0x0000019d,0x00000025,0x00000199,0x0000019b,0x0004003d,0x0000000a,0x0000019f,0x00000140,
0x0003003e,0x0000019e,0x0000019f,0x0004003d,0x0000000a,0x000001a1,0x00000145,0x0003003e,
0x000001a0,0x000001a1,0x00060039,0x00000009,0x000001a2,0x00000025,0x0000019e,0x000001a0,
0x0007000c,0x00000009,0x000001a3,0x00000001,0x00000028,0x0000019d,0x000001a2,0x0007000c,
0x00000009,0x000001a4,0x00000001,0x00000028,0x00000198,0x000001a3,0x000500ba,0x000000d6,
0x000001a5,0x000001a4,0x000000f0,0x000300f7,0x000001a7,0x00000000,0x000400fa,0x000001a5,
0x000001a6,0x000001a7,0x000200f8,0x000001a6,0x0004003d,0x0000000a,0x000001aa,0x000000ed,
0x0003003e,0x000001a9,0x000001aa,0x0004003d,0x0000000a,0x000001ac,0x00000132,0x0003003e,
0x000001ab,0x000001ac,0x00060039,0x00000009,0x000001ad,0x00000021,0x000001a9,0x000001ab,
0x0003003e,0x000001a8,0x000001ad,0x0004003d,0x00000009,0x000001ae,0x000001a8,0x0004003d,
0x00000009,0x000001af,0x00000188,0x00050083,0x00000009,0x000001b1,0x000001af,0x000001b0,
0x000500b8,0x000000d6,0x000001b2,0x000001ae,0x000001b1,0x000300f7,0x000001b4,0x00000000,
0x000400fa,0x000001b2,0x000001b3,0x000001b4,0x000200f8,0x000001b3,0x0004003d,0x0000000a,
0x000001b5,0x00000132,0x0003003e,0x00000186,0x000001b5,0x0004003d,0x00000009,0x000001b6,
0x000001a8,0x0003003e,0x00000188,0x000001b6,0x000200f9,0x000001b4,0x000200f8,0x000001b4,
0x0004003d,0x0000000a,0x000001b8,0x000000ed,0x0003003e,0x000001b7,0x000001b8,0x0004003d,
0x0000000a,0x000001ba,0x00000136,0x0003003e,0x000001b9,0x000001ba,0x00060039,0x00000009,
0x000001bb,0x00000021,0x000001b7,0x000001b9,0x0003003e,0x000001a8,0x000001bb,0x0004003d,
0x00000009,0x000001bc,0x000001a8,0x0004003d,0x00000009,0x000001bd,0x00000188,0x00050083,
0x00000009,0x000001be,0x000001bd,0x000001b0,0x000500b8,0x000000d6,0x000001bf,0x000001bc,
0x000001be,0x000300f7,0x000001c1,0x00000000,0x000400fa,0x000001bf,0x000001c0,0x000001c1,
0x000200f8,0x000001c0,0x0004003d,0x0000000a,0x000001c2,0x00000136,0x0003003e,0x00000186,
0x000001c2,0x0004003d,0x00000009,0x000001c3,0x000001a8,0x0003003e,0x00000188,0x000001c3,
0x000200f9,0x000001c1,0x000200f8,0x000001c1,0x0004003d,0x0000000a,0x000001c5,0x000000ed,
0x0003003e,0x000001c4,0x000001c5,0x0004003d,0x0000000a,0x000001c7,0x0000013b,0x0003003e,
0x000001c6,0x000001c7,0x00060039,0x00000009,0x000001c8,0x00000021,0x000001c4,0x000001c6,
0x0003003e,0x000001a8,0x000001c8,0x0004003d,0x00000009,0x000001c9,0x000001a8,0x0004003d,
0x00000009,0x000001ca,0x00000188,0x00050083,0x00000009,0x000001cb,0x000001ca,0x000001b0,
0x000500b8,0x000000d6,0x000001cc,0x000001c9,0x000001cb,0x000300f7,0x000001ce,0x00000000,
0x000400fa,0x000001cc,0x000001cd,0x000001ce,0x000200f8,0x000001cd,0x0004003d,0x0000000a,
0x000001cf,0x0000013b,0x0003003e,0x00000186,0x000001cf,0x0004003d,0x00000009,0x000001d0,
0x000001a8,0x0003003e,0x00000188,0x000001d0,0x000200f9,0x000001ce,0x000200f8,0x000001ce,
0x0004003d,0x0000000a,0x000001d2,0x000000ed,0x0003003e,0x000001d1,0x000001d2,0x0004003d,
0x0000000a,0x000001d4,0x00000140,0x0003003e,0x000001d3,0x000001d4,0x00060039,0x00000009,
0x000001d5,0x00000021,0x000001d1,0x000001d3,0x0003003e,0x000001a8,0x000001d5,0x0004003d,
0x00000009,0x000001d6,0x000001a8,0x0004003d,0x00000009,0x000001d7,0x00000188,0x00050083,
0x00000009,0x000001d8,0x000001d7,0x000001b0,0x000500b8,0x000000d6,0x000001d9,0x000001d6,
0x000001d8,0x000300f7,0x000001db,0x00000000,0x000400fa,0x000001d9,0x000001da,0x000001db,
0x000200f8,0x000001da,0x0004003d,0x0000000a,0x000001dc,0x00000140,0x0003003e,0x00000186,
0x000001dc,0x0004003d,0x00000009,0x000001dd,0x000001a8,0x0003003e,0x00000188,0x000001dd,
0x000200f9,0x000001db,0x000200f8,0x000001db,0x000200f9,0x000001a7,0x000200f8,0x000001a7,
0x0004003d,0x0000000a,0x000001e0,0x000000ed,0x0003003e,0x000001df,0x000001e0,0x0003003e,
0x000001e1,0x00000162,0x00060039,0x00000009,0x000001e2,0x00000021,0x000001df,0x000001e1,
0x0003003e,0x000001de,0x000001e2,0x0004003d,0x0000000a,0x000001e5,0x00000132,0x0003003e,
0x000001e4,0x000001e5,0x0003003e,0x000001e6,0x00000162,0x00060039,0x00000009,0x000001e7,
0x00000025,0x000001e4,0x000001e6,0x0004003d,0x0000000a,0x000001e9,0x00000136,0x0003003e,
0x000001e8,0x000001e9,0x0003003e,0x000001ea,0x00000162,0x00060039,0x00000009,0x000001eb,
0x00000025,0x000001e8,0x000001ea,0x0007000c,0x00000009,0x000001ec,0x00000001,0x00000025,
0x000001e7,0x000001eb,0x0004003d,0x0000000a,0x000001ee,0x0000013b,0x0003003e,0x000001ed,
0x000001ee,0x0003003e,0x000001ef,0x00000162,0x00060039,0x00000009,0x000001f0,0x00000025,
0x000001ed,0x000001ef,0x0004003d,0x0000000a,0x000001f2,0x00000140,0x0003003e,0x000001f1,
0x000001f2,0x0003003e,0x000001f3,0x00000162,0x00060039,0x00000009,0x000001f4,0x00000025,
0x000001f1,0x000001f3,0x0007000c,0x00000009,0x000001f5,0x00000001,0x00000025,0x000001f0,
0x000001f4,0x0007000c,0x00000009,0x000001f6,0x00000001,0x00000025,0x000001ec,0x000001f5,
0x0003003e,0x000001e3,0x000001f6,0x0004003d,0x00000009,0x000001f7,0x000001de,0x0004003d,
0x00000009,0x000001f8,0x00000188,0x00050083,0x00000009,0x000001f9,0x000001f8,0x000001b0,
0x000500b8,0x000000d6,0x000001fa,0x000001f7,0x000001f9,0x000300f7,0x000001fc,0x00000000,
0x000400fa,0x000001fa,0x000001fb,0x000001fc,0x000200f8,0x000001fb,0x0004003d,0x00000009,
0x000001fd,0x000001de,0x000500b8,0x000000d6,0x000001ff,0x000001fd,0x000001fe,0x0004003d,
0x00000009,0x00000200,0x000001e3,0x000500b8,0x000000d6,0x00000201,0x00000200,0x0000006c,
0x000500a6,0x000000d6,0x00000202,0x000001ff,0x00000201,0x000200f9,0x000001fc,0x000200f8,
0x000001fc,0x000700f5,0x000000d6,0x00000203,0x000001fa,0x000001a7,0x00000202,0x000001fb,
0x000300f7,0x00000205,0x00000000,0x000400fa,0x00000203,0x00000204,0x00000205,0x000200f8,
0x00000204,0x0003003e,0x00000186,0x00000162,0x0004003d,0x00000009,0x00000206,0x000001de,
0x0003003e,0x00000188,0x00000206,0x000200f9,0x00000205,0x000200f8,0x00000205,0x0004003d,
0x0000000a,0x00000208,0x000000ed,0x0004003d,0x00000009,0x00000209,0x00000065,0x0004003d,
0x0000000a,0x0000020a,0x00000186,0x0005008e,0x0000000a,0x0000020b,0x0000020a,0x00000209,
0x00050083,0x0000000a,0x0000020c,0x00000208,0x0000020b,0x0003003e,0x0000020d,0x0000020c,
0x00050039,0x0000000f,0x0000020e,0x00000019,0x0000020d,0x0003003e,0x00000207,0x0000020e,
0x0004003d,0x0000000a,0x00000210,0x000000ed,0x0004003d,0x00000009,0x00000211,0x00000065,
0x00050083,0x00000009,0x00000212,0x0000006c,0x00000211,0x0004003d,0x0000000a,0x00000213,
0x00000186,0x0005008e,0x0000000a,0x00000214,0x00000213,0x00000212,0x00050081,0x0000000a,
0x00000215,0x00000210,0x00000214,0x0003003e,0x00000216,0x00000215,0x00050039,0x0000000f,
0x00000217,0x0000001c,0x00000216,0x0003003e,0x0000020f,0x00000217,0x0004003d,0x00000009,
0x0000021b,0x00000188,0x0008000c,0x00000009,0x0000021c,0x00000001,0x00000031,0x00000219,
0x0000021a,0x0000021b,0x0003003e,0x00000218,0x0000021c,0x0004003d,0x000000cf,0x0000021d,
0x000000d1,0x0004003d,0x00000007,0x0000021e,0x000000c6,0x0004003d,0x0000000f,0x0000021f,
0x00000207,0x0004003d,0x0000000f,0x00000220,0x0000020f,0x0004003d,0x00000009,0x00000221,
0x00000065,0x00060050,0x0000000f,0x00000222,0x00000221,0x00000221,0x00000221,0x0008000c,
0x0000000f,0x00000223,0x00000001,0x0000002e,0x0000021f,0x00000220,0x00000222,0x0004003d,
0x00000009,0x00000224,0x00000065,0x000500bc,0x000000d6,0x00000225,0x00000224,0x000000f0,
0x0004003d,0x0000000f,0x00000226,0x00000156,0x0004003d,0x0000000f,0x00000227,0x0000015c,
0x000600a9,0x0000000f,0x00000228,0x00000225,0x00000226,0x00000227,0x0004003d,0x00000009,
0x00000229,0x00000218,0x00060050,0x0000000f,0x0000022a,0x00000229,0x00000229,0x00000229,
0x0008000c,0x0000000f,0x0000022b,0x00000001,0x0000002e,0x00000223,0x00000228,0x0000022a,
0x00050051,0x00000009,0x0000022c,0x0000022b,0x00000000,0x00050051,0x00000009,0x0000022d,
0x0000022b,0x00000001,0x00050051,0x00000009,0x0000022e,0x0000022b,0x00000002,0x00070050,
0x00000034,0x0000022f,0x0000022c,0x0000022d,0x0000022e,0x0000006c,0x00040063,0x0000021d,
0x0000021e,0x0000022f,0x000100fd,0x00010038,0x00050036,0x0000000a,0x0000000d,0x00000000,
0x0000000b,0x00030037,0x00000008,0x0000000c,0x000200f8,0x0000000e,0x0004003d,0x00000028,
0x0000002b,0x0000002a,0x0004003d,0x00000007,0x0000002c,0x0000000c,0x0004003d,0x00000007,
0x00000031,0x00000030,0x0008000c,0x00000007,0x00000032,0x00000001,0x0000002d,0x0000002c,
0x0000002e,0x00000031,0x00040064,0x00000027,0x00000033,0x0000002b,0x0007005f,0x00000034,
0x00000035,0x00000033,0x00000032,0x00000002,0x0000002d,0x0007004f,0x0000000a,0x00000036,
0x00000035,0x00000035,0x00000000,0x00000001,0x000200fe,0x00000036,0x00010038,0x00050036,
0x00000009,0x00000014,0x00000000,0x00000011,0x00030037,0x00000010,0x00000012,0x00030037,
0x00000010,0x00000013,0x000200f8,0x00000015,0x0004003b,0x00000010,0x00000039,0x00000007,
0x0004003d,0x0000000f,0x0000003a,0x00000012,0x0004003d,0x0000000f,0x0000003b,0x00000013,
0x00050083,0x0000000f,0x0000003c,0x0000003a,0x0000003b,0x0006000c,0x0000000f,0x0000003d,
0x00000001,0x00000004,0x0000003c,0x0003003e,0x00000039,0x0000003d,0x00050041,0x00000040,
0x00000041,0x00000039,0x0000003f,0x0004003d,0x00000009,0x00000042,0x00000041,0x00050041,
0x00000040,0x00000044,0x00000039,0x00000043,0x0004003d,0x00000009,0x00000045,0x00000044,
0x00050041,0x00000040,0x00000047,0x00000039,0x00000046,0x0004003d,0x00000009,0x00000048,
0x00000047,0x0007000c,0x00000009,0x00000049,0x00000001,0x00000028,0x00000045,0x00000048,
0x0007000c,0x00000009,0x0000004a,0x00000001,0x00000028,0x00000042,0x00000049,0x000200fe,
0x0000004a,0x00010038,0x00050036,0x0000000f,0x00000019,0x00000000,0x00000017,0x00030037,
0x00000016,0x00000018,0x000200f8,0x0000001a,0x0004003d,0x00000028,0x0000004e,0x0000004d,
0x0004003d,0x0000000a,0x0000004f,0x00000018,0x0004003d,0x0000000a,0x00000052,0x00000051,
0x00050085,0x0000000a,0x00000053,0x0000004f,0x00000052,0x00070058,0x00000034,0x00000055,
0x0000004e,0x00000053,0x00000002,0x00000054,0x0008004f,0x0000000f,0x00000056,0x00000055,
0x00000055,0x00000000,0x00000001,0x00000002,0x000200fe,0x00000056,0x00010038,0x00050036,
0x0000000f,0x0000001c,0x00000000,0x00000017,0x00030037,0x00000016,0x0000001b,0x000200f8,
0x0000001d,0x0004003d,0x00000028,0x0000005a,0x00000059,0x0004003d,0x0000000a,0x0000005b,
0x0000001b,0x0004003d,0x0000000a,0x0000005c,0x00000051,0x00050085,0x0000000a,0x0000005d,
0x0000005b,0x0000005c,0x00070058,0x00000034,0x0000005e,0x0000005a,0x0000005d,0x00000002,
0x00000054,0x0008004f,0x0000000f,0x0000005f,0x0000005e,0x0000005e,0x00000000,0x00000001,
0x00000002,0x000200fe,0x0000005f,0x00010038,0x00050036,0x00000009,0x00000021,0x00000000,
0x0000001e,0x00030037,0x00000016,0x0000001f,0x00030037,0x00000016,0x00000020,0x000200f8,
0x00000022,0x0004003b,0x00000016,0x00000062,0x00000007,0x0004003b,0x00000016,0x0000006a,
0x00000007,0x0004003b,0x00000040,0x00000072,0x00000007,0x0004003b,0x00000016,0x00000073,
0x00000007,0x0004003b,0x00000016,0x00000076,0x00000007,0x0004003b,0x00000010,0x00000079,
0x00000007,0x0004003b,0x00000010,0x0000007a,0x00000007,0x0004003b,0x00000016,0x0000007f,
0x00000007,0x0004003b,0x00000016,0x00000083,0x00000007,0x0004003b,0x00000010,0x00000085,
0x00000007,0x0004003b,0x00000010,0x00000086,0x00000007,0x0004003b,0x00000016,0x0000008c,
0x00000007,0x0004003b,0x00000016,0x00000090,0x00000007,0x0004003b,0x00000010,0x00000092,
0x00000007,0x0004003b,0x00000010,0x00000093,0x00000007,0x0004003b,0x00000016,0x0000009a,
0x00000007,0x0004003b,0x00000016,0x0000009e,0x00000007,0x0004003b,0x00000010,0x000000a0,
0x00000007,0x0004003b,0x00000010,0x000000a1,0x00000007,0x0004003b,0x00000016,0x000000a7,
0x00000007,0x0004003b,0x00000016,0x000000ab,0x00000007,0x0004003b,0x00000010,0x000000ad,
0x00000007,0x0004003b,0x00000010,0x000000ae,0x00000007,0x0004003d,0x0000000a,0x00000063,
0x0000001f,0x0004003d,0x00000009,0x00000066,0x00000065,0x0004003d,0x0000000a,0x00000067,
0x00000020,0x0005008e,0x0000000a,0x00000068,0x00000067,0x00000066,0x00050083,0x0000000a,
0x00000069,0x00000063,0x00000068,0x0003003e,0x00000062,0x00000069,0x0004003d,0x0000000a,
0x0000006b,0x0000001f,0x0004003d,0x00000009,0x0000006d,0x00000065,0x00050083,0x00000009,
0x0000006e,0x0000006c,0x0000006d,0x0004003d,0x0000000a,0x0000006f,0x00000020,0x0005008e,
0x0000000a,0x00000070,0x0000006f,0x0000006e,0x00050081,0x0000000a,0x00000071,0x0000006b,
0x00000070,0x0003003e,0x0000006a,0x00000071,0x0004003d,0x0000000a,0x00000074,0x00000062,
0x0003003e,0x00000073,0x00000074,0x00050039,0x0000000f,0x00000075,0x00000019,0x00000073,
0x0004003d,0x0000000a,0x00000077,0x0000006a,0x0003003e,0x00000076,0x00000077,0x00050039,
0x0000000f,0x00000078,0x0000001c,0x00000076,0x0003003e,0x00000079,0x00000075,0x0003003e,
0x0000007a,0x00000078,0x00060039,0x00000009,0x0000007b,0x00000014,0x00000079,0x0000007a,
0x0003003e,0x00000072,0x0000007b,0x0004003d,0x0000000a,0x0000007c,0x00000062,0x00050081,
0x0000000a,0x0000007e,0x0000007c,0x0000007d,0x0003003e,0x0000007f,0x0000007e,0x00050039,
0x0000000f,0x00000080,0x00000019,0x0000007f,0x0004003d,0x0000000a,0x00000081,0x0000006a,
0x00050081,0x0000000a,0x00000082,0x00000081,0x0000007d,0x0003003e,0x00000083,0x00000082,
0x00050039,0x0000000f,0x00000084,0x0000001c,0x00000083,0x0003003e,0x00000085,0x00000080,
0x0003003e,0x00000086,0x00000084,0x00060039,0x00000009,0x00000087,0x00000014,0x00000085,
0x00000086,0x0004003d,0x00000009,0x00000088,0x00000072,0x00050081,0x00000009,0x00000089,
0x00000088,0x00000087,0x0003003e,0x00000072,0x00000089,0x0004003d,0x0000000a,0x0000008a,
0x00000062,0x00050083,0x0000000a,0x0000008b,0x0000008a,0x0000007d,0x0003003e,0x0000008c,
0x0000008b,0x00050039,0x0000000f,0x0000008d,0x00000019,0x0000008c,0x0004003d,0x0000000a,
0x0000008e,0x0000006a,0x00050083,0x0000000a,0x0000008f,0x0000008e,0x0000007d,0x0003003e,
0x00000090,0x0000008f,0x00050039,0x0000000f,0x00000091,0x0000001c,0x00000090,0x0003003e,
0x00000092,0x0000008d,0x0003003e,0x00000093,0x00000091,0x00060039,0x00000009,0x00000094,
0x00000014,0x00000092,0x00000093,0x0004003d,0x00000009,0x00000095,0x00000072,0x00050081,
0x00000009,0x00000096,0x00000095,0x00000094,0x0003003e,0x00000072,0x00000096,0x0004003d,
0x0000000a,0x00000097,0x00000062,0x00050081,0x0000000a,0x00000099,0x00000097,0x00000098,
0x0003003e,0x0000009a,0x00000099,0x00050039,0x0000000f,0x0000009b,0x00000019,0x0000009a,
0x0004003d,0x0000000a,0x0000009c,0x0000006a,0x00050081,0x0000000a,0x0000009d,0x0000009c,
0x00000098,0x0003003e,0x0000009e,0x0000009d,0x00050039,0x0000000f,0x0000009f,0x0000001c,
0x0000009e,0x0003003e,0x000000a0,0x0000009b,0x0003003e,0x000000a1,0x0000009f,0x00060039,
0x00000009,0x000000a2,0x00000014,0x000000a0,0x000000a1,0x0004003d,0x00000009,0x000000a3,
0x00000072,0x00050081,0x00000009,0x000000a4,0x000000a3,0x000000a2,0x0003003e,0x00000072,
0x000000a4,0x0004003d,0x0000000a,0x000000a5,0x00000062,0x00050083,0x0000000a,0x000000a6,
0x000000a5,0x00000098,0x0003003e,0x000000a7,0x000000a6,0x00050039,0x0000000f,0x000000a8,
0x00000019,0x000000a7,0x0004003d,0x0000000a,0x000000a9,0x0000006a,0x00050083,0x0000000a,
0x000000aa,0x000000a9,0x00000098,0x0003003e,0x000000ab,0x000000aa,0x00050039,0x0000000f,
0x000000ac,0x0000001c,0x000000ab,0x0003003e,0x000000ad,0x000000a8,0x0003003e,0x000000ae,
0x000000ac,0x00060039,0x00000009,0x000000af,0x00000014,0x000000ad,0x000000ae,0x0004003d,
0x00000009,0x000000b0,0x00000072,0x00050081,0x00000009,0x000000b1,0x000000b0,0x000000af,
0x0003003e,0x00000072,0x000000b1,0x0004003d,0x00000009,0x000000b2,0x00000072,0x00050088,
0x00000009,0x000000b4,0x000000b2,0x000000b3,0x000200fe,0x000000b4,0x00010038,0x00050036,
0x00000009,0x00000025,0x00000000,0x0000001e,0x00030037,0x00000016,0x00000023,0x00030037,
0x00000016,0x00000024,0x000200f8,0x00000026,0x00050041,0x00000040,0x000000b7,0x00000023,
0x0000003f,0x0004003d,0x00000009,0x000000b8,0x000000b7,0x00050041,0x00000040,0x000000b9,
0x00000024,0x0000003f,0x0004003d,0x00000009,0x000000ba,0x000000b9,0x00050083,0x00000009,
0x000000bb,0x000000b8,0x000000ba,0x0006000c,0x00000009,0x000000bc,0x00000001,0x00000004,
0x000000bb,0x00050041,0x00000040,0x000000bd,0x00000023,0x00000043,0x0004003d,0x00000009,
0x000000be,0x000000bd,0x00050041,0x00000040,0x000000bf,0x00000024,0x00000043,0x0004003d,
0x00000009,0x000000c0,0x000000bf,0x00050083,0x00000009,0x000000c1,0x000000be,0x000000c0,
0x0006000c,0x00000009,0x000000c2,0x00000001,0x00000004,0x000000c1,0x0007000c,0x00000009,
0x000000c3,0x00000001,0x00000028,0x000000bc,0x000000c2,0x000200fe,0x000000c3,0x00010038
};

View File

@@ -2,6 +2,9 @@
// block matching: one vector per 4x4 quarter-size texels (16x16 pixels), +-RADIUS quarter texels searched
layout(local_size_x = 8, local_size_y = 8) in;
layout(constant_id = 0) const int RADIUS = 8;
// ((Triforce) how much a vector must earn its length, below: 1 for the old way's vectors; the fine vectors' coarse search
// (vk_framegen.cpp, fg_fine_flow.comp) takes less, as each fine block then tries the coarse vectors around it on its own pixels)
layout(constant_id = 1) const float BIAS = 1.0;
layout(binding = 0) uniform sampler2D la;
layout(binding = 1) uniform sampler2D lb;
layout(binding = 2, rgba16f) uniform writeonly image2D flow;
@@ -20,7 +23,7 @@ void main() {
for (int i = 0; i < 64; i++)
sad += abs(pa[i] - texelFetch(lb, clamp(c + ivec2(i % 8 - 4 + dx, i / 8 - 4 + dy), ivec2(0), lm), 0).r);
// (a vector must earn its length: flat haze and sky match anywhere, and any one chosen there swims)
const float cost = sad + 0.12 * float(abs(dx) + abs(dy)) + (dx != 0 || dy != 0 ? 0.6 : 0.0);
const float cost = sad + BIAS * (0.12 * float(abs(dx) + abs(dy)) + (dx != 0 || dy != 0 ? 0.6 : 0.0));
if (cost < best) best = cost, bv = ivec2(dx, dy);
}
imageStore(flow, g, vec4(vec2(bv) * 4.0, best / 64.0, 1.0));

View File

@@ -1,7 +1,7 @@
// 1116.6.0
#pragma once
const uint32_t FG_FLOW_COMP_SPV[] = {
0x07230203,0x00010500,0x0008000b,0x000000c8,0x00000000,0x00020011,0x00000001,0x00020011,
0x07230203,0x00010500,0x0008000b,0x000000ca,0x00000000,0x00020011,0x00000001,0x00020011,
0x00000032,0x0006000b,0x00000001,0x4c534c47,0x6474732e,0x3035342e,0x00000000,0x0003000e,
0x00000000,0x00000001,0x0009000f,0x00000005,0x00000004,0x6e69616d,0x00000000,0x0000000d,
0x00000016,0x00000024,0x0000007d,0x00060010,0x00000004,0x00000011,0x00000008,0x00000008,
@@ -13,12 +13,13 @@ const uint32_t FG_FLOW_COMP_SPV[] = {
0x74736562,0x00000000,0x00030005,0x0000005c,0x00007662,0x00030005,0x0000005d,0x00007964,
0x00040005,0x0000005e,0x49444152,0x00005355,0x00030005,0x00000067,0x00007864,0x00030005,
0x00000070,0x00646173,0x00030005,0x00000072,0x00000069,0x00030005,0x0000007d,0x0000626c,
0x00040005,0x00000097,0x74736f63,0x00000000,0x00040047,0x0000000d,0x0000000b,0x0000001c,
0x00030047,0x00000016,0x00000019,0x00040047,0x00000016,0x00000021,0x00000002,0x00040047,
0x00000016,0x00000022,0x00000000,0x00040047,0x00000024,0x00000021,0x00000000,0x00040047,
0x00000024,0x00000022,0x00000000,0x00040047,0x0000005e,0x00000001,0x00000000,0x00040047,
0x0000007d,0x00000021,0x00000001,0x00040047,0x0000007d,0x00000022,0x00000000,0x00040047,
0x000000c7,0x0000000b,0x00000019,0x00020013,0x00000002,0x00030021,0x00000003,0x00000002,
0x00040005,0x00000097,0x74736f63,0x00000000,0x00040005,0x00000099,0x53414942,0x00000000,
0x00040047,0x0000000d,0x0000000b,0x0000001c,0x00030047,0x00000016,0x00000019,0x00040047,
0x00000016,0x00000021,0x00000002,0x00040047,0x00000016,0x00000022,0x00000000,0x00040047,
0x00000024,0x00000021,0x00000000,0x00040047,0x00000024,0x00000022,0x00000000,0x00040047,
0x0000005e,0x00000001,0x00000000,0x00040047,0x0000007d,0x00000021,0x00000001,0x00040047,
0x0000007d,0x00000022,0x00000000,0x00040047,0x00000099,0x00000001,0x00000001,0x00040047,
0x000000c9,0x0000000b,0x00000019,0x00020013,0x00000002,0x00030021,0x00000003,0x00000002,
0x00040015,0x00000006,0x00000020,0x00000001,0x00040017,0x00000007,0x00000006,0x00000002,
0x00040020,0x00000008,0x00000007,0x00000007,0x00040015,0x0000000a,0x00000020,0x00000000,
0x00040017,0x0000000b,0x0000000a,0x00000003,0x00040020,0x0000000c,0x00000001,0x0000000b,
@@ -39,105 +40,106 @@ const uint32_t FG_FLOW_COMP_SPV[] = {
0x00000013,0x0000005b,0x4e6e6b28,0x00040032,0x00000006,0x0000005e,0x00000008,0x00050034,
0x00000006,0x0000005f,0x0000007e,0x0000005e,0x00050034,0x00000006,0x00000068,0x0000007e,
0x0000005e,0x0004002b,0x00000013,0x00000071,0x00000000,0x0004003b,0x00000023,0x0000007d,
0x00000000,0x0004002b,0x00000013,0x00000099,0x3df5c28f,0x0004002b,0x00000013,0x000000a7,
0x3f19999a,0x00040017,0x000000ba,0x00000013,0x00000002,0x0004002b,0x00000013,0x000000bc,
0x40800000,0x0004002b,0x00000013,0x000000bf,0x42800000,0x0004002b,0x00000013,0x000000c1,
0x3f800000,0x0004002b,0x0000000a,0x000000c5,0x00000008,0x0004002b,0x0000000a,0x000000c6,
0x00000001,0x0006002c,0x0000000b,0x000000c7,0x000000c5,0x000000c5,0x000000c6,0x00050036,
0x00000002,0x00000004,0x00000000,0x00000003,0x000200f8,0x00000005,0x0004003b,0x00000008,
0x00000009,0x00000007,0x0004003b,0x00000008,0x00000020,0x00000007,0x0004003b,0x00000008,
0x0000002c,0x00000007,0x0004003b,0x00000034,0x00000035,0x00000007,0x0004003b,0x00000040,
0x00000041,0x00000007,0x0004003b,0x00000056,0x0000005a,0x00000007,0x0004003b,0x00000008,
0x0000005c,0x00000007,0x0004003b,0x00000034,0x0000005d,0x00000007,0x0004003b,0x00000034,
0x00000067,0x00000007,0x0004003b,0x00000056,0x00000070,0x00000007,0x0004003b,0x00000034,
0x00000072,0x00000007,0x0004003b,0x00000056,0x00000097,0x00000007,0x0004003d,0x0000000b,
0x0000000f,0x0000000d,0x0007004f,0x0000000e,0x00000010,0x0000000f,0x0000000f,0x00000000,
0x00000001,0x0004007c,0x00000007,0x00000011,0x00000010,0x0003003e,0x00000009,0x00000011,
0x0004003d,0x00000007,0x00000012,0x00000009,0x0004003d,0x00000014,0x00000017,0x00000016,
0x00040068,0x00000007,0x00000018,0x00000017,0x000500af,0x0000001a,0x0000001b,0x00000012,
0x00000018,0x0004009a,0x00000019,0x0000001c,0x0000001b,0x000300f7,0x0000001e,0x00000000,
0x000400fa,0x0000001c,0x0000001d,0x0000001e,0x000200f8,0x0000001d,0x000100fd,0x000200f8,
0x0000001e,0x0004003d,0x00000022,0x00000025,0x00000024,0x00040064,0x00000021,0x00000027,
0x00000025,0x00050067,0x00000007,0x00000028,0x00000027,0x00000026,0x00050050,0x00000007,
0x0000002a,0x00000029,0x00000029,0x00050082,0x00000007,0x0000002b,0x00000028,0x0000002a,
0x0003003e,0x00000020,0x0000002b,0x0004003d,0x00000007,0x0000002d,0x00000009,0x00050050,
0x00000007,0x0000002f,0x0000002e,0x0000002e,0x00050084,0x00000007,0x00000030,0x0000002d,
0x0000002f,0x00050050,0x00000007,0x00000032,0x00000031,0x00000031,0x00050080,0x00000007,
0x00000033,0x00000030,0x00000032,0x0003003e,0x0000002c,0x00000033,0x0003003e,0x00000035,
0x00000026,0x000200f9,0x00000036,0x000200f8,0x00000036,0x000400f6,0x00000038,0x00000039,
0x00000000,0x000200f9,0x0000003a,0x000200f8,0x0000003a,0x0004003d,0x00000006,0x0000003b,
0x00000035,0x000500b1,0x00000019,0x0000003d,0x0000003b,0x0000003c,0x000400fa,0x0000003d,
0x00000037,0x00000038,0x000200f8,0x00000037,0x0004003d,0x00000006,0x00000042,0x00000035,
0x0004003d,0x00000022,0x00000043,0x00000024,0x0004003d,0x00000007,0x00000044,0x0000002c,
0x0004003d,0x00000006,0x00000045,0x00000035,0x0005008b,0x00000006,0x00000047,0x00000045,
0x00000046,0x00050082,0x00000006,0x00000048,0x00000047,0x0000002e,0x0004003d,0x00000006,
0x00000049,0x00000035,0x00050087,0x00000006,0x0000004a,0x00000049,0x00000046,0x00050082,
0x00000006,0x0000004b,0x0000004a,0x0000002e,0x00050050,0x00000007,0x0000004c,0x00000048,
0x0000004b,0x00050080,0x00000007,0x0000004d,0x00000044,0x0000004c,0x0004003d,0x00000007,
0x0000004f,0x00000020,0x0008000c,0x00000007,0x00000050,0x00000001,0x0000002d,0x0000004d,
0x0000004e,0x0000004f,0x00040064,0x00000021,0x00000051,0x00000043,0x0007005f,0x00000052,
0x00000053,0x00000051,0x00000050,0x00000002,0x00000026,0x00050051,0x00000013,0x00000055,
0x00000053,0x00000000,0x00050041,0x00000056,0x00000057,0x00000041,0x00000042,0x0003003e,
0x00000057,0x00000055,0x000200f9,0x00000039,0x000200f8,0x00000039,0x0004003d,0x00000006,
0x00000058,0x00000035,0x00050080,0x00000006,0x00000059,0x00000058,0x00000029,0x0003003e,
0x00000035,0x00000059,0x000200f9,0x00000036,0x000200f8,0x00000038,0x0003003e,0x0000005a,
0x0000005b,0x0003003e,0x0000005c,0x0000004e,0x0003003e,0x0000005d,0x0000005f,0x000200f9,
0x00000060,0x000200f8,0x00000060,0x000400f6,0x00000062,0x00000063,0x00000000,0x000200f9,
0x00000064,0x000200f8,0x00000064,0x0004003d,0x00000006,0x00000065,0x0000005d,0x000500b3,
0x00000019,0x00000066,0x00000065,0x0000005e,0x000400fa,0x00000066,0x00000061,0x00000062,
0x000200f8,0x00000061,0x0003003e,0x00000067,0x00000068,0x000200f9,0x00000069,0x000200f8,
0x00000069,0x000400f6,0x0000006b,0x0000006c,0x00000000,0x000200f9,0x0000006d,0x000200f8,
0x0000006d,0x0004003d,0x00000006,0x0000006e,0x00000067,0x000500b3,0x00000019,0x0000006f,
0x0000006e,0x0000005e,0x000400fa,0x0000006f,0x0000006a,0x0000006b,0x000200f8,0x0000006a,
0x0003003e,0x00000070,0x00000071,0x0003003e,0x00000072,0x00000026,0x000200f9,0x00000073,
0x000200f8,0x00000073,0x000400f6,0x00000075,0x00000076,0x00000000,0x000200f9,0x00000077,
0x000200f8,0x00000077,0x0004003d,0x00000006,0x00000078,0x00000072,0x000500b1,0x00000019,
0x00000079,0x00000078,0x0000003c,0x000400fa,0x00000079,0x00000074,0x00000075,0x000200f8,
0x00000074,0x0004003d,0x00000006,0x0000007a,0x00000072,0x00050041,0x00000056,0x0000007b,
0x00000041,0x0000007a,0x0004003d,0x00000013,0x0000007c,0x0000007b,0x0004003d,0x00000022,
0x0000007e,0x0000007d,0x0004003d,0x00000007,0x0000007f,0x0000002c,0x0004003d,0x00000006,
0x00000080,0x00000072,0x0005008b,0x00000006,0x00000081,0x00000080,0x00000046,0x00050082,
0x00000006,0x00000082,0x00000081,0x0000002e,0x0004003d,0x00000006,0x00000083,0x00000067,
0x00050080,0x00000006,0x00000084,0x00000082,0x00000083,0x0004003d,0x00000006,0x00000085,
0x00000072,0x00050087,0x00000006,0x00000086,0x00000085,0x00000046,0x00050082,0x00000006,
0x00000087,0x00000086,0x0000002e,0x0004003d,0x00000006,0x00000088,0x0000005d,0x00050080,
0x00000006,0x00000089,0x00000087,0x00000088,0x00050050,0x00000007,0x0000008a,0x00000084,
0x00000089,0x00050080,0x00000007,0x0000008b,0x0000007f,0x0000008a,0x0004003d,0x00000007,
0x0000008c,0x00000020,0x0008000c,0x00000007,0x0000008d,0x00000001,0x0000002d,0x0000008b,
0x0000004e,0x0000008c,0x00040064,0x00000021,0x0000008e,0x0000007e,0x0007005f,0x00000052,
0x0000008f,0x0000008e,0x0000008d,0x00000002,0x00000026,0x00050051,0x00000013,0x00000090,
0x0000008f,0x00000000,0x00050083,0x00000013,0x00000091,0x0000007c,0x00000090,0x0006000c,
0x00000013,0x00000092,0x00000001,0x00000004,0x00000091,0x0004003d,0x00000013,0x00000093,
0x00000070,0x00050081,0x00000013,0x00000094,0x00000093,0x00000092,0x0003003e,0x00000070,
0x00000094,0x000200f9,0x00000076,0x000200f8,0x00000076,0x0004003d,0x00000006,0x00000095,
0x00000072,0x00050080,0x00000006,0x00000096,0x00000095,0x00000029,0x0003003e,0x00000072,
0x00000096,0x000200f9,0x00000073,0x000200f8,0x00000075,0x0004003d,0x00000013,0x00000098,
0x00000070,0x0004003d,0x00000006,0x0000009a,0x00000067,0x0006000c,0x00000006,0x0000009b,
0x00000001,0x00000005,0x0000009a,0x0004003d,0x00000006,0x0000009c,0x0000005d,0x0006000c,
0x00000006,0x0000009d,0x00000001,0x00000005,0x0000009c,0x00050080,0x00000006,0x0000009e,
0x0000009b,0x0000009d,0x0004006f,0x00000013,0x0000009f,0x0000009e,0x00050085,0x00000013,
0x000000a0,0x00000099,0x0000009f,0x00050081,0x00000013,0x000000a1,0x00000098,0x000000a0,
0x0004003d,0x00000006,0x000000a2,0x00000067,0x000500ab,0x00000019,0x000000a3,0x000000a2,
0x00000026,0x0004003d,0x00000006,0x000000a4,0x0000005d,0x000500ab,0x00000019,0x000000a5,
0x000000a4,0x00000026,0x000500a6,0x00000019,0x000000a6,0x000000a3,0x000000a5,0x000600a9,
0x00000013,0x000000a8,0x000000a6,0x000000a7,0x00000071,0x00050081,0x00000013,0x000000a9,
0x000000a1,0x000000a8,0x0003003e,0x00000097,0x000000a9,0x0004003d,0x00000013,0x000000aa,
0x00000097,0x0004003d,0x00000013,0x000000ab,0x0000005a,0x000500b8,0x00000019,0x000000ac,
0x000000aa,0x000000ab,0x000300f7,0x000000ae,0x00000000,0x000400fa,0x000000ac,0x000000ad,
0x000000ae,0x000200f8,0x000000ad,0x0004003d,0x00000013,0x000000af,0x00000097,0x0003003e,
0x0000005a,0x000000af,0x0004003d,0x00000006,0x000000b0,0x00000067,0x0004003d,0x00000006,
0x000000b1,0x0000005d,0x00050050,0x00000007,0x000000b2,0x000000b0,0x000000b1,0x0003003e,
0x0000005c,0x000000b2,0x000200f9,0x000000ae,0x000200f8,0x000000ae,0x000200f9,0x0000006c,
0x000200f8,0x0000006c,0x0004003d,0x00000006,0x000000b3,0x00000067,0x00050080,0x00000006,
0x000000b4,0x000000b3,0x00000029,0x0003003e,0x00000067,0x000000b4,0x000200f9,0x00000069,
0x000200f8,0x0000006b,0x000200f9,0x00000063,0x000200f8,0x00000063,0x0004003d,0x00000006,
0x000000b5,0x0000005d,0x00050080,0x00000006,0x000000b6,0x000000b5,0x00000029,0x0003003e,
0x0000005d,0x000000b6,0x000200f9,0x00000060,0x000200f8,0x00000062,0x0004003d,0x00000014,
0x000000b7,0x00000016,0x0004003d,0x00000007,0x000000b8,0x00000009,0x0004003d,0x00000007,
0x000000b9,0x0000005c,0x0004006f,0x000000ba,0x000000bb,0x000000b9,0x0005008e,0x000000ba,
0x000000bd,0x000000bb,0x000000bc,0x0004003d,0x00000013,0x000000be,0x0000005a,0x00050088,
0x00000013,0x000000c0,0x000000be,0x000000bf,0x00050051,0x00000013,0x000000c2,0x000000bd,
0x00000000,0x00050051,0x00000013,0x000000c3,0x000000bd,0x00000001,0x00070050,0x00000052,
0x000000c4,0x000000c2,0x000000c3,0x000000c0,0x000000c1,0x00040063,0x000000b7,0x000000b8,
0x000000c4,0x000100fd,0x00010038
0x00000000,0x00040032,0x00000013,0x00000099,0x3f800000,0x0004002b,0x00000013,0x0000009a,
0x3df5c28f,0x0004002b,0x00000013,0x000000a7,0x3f19999a,0x00040017,0x000000bc,0x00000013,
0x00000002,0x0004002b,0x00000013,0x000000be,0x40800000,0x0004002b,0x00000013,0x000000c1,
0x42800000,0x0004002b,0x00000013,0x000000c3,0x3f800000,0x0004002b,0x0000000a,0x000000c7,
0x00000008,0x0004002b,0x0000000a,0x000000c8,0x00000001,0x0006002c,0x0000000b,0x000000c9,
0x000000c7,0x000000c7,0x000000c8,0x00050036,0x00000002,0x00000004,0x00000000,0x00000003,
0x000200f8,0x00000005,0x0004003b,0x00000008,0x00000009,0x00000007,0x0004003b,0x00000008,
0x00000020,0x00000007,0x0004003b,0x00000008,0x0000002c,0x00000007,0x0004003b,0x00000034,
0x00000035,0x00000007,0x0004003b,0x00000040,0x00000041,0x00000007,0x0004003b,0x00000056,
0x0000005a,0x00000007,0x0004003b,0x00000008,0x0000005c,0x00000007,0x0004003b,0x00000034,
0x0000005d,0x00000007,0x0004003b,0x00000034,0x00000067,0x00000007,0x0004003b,0x00000056,
0x00000070,0x00000007,0x0004003b,0x00000034,0x00000072,0x00000007,0x0004003b,0x00000056,
0x00000097,0x00000007,0x0004003d,0x0000000b,0x0000000f,0x0000000d,0x0007004f,0x0000000e,
0x00000010,0x0000000f,0x0000000f,0x00000000,0x00000001,0x0004007c,0x00000007,0x00000011,
0x00000010,0x0003003e,0x00000009,0x00000011,0x0004003d,0x00000007,0x00000012,0x00000009,
0x0004003d,0x00000014,0x00000017,0x00000016,0x00040068,0x00000007,0x00000018,0x00000017,
0x000500af,0x0000001a,0x0000001b,0x00000012,0x00000018,0x0004009a,0x00000019,0x0000001c,
0x0000001b,0x000300f7,0x0000001e,0x00000000,0x000400fa,0x0000001c,0x0000001d,0x0000001e,
0x000200f8,0x0000001d,0x000100fd,0x000200f8,0x0000001e,0x0004003d,0x00000022,0x00000025,
0x00000024,0x00040064,0x00000021,0x00000027,0x00000025,0x00050067,0x00000007,0x00000028,
0x00000027,0x00000026,0x00050050,0x00000007,0x0000002a,0x00000029,0x00000029,0x00050082,
0x00000007,0x0000002b,0x00000028,0x0000002a,0x0003003e,0x00000020,0x0000002b,0x0004003d,
0x00000007,0x0000002d,0x00000009,0x00050050,0x00000007,0x0000002f,0x0000002e,0x0000002e,
0x00050084,0x00000007,0x00000030,0x0000002d,0x0000002f,0x00050050,0x00000007,0x00000032,
0x00000031,0x00000031,0x00050080,0x00000007,0x00000033,0x00000030,0x00000032,0x0003003e,
0x0000002c,0x00000033,0x0003003e,0x00000035,0x00000026,0x000200f9,0x00000036,0x000200f8,
0x00000036,0x000400f6,0x00000038,0x00000039,0x00000000,0x000200f9,0x0000003a,0x000200f8,
0x0000003a,0x0004003d,0x00000006,0x0000003b,0x00000035,0x000500b1,0x00000019,0x0000003d,
0x0000003b,0x0000003c,0x000400fa,0x0000003d,0x00000037,0x00000038,0x000200f8,0x00000037,
0x0004003d,0x00000006,0x00000042,0x00000035,0x0004003d,0x00000022,0x00000043,0x00000024,
0x0004003d,0x00000007,0x00000044,0x0000002c,0x0004003d,0x00000006,0x00000045,0x00000035,
0x0005008b,0x00000006,0x00000047,0x00000045,0x00000046,0x00050082,0x00000006,0x00000048,
0x00000047,0x0000002e,0x0004003d,0x00000006,0x00000049,0x00000035,0x00050087,0x00000006,
0x0000004a,0x00000049,0x00000046,0x00050082,0x00000006,0x0000004b,0x0000004a,0x0000002e,
0x00050050,0x00000007,0x0000004c,0x00000048,0x0000004b,0x00050080,0x00000007,0x0000004d,
0x00000044,0x0000004c,0x0004003d,0x00000007,0x0000004f,0x00000020,0x0008000c,0x00000007,
0x00000050,0x00000001,0x0000002d,0x0000004d,0x0000004e,0x0000004f,0x00040064,0x00000021,
0x00000051,0x00000043,0x0007005f,0x00000052,0x00000053,0x00000051,0x00000050,0x00000002,
0x00000026,0x00050051,0x00000013,0x00000055,0x00000053,0x00000000,0x00050041,0x00000056,
0x00000057,0x00000041,0x00000042,0x0003003e,0x00000057,0x00000055,0x000200f9,0x00000039,
0x000200f8,0x00000039,0x0004003d,0x00000006,0x00000058,0x00000035,0x00050080,0x00000006,
0x00000059,0x00000058,0x00000029,0x0003003e,0x00000035,0x00000059,0x000200f9,0x00000036,
0x000200f8,0x00000038,0x0003003e,0x0000005a,0x0000005b,0x0003003e,0x0000005c,0x0000004e,
0x0003003e,0x0000005d,0x0000005f,0x000200f9,0x00000060,0x000200f8,0x00000060,0x000400f6,
0x00000062,0x00000063,0x00000000,0x000200f9,0x00000064,0x000200f8,0x00000064,0x0004003d,
0x00000006,0x00000065,0x0000005d,0x000500b3,0x00000019,0x00000066,0x00000065,0x0000005e,
0x000400fa,0x00000066,0x00000061,0x00000062,0x000200f8,0x00000061,0x0003003e,0x00000067,
0x00000068,0x000200f9,0x00000069,0x000200f8,0x00000069,0x000400f6,0x0000006b,0x0000006c,
0x00000000,0x000200f9,0x0000006d,0x000200f8,0x0000006d,0x0004003d,0x00000006,0x0000006e,
0x00000067,0x000500b3,0x00000019,0x0000006f,0x0000006e,0x0000005e,0x000400fa,0x0000006f,
0x0000006a,0x0000006b,0x000200f8,0x0000006a,0x0003003e,0x00000070,0x00000071,0x0003003e,
0x00000072,0x00000026,0x000200f9,0x00000073,0x000200f8,0x00000073,0x000400f6,0x00000075,
0x00000076,0x00000000,0x000200f9,0x00000077,0x000200f8,0x00000077,0x0004003d,0x00000006,
0x00000078,0x00000072,0x000500b1,0x00000019,0x00000079,0x00000078,0x0000003c,0x000400fa,
0x00000079,0x00000074,0x00000075,0x000200f8,0x00000074,0x0004003d,0x00000006,0x0000007a,
0x00000072,0x00050041,0x00000056,0x0000007b,0x00000041,0x0000007a,0x0004003d,0x00000013,
0x0000007c,0x0000007b,0x0004003d,0x00000022,0x0000007e,0x0000007d,0x0004003d,0x00000007,
0x0000007f,0x0000002c,0x0004003d,0x00000006,0x00000080,0x00000072,0x0005008b,0x00000006,
0x00000081,0x00000080,0x00000046,0x00050082,0x00000006,0x00000082,0x00000081,0x0000002e,
0x0004003d,0x00000006,0x00000083,0x00000067,0x00050080,0x00000006,0x00000084,0x00000082,
0x00000083,0x0004003d,0x00000006,0x00000085,0x00000072,0x00050087,0x00000006,0x00000086,
0x00000085,0x00000046,0x00050082,0x00000006,0x00000087,0x00000086,0x0000002e,0x0004003d,
0x00000006,0x00000088,0x0000005d,0x00050080,0x00000006,0x00000089,0x00000087,0x00000088,
0x00050050,0x00000007,0x0000008a,0x00000084,0x00000089,0x00050080,0x00000007,0x0000008b,
0x0000007f,0x0000008a,0x0004003d,0x00000007,0x0000008c,0x00000020,0x0008000c,0x00000007,
0x0000008d,0x00000001,0x0000002d,0x0000008b,0x0000004e,0x0000008c,0x00040064,0x00000021,
0x0000008e,0x0000007e,0x0007005f,0x00000052,0x0000008f,0x0000008e,0x0000008d,0x00000002,
0x00000026,0x00050051,0x00000013,0x00000090,0x0000008f,0x00000000,0x00050083,0x00000013,
0x00000091,0x0000007c,0x00000090,0x0006000c,0x00000013,0x00000092,0x00000001,0x00000004,
0x00000091,0x0004003d,0x00000013,0x00000093,0x00000070,0x00050081,0x00000013,0x00000094,
0x00000093,0x00000092,0x0003003e,0x00000070,0x00000094,0x000200f9,0x00000076,0x000200f8,
0x00000076,0x0004003d,0x00000006,0x00000095,0x00000072,0x00050080,0x00000006,0x00000096,
0x00000095,0x00000029,0x0003003e,0x00000072,0x00000096,0x000200f9,0x00000073,0x000200f8,
0x00000075,0x0004003d,0x00000013,0x00000098,0x00000070,0x0004003d,0x00000006,0x0000009b,
0x00000067,0x0006000c,0x00000006,0x0000009c,0x00000001,0x00000005,0x0000009b,0x0004003d,
0x00000006,0x0000009d,0x0000005d,0x0006000c,0x00000006,0x0000009e,0x00000001,0x00000005,
0x0000009d,0x00050080,0x00000006,0x0000009f,0x0000009c,0x0000009e,0x0004006f,0x00000013,
0x000000a0,0x0000009f,0x00050085,0x00000013,0x000000a1,0x0000009a,0x000000a0,0x0004003d,
0x00000006,0x000000a2,0x00000067,0x000500ab,0x00000019,0x000000a3,0x000000a2,0x00000026,
0x0004003d,0x00000006,0x000000a4,0x0000005d,0x000500ab,0x00000019,0x000000a5,0x000000a4,
0x00000026,0x000500a6,0x00000019,0x000000a6,0x000000a3,0x000000a5,0x000600a9,0x00000013,
0x000000a8,0x000000a6,0x000000a7,0x00000071,0x00050081,0x00000013,0x000000a9,0x000000a1,
0x000000a8,0x00050085,0x00000013,0x000000aa,0x00000099,0x000000a9,0x00050081,0x00000013,
0x000000ab,0x00000098,0x000000aa,0x0003003e,0x00000097,0x000000ab,0x0004003d,0x00000013,
0x000000ac,0x00000097,0x0004003d,0x00000013,0x000000ad,0x0000005a,0x000500b8,0x00000019,
0x000000ae,0x000000ac,0x000000ad,0x000300f7,0x000000b0,0x00000000,0x000400fa,0x000000ae,
0x000000af,0x000000b0,0x000200f8,0x000000af,0x0004003d,0x00000013,0x000000b1,0x00000097,
0x0003003e,0x0000005a,0x000000b1,0x0004003d,0x00000006,0x000000b2,0x00000067,0x0004003d,
0x00000006,0x000000b3,0x0000005d,0x00050050,0x00000007,0x000000b4,0x000000b2,0x000000b3,
0x0003003e,0x0000005c,0x000000b4,0x000200f9,0x000000b0,0x000200f8,0x000000b0,0x000200f9,
0x0000006c,0x000200f8,0x0000006c,0x0004003d,0x00000006,0x000000b5,0x00000067,0x00050080,
0x00000006,0x000000b6,0x000000b5,0x00000029,0x0003003e,0x00000067,0x000000b6,0x000200f9,
0x00000069,0x000200f8,0x0000006b,0x000200f9,0x00000063,0x000200f8,0x00000063,0x0004003d,
0x00000006,0x000000b7,0x0000005d,0x00050080,0x00000006,0x000000b8,0x000000b7,0x00000029,
0x0003003e,0x0000005d,0x000000b8,0x000200f9,0x00000060,0x000200f8,0x00000062,0x0004003d,
0x00000014,0x000000b9,0x00000016,0x0004003d,0x00000007,0x000000ba,0x00000009,0x0004003d,
0x00000007,0x000000bb,0x0000005c,0x0004006f,0x000000bc,0x000000bd,0x000000bb,0x0005008e,
0x000000bc,0x000000bf,0x000000bd,0x000000be,0x0004003d,0x00000013,0x000000c0,0x0000005a,
0x00050088,0x00000013,0x000000c2,0x000000c0,0x000000c1,0x00050051,0x00000013,0x000000c4,
0x000000bf,0x00000000,0x00050051,0x00000013,0x000000c5,0x000000bf,0x00000001,0x00070050,
0x00000052,0x000000c6,0x000000c4,0x000000c5,0x000000c2,0x000000c3,0x00040063,0x000000b9,
0x000000ba,0x000000c6,0x000100fd,0x00010038
};

View File

@@ -0,0 +1,15 @@
#version 450
// (Triforce) half-size luma of a frame (each texel the mean of 2x2 pixels), for the fine vectors (fg_fine_flow.comp)
layout(local_size_x = 8, local_size_y = 8) in;
layout(binding = 0) uniform sampler2D src;
layout(binding = 1, r16f) uniform writeonly image2D dst;
void main() {
const ivec2 g = ivec2(gl_GlobalInvocationID.xy);
if (any(greaterThanEqual(g, imageSize(dst)))) return;
const ivec2 s = textureSize(src, 0) - 1;
float sum = 0.0;
for (int y = 0; y < 2; y++)
for (int x = 0; x < 2; x++)
sum += dot(texelFetch(src, min(g * 2 + ivec2(x, y), s), 0).rgb, vec3(0.299, 0.587, 0.114));
imageStore(dst, g, vec4(sum / 4.0));
}

View File

@@ -0,0 +1,75 @@
// 1116.6.0
#pragma once
const uint32_t FG_HALF_COMP_SPV[] = {
0x07230203,0x00010500,0x0008000b,0x00000064,0x00000000,0x00020011,0x00000001,0x00020011,
0x00000031,0x00020011,0x00000032,0x0006000b,0x00000001,0x4c534c47,0x6474732e,0x3035342e,
0x00000000,0x0003000e,0x00000000,0x00000001,0x0008000f,0x00000005,0x00000004,0x6e69616d,
0x00000000,0x0000000d,0x00000016,0x00000024,0x00060010,0x00000004,0x00000011,0x00000008,
0x00000008,0x00000001,0x00030003,0x00000002,0x000001c2,0x00040005,0x00000004,0x6e69616d,
0x00000000,0x00030005,0x00000009,0x00000067,0x00080005,0x0000000d,0x475f6c67,0x61626f6c,
0x766e496c,0x7461636f,0x496e6f69,0x00000044,0x00030005,0x00000016,0x00747364,0x00030005,
0x00000020,0x00000073,0x00030005,0x00000024,0x00637273,0x00030005,0x0000002d,0x006d7573,
0x00030005,0x00000030,0x00000079,0x00030005,0x00000039,0x00000078,0x00040047,0x0000000d,
0x0000000b,0x0000001c,0x00030047,0x00000016,0x00000019,0x00040047,0x00000016,0x00000021,
0x00000001,0x00040047,0x00000016,0x00000022,0x00000000,0x00040047,0x00000024,0x00000021,
0x00000000,0x00040047,0x00000024,0x00000022,0x00000000,0x00040047,0x00000063,0x0000000b,
0x00000019,0x00020013,0x00000002,0x00030021,0x00000003,0x00000002,0x00040015,0x00000006,
0x00000020,0x00000001,0x00040017,0x00000007,0x00000006,0x00000002,0x00040020,0x00000008,
0x00000007,0x00000007,0x00040015,0x0000000a,0x00000020,0x00000000,0x00040017,0x0000000b,
0x0000000a,0x00000003,0x00040020,0x0000000c,0x00000001,0x0000000b,0x0004003b,0x0000000c,
0x0000000d,0x00000001,0x00040017,0x0000000e,0x0000000a,0x00000002,0x00030016,0x00000013,
0x00000020,0x00090019,0x00000014,0x00000013,0x00000001,0x00000000,0x00000000,0x00000000,
0x00000002,0x00000009,0x00040020,0x00000015,0x00000000,0x00000014,0x0004003b,0x00000015,
0x00000016,0x00000000,0x00020014,0x00000019,0x00040017,0x0000001a,0x00000019,0x00000002,
0x00090019,0x00000021,0x00000013,0x00000001,0x00000000,0x00000000,0x00000000,0x00000001,
0x00000000,0x0003001b,0x00000022,0x00000021,0x00040020,0x00000023,0x00000000,0x00000022,
0x0004003b,0x00000023,0x00000024,0x00000000,0x0004002b,0x00000006,0x00000026,0x00000000,
0x0004002b,0x00000006,0x00000029,0x00000001,0x00040020,0x0000002c,0x00000007,0x00000013,
0x0004002b,0x00000013,0x0000002e,0x00000000,0x00040020,0x0000002f,0x00000007,0x00000006,
0x0004002b,0x00000006,0x00000037,0x00000002,0x00040017,0x0000004c,0x00000013,0x00000004,
0x00040017,0x0000004e,0x00000013,0x00000003,0x0004002b,0x00000013,0x00000050,0x3e991687,
0x0004002b,0x00000013,0x00000051,0x3f1645a2,0x0004002b,0x00000013,0x00000052,0x3de978d5,
0x0006002c,0x0000004e,0x00000053,0x00000050,0x00000051,0x00000052,0x0004002b,0x00000013,
0x0000005e,0x40800000,0x0004002b,0x0000000a,0x00000061,0x00000008,0x0004002b,0x0000000a,
0x00000062,0x00000001,0x0006002c,0x0000000b,0x00000063,0x00000061,0x00000061,0x00000062,
0x00050036,0x00000002,0x00000004,0x00000000,0x00000003,0x000200f8,0x00000005,0x0004003b,
0x00000008,0x00000009,0x00000007,0x0004003b,0x00000008,0x00000020,0x00000007,0x0004003b,
0x0000002c,0x0000002d,0x00000007,0x0004003b,0x0000002f,0x00000030,0x00000007,0x0004003b,
0x0000002f,0x00000039,0x00000007,0x0004003d,0x0000000b,0x0000000f,0x0000000d,0x0007004f,
0x0000000e,0x00000010,0x0000000f,0x0000000f,0x00000000,0x00000001,0x0004007c,0x00000007,
0x00000011,0x00000010,0x0003003e,0x00000009,0x00000011,0x0004003d,0x00000007,0x00000012,
0x00000009,0x0004003d,0x00000014,0x00000017,0x00000016,0x00040068,0x00000007,0x00000018,
0x00000017,0x000500af,0x0000001a,0x0000001b,0x00000012,0x00000018,0x0004009a,0x00000019,
0x0000001c,0x0000001b,0x000300f7,0x0000001e,0x00000000,0x000400fa,0x0000001c,0x0000001d,
0x0000001e,0x000200f8,0x0000001d,0x000100fd,0x000200f8,0x0000001e,0x0004003d,0x00000022,
0x00000025,0x00000024,0x00040064,0x00000021,0x00000027,0x00000025,0x00050067,0x00000007,
0x00000028,0x00000027,0x00000026,0x00050050,0x00000007,0x0000002a,0x00000029,0x00000029,
0x00050082,0x00000007,0x0000002b,0x00000028,0x0000002a,0x0003003e,0x00000020,0x0000002b,
0x0003003e,0x0000002d,0x0000002e,0x0003003e,0x00000030,0x00000026,0x000200f9,0x00000031,
0x000200f8,0x00000031,0x000400f6,0x00000033,0x00000034,0x00000000,0x000200f9,0x00000035,
0x000200f8,0x00000035,0x0004003d,0x00000006,0x00000036,0x00000030,0x000500b1,0x00000019,
0x00000038,0x00000036,0x00000037,0x000400fa,0x00000038,0x00000032,0x00000033,0x000200f8,
0x00000032,0x0003003e,0x00000039,0x00000026,0x000200f9,0x0000003a,0x000200f8,0x0000003a,
0x000400f6,0x0000003c,0x0000003d,0x00000000,0x000200f9,0x0000003e,0x000200f8,0x0000003e,
0x0004003d,0x00000006,0x0000003f,0x00000039,0x000500b1,0x00000019,0x00000040,0x0000003f,
0x00000037,0x000400fa,0x00000040,0x0000003b,0x0000003c,0x000200f8,0x0000003b,0x0004003d,
0x00000022,0x00000041,0x00000024,0x0004003d,0x00000007,0x00000042,0x00000009,0x00050050,
0x00000007,0x00000043,0x00000037,0x00000037,0x00050084,0x00000007,0x00000044,0x00000042,
0x00000043,0x0004003d,0x00000006,0x00000045,0x00000039,0x0004003d,0x00000006,0x00000046,
0x00000030,0x00050050,0x00000007,0x00000047,0x00000045,0x00000046,0x00050080,0x00000007,
0x00000048,0x00000044,0x00000047,0x0004003d,0x00000007,0x00000049,0x00000020,0x0007000c,
0x00000007,0x0000004a,0x00000001,0x00000027,0x00000048,0x00000049,0x00040064,0x00000021,
0x0000004b,0x00000041,0x0007005f,0x0000004c,0x0000004d,0x0000004b,0x0000004a,0x00000002,
0x00000026,0x0008004f,0x0000004e,0x0000004f,0x0000004d,0x0000004d,0x00000000,0x00000001,
0x00000002,0x00050094,0x00000013,0x00000054,0x0000004f,0x00000053,0x0004003d,0x00000013,
0x00000055,0x0000002d,0x00050081,0x00000013,0x00000056,0x00000055,0x00000054,0x0003003e,
0x0000002d,0x00000056,0x000200f9,0x0000003d,0x000200f8,0x0000003d,0x0004003d,0x00000006,
0x00000057,0x00000039,0x00050080,0x00000006,0x00000058,0x00000057,0x00000029,0x0003003e,
0x00000039,0x00000058,0x000200f9,0x0000003a,0x000200f8,0x0000003c,0x000200f9,0x00000034,
0x000200f8,0x00000034,0x0004003d,0x00000006,0x00000059,0x00000030,0x00050080,0x00000006,
0x0000005a,0x00000059,0x00000029,0x0003003e,0x00000030,0x0000005a,0x000200f9,0x00000031,
0x000200f8,0x00000033,0x0004003d,0x00000014,0x0000005b,0x00000016,0x0004003d,0x00000007,
0x0000005c,0x00000009,0x0004003d,0x00000013,0x0000005d,0x0000002d,0x00050088,0x00000013,
0x0000005f,0x0000005d,0x0000005e,0x00070050,0x0000004c,0x00000060,0x0000005f,0x0000005f,
0x0000005f,0x0000005f,0x00040063,0x0000005b,0x0000005c,0x00000060,0x000100fd,0x00010038
};

View File

@@ -0,0 +1,28 @@
#ifndef TRIFORCE_RENODX_OUTPUT_SLANGI
#define TRIFORCE_RENODX_OUTPUT_SLANGI
// (Triforce) The gamma correction of NX2navi's HDR on RenoDX's Zelda OoT output encoder. scripts/renodx_shaders.py
// compiles this file in front of RenoDX's encoder, the frame's last draw: it reads the finished picture (the scene and
// the HUD over it, linear, white at 1) once and writes it as HDR10. The game's picture is made for a display of gamma
// about 2.2, and the sRGB curve it is decoded with is linear near black, which leaves the darkest tones lighter than
// such a display shows them. With the correction on, the encoder is given the picture as a display of gamma 2.2 or 2.4
// shows its sRGB code (RenoDX's renodx::color::correct::GammaSafe); off, the picture as it is, and the encoder does what
// it did without this file. The tone mappers are given a peak with the correction taken out of it, so that the brightest
// light still lands on the display's peak (gpu/vk/vk_renodx.cpp peak_before_gamma).
#include "triforce_settings.slangi"
float3 TriforceGammaCorrect(float3 color, float gamma_correction) {
if (gamma_correction == 1.f) return renodx::color::correct::GammaSafe(color, false, 2.2f);
if (gamma_correction == 2.f) return renodx::color::correct::GammaSafe(color, false, 2.4f);
return color;
}
/// The encoder's one read of the picture
vec4 TriforceOutputPicture(sampler2D picture, vec2 at) {
vec4 color = texture(picture, at);
color.rgb = TriforceGammaCorrect(color.rgb, triforce_constants.settings.gamma_correction);
return color;
}
// (from here on, which is RenoDX's encoder: its texture() is the read above)
#define texture TriforceOutputPicture
#endif

View File

@@ -0,0 +1,54 @@
#ifndef TRIFORCE_RENODX_SCENE_SLANGI
#define TRIFORCE_RENODX_SCENE_SLANGI
// (Triforce) RenoDRT as a third tone mapper of RenoDX's Zelda OoT scene shaders, which have Vanilla and PsychoV35.
// scripts/renodx_shaders.py compiles this file in front of each of RenoDX's two scene shaders. Those call
// ApplySceneToneMap (RenoDX's scene_tonemap.slangi) for every tone mapper but Vanilla; here RenoDX's own keeps its body
// under another name and the one the shaders call chooses: RenoDRT (tone_map_type 3, RenoDX's number for it) goes to
// RenoDX's library, anything else to RenoDX's mapper as it is. So with Vanilla or PsychoV35 a scene shader does what it
// did without this file.
//
// What RenoDRT does with the scene is what NX2navi's HDR does with it (runtime/shaders/hdr_tonemap.frag, tone mapper 3)
// and RenoDX's own games do: the game's SDR picture of the scene is the colour grade, carried up to the display's range
// by brightness against a neutral SDR curve of the same scene (renodx::draw::ToneMapPass), with RenoDRT's rows.
#include "triforce_settings.slangi"
#define ApplySceneToneMap RenoDXApplySceneToneMap
#include "scene_tonemap.slangi" // (RenoDX's)
#undef ApplySceneToneMap
/// The game's own SDR picture of what it feeds its tone curve: that curve (Narkowicz's fit of ACES, with the numbers the
/// game's shader has) and its clamp to 0..1
float3 TriforceGameSdr(float3 x) {
return saturate(x * (2.51f * x + 0.03f) / (x * (2.43f * x + 0.59f) + 0.14f));
}
/// x: what the game feeds its tone curve (the scene with bloom, times the game's exposure). config: RenoDX's, with the
/// peak, the whites and the four grading rows in it. Returns linear Rec. 709 with the menus' white at 1, as RenoDX's own
/// scene mapper does
float3 TriforceRenoDRT(float3 x, renodx::draw::Config config, TriforceSettings settings) {
config.tone_map_type = renodx::draw::TONE_MAP_TYPE_RENO_DRT;
// (the peak comes with the gamma correction taken out of it already: vk_renodx.cpp peak_before_gamma)
config.gamma_correction = renodx::draw::GAMMA_CORRECTION_NONE;
config.color_grade_strength = settings.color_grade_strength;
config.tone_map_highlight_saturation = settings.highlight_saturation;
config.tone_map_blowout = settings.blowout;
config.tone_map_flare = settings.flare;
config.tone_map_hue_correction = settings.hue_correction;
float3 mapped = renodx::draw::ToneMapPass(x, TriforceGameSdr(x), config);
mapped *= config.diffuse_white_nits / config.graphics_white_nits;
// (as NX2navi's: a channel that is not a number is black)
return float3(isnan(mapped.x) ? 0.f : mapped.x, isnan(mapped.y) ? 0.f : mapped.y, isnan(mapped.z) ? 0.f : mapped.z);
}
float3 ApplySceneToneMap(float3 custom_untonemapped, float3 custom_native_gain) {
if (RENODX_TONE_MAP_TYPE == renodx::draw::TONE_MAP_TYPE_RENO_DRT) {
// (the game's exposure is the gain's first entry: both of the game's scene shaders multiply each channel by that one
// number before their curve. RenoDX's shader with local exposure hands its own mapper a gain whose blue is that
// number squared; RenoDRT starts from what the game's curve takes, so it takes the number itself for all three)
return TriforceRenoDRT(custom_untonemapped * custom_native_gain.x, renodx::draw::BuildConfig(),
triforce_constants.settings);
}
return RenoDXApplySceneToneMap(custom_untonemapped, custom_native_gain);
}
#endif

View File

@@ -0,0 +1,26 @@
#ifndef TRIFORCE_RENODX_SETTINGS_SLANGI
#define TRIFORCE_RENODX_SETTINGS_SLANGI
// (Triforce) The settings RenoDX's Zelda OoT mod has no place for in its own block (shared.h ShaderInjectData, set 1
// binding 4): the gamma correction and what RenoDRT alone reads. gpu/vk/vk_renodx.cpp fills them from the Insert menu's rows
// (struct Extra, the same six floats in the same order) and gives them to every program with one of RenoDX's stages as
// set 1 binding 5.
//
// This file and the two beside it are Triforce's own. They hold none of RenoDX's code: scripts/renodx_shaders.py compiles
// them together with RenoDX's sources as it fetched them (github.com/clshortfuse/renodx, MIT, Carlos Lopez), and what they
// call of RenoDX is its library as it is there.
#include "shared.h" // (RenoDX's: its settings and, through renodx.hlsl, its library)
struct TriforceSettings {
float gamma_correction; // 0 none, 1 a display of gamma 2.2, 2 of 2.4 (triforce_output.slangi)
float highlight_saturation; // RenoDRT: 1 unchanged
float blowout; // RenoDRT: 0 none
float flare; // RenoDRT: 0 none
float hue_correction; // RenoDRT: 1 the hue from before the tone mapper
float color_grade_strength; // RenoDRT: 1 all of the game's own grading
};
layout(set = 1, binding = 5, scalar) uniform TriforceShaderConstants {
TriforceSettings settings;
} triforce_constants;
#endif

View File

@@ -0,0 +1,20 @@
#version 450
// (Triforce) SMAA pass 3 (gpu/vk/vk_aa.cpp): neighbourhood blending of the tone mapped picture by the weights, in linear light
// (as SMAA asks of a picture read through an sRGB view), stored into the game's target as its raw RGB9E5 words of the square
// root (aa_common.glsl). After NX2navi's shaders/smaa_blend.comp.
#extension GL_GOOGLE_include_directive : require
#include "smaa_setup.glsl"
#include "aa_common.glsl"
layout(binding = 0) uniform sampler2D color; // (the picture the game's pass samples, linear filtering)
layout(binding = 1) uniform sampler2D weights; // (linear filtering)
layout(binding = 2, r32ui) uniform writeonly uimage2D dst;
void main() {
ivec2 p;
vec2 uv;
if (!smaa_pixel(p, uv)) return;
vec4 offset;
SMAANeighborhoodBlendingVS(uv, offset);
vec3 c = SMAANeighborhoodBlendingPS(uv, offset, color, weights).rgb;
imageStore(dst, p, uvec4(aa_pack_rgb9e5(aa_encode(c)), 0u, 0u, 0u));
}

View File

@@ -0,0 +1,326 @@
// smaa_blend.comp + smaa_setup.glsl + SMAA.hlsl + aa_common.glsl sha256 710e81e6db30bd4dbeb585f7dbef8dc2434b0b5fdcdb979415cc642ae980aa00 (tests/test_aa_shaders.py --write: glslangValidator -V --target-env vulkan1.2 -I third_party/smaa --vn SMAA_BLEND_COMP_SPV)
// 1116.6.0
#pragma once
const uint32_t SMAA_BLEND_COMP_SPV[] = {
0x07230203,0x00010500,0x0008000b,0x000001aa,0x00000000,0x00020011,0x00000001,0x0006000b,
0x00000001,0x4c534c47,0x6474732e,0x3035342e,0x00000000,0x0003000e,0x00000000,0x00000001,
0x000a000f,0x00000005,0x00000004,0x6e69616d,0x00000000,0x00000076,0x000000fe,0x00000194,
0x00000195,0x0000019e,0x00060010,0x00000004,0x00000011,0x00000008,0x00000008,0x00000001,
0x00030003,0x00000002,0x000001c2,0x000a0004,0x475f4c47,0x4c474f4f,0x70635f45,0x74735f70,
0x5f656c79,0x656e696c,0x7269645f,0x69746365,0x00006576,0x00080004,0x475f4c47,0x4c474f4f,
0x6e695f45,0x64756c63,0x69645f65,0x74636572,0x00657669,0x00040005,0x00000004,0x6e69616d,
0x00000000,0x00080005,0x00000010,0x41414d53,0x63766f4d,0x32627628,0x3266763b,0x3266763b,
0x0000003b,0x00040005,0x0000000d,0x646e6f63,0x00000000,0x00050005,0x0000000e,0x69726176,
0x656c6261,0x00000000,0x00040005,0x0000000f,0x756c6176,0x00000065,0x00080005,0x0000001a,
0x41414d53,0x63766f4d,0x34627628,0x3466763b,0x3466763b,0x0000003b,0x00040005,0x00000017,
0x646e6f63,0x00000000,0x00050005,0x00000018,0x69726176,0x656c6261,0x00000000,0x00040005,
0x00000019,0x756c6176,0x00000065,0x000b0005,0x0000001f,0x41414d53,0x6769654e,0x726f6268,
0x646f6f68,0x6e656c42,0x676e6964,0x76285356,0x763b3266,0x003b3466,0x00050005,0x0000001d,
0x63786574,0x64726f6f,0x00000000,0x00040005,0x0000001e,0x7366666f,0x00007465,0x000d0005,
0x00000029,0x41414d53,0x6769654e,0x726f6268,0x646f6f68,0x6e656c42,0x676e6964,0x76285350,
0x763b3266,0x733b3466,0x733b3132,0x003b3132,0x00050005,0x00000025,0x63786574,0x64726f6f,
0x00000000,0x00040005,0x00000026,0x7366666f,0x00007465,0x00050005,0x00000027,0x6f6c6f63,
0x78655472,0x00000000,0x00050005,0x00000028,0x6e656c62,0x78655464,0x00000000,0x00070005,
0x00000031,0x61616d73,0x7869705f,0x76286c65,0x763b3269,0x003b3266,0x00030005,0x0000002f,
0x00000070,0x00030005,0x00000030,0x00007675,0x00070005,0x00000036,0x635f6161,0x6e6e6168,
0x395f6c65,0x66283565,0x00003b31,0x00030005,0x00000035,0x00000076,0x00070005,0x0000003d,
0x705f6161,0x5f6b6361,0x39626772,0x76283565,0x003b3366,0x00030005,0x0000003c,0x00000063,
0x00060005,0x00000041,0x655f6161,0x646f636e,0x66762865,0x00003b33,0x00030005,0x00000040,
0x00000063,0x00040005,0x00000054,0x61726170,0x0000006d,0x00040005,0x00000057,0x61726170,
0x0000006d,0x00040005,0x0000005a,0x61726170,0x0000006d,0x00040005,0x00000063,0x61726170,
0x0000006d,0x00040005,0x00000066,0x61726170,0x0000006d,0x00040005,0x00000069,0x61726170,
0x0000006d,0x00040005,0x00000074,0x68737550,0x00000000,0x00050006,0x00000074,0x00000000,
0x7274656d,0x00736369,0x00030005,0x00000076,0x00006370,0x00030005,0x00000082,0x00000061,
0x00040005,0x0000009e,0x6f6c6f63,0x00000072,0x00030005,0x000000a5,0x00000068,0x00060005,
0x000000b1,0x6e656c62,0x676e6964,0x7366664f,0x00007465,0x00060005,0x000000b7,0x6e656c62,
0x676e6964,0x67696557,0x00007468,0x00040005,0x000000c1,0x61726170,0x0000006d,0x00040005,
0x000000c2,0x61726170,0x0000006d,0x00040005,0x000000c4,0x61726170,0x0000006d,0x00040005,
0x000000c9,0x61726170,0x0000006d,0x00040005,0x000000ca,0x61726170,0x0000006d,0x00040005,
0x000000cc,0x61726170,0x0000006d,0x00060005,0x000000d7,0x6e656c62,0x676e6964,0x726f6f43,
0x00000064,0x00040005,0x000000e8,0x6f6c6f63,0x00000072,0x00080005,0x000000fe,0x475f6c67,
0x61626f6c,0x766e496c,0x7461636f,0x496e6f69,0x00000044,0x00030005,0x0000012b,0x00000072,
0x00040005,0x0000012c,0x61726170,0x0000006d,0x00030005,0x00000130,0x00000067,0x00040005,
0x00000131,0x61726170,0x0000006d,0x00030005,0x00000135,0x00000062,0x00040005,0x00000136,
0x61726170,0x0000006d,0x00030005,0x0000013a,0x0000006d,0x00030005,0x00000140,0x00000065,
0x00040005,0x0000014a,0x6c616373,0x00000065,0x00030005,0x00000182,0x00000070,0x00030005,
0x00000183,0x00007675,0x00040005,0x00000184,0x61726170,0x0000006d,0x00040005,0x00000185,
0x61726170,0x0000006d,0x00040005,0x0000018d,0x7366666f,0x00007465,0x00040005,0x0000018e,
0x61726170,0x0000006d,0x00040005,0x00000190,0x61726170,0x0000006d,0x00030005,0x00000193,
0x00000063,0x00040005,0x00000194,0x6f6c6f63,0x00000072,0x00040005,0x00000195,0x67696577,
0x00737468,0x00040005,0x00000196,0x61726170,0x0000006d,0x00040005,0x00000198,0x61726170,
0x0000006d,0x00030005,0x0000019e,0x00747364,0x00040005,0x000001a1,0x61726170,0x0000006d,
0x00040005,0x000001a4,0x61726170,0x0000006d,0x00030047,0x00000074,0x00000002,0x00050048,
0x00000074,0x00000000,0x00000023,0x00000000,0x00040047,0x000000fe,0x0000000b,0x0000001c,
0x00040047,0x00000194,0x00000021,0x00000000,0x00040047,0x00000194,0x00000022,0x00000000,
0x00040047,0x00000195,0x00000021,0x00000001,0x00040047,0x00000195,0x00000022,0x00000000,
0x00030047,0x0000019e,0x00000019,0x00040047,0x0000019e,0x00000021,0x00000002,0x00040047,
0x0000019e,0x00000022,0x00000000,0x00040047,0x000001a9,0x0000000b,0x00000019,0x00020013,
0x00000002,0x00030021,0x00000003,0x00000002,0x00020014,0x00000006,0x00040017,0x00000007,
0x00000006,0x00000002,0x00040020,0x00000008,0x00000007,0x00000007,0x00030016,0x00000009,
0x00000020,0x00040017,0x0000000a,0x00000009,0x00000002,0x00040020,0x0000000b,0x00000007,
0x0000000a,0x00060021,0x0000000c,0x00000002,0x00000008,0x0000000b,0x0000000b,0x00040017,
0x00000012,0x00000006,0x00000004,0x00040020,0x00000013,0x00000007,0x00000012,0x00040017,
0x00000014,0x00000009,0x00000004,0x00040020,0x00000015,0x00000007,0x00000014,0x00060021,
0x00000016,0x00000002,0x00000013,0x00000015,0x00000015,0x00050021,0x0000001c,0x00000002,
0x0000000b,0x00000015,0x00090019,0x00000021,0x00000009,0x00000001,0x00000000,0x00000000,
0x00000000,0x00000001,0x00000000,0x0003001b,0x00000022,0x00000021,0x00040020,0x00000023,
0x00000000,0x00000022,0x00070021,0x00000024,0x00000014,0x0000000b,0x00000015,0x00000023,
0x00000023,0x00040015,0x0000002b,0x00000020,0x00000001,0x00040017,0x0000002c,0x0000002b,
0x00000002,0x00040020,0x0000002d,0x00000007,0x0000002c,0x00050021,0x0000002e,0x00000006,
0x0000002d,0x0000000b,0x00040020,0x00000033,0x00000007,0x00000009,0x00040021,0x00000034,
0x00000009,0x00000033,0x00040017,0x00000038,0x00000009,0x00000003,0x00040020,0x00000039,
0x00000007,0x00000038,0x00040015,0x0000003a,0x00000020,0x00000000,0x00040021,0x0000003b,
0x0000003a,0x00000039,0x00040021,0x0000003f,0x00000038,0x00000039,0x0004002b,0x0000003a,
0x00000043,0x00000000,0x00040020,0x00000044,0x00000007,0x00000006,0x0004002b,0x0000003a,
0x0000004c,0x00000001,0x0004002b,0x0000003a,0x0000006e,0x00000002,0x0004002b,0x0000003a,
0x00000071,0x00000003,0x0003001e,0x00000074,0x00000014,0x00040020,0x00000075,0x00000009,
0x00000074,0x0004003b,0x00000075,0x00000076,0x00000009,0x0004002b,0x0000002b,0x00000077,
0x00000000,0x00040020,0x00000078,0x00000009,0x00000014,0x0004002b,0x00000009,0x0000007c,
0x3f800000,0x0004002b,0x00000009,0x0000007d,0x00000000,0x0007002c,0x00000014,0x0000007e,
0x0000007c,0x0000007d,0x0000007d,0x0000007c,0x0007002c,0x00000014,0x00000098,0x0000007c,
0x0000007c,0x0000007c,0x0000007c,0x0004002b,0x00000009,0x0000009a,0x3727c5ac,0x0005002c,
0x0000000a,0x000000d2,0x0000007c,0x0000007c,0x00040017,0x000000fc,0x0000003a,0x00000003,
0x00040020,0x000000fd,0x00000001,0x000000fc,0x0004003b,0x000000fd,0x000000fe,0x00000001,
0x00040017,0x000000ff,0x0000003a,0x00000002,0x0004002b,0x00000009,0x00000105,0x3f000000,
0x00040020,0x0000010c,0x00000007,0x0000002b,0x00040020,0x0000010f,0x00000009,0x00000009,
0x0004002b,0x00000009,0x00000125,0x477f8000,0x0004002b,0x00000009,0x00000141,0xc1800000,
0x0004002b,0x00000009,0x00000143,0x0da24260,0x0004002b,0x00000009,0x00000148,0x41800000,
0x0004002b,0x00000009,0x0000014c,0x41c00000,0x0004002b,0x00000009,0x00000154,0x44000000,
0x0004002b,0x00000009,0x0000015a,0x40000000,0x0004002b,0x0000002b,0x00000169,0x00000009,
0x0004002b,0x0000002b,0x00000172,0x00000012,0x0004002b,0x0000002b,0x00000177,0x0000001b,
0x0006002c,0x00000038,0x0000017d,0x0000007d,0x0000007d,0x0000007d,0x0004003b,0x00000023,
0x00000194,0x00000000,0x0004003b,0x00000023,0x00000195,0x00000000,0x00090019,0x0000019c,
0x0000003a,0x00000001,0x00000000,0x00000000,0x00000000,0x00000002,0x00000021,0x00040020,
0x0000019d,0x00000000,0x0000019c,0x0004003b,0x0000019d,0x0000019e,0x00000000,0x00040017,
0x000001a6,0x0000003a,0x00000004,0x0004002b,0x0000003a,0x000001a8,0x00000008,0x0006002c,
0x000000fc,0x000001a9,0x000001a8,0x000001a8,0x0000004c,0x00050036,0x00000002,0x00000004,
0x00000000,0x00000003,0x000200f8,0x00000005,0x0004003b,0x0000002d,0x00000182,0x00000007,
0x0004003b,0x0000000b,0x00000183,0x00000007,0x0004003b,0x0000002d,0x00000184,0x00000007,
0x0004003b,0x0000000b,0x00000185,0x00000007,0x0004003b,0x00000015,0x0000018d,0x00000007,
0x0004003b,0x0000000b,0x0000018e,0x00000007,0x0004003b,0x00000015,0x00000190,0x00000007,
0x0004003b,0x00000039,0x00000193,0x00000007,0x0004003b,0x0000000b,0x00000196,0x00000007,
0x0004003b,0x00000015,0x00000198,0x00000007,0x0004003b,0x00000039,0x000001a1,0x00000007,
0x0004003b,0x00000039,0x000001a4,0x00000007,0x00060039,0x00000006,0x00000186,0x00000031,
0x00000184,0x00000185,0x0004003d,0x0000002c,0x00000187,0x00000184,0x0003003e,0x00000182,
0x00000187,0x0004003d,0x0000000a,0x00000188,0x00000185,0x0003003e,0x00000183,0x00000188,
0x000400a8,0x00000006,0x00000189,0x00000186,0x000300f7,0x0000018b,0x00000000,0x000400fa,
0x00000189,0x0000018a,0x0000018b,0x000200f8,0x0000018a,0x000100fd,0x000200f8,0x0000018b,
0x0004003d,0x0000000a,0x0000018f,0x00000183,0x0003003e,0x0000018e,0x0000018f,0x00060039,
0x00000002,0x00000191,0x0000001f,0x0000018e,0x00000190,0x0004003d,0x00000014,0x00000192,
0x00000190,0x0003003e,0x0000018d,0x00000192,0x0004003d,0x0000000a,0x00000197,0x00000183,
0x0003003e,0x00000196,0x00000197,0x0004003d,0x00000014,0x00000199,0x0000018d,0x0003003e,
0x00000198,0x00000199,0x00080039,0x00000014,0x0000019a,0x00000029,0x00000196,0x00000198,
0x00000194,0x00000195,0x0008004f,0x00000038,0x0000019b,0x0000019a,0x0000019a,0x00000000,
0x00000001,0x00000002,0x0003003e,0x00000193,0x0000019b,0x0004003d,0x0000019c,0x0000019f,
0x0000019e,0x0004003d,0x0000002c,0x000001a0,0x00000182,0x0004003d,0x00000038,0x000001a2,
0x00000193,0x0003003e,0x000001a1,0x000001a2,0x00050039,0x00000038,0x000001a3,0x00000041,
0x000001a1,0x0003003e,0x000001a4,0x000001a3,0x00050039,0x0000003a,0x000001a5,0x0000003d,
0x000001a4,0x00070050,0x000001a6,0x000001a7,0x000001a5,0x00000043,0x00000043,0x00000043,
0x00050063,0x0000019f,0x000001a0,0x000001a7,0x00002000,0x000100fd,0x00010038,0x00050036,
0x00000002,0x00000010,0x00000000,0x0000000c,0x00030037,0x00000008,0x0000000d,0x00030037,
0x0000000b,0x0000000e,0x00030037,0x0000000b,0x0000000f,0x000200f8,0x00000011,0x00050041,
0x00000044,0x00000045,0x0000000d,0x00000043,0x0004003d,0x00000006,0x00000046,0x00000045,
0x000300f7,0x00000048,0x00000000,0x000400fa,0x00000046,0x00000047,0x00000048,0x000200f8,
0x00000047,0x00050041,0x00000033,0x00000049,0x0000000f,0x00000043,0x0004003d,0x00000009,
0x0000004a,0x00000049,0x00050041,0x00000033,0x0000004b,0x0000000e,0x00000043,0x0003003e,
0x0000004b,0x0000004a,0x000200f9,0x00000048,0x000200f8,0x00000048,0x00050041,0x00000044,
0x0000004d,0x0000000d,0x0000004c,0x0004003d,0x00000006,0x0000004e,0x0000004d,0x000300f7,
0x00000050,0x00000000,0x000400fa,0x0000004e,0x0000004f,0x00000050,0x000200f8,0x0000004f,
0x00050041,0x00000033,0x00000051,0x0000000f,0x0000004c,0x0004003d,0x00000009,0x00000052,
0x00000051,0x00050041,0x00000033,0x00000053,0x0000000e,0x0000004c,0x0003003e,0x00000053,
0x00000052,0x000200f9,0x00000050,0x000200f8,0x00000050,0x000100fd,0x00010038,0x00050036,
0x00000002,0x0000001a,0x00000000,0x00000016,0x00030037,0x00000013,0x00000017,0x00030037,
0x00000015,0x00000018,0x00030037,0x00000015,0x00000019,0x000200f8,0x0000001b,0x0004003b,
0x00000008,0x00000054,0x00000007,0x0004003b,0x0000000b,0x00000057,0x00000007,0x0004003b,
0x0000000b,0x0000005a,0x00000007,0x0004003b,0x00000008,0x00000063,0x00000007,0x0004003b,
0x0000000b,0x00000066,0x00000007,0x0004003b,0x0000000b,0x00000069,0x00000007,0x0004003d,
0x00000012,0x00000055,0x00000017,0x0007004f,0x00000007,0x00000056,0x00000055,0x00000055,
0x00000000,0x00000001,0x0003003e,0x00000054,0x00000056,0x0004003d,0x00000014,0x00000058,
0x00000018,0x0007004f,0x0000000a,0x00000059,0x00000058,0x00000058,0x00000000,0x00000001,
0x0003003e,0x00000057,0x00000059,0x0004003d,0x00000014,0x0000005b,0x00000019,0x0007004f,
0x0000000a,0x0000005c,0x0000005b,0x0000005b,0x00000000,0x00000001,0x0003003e,0x0000005a,
0x0000005c,0x00070039,0x00000002,0x0000005d,0x00000010,0x00000054,0x00000057,0x0000005a,
0x0004003d,0x0000000a,0x0000005e,0x00000057,0x00050041,0x00000033,0x0000005f,0x00000018,
0x00000043,0x00050051,0x00000009,0x00000060,0x0000005e,0x00000000,0x0003003e,0x0000005f,
0x00000060,0x00050041,0x00000033,0x00000061,0x00000018,0x0000004c,0x00050051,0x00000009,
0x00000062,0x0000005e,0x00000001,0x0003003e,0x00000061,0x00000062,0x0004003d,0x00000012,
0x00000064,0x00000017,0x0007004f,0x00000007,0x00000065,0x00000064,0x00000064,0x00000002,
0x00000003,0x0003003e,0x00000063,0x00000065,0x0004003d,0x00000014,0x00000067,0x00000018,
0x0007004f,0x0000000a,0x00000068,0x00000067,0x00000067,0x00000002,0x00000003,0x0003003e,
0x00000066,0x00000068,0x0004003d,0x00000014,0x0000006a,0x00000019,0x0007004f,0x0000000a,
0x0000006b,0x0000006a,0x0000006a,0x00000002,0x00000003,0x0003003e,0x00000069,0x0000006b,
0x00070039,0x00000002,0x0000006c,0x00000010,0x00000063,0x00000066,0x00000069,0x0004003d,
0x0000000a,0x0000006d,0x00000066,0x00050041,0x00000033,0x0000006f,0x00000018,0x0000006e,
0x00050051,0x00000009,0x00000070,0x0000006d,0x00000000,0x0003003e,0x0000006f,0x00000070,
0x00050041,0x00000033,0x00000072,0x00000018,0x00000071,0x00050051,0x00000009,0x00000073,
0x0000006d,0x00000001,0x0003003e,0x00000072,0x00000073,0x000100fd,0x00010038,0x00050036,
0x00000002,0x0000001f,0x00000000,0x0000001c,0x00030037,0x0000000b,0x0000001d,0x00030037,
0x00000015,0x0000001e,0x000200f8,0x00000020,0x00050041,0x00000078,0x00000079,0x00000076,
0x00000077,0x0004003d,0x00000014,0x0000007a,0x00000079,0x0009004f,0x00000014,0x0000007b,
0x0000007a,0x0000007a,0x00000000,0x00000001,0x00000000,0x00000001,0x0004003d,0x0000000a,
0x0000007f,0x0000001d,0x0009004f,0x00000014,0x00000080,0x0000007f,0x0000007f,0x00000000,
0x00000001,0x00000000,0x00000001,0x0008000c,0x00000014,0x00000081,0x00000001,0x00000032,
0x0000007b,0x0000007e,0x00000080,0x0003003e,0x0000001e,0x00000081,0x000100fd,0x00010038,
0x00050036,0x00000014,0x00000029,0x00000000,0x00000024,0x00030037,0x0000000b,0x00000025,
0x00030037,0x00000015,0x00000026,0x00030037,0x00000023,0x00000027,0x00030037,0x00000023,
0x00000028,0x000200f8,0x0000002a,0x0004003b,0x00000015,0x00000082,0x00000007,0x0004003b,
0x00000015,0x0000009e,0x00000007,0x0004003b,0x00000044,0x000000a5,0x00000007,0x0004003b,
0x00000015,0x000000b1,0x00000007,0x0004003b,0x0000000b,0x000000b7,0x00000007,0x0004003b,
0x00000013,0x000000c1,0x00000007,0x0004003b,0x00000015,0x000000c2,0x00000007,0x0004003b,
0x00000015,0x000000c4,0x00000007,0x0004003b,0x00000008,0x000000c9,0x00000007,0x0004003b,
0x0000000b,0x000000ca,0x00000007,0x0004003b,0x0000000b,0x000000cc,0x00000007,0x0004003b,
0x00000015,0x000000d7,0x00000007,0x0004003b,0x00000015,0x000000e8,0x00000007,0x0004003d,
0x00000022,0x00000083,0x00000028,0x0004003d,0x00000014,0x00000084,0x00000026,0x0007004f,
0x0000000a,0x00000085,0x00000084,0x00000084,0x00000000,0x00000001,0x00070058,0x00000014,
0x00000086,0x00000083,0x00000085,0x00000002,0x0000007d,0x00050051,0x00000009,0x00000087,
0x00000086,0x00000003,0x00050041,0x00000033,0x00000088,0x00000082,0x00000043,0x0003003e,
0x00000088,0x00000087,0x0004003d,0x00000022,0x00000089,0x00000028,0x0004003d,0x00000014,
0x0000008a,0x00000026,0x0007004f,0x0000000a,0x0000008b,0x0000008a,0x0000008a,0x00000002,
0x00000003,0x00070058,0x00000014,0x0000008c,0x00000089,0x0000008b,0x00000002,0x0000007d,
0x00050051,0x00000009,0x0000008d,0x0000008c,0x00000001,0x00050041,0x00000033,0x0000008e,
0x00000082,0x0000004c,0x0003003e,0x0000008e,0x0000008d,0x0004003d,0x00000022,0x0000008f,
0x00000028,0x0004003d,0x0000000a,0x00000090,0x00000025,0x00070058,0x00000014,0x00000091,
0x0000008f,0x00000090,0x00000002,0x0000007d,0x0007004f,0x0000000a,0x00000092,0x00000091,
0x00000091,0x00000000,0x00000002,0x00050041,0x00000033,0x00000093,0x00000082,0x00000071,
0x00050051,0x00000009,0x00000094,0x00000092,0x00000000,0x0003003e,0x00000093,0x00000094,
0x00050041,0x00000033,0x00000095,0x00000082,0x0000006e,0x00050051,0x00000009,0x00000096,
0x00000092,0x00000001,0x0003003e,0x00000095,0x00000096,0x0004003d,0x00000014,0x00000097,
0x00000082,0x00050094,0x00000009,0x00000099,0x00000097,0x00000098,0x000500b8,0x00000006,
0x0000009b,0x00000099,0x0000009a,0x000300f7,0x0000009d,0x00000000,0x000400fa,0x0000009b,
0x0000009c,0x000000a4,0x000200f8,0x0000009c,0x0004003d,0x00000022,0x0000009f,0x00000027,
0x0004003d,0x0000000a,0x000000a0,0x00000025,0x00070058,0x00000014,0x000000a1,0x0000009f,
0x000000a0,0x00000002,0x0000007d,0x0003003e,0x0000009e,0x000000a1,0x0004003d,0x00000014,
0x000000a2,0x0000009e,0x000200fe,0x000000a2,0x000200f8,0x000000a4,0x00050041,0x00000033,
0x000000a6,0x00000082,0x00000043,0x0004003d,0x00000009,0x000000a7,0x000000a6,0x00050041,
0x00000033,0x000000a8,0x00000082,0x0000006e,0x0004003d,0x00000009,0x000000a9,0x000000a8,
0x0007000c,0x00000009,0x000000aa,0x00000001,0x00000028,0x000000a7,0x000000a9,0x00050041,
0x00000033,0x000000ab,0x00000082,0x0000004c,0x0004003d,0x00000009,0x000000ac,0x000000ab,
0x00050041,0x00000033,0x000000ad,0x00000082,0x00000071,0x0004003d,0x00000009,0x000000ae,
0x000000ad,0x0007000c,0x00000009,0x000000af,0x00000001,0x00000028,0x000000ac,0x000000ae,
0x000500ba,0x00000006,0x000000b0,0x000000aa,0x000000af,0x0003003e,0x000000a5,0x000000b0,
0x00050041,0x00000033,0x000000b2,0x00000082,0x0000004c,0x0004003d,0x00000009,0x000000b3,
0x000000b2,0x00050041,0x00000033,0x000000b4,0x00000082,0x00000071,0x0004003d,0x00000009,
0x000000b5,0x000000b4,0x00070050,0x00000014,0x000000b6,0x0000007d,0x000000b3,0x0000007d,
0x000000b5,0x0003003e,0x000000b1,0x000000b6,0x0004003d,0x00000014,0x000000b8,0x00000082,
0x0007004f,0x0000000a,0x000000b9,0x000000b8,0x000000b8,0x00000001,0x00000003,0x0003003e,
0x000000b7,0x000000b9,0x0004003d,0x00000006,0x000000ba,0x000000a5,0x00070050,0x00000012,
0x000000bb,0x000000ba,0x000000ba,0x000000ba,0x000000ba,0x00050041,0x00000033,0x000000bc,
0x00000082,0x00000043,0x0004003d,0x00000009,0x000000bd,0x000000bc,0x00050041,0x00000033,
0x000000be,0x00000082,0x0000006e,0x0004003d,0x00000009,0x000000bf,0x000000be,0x00070050,
0x00000014,0x000000c0,0x000000bd,0x0000007d,0x000000bf,0x0000007d,0x0003003e,0x000000c1,
0x000000bb,0x0004003d,0x00000014,0x000000c3,0x000000b1,0x0003003e,0x000000c2,0x000000c3,
0x0003003e,0x000000c4,0x000000c0,0x00070039,0x00000002,0x000000c5,0x0000001a,0x000000c1,
0x000000c2,0x000000c4,0x0004003d,0x00000014,0x000000c6,0x000000c2,0x0003003e,0x000000b1,
0x000000c6,0x0004003d,0x00000006,0x000000c7,0x000000a5,0x00050050,0x00000007,0x000000c8,
0x000000c7,0x000000c7,0x0003003e,0x000000c9,0x000000c8,0x0004003d,0x0000000a,0x000000cb,
0x000000b7,0x0003003e,0x000000ca,0x000000cb,0x0004003d,0x00000014,0x000000cd,0x00000082,
0x0007004f,0x0000000a,0x000000ce,0x000000cd,0x000000cd,0x00000000,0x00000002,0x0003003e,
0x000000cc,0x000000ce,0x00070039,0x00000002,0x000000cf,0x00000010,0x000000c9,0x000000ca,
0x000000cc,0x0004003d,0x0000000a,0x000000d0,0x000000ca,0x0003003e,0x000000b7,0x000000d0,
0x0004003d,0x0000000a,0x000000d1,0x000000b7,0x00050094,0x00000009,0x000000d3,0x000000d1,
0x000000d2,0x0004003d,0x0000000a,0x000000d4,0x000000b7,0x00050050,0x0000000a,0x000000d5,
0x000000d3,0x000000d3,0x00050088,0x0000000a,0x000000d6,0x000000d4,0x000000d5,0x0003003e,
0x000000b7,0x000000d6,0x0004003d,0x00000014,0x000000d8,0x000000b1,0x00050041,0x00000078,
0x000000d9,0x00000076,0x00000077,0x0004003d,0x00000014,0x000000da,0x000000d9,0x0007004f,
0x0000000a,0x000000db,0x000000da,0x000000da,0x00000000,0x00000001,0x00050041,0x00000078,
0x000000dc,0x00000076,0x00000077,0x0004003d,0x00000014,0x000000dd,0x000000dc,0x0007004f,
0x0000000a,0x000000de,0x000000dd,0x000000dd,0x00000000,0x00000001,0x0004007f,0x0000000a,
0x000000df,0x000000de,0x00050051,0x00000009,0x000000e0,0x000000db,0x00000000,0x00050051,
0x00000009,0x000000e1,0x000000db,0x00000001,0x00050051,0x00000009,0x000000e2,0x000000df,
0x00000000,0x00050051,0x00000009,0x000000e3,0x000000df,0x00000001,0x00070050,0x00000014,
0x000000e4,0x000000e0,0x000000e1,0x000000e2,0x000000e3,0x0004003d,0x0000000a,0x000000e5,
0x00000025,0x0009004f,0x00000014,0x000000e6,0x000000e5,0x000000e5,0x00000000,0x00000001,
0x00000000,0x00000001,0x0008000c,0x00000014,0x000000e7,0x00000001,0x00000032,0x000000d8,
0x000000e4,0x000000e6,0x0003003e,0x000000d7,0x000000e7,0x00050041,0x00000033,0x000000e9,
0x000000b7,0x00000043,0x0004003d,0x00000009,0x000000ea,0x000000e9,0x0004003d,0x00000022,
0x000000eb,0x00000027,0x0004003d,0x00000014,0x000000ec,0x000000d7,0x0007004f,0x0000000a,
0x000000ed,0x000000ec,0x000000ec,0x00000000,0x00000001,0x00070058,0x00000014,0x000000ee,
0x000000eb,0x000000ed,0x00000002,0x0000007d,0x0005008e,0x00000014,0x000000ef,0x000000ee,
0x000000ea,0x0003003e,0x000000e8,0x000000ef,0x00050041,0x00000033,0x000000f0,0x000000b7,
0x0000004c,0x0004003d,0x00000009,0x000000f1,0x000000f0,0x0004003d,0x00000022,0x000000f2,
0x00000027,0x0004003d,0x00000014,0x000000f3,0x000000d7,0x0007004f,0x0000000a,0x000000f4,
0x000000f3,0x000000f3,0x00000002,0x00000003,0x00070058,0x00000014,0x000000f5,0x000000f2,
0x000000f4,0x00000002,0x0000007d,0x0005008e,0x00000014,0x000000f6,0x000000f5,0x000000f1,
0x0004003d,0x00000014,0x000000f7,0x000000e8,0x00050081,0x00000014,0x000000f8,0x000000f7,
0x000000f6,0x0003003e,0x000000e8,0x000000f8,0x0004003d,0x00000014,0x000000f9,0x000000e8,
0x000200fe,0x000000f9,0x000200f8,0x0000009d,0x000100ff,0x00010038,0x00050036,0x00000006,
0x00000031,0x00000000,0x0000002e,0x00030037,0x0000002d,0x0000002f,0x00030037,0x0000000b,
0x00000030,0x000200f8,0x00000032,0x0004003d,0x000000fc,0x00000100,0x000000fe,0x0007004f,
0x000000ff,0x00000101,0x00000100,0x00000100,0x00000000,0x00000001,0x0004007c,0x0000002c,
0x00000102,0x00000101,0x0003003e,0x0000002f,0x00000102,0x0004003d,0x0000002c,0x00000103,
0x0000002f,0x0004006f,0x0000000a,0x00000104,0x00000103,0x00050050,0x0000000a,0x00000106,
0x00000105,0x00000105,0x00050081,0x0000000a,0x00000107,0x00000104,0x00000106,0x00050041,
0x00000078,0x00000108,0x00000076,0x00000077,0x0004003d,0x00000014,0x00000109,0x00000108,
0x0007004f,0x0000000a,0x0000010a,0x00000109,0x00000109,0x00000000,0x00000001,0x00050085,
0x0000000a,0x0000010b,0x00000107,0x0000010a,0x0003003e,0x00000030,0x0000010b,0x00050041,
0x0000010c,0x0000010d,0x0000002f,0x00000043,0x0004003d,0x0000002b,0x0000010e,0x0000010d,
0x00060041,0x0000010f,0x00000110,0x00000076,0x00000077,0x0000006e,0x0004003d,0x00000009,
0x00000111,0x00000110,0x0004006e,0x0000002b,0x00000112,0x00000111,0x000500b1,0x00000006,
0x00000113,0x0000010e,0x00000112,0x000300f7,0x00000115,0x00000000,0x000400fa,0x00000113,
0x00000114,0x00000115,0x000200f8,0x00000114,0x00050041,0x0000010c,0x00000116,0x0000002f,
0x0000004c,0x0004003d,0x0000002b,0x00000117,0x00000116,0x00060041,0x0000010f,0x00000118,
0x00000076,0x00000077,0x00000071,0x0004003d,0x00000009,0x00000119,0x00000118,0x0004006e,
0x0000002b,0x0000011a,0x00000119,0x000500b1,0x00000006,0x0000011b,0x00000117,0x0000011a,
0x000200f9,0x00000115,0x000200f8,0x00000115,0x000700f5,0x00000006,0x0000011c,0x00000113,
0x00000032,0x0000011b,0x00000114,0x000200fe,0x0000011c,0x00010038,0x00050036,0x00000009,
0x00000036,0x00000000,0x00000034,0x00030037,0x00000033,0x00000035,0x000200f8,0x00000037,
0x0004003b,0x00000033,0x00000121,0x00000007,0x0004003d,0x00000009,0x0000011f,0x00000035,
0x000500ba,0x00000006,0x00000120,0x0000011f,0x0000007d,0x000300f7,0x00000123,0x00000000,
0x000400fa,0x00000120,0x00000122,0x00000127,0x000200f8,0x00000122,0x0004003d,0x00000009,
0x00000124,0x00000035,0x0007000c,0x00000009,0x00000126,0x00000001,0x00000025,0x00000124,
0x00000125,0x0003003e,0x00000121,0x00000126,0x000200f9,0x00000123,0x000200f8,0x00000127,
0x0003003e,0x00000121,0x0000007d,0x000200f9,0x00000123,0x000200f8,0x00000123,0x0004003d,
0x00000009,0x00000128,0x00000121,0x000200fe,0x00000128,0x00010038,0x00050036,0x0000003a,
0x0000003d,0x00000000,0x0000003b,0x00030037,0x00000039,0x0000003c,0x000200f8,0x0000003e,
0x0004003b,0x00000033,0x0000012b,0x00000007,0x0004003b,0x00000033,0x0000012c,0x00000007,
0x0004003b,0x00000033,0x00000130,0x00000007,0x0004003b,0x00000033,0x00000131,0x00000007,
0x0004003b,0x00000033,0x00000135,0x00000007,0x0004003b,0x00000033,0x00000136,0x00000007,
0x0004003b,0x00000033,0x0000013a,0x00000007,0x0004003b,0x00000033,0x00000140,0x00000007,
0x0004003b,0x00000033,0x0000014a,0x00000007,0x00050041,0x00000033,0x0000012d,0x0000003c,
0x00000043,0x0004003d,0x00000009,0x0000012e,0x0000012d,0x0003003e,0x0000012c,0x0000012e,
0x00050039,0x00000009,0x0000012f,0x00000036,0x0000012c,0x0003003e,0x0000012b,0x0000012f,
0x00050041,0x00000033,0x00000132,0x0000003c,0x0000004c,0x0004003d,0x00000009,0x00000133,
0x00000132,0x0003003e,0x00000131,0x00000133,0x00050039,0x00000009,0x00000134,0x00000036,
0x00000131,0x0003003e,0x00000130,0x00000134,0x00050041,0x00000033,0x00000137,0x0000003c,
0x0000006e,0x0004003d,0x00000009,0x00000138,0x00000137,0x0003003e,0x00000136,0x00000138,
0x00050039,0x00000009,0x00000139,0x00000036,0x00000136,0x0003003e,0x00000135,0x00000139,
0x0004003d,0x00000009,0x0000013b,0x0000012b,0x0004003d,0x00000009,0x0000013c,0x00000130,
0x0007000c,0x00000009,0x0000013d,0x00000001,0x00000028,0x0000013b,0x0000013c,0x0004003d,
0x00000009,0x0000013e,0x00000135,0x0007000c,0x00000009,0x0000013f,0x00000001,0x00000028,
0x0000013d,0x0000013e,0x0003003e,0x0000013a,0x0000013f,0x0004003d,0x00000009,0x00000142,
0x0000013a,0x0007000c,0x00000009,0x00000144,0x00000001,0x00000028,0x00000142,0x00000143,
0x0006000c,0x00000009,0x00000145,0x00000001,0x0000001e,0x00000144,0x0006000c,0x00000009,
0x00000146,0x00000001,0x00000008,0x00000145,0x0007000c,0x00000009,0x00000147,0x00000001,
0x00000028,0x00000141,0x00000146,0x00050081,0x00000009,0x00000149,0x00000147,0x00000148,
0x0003003e,0x00000140,0x00000149,0x0004003d,0x00000009,0x0000014b,0x00000140,0x00050083,
0x00000009,0x0000014d,0x0000014b,0x0000014c,0x0006000c,0x00000009,0x0000014e,0x00000001,
0x0000001d,0x0000014d,0x0003003e,0x0000014a,0x0000014e,0x0004003d,0x00000009,0x0000014f,
0x0000013a,0x0004003d,0x00000009,0x00000150,0x0000014a,0x00050088,0x00000009,0x00000151,
0x0000014f,0x00000150,0x00050081,0x00000009,0x00000152,0x00000151,0x00000105,0x0006000c,
0x00000009,0x00000153,0x00000001,0x00000008,0x00000152,0x000500be,0x00000006,0x00000155,
0x00000153,0x00000154,0x000300f7,0x00000157,0x00000000,0x000400fa,0x00000155,0x00000156,
0x00000157,0x000200f8,0x00000156,0x0004003d,0x00000009,0x00000158,0x00000140,0x00050081,
0x00000009,0x00000159,0x00000158,0x0000007c,0x0003003e,0x00000140,0x00000159,0x0004003d,
0x00000009,0x0000015b,0x0000014a,0x00050085,0x00000009,0x0000015c,0x0000015b,0x0000015a,
0x0003003e,0x0000014a,0x0000015c,0x000200f9,0x00000157,0x000200f8,0x00000157,0x0004003d,
0x00000009,0x0000015d,0x0000012b,0x0004003d,0x00000009,0x0000015e,0x0000014a,0x00050088,
0x00000009,0x0000015f,0x0000015d,0x0000015e,0x00050081,0x00000009,0x00000160,0x0000015f,
0x00000105,0x0006000c,0x00000009,0x00000161,0x00000001,0x00000008,0x00000160,0x0004006d,
0x0000003a,0x00000162,0x00000161,0x0004003d,0x00000009,0x00000163,0x00000130,0x0004003d,
0x00000009,0x00000164,0x0000014a,0x00050088,0x00000009,0x00000165,0x00000163,0x00000164,
0x00050081,0x00000009,0x00000166,0x00000165,0x00000105,0x0006000c,0x00000009,0x00000167,
0x00000001,0x00000008,0x00000166,0x0004006d,0x0000003a,0x00000168,0x00000167,0x000500c4,
0x0000003a,0x0000016a,0x00000168,0x00000169,0x000500c5,0x0000003a,0x0000016b,0x00000162,
0x0000016a,0x0004003d,0x00000009,0x0000016c,0x00000135,0x0004003d,0x00000009,0x0000016d,
0x0000014a,0x00050088,0x00000009,0x0000016e,0x0000016c,0x0000016d,0x00050081,0x00000009,
0x0000016f,0x0000016e,0x00000105,0x0006000c,0x00000009,0x00000170,0x00000001,0x00000008,
0x0000016f,0x0004006d,0x0000003a,0x00000171,0x00000170,0x000500c4,0x0000003a,0x00000173,
0x00000171,0x00000172,0x000500c5,0x0000003a,0x00000174,0x0000016b,0x00000173,0x0004003d,
0x00000009,0x00000175,0x00000140,0x0004006d,0x0000003a,0x00000176,0x00000175,0x000500c4,
0x0000003a,0x00000178,0x00000176,0x00000177,0x000500c5,0x0000003a,0x00000179,0x00000174,
0x00000178,0x000200fe,0x00000179,0x00010038,0x00050036,0x00000038,0x00000041,0x00000000,
0x0000003f,0x00030037,0x00000039,0x00000040,0x000200f8,0x00000042,0x0004003d,0x00000038,
0x0000017c,0x00000040,0x0007000c,0x00000038,0x0000017e,0x00000001,0x00000028,0x0000017c,
0x0000017d,0x0006000c,0x00000038,0x0000017f,0x00000001,0x0000001f,0x0000017e,0x000200fe,
0x0000017f,0x00010038
};

View File

@@ -0,0 +1,16 @@
#version 450
// (Triforce) SMAA pass 1 (gpu/vk/vk_aa.cpp): luma edge detection over pass 0's luma into the edges image (red: an edge on the
// pixel's left, green: on its top). After NX2navi's shaders/smaa_edge.comp.
#extension GL_GOOGLE_include_directive : require
#include "smaa_setup.glsl"
layout(binding = 0) uniform sampler2D luma; // (smaa_luma.comp's, linear filtering)
layout(binding = 1, rgba8) uniform writeonly image2D edges;
void main() {
ivec2 p;
vec2 uv;
if (!smaa_pixel(p, uv)) return;
vec4 offset[3];
SMAAEdgeDetectionVS(uv, offset);
imageStore(edges, p, vec4(SMAALumaEdgeDetectionPS(uv, offset, luma), 0.0, 0.0));
}

View File

@@ -0,0 +1,225 @@
// smaa_edge.comp + smaa_setup.glsl + SMAA.hlsl sha256 e6dd83ec2989025609557990fb28892b814e6a4fa1c15c9641890fa440d2de2a (tests/test_aa_shaders.py --write: glslangValidator -V --target-env vulkan1.2 -I third_party/smaa --vn SMAA_EDGE_COMP_SPV)
// 1116.6.0
#pragma once
const uint32_t SMAA_EDGE_COMP_SPV[] = {
0x07230203,0x00010500,0x0008000b,0x00000122,0x00000000,0x00020011,0x00000001,0x0006000b,
0x00000001,0x4c534c47,0x6474732e,0x3035342e,0x00000000,0x0003000e,0x00000000,0x00000001,
0x0009000f,0x00000005,0x00000004,0x6e69616d,0x00000000,0x00000028,0x000000e0,0x00000114,
0x00000117,0x00060010,0x00000004,0x00000011,0x00000008,0x00000008,0x00000001,0x00030003,
0x00000002,0x000001c2,0x000a0004,0x475f4c47,0x4c474f4f,0x70635f45,0x74735f70,0x5f656c79,
0x656e696c,0x7269645f,0x69746365,0x00006576,0x00080004,0x475f4c47,0x4c474f4f,0x6e695f45,
0x64756c63,0x69645f65,0x74636572,0x00657669,0x00040005,0x00000004,0x6e69616d,0x00000000,
0x000a0005,0x00000011,0x41414d53,0x65676445,0x65746544,0x6f697463,0x2853566e,0x3b326676,
0x5b346676,0x003b5d33,0x00050005,0x0000000f,0x63786574,0x64726f6f,0x00000000,0x00040005,
0x00000010,0x7366666f,0x00007465,0x000c0005,0x0000001a,0x41414d53,0x616d754c,0x65676445,
0x65746544,0x6f697463,0x2853506e,0x3b326676,0x5b346676,0x733b5d33,0x003b3132,0x00050005,
0x00000017,0x63786574,0x64726f6f,0x00000000,0x00040005,0x00000018,0x7366666f,0x00007465,
0x00050005,0x00000019,0x6f6c6f63,0x78655472,0x00000000,0x00070005,0x00000023,0x61616d73,
0x7869705f,0x76286c65,0x763b3269,0x003b3266,0x00030005,0x00000021,0x00000070,0x00030005,
0x00000022,0x00007675,0x00040005,0x00000026,0x68737550,0x00000000,0x00050006,0x00000026,
0x00000000,0x7274656d,0x00736369,0x00030005,0x00000028,0x00006370,0x00050005,0x00000049,
0x65726874,0x6c6f6873,0x00000064,0x00040005,0x0000004e,0x67696577,0x00737468,0x00030005,
0x00000054,0x0000004c,0x00040005,0x0000005b,0x66656c4c,0x00000074,0x00040005,0x00000064,
0x706f744c,0x00000000,0x00040005,0x0000006d,0x746c6564,0x00000061,0x00040005,0x0000007b,
0x65676465,0x00000073,0x00040005,0x00000086,0x6769724c,0x00007468,0x00040005,0x0000008f,
0x746f624c,0x006d6f74,0x00050005,0x000000a4,0x4478616d,0x61746c65,0x00000000,0x00050005,
0x000000aa,0x66656c4c,0x66656c74,0x00000074,0x00040005,0x000000b3,0x706f744c,0x00706f74,
0x00050005,0x000000cc,0x616e6966,0x6c65446c,0x00006174,0x00080005,0x000000e0,0x475f6c67,
0x61626f6c,0x766e496c,0x7461636f,0x496e6f69,0x00000044,0x00030005,0x00000101,0x00000070,
0x00030005,0x00000102,0x00007675,0x00040005,0x00000103,0x61726170,0x0000006d,0x00040005,
0x00000104,0x61726170,0x0000006d,0x00040005,0x0000010c,0x7366666f,0x00007465,0x00040005,
0x0000010d,0x61726170,0x0000006d,0x00040005,0x0000010f,0x61726170,0x0000006d,0x00040005,
0x00000114,0x65676465,0x00000073,0x00040005,0x00000117,0x616d756c,0x00000000,0x00040005,
0x00000118,0x61726170,0x0000006d,0x00040005,0x0000011a,0x61726170,0x0000006d,0x00030047,
0x00000026,0x00000002,0x00050048,0x00000026,0x00000000,0x00000023,0x00000000,0x00040047,
0x000000e0,0x0000000b,0x0000001c,0x00030047,0x00000114,0x00000019,0x00040047,0x00000114,
0x00000021,0x00000001,0x00040047,0x00000114,0x00000022,0x00000000,0x00040047,0x00000117,
0x00000021,0x00000000,0x00040047,0x00000117,0x00000022,0x00000000,0x00040047,0x00000121,
0x0000000b,0x00000019,0x00020013,0x00000002,0x00030021,0x00000003,0x00000002,0x00030016,
0x00000006,0x00000020,0x00040017,0x00000007,0x00000006,0x00000002,0x00040020,0x00000008,
0x00000007,0x00000007,0x00040017,0x00000009,0x00000006,0x00000004,0x00040015,0x0000000a,
0x00000020,0x00000000,0x0004002b,0x0000000a,0x0000000b,0x00000003,0x0004001c,0x0000000c,
0x00000009,0x0000000b,0x00040020,0x0000000d,0x00000007,0x0000000c,0x00050021,0x0000000e,
0x00000002,0x00000008,0x0000000d,0x00090019,0x00000013,0x00000006,0x00000001,0x00000000,
0x00000000,0x00000000,0x00000001,0x00000000,0x0003001b,0x00000014,0x00000013,0x00040020,
0x00000015,0x00000000,0x00000014,0x00060021,0x00000016,0x00000007,0x00000008,0x0000000d,
0x00000015,0x00040015,0x0000001c,0x00000020,0x00000001,0x00040017,0x0000001d,0x0000001c,
0x00000002,0x00040020,0x0000001e,0x00000007,0x0000001d,0x00020014,0x0000001f,0x00050021,
0x00000020,0x0000001f,0x0000001e,0x00000008,0x0004002b,0x0000001c,0x00000025,0x00000000,
0x0003001e,0x00000026,0x00000009,0x00040020,0x00000027,0x00000009,0x00000026,0x0004003b,
0x00000027,0x00000028,0x00000009,0x00040020,0x00000029,0x00000009,0x00000009,0x0004002b,
0x00000006,0x0000002d,0xbf800000,0x0004002b,0x00000006,0x0000002e,0x00000000,0x0007002c,
0x00000009,0x0000002f,0x0000002d,0x0000002e,0x0000002e,0x0000002d,0x00040020,0x00000033,
0x00000007,0x00000009,0x0004002b,0x0000001c,0x00000035,0x00000001,0x0004002b,0x00000006,
0x00000039,0x3f800000,0x0007002c,0x00000009,0x0000003a,0x00000039,0x0000002e,0x0000002e,
0x00000039,0x0004002b,0x0000001c,0x0000003f,0x00000002,0x0004002b,0x00000006,0x00000043,
0xc0000000,0x0007002c,0x00000009,0x00000044,0x00000043,0x0000002e,0x0000002e,0x00000043,
0x0004002b,0x00000006,0x0000004a,0x3d4ccccd,0x0005002c,0x00000007,0x0000004b,0x0000004a,
0x0000004a,0x00040017,0x0000004c,0x00000006,0x00000003,0x00040020,0x0000004d,0x00000007,
0x0000004c,0x0004002b,0x00000006,0x0000004f,0x3e59b3d0,0x0004002b,0x00000006,0x00000050,
0x3f371759,0x0004002b,0x00000006,0x00000051,0x3d93dd98,0x0006002c,0x0000004c,0x00000052,
0x0000004f,0x00000050,0x00000051,0x00040020,0x00000053,0x00000007,0x00000006,0x0004002b,
0x0000000a,0x00000075,0x00000000,0x0004002b,0x0000000a,0x00000078,0x00000001,0x0005002c,
0x00000007,0x00000081,0x00000039,0x00000039,0x0004002b,0x0000000a,0x0000009f,0x00000002,
0x0004002b,0x00000006,0x000000d3,0x40000000,0x00040017,0x000000de,0x0000000a,0x00000003,
0x00040020,0x000000df,0x00000001,0x000000de,0x0004003b,0x000000df,0x000000e0,0x00000001,
0x00040017,0x000000e1,0x0000000a,0x00000002,0x0004002b,0x00000006,0x000000e7,0x3f000000,
0x00040020,0x000000ee,0x00000007,0x0000001c,0x00040020,0x000000f1,0x00000009,0x00000006,
0x00090019,0x00000112,0x00000006,0x00000001,0x00000000,0x00000000,0x00000000,0x00000002,
0x00000004,0x00040020,0x00000113,0x00000000,0x00000112,0x0004003b,0x00000113,0x00000114,
0x00000000,0x0004003b,0x00000015,0x00000117,0x00000000,0x0004002b,0x0000000a,0x00000120,
0x00000008,0x0006002c,0x000000de,0x00000121,0x00000120,0x00000120,0x00000078,0x00050036,
0x00000002,0x00000004,0x00000000,0x00000003,0x000200f8,0x00000005,0x0004003b,0x0000001e,
0x00000101,0x00000007,0x0004003b,0x00000008,0x00000102,0x00000007,0x0004003b,0x0000001e,
0x00000103,0x00000007,0x0004003b,0x00000008,0x00000104,0x00000007,0x0004003b,0x0000000d,
0x0000010c,0x00000007,0x0004003b,0x00000008,0x0000010d,0x00000007,0x0004003b,0x0000000d,
0x0000010f,0x00000007,0x0004003b,0x00000008,0x00000118,0x00000007,0x0004003b,0x0000000d,
0x0000011a,0x00000007,0x00060039,0x0000001f,0x00000105,0x00000023,0x00000103,0x00000104,
0x0004003d,0x0000001d,0x00000106,0x00000103,0x0003003e,0x00000101,0x00000106,0x0004003d,
0x00000007,0x00000107,0x00000104,0x0003003e,0x00000102,0x00000107,0x000400a8,0x0000001f,
0x00000108,0x00000105,0x000300f7,0x0000010a,0x00000000,0x000400fa,0x00000108,0x00000109,
0x0000010a,0x000200f8,0x00000109,0x000100fd,0x000200f8,0x0000010a,0x0004003d,0x00000007,
0x0000010e,0x00000102,0x0003003e,0x0000010d,0x0000010e,0x00060039,0x00000002,0x00000110,
0x00000011,0x0000010d,0x0000010f,0x0004003d,0x0000000c,0x00000111,0x0000010f,0x0003003e,
0x0000010c,0x00000111,0x0004003d,0x00000112,0x00000115,0x00000114,0x0004003d,0x0000001d,
0x00000116,0x00000101,0x0004003d,0x00000007,0x00000119,0x00000102,0x0003003e,0x00000118,
0x00000119,0x0004003d,0x0000000c,0x0000011b,0x0000010c,0x0003003e,0x0000011a,0x0000011b,
0x00070039,0x00000007,0x0000011c,0x0000001a,0x00000118,0x0000011a,0x00000117,0x00050051,
0x00000006,0x0000011d,0x0000011c,0x00000000,0x00050051,0x00000006,0x0000011e,0x0000011c,
0x00000001,0x00070050,0x00000009,0x0000011f,0x0000011d,0x0000011e,0x0000002e,0x0000002e,
0x00040063,0x00000115,0x00000116,0x0000011f,0x000100fd,0x00010038,0x00050036,0x00000002,
0x00000011,0x00000000,0x0000000e,0x00030037,0x00000008,0x0000000f,0x00030037,0x0000000d,
0x00000010,0x000200f8,0x00000012,0x00050041,0x00000029,0x0000002a,0x00000028,0x00000025,
0x0004003d,0x00000009,0x0000002b,0x0000002a,0x0009004f,0x00000009,0x0000002c,0x0000002b,
0x0000002b,0x00000000,0x00000001,0x00000000,0x00000001,0x0004003d,0x00000007,0x00000030,
0x0000000f,0x0009004f,0x00000009,0x00000031,0x00000030,0x00000030,0x00000000,0x00000001,
0x00000000,0x00000001,0x0008000c,0x00000009,0x00000032,0x00000001,0x00000032,0x0000002c,
0x0000002f,0x00000031,0x00050041,0x00000033,0x00000034,0x00000010,0x00000025,0x0003003e,
0x00000034,0x00000032,0x00050041,0x00000029,0x00000036,0x00000028,0x00000025,0x0004003d,
0x00000009,0x00000037,0x00000036,0x0009004f,0x00000009,0x00000038,0x00000037,0x00000037,
0x00000000,0x00000001,0x00000000,0x00000001,0x0004003d,0x00000007,0x0000003b,0x0000000f,
0x0009004f,0x00000009,0x0000003c,0x0000003b,0x0000003b,0x00000000,0x00000001,0x00000000,
0x00000001,0x0008000c,0x00000009,0x0000003d,0x00000001,0x00000032,0x00000038,0x0000003a,
0x0000003c,0x00050041,0x00000033,0x0000003e,0x00000010,0x00000035,0x0003003e,0x0000003e,
0x0000003d,0x00050041,0x00000029,0x00000040,0x00000028,0x00000025,0x0004003d,0x00000009,
0x00000041,0x00000040,0x0009004f,0x00000009,0x00000042,0x00000041,0x00000041,0x00000000,
0x00000001,0x00000000,0x00000001,0x0004003d,0x00000007,0x00000045,0x0000000f,0x0009004f,
0x00000009,0x00000046,0x00000045,0x00000045,0x00000000,0x00000001,0x00000000,0x00000001,
0x0008000c,0x00000009,0x00000047,0x00000001,0x00000032,0x00000042,0x00000044,0x00000046,
0x00050041,0x00000033,0x00000048,0x00000010,0x0000003f,0x0003003e,0x00000048,0x00000047,
0x000100fd,0x00010038,0x00050036,0x00000007,0x0000001a,0x00000000,0x00000016,0x00030037,
0x00000008,0x00000017,0x00030037,0x0000000d,0x00000018,0x00030037,0x00000015,0x00000019,
0x000200f8,0x0000001b,0x0004003b,0x00000008,0x00000049,0x00000007,0x0004003b,0x0000004d,
0x0000004e,0x00000007,0x0004003b,0x00000053,0x00000054,0x00000007,0x0004003b,0x00000053,
0x0000005b,0x00000007,0x0004003b,0x00000053,0x00000064,0x00000007,0x0004003b,0x00000033,
0x0000006d,0x00000007,0x0004003b,0x00000008,0x0000007b,0x00000007,0x0004003b,0x00000053,
0x00000086,0x00000007,0x0004003b,0x00000053,0x0000008f,0x00000007,0x0004003b,0x00000008,
0x000000a4,0x00000007,0x0004003b,0x00000053,0x000000aa,0x00000007,0x0004003b,0x00000053,
0x000000b3,0x00000007,0x0004003b,0x00000053,0x000000cc,0x00000007,0x0003003e,0x00000049,
0x0000004b,0x0003003e,0x0000004e,0x00000052,0x0004003d,0x00000014,0x00000055,0x00000019,
0x0004003d,0x00000007,0x00000056,0x00000017,0x00070058,0x00000009,0x00000057,0x00000055,
0x00000056,0x00000002,0x0000002e,0x0008004f,0x0000004c,0x00000058,0x00000057,0x00000057,
0x00000000,0x00000001,0x00000002,0x0004003d,0x0000004c,0x00000059,0x0000004e,0x00050094,
0x00000006,0x0000005a,0x00000058,0x00000059,0x0003003e,0x00000054,0x0000005a,0x0004003d,
0x00000014,0x0000005c,0x00000019,0x00050041,0x00000033,0x0000005d,0x00000018,0x00000025,
0x0004003d,0x00000009,0x0000005e,0x0000005d,0x0007004f,0x00000007,0x0000005f,0x0000005e,
0x0000005e,0x00000000,0x00000001,0x00070058,0x00000009,0x00000060,0x0000005c,0x0000005f,
0x00000002,0x0000002e,0x0008004f,0x0000004c,0x00000061,0x00000060,0x00000060,0x00000000,
0x00000001,0x00000002,0x0004003d,0x0000004c,0x00000062,0x0000004e,0x00050094,0x00000006,
0x00000063,0x00000061,0x00000062,0x0003003e,0x0000005b,0x00000063,0x0004003d,0x00000014,
0x00000065,0x00000019,0x00050041,0x00000033,0x00000066,0x00000018,0x00000025,0x0004003d,
0x00000009,0x00000067,0x00000066,0x0007004f,0x00000007,0x00000068,0x00000067,0x00000067,
0x00000002,0x00000003,0x00070058,0x00000009,0x00000069,0x00000065,0x00000068,0x00000002,
0x0000002e,0x0008004f,0x0000004c,0x0000006a,0x00000069,0x00000069,0x00000000,0x00000001,
0x00000002,0x0004003d,0x0000004c,0x0000006b,0x0000004e,0x00050094,0x00000006,0x0000006c,
0x0000006a,0x0000006b,0x0003003e,0x00000064,0x0000006c,0x0004003d,0x00000006,0x0000006e,
0x00000054,0x0004003d,0x00000006,0x0000006f,0x0000005b,0x0004003d,0x00000006,0x00000070,
0x00000064,0x00050050,0x00000007,0x00000071,0x0000006f,0x00000070,0x00050050,0x00000007,
0x00000072,0x0000006e,0x0000006e,0x00050083,0x00000007,0x00000073,0x00000072,0x00000071,
0x0006000c,0x00000007,0x00000074,0x00000001,0x00000004,0x00000073,0x00050041,0x00000053,
0x00000076,0x0000006d,0x00000075,0x00050051,0x00000006,0x00000077,0x00000074,0x00000000,
0x0003003e,0x00000076,0x00000077,0x00050041,0x00000053,0x00000079,0x0000006d,0x00000078,
0x00050051,0x00000006,0x0000007a,0x00000074,0x00000001,0x0003003e,0x00000079,0x0000007a,
0x0004003d,0x00000007,0x0000007c,0x00000049,0x0004003d,0x00000009,0x0000007d,0x0000006d,
0x0007004f,0x00000007,0x0000007e,0x0000007d,0x0000007d,0x00000000,0x00000001,0x0007000c,
0x00000007,0x0000007f,0x00000001,0x00000030,0x0000007c,0x0000007e,0x0003003e,0x0000007b,
0x0000007f,0x0004003d,0x00000007,0x00000080,0x0000007b,0x00050094,0x00000006,0x00000082,
0x00000080,0x00000081,0x000500b4,0x0000001f,0x00000083,0x00000082,0x0000002e,0x000300f7,
0x00000085,0x00000000,0x000400fa,0x00000083,0x00000084,0x00000085,0x000200f8,0x00000084,
0x000200f9,0x00000085,0x000200f8,0x00000085,0x0004003d,0x00000014,0x00000087,0x00000019,
0x00050041,0x00000033,0x00000088,0x00000018,0x00000035,0x0004003d,0x00000009,0x00000089,
0x00000088,0x0007004f,0x00000007,0x0000008a,0x00000089,0x00000089,0x00000000,0x00000001,
0x00070058,0x00000009,0x0000008b,0x00000087,0x0000008a,0x00000002,0x0000002e,0x0008004f,
0x0000004c,0x0000008c,0x0000008b,0x0000008b,0x00000000,0x00000001,0x00000002,0x0004003d,
0x0000004c,0x0000008d,0x0000004e,0x00050094,0x00000006,0x0000008e,0x0000008c,0x0000008d,
0x0003003e,0x00000086,0x0000008e,0x0004003d,0x00000014,0x00000090,0x00000019,0x00050041,
0x00000033,0x00000091,0x00000018,0x00000035,0x0004003d,0x00000009,0x00000092,0x00000091,
0x0007004f,0x00000007,0x00000093,0x00000092,0x00000092,0x00000002,0x00000003,0x00070058,
0x00000009,0x00000094,0x00000090,0x00000093,0x00000002,0x0000002e,0x0008004f,0x0000004c,
0x00000095,0x00000094,0x00000094,0x00000000,0x00000001,0x00000002,0x0004003d,0x0000004c,
0x00000096,0x0000004e,0x00050094,0x00000006,0x00000097,0x00000095,0x00000096,0x0003003e,
0x0000008f,0x00000097,0x0004003d,0x00000006,0x00000098,0x00000054,0x0004003d,0x00000006,
0x00000099,0x00000086,0x0004003d,0x00000006,0x0000009a,0x0000008f,0x00050050,0x00000007,
0x0000009b,0x00000099,0x0000009a,0x00050050,0x00000007,0x0000009c,0x00000098,0x00000098,
0x00050083,0x00000007,0x0000009d,0x0000009c,0x0000009b,0x0006000c,0x00000007,0x0000009e,
0x00000001,0x00000004,0x0000009d,0x00050041,0x00000053,0x000000a0,0x0000006d,0x0000009f,
0x00050051,0x00000006,0x000000a1,0x0000009e,0x00000000,0x0003003e,0x000000a0,0x000000a1,
0x00050041,0x00000053,0x000000a2,0x0000006d,0x0000000b,0x00050051,0x00000006,0x000000a3,
0x0000009e,0x00000001,0x0003003e,0x000000a2,0x000000a3,0x0004003d,0x00000009,0x000000a5,
0x0000006d,0x0007004f,0x00000007,0x000000a6,0x000000a5,0x000000a5,0x00000000,0x00000001,
0x0004003d,0x00000009,0x000000a7,0x0000006d,0x0007004f,0x00000007,0x000000a8,0x000000a7,
0x000000a7,0x00000002,0x00000003,0x0007000c,0x00000007,0x000000a9,0x00000001,0x00000028,
0x000000a6,0x000000a8,0x0003003e,0x000000a4,0x000000a9,0x0004003d,0x00000014,0x000000ab,
0x00000019,0x00050041,0x00000033,0x000000ac,0x00000018,0x0000003f,0x0004003d,0x00000009,
0x000000ad,0x000000ac,0x0007004f,0x00000007,0x000000ae,0x000000ad,0x000000ad,0x00000000,
0x00000001,0x00070058,0x00000009,0x000000af,0x000000ab,0x000000ae,0x00000002,0x0000002e,
0x0008004f,0x0000004c,0x000000b0,0x000000af,0x000000af,0x00000000,0x00000001,0x00000002,
0x0004003d,0x0000004c,0x000000b1,0x0000004e,0x00050094,0x00000006,0x000000b2,0x000000b0,
0x000000b1,0x0003003e,0x000000aa,0x000000b2,0x0004003d,0x00000014,0x000000b4,0x00000019,
0x00050041,0x00000033,0x000000b5,0x00000018,0x0000003f,0x0004003d,0x00000009,0x000000b6,
0x000000b5,0x0007004f,0x00000007,0x000000b7,0x000000b6,0x000000b6,0x00000002,0x00000003,
0x00070058,0x00000009,0x000000b8,0x000000b4,0x000000b7,0x00000002,0x0000002e,0x0008004f,
0x0000004c,0x000000b9,0x000000b8,0x000000b8,0x00000000,0x00000001,0x00000002,0x0004003d,
0x0000004c,0x000000ba,0x0000004e,0x00050094,0x00000006,0x000000bb,0x000000b9,0x000000ba,
0x0003003e,0x000000b3,0x000000bb,0x0004003d,0x00000006,0x000000bc,0x0000005b,0x0004003d,
0x00000006,0x000000bd,0x00000064,0x00050050,0x00000007,0x000000be,0x000000bc,0x000000bd,
0x0004003d,0x00000006,0x000000bf,0x000000aa,0x0004003d,0x00000006,0x000000c0,0x000000b3,
0x00050050,0x00000007,0x000000c1,0x000000bf,0x000000c0,0x00050083,0x00000007,0x000000c2,
0x000000be,0x000000c1,0x0006000c,0x00000007,0x000000c3,0x00000001,0x00000004,0x000000c2,
0x00050041,0x00000053,0x000000c4,0x0000006d,0x0000009f,0x00050051,0x00000006,0x000000c5,
0x000000c3,0x00000000,0x0003003e,0x000000c4,0x000000c5,0x00050041,0x00000053,0x000000c6,
0x0000006d,0x0000000b,0x00050051,0x00000006,0x000000c7,0x000000c3,0x00000001,0x0003003e,
0x000000c6,0x000000c7,0x0004003d,0x00000007,0x000000c8,0x000000a4,0x0004003d,0x00000009,
0x000000c9,0x0000006d,0x0007004f,0x00000007,0x000000ca,0x000000c9,0x000000c9,0x00000002,
0x00000003,0x0007000c,0x00000007,0x000000cb,0x00000001,0x00000028,0x000000c8,0x000000ca,
0x0003003e,0x000000a4,0x000000cb,0x00050041,0x00000053,0x000000cd,0x000000a4,0x00000075,
0x0004003d,0x00000006,0x000000ce,0x000000cd,0x00050041,0x00000053,0x000000cf,0x000000a4,
0x00000078,0x0004003d,0x00000006,0x000000d0,0x000000cf,0x0007000c,0x00000006,0x000000d1,
0x00000001,0x00000028,0x000000ce,0x000000d0,0x0003003e,0x000000cc,0x000000d1,0x0004003d,
0x00000006,0x000000d2,0x000000cc,0x0004003d,0x00000009,0x000000d4,0x0000006d,0x0007004f,
0x00000007,0x000000d5,0x000000d4,0x000000d4,0x00000000,0x00000001,0x0005008e,0x00000007,
0x000000d6,0x000000d5,0x000000d3,0x00050050,0x00000007,0x000000d7,0x000000d2,0x000000d2,
0x0007000c,0x00000007,0x000000d8,0x00000001,0x00000030,0x000000d7,0x000000d6,0x0004003d,
0x00000007,0x000000d9,0x0000007b,0x00050085,0x00000007,0x000000da,0x000000d9,0x000000d8,
0x0003003e,0x0000007b,0x000000da,0x0004003d,0x00000007,0x000000db,0x0000007b,0x000200fe,
0x000000db,0x00010038,0x00050036,0x0000001f,0x00000023,0x00000000,0x00000020,0x00030037,
0x0000001e,0x00000021,0x00030037,0x00000008,0x00000022,0x000200f8,0x00000024,0x0004003d,
0x000000de,0x000000e2,0x000000e0,0x0007004f,0x000000e1,0x000000e3,0x000000e2,0x000000e2,
0x00000000,0x00000001,0x0004007c,0x0000001d,0x000000e4,0x000000e3,0x0003003e,0x00000021,
0x000000e4,0x0004003d,0x0000001d,0x000000e5,0x00000021,0x0004006f,0x00000007,0x000000e6,
0x000000e5,0x00050050,0x00000007,0x000000e8,0x000000e7,0x000000e7,0x00050081,0x00000007,
0x000000e9,0x000000e6,0x000000e8,0x00050041,0x00000029,0x000000ea,0x00000028,0x00000025,
0x0004003d,0x00000009,0x000000eb,0x000000ea,0x0007004f,0x00000007,0x000000ec,0x000000eb,
0x000000eb,0x00000000,0x00000001,0x00050085,0x00000007,0x000000ed,0x000000e9,0x000000ec,
0x0003003e,0x00000022,0x000000ed,0x00050041,0x000000ee,0x000000ef,0x00000021,0x00000075,
0x0004003d,0x0000001c,0x000000f0,0x000000ef,0x00060041,0x000000f1,0x000000f2,0x00000028,
0x00000025,0x0000009f,0x0004003d,0x00000006,0x000000f3,0x000000f2,0x0004006e,0x0000001c,
0x000000f4,0x000000f3,0x000500b1,0x0000001f,0x000000f5,0x000000f0,0x000000f4,0x000300f7,
0x000000f7,0x00000000,0x000400fa,0x000000f5,0x000000f6,0x000000f7,0x000200f8,0x000000f6,
0x00050041,0x000000ee,0x000000f8,0x00000021,0x00000078,0x0004003d,0x0000001c,0x000000f9,
0x000000f8,0x00060041,0x000000f1,0x000000fa,0x00000028,0x00000025,0x0000000b,0x0004003d,
0x00000006,0x000000fb,0x000000fa,0x0004006e,0x0000001c,0x000000fc,0x000000fb,0x000500b1,
0x0000001f,0x000000fd,0x000000f9,0x000000fc,0x000200f9,0x000000f7,0x000200f8,0x000000f7,
0x000700f5,0x0000001f,0x000000fe,0x000000f5,0x00000024,0x000000fd,0x000000f6,0x000200fe,
0x000000fe,0x00010038
};

View File

@@ -0,0 +1,18 @@
#version 450
// (Triforce) SMAA pass 0 (gpu/vk/vk_aa.cpp, the Anti-aliasing row's SMAA Ultra): the luma SMAA looks for edges in, from the
// tone mapped picture, in the square-root space the target stores (aa_common.glsl). One channel: SMAA's luma edge detection
// takes the dot product of the colour it reads with its luma weights, and a one-channel image reads as (r, 0, 0), so r holds
// the luma over the red weight. After NX2navi's shaders/smaa_encode.comp.
#extension GL_GOOGLE_include_directive : require
layout(local_size_x = 8, local_size_y = 8) in;
layout(binding = 0) uniform sampler2D src;
layout(binding = 1, r16f) uniform writeonly image2D dst;
layout(push_constant) uniform Push { vec4 metrics; } pc;
#include "aa_common.glsl"
void main() {
ivec2 p = ivec2(gl_GlobalInvocationID.xy);
if (p.x >= int(pc.metrics.z) || p.y >= int(pc.metrics.w)) return;
const vec3 W = vec3(0.2126, 0.7152, 0.0722); // (SMAA.hlsl SMAALumaEdgeDetectionPS's weights)
imageStore(dst, p, vec4(dot(aa_encode(texelFetch(src, p, 0).rgb), W) / W.r, 0.0, 0.0, 1.0));
}

View File

@@ -0,0 +1,72 @@
// smaa_luma.comp + aa_common.glsl sha256 5d78cd804ec94dee0853d30a3733a3c3db861e69005b6bb0391bdd2e6dd61df4 (tests/test_aa_shaders.py --write: glslangValidator -V --target-env vulkan1.2 -I third_party/smaa --vn SMAA_LUMA_COMP_SPV)
// 1116.6.0
#pragma once
const uint32_t SMAA_LUMA_COMP_SPV[] = {
0x07230203,0x00010500,0x0008000b,0x0000005a,0x00000000,0x00020011,0x00000001,0x00020011,
0x00000031,0x0006000b,0x00000001,0x4c534c47,0x6474732e,0x3035342e,0x00000000,0x0003000e,
0x00000000,0x00000001,0x0009000f,0x00000005,0x00000004,0x6e69616d,0x00000000,0x0000001b,
0x00000028,0x00000041,0x00000047,0x00060010,0x00000004,0x00000011,0x00000008,0x00000008,
0x00000001,0x00030003,0x00000002,0x000001c2,0x000a0004,0x475f4c47,0x4c474f4f,0x70635f45,
0x74735f70,0x5f656c79,0x656e696c,0x7269645f,0x69746365,0x00006576,0x00080004,0x475f4c47,
0x4c474f4f,0x6e695f45,0x64756c63,0x69645f65,0x74636572,0x00657669,0x00040005,0x00000004,
0x6e69616d,0x00000000,0x00060005,0x0000000b,0x655f6161,0x646f636e,0x66762865,0x00003b33,
0x00030005,0x0000000a,0x00000063,0x00030005,0x00000017,0x00000070,0x00080005,0x0000001b,
0x475f6c67,0x61626f6c,0x766e496c,0x7461636f,0x496e6f69,0x00000044,0x00040005,0x00000026,
0x68737550,0x00000000,0x00050006,0x00000026,0x00000000,0x7274656d,0x00736369,0x00030005,
0x00000028,0x00006370,0x00030005,0x00000041,0x00747364,0x00030005,0x00000047,0x00637273,
0x00040005,0x0000004c,0x61726170,0x0000006d,0x00040047,0x0000001b,0x0000000b,0x0000001c,
0x00030047,0x00000026,0x00000002,0x00050048,0x00000026,0x00000000,0x00000023,0x00000000,
0x00030047,0x00000041,0x00000019,0x00040047,0x00000041,0x00000021,0x00000001,0x00040047,
0x00000041,0x00000022,0x00000000,0x00040047,0x00000047,0x00000021,0x00000000,0x00040047,
0x00000047,0x00000022,0x00000000,0x00040047,0x00000058,0x0000000b,0x00000019,0x00020013,
0x00000002,0x00030021,0x00000003,0x00000002,0x00030016,0x00000006,0x00000020,0x00040017,
0x00000007,0x00000006,0x00000003,0x00040020,0x00000008,0x00000007,0x00000007,0x00040021,
0x00000009,0x00000007,0x00000008,0x0004002b,0x00000006,0x0000000e,0x00000000,0x0006002c,
0x00000007,0x0000000f,0x0000000e,0x0000000e,0x0000000e,0x00040015,0x00000014,0x00000020,
0x00000001,0x00040017,0x00000015,0x00000014,0x00000002,0x00040020,0x00000016,0x00000007,
0x00000015,0x00040015,0x00000018,0x00000020,0x00000000,0x00040017,0x00000019,0x00000018,
0x00000003,0x00040020,0x0000001a,0x00000001,0x00000019,0x0004003b,0x0000001a,0x0000001b,
0x00000001,0x00040017,0x0000001c,0x00000018,0x00000002,0x00020014,0x00000020,0x0004002b,
0x00000018,0x00000021,0x00000000,0x00040020,0x00000022,0x00000007,0x00000014,0x00040017,
0x00000025,0x00000006,0x00000004,0x0003001e,0x00000026,0x00000025,0x00040020,0x00000027,
0x00000009,0x00000026,0x0004003b,0x00000027,0x00000028,0x00000009,0x0004002b,0x00000014,
0x00000029,0x00000000,0x0004002b,0x00000018,0x0000002a,0x00000002,0x00040020,0x0000002b,
0x00000009,0x00000006,0x0004002b,0x00000018,0x00000033,0x00000001,0x0004002b,0x00000018,
0x00000036,0x00000003,0x00090019,0x0000003f,0x00000006,0x00000001,0x00000000,0x00000000,
0x00000000,0x00000002,0x00000009,0x00040020,0x00000040,0x00000000,0x0000003f,0x0004003b,
0x00000040,0x00000041,0x00000000,0x00090019,0x00000044,0x00000006,0x00000001,0x00000000,
0x00000000,0x00000000,0x00000001,0x00000000,0x0003001b,0x00000045,0x00000044,0x00040020,
0x00000046,0x00000000,0x00000045,0x0004003b,0x00000046,0x00000047,0x00000000,0x0004002b,
0x00000006,0x0000004f,0x3e59b3d0,0x0004002b,0x00000006,0x00000050,0x3f371759,0x0004002b,
0x00000006,0x00000051,0x3d93dd98,0x0006002c,0x00000007,0x00000052,0x0000004f,0x00000050,
0x00000051,0x0004002b,0x00000006,0x00000055,0x3f800000,0x0004002b,0x00000018,0x00000057,
0x00000008,0x0006002c,0x00000019,0x00000058,0x00000057,0x00000057,0x00000033,0x0004002b,
0x00000006,0x00000059,0x477f8000,0x00050036,0x00000002,0x00000004,0x00000000,0x00000003,
0x000200f8,0x00000005,0x0004003b,0x00000016,0x00000017,0x00000007,0x0004003b,0x00000008,
0x0000004c,0x00000007,0x0004003d,0x00000019,0x0000001d,0x0000001b,0x0007004f,0x0000001c,
0x0000001e,0x0000001d,0x0000001d,0x00000000,0x00000001,0x0004007c,0x00000015,0x0000001f,
0x0000001e,0x0003003e,0x00000017,0x0000001f,0x00050041,0x00000022,0x00000023,0x00000017,
0x00000021,0x0004003d,0x00000014,0x00000024,0x00000023,0x00060041,0x0000002b,0x0000002c,
0x00000028,0x00000029,0x0000002a,0x0004003d,0x00000006,0x0000002d,0x0000002c,0x0004006e,
0x00000014,0x0000002e,0x0000002d,0x000500af,0x00000020,0x0000002f,0x00000024,0x0000002e,
0x000400a8,0x00000020,0x00000030,0x0000002f,0x000300f7,0x00000032,0x00000000,0x000400fa,
0x00000030,0x00000031,0x00000032,0x000200f8,0x00000031,0x00050041,0x00000022,0x00000034,
0x00000017,0x00000033,0x0004003d,0x00000014,0x00000035,0x00000034,0x00060041,0x0000002b,
0x00000037,0x00000028,0x00000029,0x00000036,0x0004003d,0x00000006,0x00000038,0x00000037,
0x0004006e,0x00000014,0x00000039,0x00000038,0x000500af,0x00000020,0x0000003a,0x00000035,
0x00000039,0x000200f9,0x00000032,0x000200f8,0x00000032,0x000700f5,0x00000020,0x0000003b,
0x0000002f,0x00000005,0x0000003a,0x00000031,0x000300f7,0x0000003d,0x00000000,0x000400fa,
0x0000003b,0x0000003c,0x0000003d,0x000200f8,0x0000003c,0x000100fd,0x000200f8,0x0000003d,
0x0004003d,0x0000003f,0x00000042,0x00000041,0x0004003d,0x00000015,0x00000043,0x00000017,
0x0004003d,0x00000045,0x00000048,0x00000047,0x0004003d,0x00000015,0x00000049,0x00000017,
0x00040064,0x00000044,0x0000004a,0x00000048,0x0007005f,0x00000025,0x0000004b,0x0000004a,
0x00000049,0x00000002,0x00000029,0x0008004f,0x00000007,0x0000004d,0x0000004b,0x0000004b,
0x00000000,0x00000001,0x00000002,0x0003003e,0x0000004c,0x0000004d,0x00050039,0x00000007,
0x0000004e,0x0000000b,0x0000004c,0x00050094,0x00000006,0x00000053,0x0000004e,0x00000052,
0x00050088,0x00000006,0x00000054,0x00000053,0x0000004f,0x00070050,0x00000025,0x00000056,
0x00000054,0x0000000e,0x0000000e,0x00000055,0x00040063,0x00000042,0x00000043,0x00000056,
0x000100fd,0x00010038,0x00050036,0x00000007,0x0000000b,0x00000000,0x00000009,0x00030037,
0x00000008,0x0000000a,0x000200f8,0x0000000c,0x0004003d,0x00000007,0x0000000d,0x0000000a,
0x0007000c,0x00000007,0x00000010,0x00000001,0x00000028,0x0000000d,0x0000000f,0x0006000c,
0x00000007,0x00000011,0x00000001,0x0000001f,0x00000010,0x000200fe,0x00000011,0x00010038
};

View File

@@ -0,0 +1,22 @@
// (Triforce) Settings and resources shared by the SMAA passes (gpu/vk/vk_aa.cpp; smaa_edge.comp, smaa_weight.comp,
// smaa_blend.comp). After NX2navi's shaders/smaa_setup.glsl. SMAA.hlsl (third_party/smaa, MIT: its PROVENANCE.md) is the SMAA
// reference implementation, unmodified, found through glslangValidator's -I; its pixel- and vertex-shader functions run here
// inside compute shaders, one invocation a pixel, with the texture coordinate of the pixel's centre. Its "discard" of a pixel
// without edges becomes nothing: the edge pass then stores the zero edges it found, which is what the next pass reads for
// such a pixel anyway. (The pass that includes this asks for GL_GOOGLE_include_directive first.)
layout(local_size_x = 8, local_size_y = 8) in;
layout(push_constant) uniform Push {
vec4 metrics; // (1 / width, 1 / height, width, height)
} pc;
#define SMAA_RT_METRICS pc.metrics
#define SMAA_GLSL_4 1
#define SMAA_PRESET_ULTRA 1
#define discard
#include "SMAA.hlsl"
// (the pixel this invocation works on and its centre's texture coordinate; false past the image's edge)
bool smaa_pixel(out ivec2 p, out vec2 uv) {
p = ivec2(gl_GlobalInvocationID.xy);
uv = (vec2(p) + 0.5) * pc.metrics.xy;
return p.x < int(pc.metrics.z) && p.y < int(pc.metrics.w);
}

View File

@@ -0,0 +1,19 @@
#version 450
// (Triforce) SMAA pass 2 (gpu/vk/vk_aa.cpp): blending weights from the edges, with SMAA's area and search lookup textures
// (third_party/smaa AreaTex.h, SearchTex.h). After NX2navi's shaders/smaa_weight.comp.
#extension GL_GOOGLE_include_directive : require
#include "smaa_setup.glsl"
layout(binding = 0) uniform sampler2D edges; // (linear filtering)
layout(binding = 1) uniform sampler2D area; // (linear filtering)
layout(binding = 2) uniform sampler2D search; // (nearest)
layout(binding = 3, rgba8) uniform writeonly image2D weights;
void main() {
ivec2 p;
vec2 uv;
if (!smaa_pixel(p, uv)) return;
vec2 pix;
vec4 offset[3];
SMAABlendingWeightCalculationVS(uv, pix, offset);
imageStore(weights, p, SMAABlendingWeightCalculationPS(uv, pix, offset, edges, area, search, vec4(0.0)));
}

File diff suppressed because it is too large Load Diff

278
gpu/vk/shadow_prev.h Normal file
View File

@@ -0,0 +1,278 @@
// (Triforce) The texture shadow's pages that a capture takes again with bytes the shadow holds already (vk_memory.cpp
// take_capture's shadow loop), apart from Vulkan so that tests/test_shadow_prev.py runs the same steps.
//
// The shadow (vk_memory.cpp ShadowPage) is a 4 KB copy of each texture storage page the last capture took, which the
// uploads read (memory_as_submitted) and the re-upload checks hash (memory_hash_as_submitted, its hash kept with the copy
// until a capture copies other bytes into it: upload_same.h). After some tours of the lab's S2 the game rewrites the Market's
// textures every few frames, and about half of the pages a capture takes hold the bytes of their copy (RX_VK_TEX_SAME: no
// upload for them) and the other half other bytes, which the next change takes back (round46: as many pages the same as
// changed, to the page, in every looping tour, and the checks after them found the units the same: 2.5 GB looked at against
// 12 MB uploaded in tour-1's last window). take_capture copied every one of them into the shadow again, about 400 pages a
// take, six takes a frame, and a page copied with other bytes lost the hash a check had kept with it, so the check after the
// change back hashed it again (the checks' 0.2 to 0.3 ms a frame there; the take's "retires and stamps" 0.13 ms against
// 0.016 in a clean tour).
//
// Now, for a page tex_same compared (one with a copy, not a late page found the same):
// - RX_VK_SHADOW_KEEP: one whose copy holds the capture's bytes (tex_same's memcmp, SAME) keeps it as it is: no copy,
// its hash kept, as the copy over it with the same bytes kept it before;
// - RX_VK_SHADOW_PREV: one found with other bytes keeps the copy it had and that copy's hash (kept with the page:
// its buffer is the one the new bytes go into, CHANGED), and one whose capture holds the bytes of that kept copy (memcmp
// against it, BACK) takes it back, hash and all: no copy, and the check after it finds the hashes kept. A kept copy that
// came back none of kMisses changes in a row goes, and the page keeps none for its next kRetry changes; one not given back
// within kStale takes (a page that changed once and is then taken with the same bytes: its copy kept for nothing, the
// slot's seq refreshed by every take so the sweeps leave it) goes at its next take that looks at it, or at the sweep every
// kSweep takes (age); at most kCapPages are kept (64 MB).
// Both only while the page is the capture's once and its slot holds the buffers the look compared (act); any other page is
// copied as before. Exact: after the take the shadow holds what the capture holds in every case (each skip is a memcmp
// equal), a hash is content_hash of the bytes it is kept with or 0, and the stamps, the capture entries, the mirrors and the
// reads as submitted are what they were. =0 each: as before (RX_VK_SHADOW_CHECK=1: each page left uncopied compared with
// the capture's copy again after the take, and the hashes recomputed: the shadow line's "check" counts must stay 0).
//
// ((Triforce, 11-17) RX_VK_SHADOW_VERS: the copies a page keeps, not one. A page whose captures bring more than two versions
// (A, B, C, A every few frames) is given back its copy only when the version it brings is the one before the last (BACK
// above): the third one, A again, is other bytes and copied, and so is every such page's every third take. The kept copies
// are the page's last few versions then, newest first: a capture that brings the bytes of any of them takes that one back
// and the copy it had becomes the newest (back), so every version a page cycles through is copied once and given back after.
// A capture's bytes are compared with the kept copies newest first while they were kept (look, one memcmp each), so at
// RX_VK_SHADOW_VERS=1 (the default, as before) the one copy of PREV is compared exactly as it was; 2 to 4 keep that many, at
// 4 KB each, under the same kCapPages. keep_old keeps the copy the page had as the newest and the oldest goes back to the
// copy's buffer (no allocation at a ring full); a version not given back within kStale takes goes as one does; and the misses
// and the kRetry are the page's, not a version's: a page whose versions none came back for kMisses changes lets them all go
// and keeps none for kRetry changes. A cycle longer than the copies kept still copies: only the versions still among them
// come back. The lab's A/B of it, and the counts that say how many pages were given a copy two or three versions back
// (vk_memory.cpp's shadow line), in the owner's reports folder, triforce-loop-versions.md.
#pragma once
#include <algorithm>
#include <atomic>
#include <cstdint>
#include <cstdlib>
#include <cstring>
#include <memory>
#include <new>
#include <utility>
#include <vector>
namespace shadow_prev {
inline constexpr uint64_t kPage = 4096;
inline constexpr int64_t kCapPages = 16384; // (kept copies at most: 64 MB beside the shadow's 256, any number of versions)
inline constexpr int kMaxVers = 4; // (copies beside the page's own, at most: RX_VK_SHADOW_VERS's range)
inline constexpr uint8_t kMisses = 8; // (changes in a row without the kept copy coming back: it goes,
inline constexpr uint8_t kRetry = 32; // and none is kept for this many changes of the page)
inline constexpr uint64_t kStale = 64; // (takes: a kept copy not given back within this many goes, age)
inline constexpr uint64_t kSweep = 256; // (takes between the sweeps that age every kept copy: vk_memory.cpp take_capture)
inline constexpr int kVersDefault = 1; // (RX_VK_SHADOW_VERS unset: one copy, as before)
/// RX_VK_<name>: on unless set to a number that is 0 (as RX_VK_TEX_SAME)
inline bool env_on(const char* name) {
const char* e = getenv(name);
return !(e && !atoi(e));
}
/// RX_VK_SHADOW_VERS: the copies a page keeps beside its own (1 to kMaxVers; unset, empty or below 1: kVersDefault; more: kMaxVers)
inline int env_vers() {
const char* e = getenv("RX_VK_SHADOW_VERS");
if (!e || !*e) return kVersDefault;
const int v = atoi(e);
return v < 1 ? kVersDefault : v > kMaxVers ? kMaxVers : v;
}
/// The copies a page keeps, as the number the steps below compare and keep (1 to kMaxVers)
inline int vers_count(int vers) { return vers < 1 ? 1 : vers > kMaxVers ? kMaxVers : vers; }
/// The kept copies alive (each one with a buffer counts once)
inline std::atomic<int64_t>& alive() {
static std::atomic<int64_t> n{0};
return n;
}
/// A page's copy before one of its changes, with that copy's hash (content_hash of b, or 0: none taken)
struct Prev {
std::unique_ptr<uint8_t[]> b;
uint64_t hash = 0;
uint64_t at = 0; // (the take, shadow_seq, that kept it or gave it back; age)
Prev() = default;
Prev(const Prev&) = delete;
Prev& operator=(const Prev&) = delete;
Prev(Prev&& o) noexcept : b(std::move(o.b)), hash(o.hash), at(o.at) { o.hash = 0, o.at = 0; }
Prev& operator=(Prev&& o) noexcept {
if (this != &o) drop(), b = std::move(o.b), hash = o.hash, at = o.at, o.hash = 0, o.at = 0;
return *this;
}
~Prev() { drop(); }
void drop() {
if (b) b.reset(), alive().fetch_sub(1, std::memory_order_relaxed);
hash = 0;
}
};
/// A page's last few versions, newest first (RX_VK_SHADOW_VERS): p[0..n-1] with a buffer each, the misses and the retry the
/// page's (as one Prev's were: changes since a copy came back, and changes left before one is kept again)
struct Vers {
Prev p[kMaxVers];
uint8_t n = 0; // (kept now, newest first)
uint8_t misses = 0; // (with n: changes since one came back; without: changes left before one is kept again)
Vers() = default;
Vers(const Vers&) = delete;
Vers& operator=(const Vers&) = delete;
Vers(Vers&& o) noexcept { *this = std::move(o); }
Vers& operator=(Vers&& o) noexcept {
if (this != &o) {
for (int k = 0; k < kMaxVers; k++) p[k] = std::move(o.p[k]); // (each buffer's count moves with it)
n = o.n, misses = o.misses;
o.n = 0, o.misses = 0;
}
return *this;
}
};
/// The newest copy's bytes (nullptr: none)
inline const uint8_t* newest(const Vers& v) { return v.n ? v.p[0].b.get() : nullptr; }
/// Let go of every copy the page keeps (its misses and its retry are the caller's)
inline void drop_all(Vers& v) {
for (int k = 0; k < (int)v.n; k++) v.p[k].drop();
v.n = 0;
}
enum : uint8_t {
NONE = 0, // (not compared, or not to be left: copied as before)
SAME = 1, // (its copy holds the capture's bytes: left as it is)
CHANGED = 2, // (other bytes: its copy kept as the page's newest version, then copied over)
BACK = 3, // (the bytes of one of its versions: that one taken back, no copy)
};
/// tex_same's look at a page that has a copy: what it found, the buffers it compared (for act), and which kept version
/// holds the capture's bytes (BACK: 0 the newest, 1 the one before it...)
struct Look {
uint8_t v = NONE;
const uint8_t* b = nullptr;
const uint8_t* prev = nullptr;
uint8_t depth = 0;
uint64_t at = 0; // (BACK: the take that kept that version: with depth, the version's identity)
};
/// same: tex_same's memcmp of the copy b against the capture's copy cap found them equal. A changed page whose kept
/// versions hold cap's bytes is BACK, at the newest that does (prev_on only); cap is read only then
inline Look look(bool same, const uint8_t* b, const Vers& v, const uint8_t* cap, bool prev_on, int vers) {
Look l{same ? SAME : CHANGED, b, newest(v), 0};
if (!same && prev_on && cap) {
const int want = vers_count(vers);
const int n = want < (int)v.n ? want : (int)v.n;
for (int d = 0; d < n; d++)
if (v.p[d].b && std::memcmp(v.p[d].b.get(), cap, kPage) == 0) {
l.v = BACK, l.depth = (uint8_t)d, l.prev = v.p[d].b.get(), l.at = v.p[d].at;
break;
}
}
return l;
}
/// What the shadow's loop does with the page now: SAME and BACK only while the slot still holds the buffers the look
/// compared (b_now, and for BACK the version's) and the page is the capture's once (twice: its copies may hold other bytes
/// than the one compared); CHANGED (prev_on) when it still has a copy; else NONE (copied as before). BACK's version is
/// compared by its buffer and the take it was kept at: a buffer given back into the copy and filled with other bytes, then
/// kept again, is not the version the look compared
inline uint8_t act(const Look& l, bool keep_on, bool prev_on, bool twice, const uint8_t* b_now, const Vers& v_now) {
if (l.v == NONE || twice || !b_now || b_now != l.b) return NONE;
if (l.v == SAME) return keep_on ? SAME : NONE;
if (!prev_on) return NONE;
if (l.v == BACK && l.depth < v_now.n && v_now.p[l.depth].b.get() == l.prev && v_now.p[l.depth].at == l.at) return BACK;
return CHANGED;
}
/// BACK at take now, depth d: that version holds the capture's bytes: it and its hash become the page's copy, the copy it
/// had and its hash the newest version (given back: the page's misses and the newest's age start again)
inline void back(std::unique_ptr<uint8_t[]>& b, uint64_t& hash, Vers& v, uint8_t d, uint64_t now) {
if (d >= v.n || !v.p[d].b) return;
std::swap(b, v.p[d].b);
std::swap(hash, v.p[d].hash);
Prev took = std::move(v.p[d]); // (the bytes the page had, now a version)
for (int k = (int)d; k > 0; k--) v.p[k] = std::move(v.p[k - 1]);
v.p[0] = std::move(took);
v.p[0].at = now; // (it was the page's copy until this take)
v.misses = 0;
}
/// CHANGED at take now, before the capture's bytes are copied into b: b's bytes and hash kept as the newest version, b
/// given the oldest version's buffer (or a new one while fewer than the cap are kept), b's hash 0 (the copy brings other
/// bytes). False when none is kept: b is copied over as before (its hash let go by the loop)
inline bool keep_old(std::unique_ptr<uint8_t[]>& b, uint64_t& hash, Vers& v, int64_t cap, uint64_t now, int vers) {
if (!b) return false;
if (v.n) {
if (++v.misses >= kMisses) { // (none comes back: they all go, and none kept for a while)
drop_all(v);
v.misses = kRetry;
return false;
}
} else if (v.misses) {
v.misses--;
return false;
}
const int maxn = vers_count(vers);
std::unique_ptr<uint8_t[]> fresh;
if ((int)v.n >= maxn) { // (the oldest's buffer is the copy's: the ring is never more than maxn)
fresh = std::move(v.p[maxn - 1].b);
v.n = (uint8_t)(maxn - 1);
} else {
if (alive().load(std::memory_order_relaxed) >= cap) return false;
fresh.reset(new (std::nothrow) uint8_t[kPage]);
if (!fresh) return false;
alive().fetch_add(1, std::memory_order_relaxed);
}
for (int k = (int)v.n; k > 0; k--) v.p[k] = std::move(v.p[k - 1]);
v.p[0].b = std::move(b);
v.p[0].hash = hash;
v.p[0].at = now;
v.n++;
b = std::move(fresh);
hash = 0;
return true;
}
/// At take now, a page taken without a change (SAME, or copied with a look that kept nothing), and every kept copy at the
/// sweep: a version not given back within kStale takes goes (the others stay, newest first), and the next change keeps one
/// again. Without this a page changed once and taken the same ever after held its copy for good, and once kCapPages were
/// held so, keep_old kept none for any page. True when one went
inline bool age(Vers& v, uint64_t now) {
bool any = false;
for (int k = 0; k < (int)v.n;) {
if (!v.p[k].b || now - v.p[k].at <= kStale) {
k++;
continue;
}
v.p[k].drop();
for (int j = k; j + 1 < (int)v.n; j++) v.p[j] = std::move(v.p[j + 1]);
v.n--;
any = true;
}
// ((Triforce) as HEAD's: the page's misses start again once the copy it kept has gone. Not when nothing went: the take
// that set keep_old's kRetry cooldown calls age too, and that must survive it)
if (any && !v.n) v.misses = 0;
return any;
}
/// Whether page p is in sorted more than once (the capture's twice)
inline bool twice(const std::vector<uint32_t>& sorted, uint32_t p) {
const auto r = std::equal_range(sorted.begin(), sorted.end(), p);
return r.second - r.first > 1;
}
/// (RX_VK_SHADOW_CHECK) Whether a copy b with hash holds cap's bytes (cap null: not compared) and its hash is theirs or 0
template <class Hash>
bool holds(const uint8_t* b, uint64_t hash, const uint8_t* cap, Hash&& content_hash) {
if (!b) return false;
if (cap && std::memcmp(b, cap, kPage) != 0) return false;
return !hash || hash == content_hash(b, kPage);
}
/// (RX_VK_SHADOW_CHECK) Every copy the page keeps holds a hash of its bytes or 0. False: one of them does not
template <class Hash>
bool holds_all(const Vers& v, Hash&& content_hash) {
for (int k = 0; k < (int)v.n; k++)
if (!holds(v.p[k].b.get(), v.p[k].hash, nullptr, content_hash)) return false;
return true;
}
} // namespace shadow_prev

216
gpu/vk/spv_store.h Normal file
View File

@@ -0,0 +1,216 @@
// (Triforce) A shader stage's SPIR-V kept packed while no pipeline is being made from it, and the shader module made from
// it again for each pipeline's creation (vk_shader.cpp StageModule; on unless RX_VK_SPV_PACK=0). Apart from Vulkan so that
// tests/test_spv_store.py runs it alone. The idea is NX2navi's (runtime/spv_store.h, "SPIR-V at rest": 777 to 229 MiB
// there); the code is written for this back end.
//
// Why: this back end keeps no SPIR-V of its own in play, the driver does. Every stage of every program variant in memory
// has its VkShaderModule from the moment it is read from triforce_programs.bin or translated until the game ends, and a
// module holds the SPIR-V it was made from (and what the driver adds). The SPIR-V here is large: measured over the kept
// variants of an install on the lab's PC (54,290 records, 6.8 GiB of SPIR-V; a sample of every 25th), 128 KiB a variant
// on average (graphics 124, compute 98, mesh 459). A module is needed only while a pipeline is created with it: after
// that the pipeline has what it needs, and Vulkan lets the module go.
//
// So a stage's SPIR-V is packed when its program becomes ready, its module destroyed, and a module made again (unpack,
// vkCreateShaderModule) for the time one pipeline's creation takes: pin, the creation, unpin. What is held between two
// pipelines of a program is the packed copy, a little over a third of the SPIR-V.
//
// The packing is this file's own: the repository carries zstd's decoder only (third_party/zstd: lib/common and
// lib/decompress), so there is no zstd encoder to link. The four bytes of the words are stored apart (all the low bytes,
// then the next ones, and so on: SPIR-V's words are small numbers, so three of the four runs are mostly zeros), then an
// LZ77 coding of that in the layout of an LZ4 block: a token (literal count, match length less four, a nibble each, 15
// meaning more bytes follow), the literals, a two-byte distance back, and literals alone to end with. Measured on 2,000
// stages of that install (136 MiB of SPIR-V, every one unpacked again and compared): 0.37 of its size, where zstd at
// level 3 gives 0.27; 0.14 ms to pack a stage of 70 KiB and 0.06 ms to unpack it, on one thread.
//
// The rules that make it safe. The test runs the packing by itself and the second and third rule; the first is a second
// guard behind the packing, and no test can make it answer no while pack and unpack are right:
// - a store is filled only once its packed copy has been unpacked again and found the same, word for word: a stage whose
// copy does not come back keeps its module as it is and is never packed;
// - unpack writes no byte outside its block and reads none outside the packed copy, whatever the bytes are, and says no
// to anything that does not give exactly as many words as the store was filled with; and a hash of the words, taken
// when the store was filled, is asked again of every copy unpacked for a module: memory gone bad makes no module,
// never a module of other code;
// - the module is made and let go under the store's lock, and counted by its pins: two builders of one program's
// pipelines get the one module, and it is destroyed when the last of them is done, never under the other.
#pragma once
#include <cstdint>
#include <cstring>
#include <mutex>
#include <vector>
namespace spv_store {
namespace detail {
inline constexpr size_t kMinMatch = 4, kHashBits = 15, kMaxDistance = 65535;
inline uint32_t read32(const uint8_t* p) {
uint32_t v;
std::memcpy(&v, p, 4);
return v;
}
inline void put_count(std::vector<uint8_t>& out, size_t n) { // (what a nibble of 15 leaves over)
for (; n >= 255; n -= 255) out.push_back(255);
out.push_back((uint8_t)n);
}
/// src as LZ77 sequences, appended to out
inline void lz_pack(const uint8_t* src, size_t n, std::vector<uint8_t>& out) {
std::vector<uint32_t> table((size_t)1 << kHashBits, 0); // (position + 1 of the last four bytes with this hash; 0: none)
size_t at = 0, lit = 0; // (lit: where the literals not written yet begin)
const auto emit = [&](size_t match_at, size_t match_len) {
const size_t nlit = at - lit, m = match_len ? match_len - kMinMatch : 0;
out.push_back((uint8_t)((nlit < 15 ? nlit : 15) << 4 | (m < 15 ? m : 15)));
if (nlit >= 15) put_count(out, nlit - 15);
out.insert(out.end(), src + lit, src + at);
if (!match_len) return;
const size_t d = at - match_at;
out.push_back((uint8_t)d), out.push_back((uint8_t)(d >> 8));
if (m >= 15) put_count(out, m - 15);
};
while (at + kMinMatch <= n) {
const uint32_t h = (read32(src + at) * 2654435761u) >> (32 - kHashBits);
const size_t was = table[h];
table[h] = (uint32_t)(at + 1);
if (was && at + 1 - was <= kMaxDistance && read32(src + was - 1) == read32(src + at)) {
const size_t from = was - 1;
size_t len = kMinMatch;
while (at + len < n && src[from + len] == src[at + len]) len++;
emit(from, len);
at += len, lit = at;
} else {
at++;
}
}
at = n;
emit(0, 0); // (the literals left, alone: every packed copy ends with such a sequence, an empty one too)
}
/// The sequences of z into out, which must come out exactly n bytes long. False: z is not what lz_pack wrote for n bytes
inline bool lz_unpack(const uint8_t* z, size_t zn, uint8_t* out, size_t n) {
size_t in = 0, at = 0;
const auto count = [&](size_t* v) { // (the bytes after a nibble of 15)
for (;;) {
if (in >= zn) return false;
const uint8_t b = z[in++];
*v += b;
if (*v > n) return false; // (no count in a copy of n bytes is past n: and no sum of them wraps)
if (b != 255) return true;
}
};
for (;;) {
if (in >= zn) return false;
const uint8_t token = z[in++];
size_t nlit = token >> 4, m = token & 15;
if (nlit == 15 && !count(&nlit)) return false;
if (nlit > zn - in || nlit > n - at) return false;
std::memcpy(out + at, z + in, nlit);
in += nlit, at += nlit;
if (in == zn) return at == n && !m; // (the last sequence: literals alone)
if (zn - in < 2) return false;
const size_t d = (size_t)z[in] | (size_t)z[in + 1] << 8;
in += 2;
if (m == 15 && !count(&m)) return false;
m += kMinMatch;
if (!d || d > at || m > n - at) return false;
for (size_t i = 0; i < m; i++) out[at + i] = out[at + i - d]; // (byte by byte: a match may run into itself)
at += m;
}
}
} // namespace detail
/// The words packed into out (emptied first)
inline void pack(const uint32_t* w, size_t words, std::vector<uint8_t>& out) {
out.clear();
std::vector<uint8_t> planes(words * 4);
const uint8_t* b = (const uint8_t*)w;
for (size_t k = 0; k < 4; k++)
for (size_t i = 0; i < words; i++) planes[k * words + i] = b[i * 4 + k];
detail::lz_pack(planes.data(), planes.size(), out);
}
/// What pack made of that many words, back into out (resized to them). False, and out empty: z is not such a copy
inline bool unpack(const uint8_t* z, size_t zn, size_t words, std::vector<uint32_t>& out) {
out.clear();
if (words > SIZE_MAX / 8) return false;
std::vector<uint8_t> planes(words * 4);
if (!detail::lz_unpack(z, zn, planes.data(), planes.size())) return false;
out.resize(words);
uint8_t* b = (uint8_t*)out.data();
for (size_t k = 0; k < 4; k++)
for (size_t i = 0; i < words; i++) b[i * 4 + k] = planes[k * words + i];
return true;
}
/// A hash of the words (FNV-1a's steps, a word at a time)
inline uint64_t hash(const uint32_t* w, size_t words) {
uint64_t h = 0xCBF29CE484222325ull;
for (size_t i = 0; i < words; i++) h = (h ^ w[i]) * 1099511628211ull;
return h;
}
/// One stage's SPIR-V, packed, and the module made from it while a pipeline is being created. Module: the back end's
/// VkShaderModule (a stand-in in the test); a value-initialised one is "none"
template <class Module>
class Code {
public:
Code() = default;
Code(const Code&) = delete;
Code& operator=(const Code&) = delete;
/// Takes the words. True: packed, and the packed copy gave them back when unpacked; false: nothing is kept (the caller
/// goes on with the module it has)
bool fill(const uint32_t* w, size_t words) {
std::vector<uint8_t> z;
std::vector<uint32_t> back;
pack(w, words, z);
if (!words || !unpack(z.data(), z.size(), words, back) || std::memcmp(back.data(), w, words * 4)) return false;
z.shrink_to_fit();
std::lock_guard lk(mu_);
z_ = std::move(z), words_ = words, hash_ = hash(w, words);
return true;
}
bool filled() const { return words_ != 0; }
size_t words() const { return words_; }
size_t packed_bytes() const { return z_.size(); }
/// The module, made by make(const uint32_t* words, size_t n) if no pin holds one; kept until as many unpins as pins.
/// "None" when the store is not filled, the copy does not unpack to the words it was filled with (it was checked then:
/// memory gone bad) or make gives none: then nothing is pinned and no unpin is owed
template <class Make>
Module pin(Make&& make) {
std::lock_guard lk(mu_);
if (pins_) return pins_++, live_;
std::vector<uint32_t> w;
if (!words_ || !unpack(z_.data(), z_.size(), words_, w) || hash(w.data(), w.size()) != hash_) return Module{};
const Module m = make((const uint32_t*)w.data(), w.size());
if (m == Module{}) return m;
live_ = m, pins_ = 1, made_++;
return m;
}
/// The end of a pin that gave a module: the last one's has drop(module) called
template <class Drop>
void unpin(Drop&& drop) {
std::lock_guard lk(mu_);
if (!pins_ || --pins_) return;
const Module m = live_;
live_ = Module{};
drop(m);
}
uint32_t pins() const {
std::lock_guard lk(mu_);
return pins_;
}
uint64_t made() const { // (the modules made so far: one a pipeline, or fewer where builders overlapped)
std::lock_guard lk(mu_);
return made_;
}
private:
mutable std::mutex mu_;
std::vector<uint8_t> z_; // (set once, by fill, before any pin)
size_t words_ = 0;
uint64_t hash_ = 0;
Module live_{};
uint32_t pins_ = 0;
uint64_t made_ = 0;
};
} // namespace spv_store

165
gpu/vk/stage_code_cache.h Normal file
View File

@@ -0,0 +1,165 @@
/* (Triforce) The stages' decoded code kept by their bytes, for the program picks (vk_shader.cpp read_code, codes_of).
A program pick that meets a program object (its serial) for the first time reads each of its stages from guest memory, the
control section and the code area, takes their code key (FNV-1a over both) and decodes the code: its Ampere instructions
captured and lowered to Maxwell (parse_code). All of it under the shaders' lock: the stall line of the save's load-in frame
said "the program picks read the stages' code 45.6 ms" (lab run r33-t28-s1-1 frame 14966; 37 to 67 ms in the lab and the
owner's sessions, 37 ms at the Market's arrival). Measured over the game's own shaders (triforce_shaders.bin, a stage 12.2 KB
on average, every 20th): the read 1.5 us a stage, the key 9.3 us, the decoding 48.3 us. The game makes program objects again
for code it has had before (another serial for the same stages, at the same address or another), and each was decoded again.
This cache keeps what was made of a stage's bytes: a pick that reads bytes met before takes a copy of it, and neither keys
nor decodes them again. The lookup is by a quick hash of the bytes (quick_hash, four lanes of 64-bit words: 1.2 us a stage),
and an entry is taken only when its bytes are the same, byte for byte (0.2 us): two stages whose quick hashes are the same
cost a decoding, never one's code for the other's. What is made of the bytes is made of them alone (and of the switches,
read once, and the device), so the copy is what decoding them again would give.
Bounded: the entries' bytes and what was made of them, budget bytes at most (RX_VK_STAGE_CODE_CACHE=<MB>, 64: some 3000 stages
decoded), the least recently used going first. RX_VK_STAGE_CODE_CACHE=0: nothing kept, each stage decoded again for each
program object, as before. Its own lock, taken alone or under the shaders' lock, nothing taken under it. Plain code, what is
kept a template parameter: tests/test_stage_code_cache.py runs it.
And the lowering only where a pick translates (RX_VK_STAGE_CODE_LAZY, vk_shader.cpp stage_head, codes_whole). A pick needs a
stage's key, NVN stage and header to find its program, the lowered code only to translate one, and the shaders are prepared
before the title: the save's load-in frame finds its programs in the program cache, read ahead or read there. So a pick reads
a stage's head (the key: 9.3 us, of the 60) and the cache keeps that; the code is lowered (and kept whole) when the pick
translates a variant. A stage that does not lower leaves the program without it, as before: no program in any cache was made
from such a stage (its key names its stages: a translation lowers each first, and so did the prepare), so a pick that finds
its program needs no lowering, and one that translates lowers the stages first and, should one not lower, picks again from
those that do. tests/test_stage_code_lazy.py runs the picks with the switch on and off.
And read with the shaders' lock let go (RX_VK_STAGE_CODE_UNLOCKED, vk_shader.cpp codes_read): what a pick reads of a program
object met for the first time (guest memory, the key, this cache) needs nothing the lock keeps, so the pick lets it go
meanwhile as a draw's own read of a program does (program_read_once.h: a program's release waits until no pick has the lock
let go), and the replay's other threads go on picking. */
#pragma once
#include <cstddef>
#include <cstdint>
#include <cstdlib>
#include <cstring>
#include <list>
#include <memory>
#include <mutex>
#include <unordered_map>
#include <vector>
namespace mkwvk::stage_code_cache {
constexpr size_t kDefaultMb = 64;
/// RX_VK_STAGE_CODE_CACHE: the budget in bytes (MB given, kDefaultMb unset or unreadable, at most 4096 MB); 0: none
inline size_t budget(const char* env) {
size_t mb = kDefaultMb;
if (env && *env) {
char* end = nullptr;
const long long v = std::strtoll(env, &end, 10);
if (end != env && !*end && v >= 0) mb = (size_t)(v < 4096 ? v : 4096);
}
return mb << 20;
}
/// RX_VK_STAGE_CODE_LAZY: on unless "0"
inline bool lazy_on(const char* env) { return !(env && env[0] == '0' && !env[1]); }
/// RX_VK_STAGE_CODE_UNLOCKED: on unless "0"
inline bool unlocked_on(const char* env) { return !(env && env[0] == '0' && !env[1]); }
/// The cache's lookup hash of bytes: four lanes of 64-bit words, the tail and the length folded in. Only a lookup's: an entry is
/// taken for bytes equal to its own
inline uint64_t quick_hash(const uint8_t* p, size_t n) {
constexpr uint64_t m = 0x9E3779B97F4A7C15ull;
uint64_t lane[4] = {0x243F6A8885A308D3ull, 0x13198A2E03707344ull, 0xA4093822299F31D0ull, 0x082EFA98EC4E6C89ull};
size_t i = 0;
for (; i + 32 <= n; i += 32)
for (int k = 0; k < 4; k++) {
uint64_t w;
std::memcpy(&w, p + i + 8 * k, 8);
lane[k] = (lane[k] ^ w) * m;
lane[k] ^= lane[k] >> 29;
}
uint64_t h = 0xCBF29CE484222325ull ^ n;
for (; i < n; i++) h = (h ^ p[i]) * 0x100000001B3ull;
for (int k = 0; k < 4; k++) {
h = (h ^ lane[k]) * m;
h ^= h >> 31;
}
return h;
}
/// What was made of stages' bytes (Value), by the bytes (Hash: the lookup's, quick_hash; another only in the tests)
template <class Value, uint64_t (*Hash)(const uint8_t*, size_t) = quick_hash> class Cache {
public:
using Bytes = std::shared_ptr<const std::vector<uint8_t>>;
explicit Cache(size_t budget) : budget_(budget) {}
size_t budget() const { return budget_; }
/// What was made of bytes equal to these (now the most recently used), or null
std::shared_ptr<const Value> find(const std::vector<uint8_t>& bytes) {
if (!budget_) return nullptr;
const uint64_t h = Hash(bytes.data(), bytes.size());
std::lock_guard lk(mu_);
const auto range = by_hash_.equal_range(h);
for (auto it = range.first; it != range.second; ++it)
if (*it->second->bytes == bytes) {
lru_.splice(lru_.begin(), lru_, it->second);
return it->second->value;
}
return nullptr;
}
/// value, made of bytes, kept (in place of what was kept for bytes equal to them); cost: its own size, besides the bytes'.
/// The least recently used go while the whole is over the budget (this one too, were it alone over it)
void put(const Bytes& bytes, std::shared_ptr<const Value> value, size_t cost) {
if (!budget_ || !bytes || !value) return;
const uint64_t h = Hash(bytes->data(), bytes->size());
std::lock_guard lk(mu_);
const auto range = by_hash_.equal_range(h);
auto at = lru_.end();
for (auto it = range.first; it != range.second; ++it)
if (*it->second->bytes == *bytes) at = it->second;
if (at != lru_.end()) {
used_ -= at->cost;
at->value = std::move(value), at->cost = bytes->size() + cost;
used_ += at->cost;
lru_.splice(lru_.begin(), lru_, at);
} else {
lru_.push_front(Entry{bytes, h, std::move(value), bytes->size() + cost});
by_hash_.emplace(h, lru_.begin());
used_ += lru_.front().cost;
}
while (used_ > budget_ && !lru_.empty()) {
const auto last = std::prev(lru_.end());
const auto range2 = by_hash_.equal_range(last->hash);
for (auto it = range2.first; it != range2.second; ++it)
if (it->second == last) {
by_hash_.erase(it);
break;
}
used_ -= last->cost;
lru_.erase(last);
}
}
/// Entries kept and their bytes in all (for the log)
void usage(size_t* entries, size_t* bytes) const {
std::lock_guard lk(mu_);
*entries = lru_.size(), *bytes = used_;
}
private:
struct Entry {
Bytes bytes;
uint64_t hash;
std::shared_ptr<const Value> value;
size_t cost; // (the bytes' and the value's)
};
const size_t budget_;
mutable std::mutex mu_;
std::list<Entry> lru_; // (the most recently used first)
std::unordered_multimap<uint64_t, typename std::list<Entry>::iterator> by_hash_;
size_t used_ = 0;
};
} // namespace mkwvk::stage_code_cache

243
gpu/vk/stall_watch.h Normal file
View File

@@ -0,0 +1,243 @@
/* (Triforce) The stalls in the log (vk_replay.cpp status_tick). A public benchmark of v0.4.0 had a 0.1 % low of 6 fps: frames
of over 150 ms now and then, which the log's 120- and 600-present averages cannot show. So every 600 of the game's frames a
line of how long they were, present to present and in the replay's own time, and for a frame over 50 ms one line of where
its time went: the replay's phases, the memory sync's parts and its waits for write-backs still on the GPU, the waits for
shaders and pipelines, the uploads, the waits for the GPU, the present and the game's own fence waits. At most three such
lines a second and 1500 in a session (Bound): a load of many slow frames in a row says so in the 600-frame line instead.
Plain functions: tests/test_stall_watch.cpp checks them. RX_VK_STALL_LOG=0: neither line; =<ms>: the stall line's
threshold (default 50). */
#pragma once
#include <cstdint>
#include <cstdio>
namespace mkwvk::stall_watch {
constexpr int kEvery = 600; // (the game's frames between two lines of frame times)
constexpr int kEdges = 4;
constexpr double kEdgeMs[kEdges] = {33.0, 50.0, 100.0, 250.0}; // (the counts of the frames over each)
constexpr double kStallMs = 50.0; // (a frame this long gets a line of its own)
/// The frames of one stretch: how many, the longest, and how many over each edge
struct Histogram {
int64_t frames = 0;
double longest = 0;
int64_t over[kEdges] = {};
void add(double ms) {
frames++;
if (ms > longest) longest = ms;
for (int i = 0; i < kEdges; i++)
if (ms > kEdgeMs[i]) over[i]++;
}
void reset() { *this = Histogram{}; }
};
/// "the longest 252.0 ms, 12 over 33 ms, 3 over 50, 1 over 100, 1 over 250"
inline int describe(const Histogram& h, char* out, size_t cap) {
return snprintf(out, cap, "the longest %.1f ms, %lld over 33 ms, %lld over 50, %lld over 100, %lld over 250", h.longest,
(long long)h.over[0], (long long)h.over[1], (long long)h.over[2], (long long)h.over[3]);
}
/// At most per_second lines a second (a burst of that many after a quiet stretch), and at most total in a session; held: the
/// lines it refused since the last look (taken by held_since)
struct Bound {
double per_second = 3.0, tokens = 3.0;
int64_t last_ns = 0;
int64_t total = 1500, told = 0, held = 0;
bool take(int64_t now_ns) {
if (last_ns) {
tokens += (double)(now_ns - last_ns) / 1e9 * per_second;
if (tokens > per_second) tokens = per_second;
}
last_ns = now_ns;
if (told >= total || tokens < 1.0) {
held++;
return false;
}
tokens -= 1.0;
told++;
return true;
}
int64_t held_since() {
const int64_t h = held;
held = 0;
return h;
}
};
/// One frame's account, in milliseconds unless named a count: what the stall line says
struct Frame {
int64_t no = 0; // (the game's frame, as the status line counts them)
double interval = 0; // (present to present)
double replay = 0; // (the replay's time for it: its submits and its present, on every thread that replays)
int phases = 0; // (the replay's phases: names[i], ms[i]; the first two, submit and present, are the replay's own)
const char* const* names = nullptr;
const double* ms = nullptr;
double maps = 0, capture = 0, collect = 0; // (memory sync's parts: the pools' refresh, the capture taken, the writes collected)
double wb_wait = 0; // (waits for write-backs still on the GPU: the captures' late pages, and the reads and bindings of pages put off)
int64_t wb_waits = 0, wb_ahead = 0, wb_behind_present = 0; // (how many; the most submissions ahead of the GPU one waited for; how many waited behind a present's blit)
int64_t wb_unsent = 0; // ((Triforce) how many waited for a submission not on the queue yet: the replay's, not the GPU's work)
double wb_unsent_ms = 0;
int64_t wb_held = 0; // ((Triforce, wp20) how many went on after the writer's recording was done on the GPU, and
double wb_held_ms = 0; // that part of them: the submissions' timeline held by work stopped at a wait, wb_rec_wait.h)
double pipe_wait = 0; // (waits for shaders and pipelines)
int64_t pipe_waits = 0, left_out = 0; // (how many; draws left out while theirs were built)
int64_t pipes_first = 0, pipes_ahead = 0, pipes_order = 0; // ((Triforce, wp18) vk_draw.cpp graphics_pipeline: the pipelines the
int64_t pipes_links = 0, pipes_libs = 0; // draws needed first, those the warm had linked for them (of those
// the last sessions' order's, pipe_order.h), and the links and the
// stage libraries the draws or their jobs made from the caches)
int64_t uploads = 0;
double upload_mb = 0;
double gpu_bound = 0; // (the replay waited for a recording slot: the GPU behind)
double present = 0; // (the present's own time: the swapchain's image, its fence, the present call, Reflex's sleep)
double present_wait = 0; // ((Triforce) of it, the wait for the present before to go out: with the present thread
// (RX_VK_PRESENT_THREAD) the present call is that thread's, present the presenting thread's part)
double game_wait = 0; // (the game's fence waits, every thread)
int64_t game_waits = 0;
int64_t imports = 0; // (vk_memory.cpp: pools imported where they were bound, on the replay's threads, and their time)
double import_ms = 0;
int64_t prog_reads = 0; // (vk_shader.cpp: programs a draw read from the program cache itself, their time; and those it
double prog_read_ms = 0; // took read ahead by the worker, program_readahead.h)
int64_t prog_ahead = 0;
int64_t uploads_later = 0; // (vk_texture.cpp: textures uploaded before whose new upload was left for the next frame)
int trial_phase = 0; // (vk_memory.cpp demote_trials: a pool's trial in guest memory under way, 1 to 4; 0: none)
uint64_t trial_pool = 0;
int64_t new_later = 0; // ((Triforce) vk_texture.cpp: new textures whose first upload was left for the next frame)
bool held = false; // ((Triforce) vk_replay.cpp, present_hold.h: this frame was not shown, the last picture held,
double hold_ms = 0; // for so long by now)
int64_t split_parts = 0; // ((Triforce) vk_texture.cpp, layer_split.h: parts of new layered textures' first uploads)
int64_t view_lookups = 0; // ((Triforce) vk_draw.cpp texture_view_of: texture views resolved anew, not from its cache)
double pick_lock_ms = 0, pick_codes_ms = 0; // ((Triforce) vk_shader.cpp program_pick: waits for shader_mu, code reads,
int64_t pick_inline = 0; // and programs translated where asked (compute: in the dispatch), their time)
double pick_inline_ms = 0;
int64_t once_waits = 0; // ((Triforce) vk_shader.cpp program_cache_load, program_read_once.h: a draw's waits for a
double once_wait_ms = 0; // program another draw was reading with the shaders' lock let go, their time)
int64_t why_unnamed = 0, why_let_go = 0, why_ahead = 0; // ((Triforce) vk_shader.cpp readahead_why: of prog_reads, those no
int64_t order_taken = 0; // order named, those read ahead and let go before, those not read ahead yet; of prog_ahead,
// those taken read ahead in the sessions' order, program_order.h)
int64_t made = 0, made_later = 0; // ((Triforce) vk_texture.cpp texture_get: textures made, their time, those not made yet for
double made_ms = 0; // the budget (create_later); vk_vma.cpp: the images made, render targets apart, their time)
int64_t images = 0, targets = 0;
double images_ms = 0, targets_ms = 0;
int64_t qs_bg = 0; // ((Triforce) vk_replay.cpp submit_recording: queue submits made while the background thread
double qs_bg_ms = 0; // imported, their time; vk_memory.cpp: imports given back for the RAM or the heap, and ranges
int64_t given_back = 0, ahead_sent = 0; // sent to be imported ahead)
double disp_pick_ms = 0, disp_pipe_ms = 0; // ((Triforce, test19) vk_draw.cpp dispatch: the programs' picks (their reads, the
int64_t disp_ready = 0, disp_made = 0; // waits for the read-ahead's) and the first layouts and pipelines; the first
int64_t ra_waits = 0; // dispatches that found their pipeline made, those that made one; vk_shader.cpp:
double ra_wait_ms = 0; // the draws' and dispatches' waits for a program the read-ahead was reading)
double captures = 0; // ((Triforce) vk_memory.cpp: the submit-time captures on the game's threads, the copies behind
// them made or waited for there included; outside the replay's submit time)
};
/// (Triforce) The holds of the last picture in a stretch of frames (present_hold.h), for the 600-frame line
struct Holds {
int64_t holds = 0, frames = 0, bounded = 0, new_later = 0;
double ms = 0, longest = 0; // (the holds that ended in the stretch: their time in all, the longest)
};
/// "2 pictures held (14 frames, 230 ms in all, the longest 180 ms, 0 at the 250 ms bound), 312 new textures left for the next frame"
inline int describe_holds(const Holds& h, double bound_ms, char* out, size_t cap) {
return snprintf(out, cap, "%lld pictures held (%lld frames, %.0f ms in all, the longest %.0f ms, %lld at the %.0f ms bound), %lld "
"new textures left for the next frame", (long long)h.holds, (long long)h.frames, h.ms, h.longest,
(long long)h.bounded, bound_ms, (long long)h.new_later);
}
/// A part worth naming: 1 ms or more (the line stays short; the 120-present lines have the rest)
inline bool worth(double ms) { return ms >= 1.0; }
/// The stall line: "vk: stall: frame 12345: 123.4 ms present to present; replay 98.1 ms (draw 5.1, memory sync 90.0 [capture
/// taken 89.9, of it 89.5 waiting for 1 write-back 3 submissions ahead of the GPU, 1 behind a present]); ..."
inline int stall_line(const Frame& f, char* out, size_t cap) {
if (!cap) return 0;
size_t k = 0;
auto put = [&](const char* fmt, auto... a) {
if (k >= cap) return;
const int n = snprintf(out + k, cap - k, fmt, a...);
if (n > 0) k += (size_t)n < cap - k ? (size_t)n : cap - k - 1;
};
put("vk: stall: frame %lld: %.1f ms present to present; replay %.1f ms", (long long)f.no, f.interval, f.replay);
bool open = false;
for (int i = 2; i < f.phases; i++) { // (from draw on: submit and present hold them)
if (!worth(f.ms[i])) continue;
put("%s%s %.1f", open ? ", " : " (", f.names[i], f.ms[i]);
open = true;
const bool memsync = f.names[i][0] == 'm' && f.names[i][1] == 'e' && f.names[i][2] == 'm'; // ("memory sync")
if (!memsync) continue;
bool in = false;
auto part = [&](const char* name, double ms) {
if (!worth(ms)) return;
put("%s%s %.1f", in ? ", " : " [", name, ms);
in = true;
};
part("maps", f.maps);
part("capture taken", f.capture);
part("writes collected", f.collect);
if (in) put("%s", "]");
}
if (open) put("%s", ")");
// (the captures' late pages wait in "capture taken"; a binding or a read of a page put off waits in its own phase)
if (f.wb_waits && worth(f.wb_wait)) {
put("; %lld waits for write-backs still on the GPU %.1f ms (up to %lld submissions ahead of it", (long long)f.wb_waits,
f.wb_wait, (long long)f.wb_ahead);
if (f.wb_behind_present) put(", %lld of the waits behind a present's blit", (long long)f.wb_behind_present);
if (f.wb_unsent) put(", %lld of them for a submission not on the queue yet, %.1f ms", (long long)f.wb_unsent, f.wb_unsent_ms);
if (f.wb_held) // ((Triforce, wp20) the timeline held past the writer's recording: wb_rec_wait.h)
put(", %lld of them %.1f ms after their writer's recording was done (the timeline held by work stopped at a wait)",
(long long)f.wb_held, f.wb_held_ms);
put("%s", ")");
}
if (f.pipe_waits || f.left_out)
put("; %lld waits for shaders or pipelines (%.1f ms), %lld draws left out", (long long)f.pipe_waits, f.pipe_wait,
(long long)f.left_out);
if (f.pipes_first || f.pipes_links) // ((Triforce, wp18) in "pipelines" above)
put("; %lld pipelines new to the draws, %lld of them linked ahead (%lld in the last sessions' order), %lld linked by the "
"draws or their jobs, %lld stage libraries made for them from the caches", (long long)f.pipes_first,
(long long)f.pipes_ahead, (long long)f.pipes_order, (long long)f.pipes_links, (long long)f.pipes_libs);
if (f.uploads) put("; %lld uploads (%.1f MB)", (long long)f.uploads, f.upload_mb);
if (f.uploads_later) put("; %lld uploads of textures held before left for the next frame", (long long)f.uploads_later);
if (f.new_later) put("; %lld new textures left for the next frame", (long long)f.new_later);
if (f.held) put("; not shown: the last picture held (%.0f ms so far)", f.hold_ms);
if (f.imports) put("; %lld pools imported where bound (%.1f ms)", (long long)f.imports, f.import_ms); // (in draw or dispatch above)
if (f.prog_reads || f.prog_ahead) // (in "programs" above)
put("; %lld programs read from the cache by the draws (%.1f ms), %lld taken read ahead", (long long)f.prog_reads,
f.prog_read_ms, (long long)f.prog_ahead);
if (f.why_unnamed || f.why_let_go || f.why_ahead) // ((Triforce) why the read-ahead had not read those)
put("; of the programs the draws read, %lld named by no order, %lld read ahead and let go before, %lld not read ahead yet",
(long long)f.why_unnamed, (long long)f.why_let_go, (long long)f.why_ahead);
if (f.order_taken) put("; %lld taken read ahead in the last sessions' order", (long long)f.order_taken);
if (f.pick_inline) put("; %lld programs translated where asked (%.1f ms)", (long long)f.pick_inline, f.pick_inline_ms);
if (f.ra_waits) put("; %lld waits for programs being read ahead (%.1f ms)", (long long)f.ra_waits, f.ra_wait_ms); // (in "programs")
if (f.once_waits) // ((Triforce) in "programs")
put("; %lld waits for a program another draw was reading (%.1f ms)", (long long)f.once_waits, f.once_wait_ms);
if (worth(f.disp_pick_ms) || worth(f.disp_pipe_ms) || f.disp_made || f.disp_ready) // (in "dispatch" above)
put("; the dispatches' program picks %.1f ms, their first layouts and pipelines %.1f ms (%lld made there, %lld found made)",
f.disp_pick_ms, f.disp_pipe_ms, (long long)f.disp_made, (long long)f.disp_ready);
if (worth(f.pick_lock_ms)) put("; the program picks waited %.1f ms for the shaders' lock", f.pick_lock_ms);
if (worth(f.pick_codes_ms)) put("; the program picks read the stages' code %.1f ms", f.pick_codes_ms);
if (f.split_parts) put("; %lld parts of layered textures' first uploads", (long long)f.split_parts);
if (f.view_lookups) put("; %lld texture views resolved anew", (long long)f.view_lookups); // (in "descriptors" above)
if (f.made || f.made_later) // (in "texture lookups" above)
put("; %lld textures made (%.1f ms), %lld not made yet", (long long)f.made, f.made_ms, (long long)f.made_later);
if (f.qs_bg) put("; %lld queue submits while a background import ran (%.1f ms)", (long long)f.qs_bg, f.qs_bg_ms); // (in "queue submit")
if (f.given_back || f.ahead_sent)
put("; %lld imports given back, %lld ranges sent to be imported ahead", (long long)f.given_back, (long long)f.ahead_sent);
if (f.images || f.targets) // (wherever they were made: texture lookups, or a pass's targets in "commands")
put("; images made: %lld textures (%.1f ms), %lld render targets (%.1f ms)", (long long)f.images, f.images_ms,
(long long)f.targets, f.targets_ms);
if (worth(f.gpu_bound)) put("; the GPU behind %.1f ms", f.gpu_bound);
if (worth(f.present)) put("; present %.1f ms", f.present);
if (worth(f.present) && worth(f.present_wait)) put(" (%.1f of it waiting for the present before on the present thread)", f.present_wait);
if (f.game_waits && worth(f.game_wait)) put("; the game's fence waits %.1f ms in %lld", f.game_wait, (long long)f.game_waits);
const double outside = f.interval - f.replay - f.present;
if (worth(outside)) put("; outside the replay and the present %.1f ms", outside);
if (worth(f.captures)) put("; the captures at the game's submits %.1f ms", f.captures); // ((Triforce) mostly in "outside")
// (the GPU's time while a pool both write is tried in guest memory is the trial's: its waits land in memory sync above)
if (f.trial_phase >= 1 && f.trial_phase <= 4) {
static const char* const what[4] = {"settling in guest memory", "measured in guest memory", "settling mirrored again",
"measured mirrored again"};
put("; pool %#llx on trial (%s)", (unsigned long long)f.trial_pool, what[f.trial_phase - 1]);
}
return (int)k;
}
} // namespace mkwvk::stall_watch

125
gpu/vk/stream_order.h Normal file
View File

@@ -0,0 +1,125 @@
// (Triforce) The parallel replay with a temporal upscaler or a frame generation on: what their hooks in the draws decide
// "first" or "n-th", decided in stream order, whichever thread records which segment. tests/test_stream_order.py
// (tests/stream_order_test.cpp)
//
// NVIDIA DLSS, AMD FSR 4 and 3.1 and the frame generations that take the scene's camera (FSR's, DLSS's) are told things
// by the draws, and decide by the order they are told in (vk_temporal.cpp, vk_fsr3_inputs.cpp):
// - the scene's camera: the largest target with a camera this frame, the first of equal ones (temporal_note_scene_draw);
// - the engine's point before the game's bloom: the first draw after the scene's last (temporal_pre_point, below);
// - the skinned draws and the draws that store their positions, each named by its model and which draw of that model in
// the frame it is (the n-th of a model this frame is matched with the n-th the frame before: dlss_actor::key);
// - and the upscale's dispatch, the engine's point and the draws after it take what those noted.
// Recorded in segments on several threads (vk_replay.cpp replay_handle) the draws came in another order, so the replay
// recorded such a frame on the replay thread alone (par_stream_order_only). Now:
// - the dry pass, which goes through the command buffer in stream order on the replay thread before any segment is
// recorded, tells the camera and the point's choice (vk_draw.cpp dry_temporal_draw), up to the first place that takes
// what they noted: the engine's point, or a dispatch that may be the upscale (a network, or a program that did not
// translate: dry_temporal_dispatch). The segments' workers do not tell them again (replay_hooks_told);
// - from that place to the command buffer's end is the tail: one segment, recorded once every segment before it is
// recorded (InOrder below), with the hooks told as the draws come, as on the replay thread alone;
// - the skinned and stored draws are noted with their place in the stream (replay_stream_pos) and put back in stream
// order, and named again, where they are taken (renumber).
// A command buffer that starts with the engine's work pending from the one before (the engine drew, the upscale's dispatch
// still to come) is recorded on the replay thread alone, as all of them were.
//
// What is not the same as on the replay thread alone: the dry pass asks a draw's program only when the point's choice
// needs it (the first draw after a draw into the scene), so a draw that has no program at all (the log's "a draw without
// a program", which is not drawn) still counts as a draw there; and RX_VK_SKIP_FILE and RX_VK_NO_MESH, which leave draws
// out before the hooks, are not asked by the dry pass. When the stored positions' buffer runs out in a frame, which draws
// were given a slot before it did follows the threads' order (it is made larger for the next frame either way); the same
// for the skinned draws kept past 512 without an upscale (they are dropped then either way).
// RX_VK_PARALLEL_TEMPORAL=0: such a frame on the replay thread alone, as before.
#pragma once
#include <algorithm>
#include <condition_variable>
#include <cstdint>
#include <mutex>
#include <unordered_map>
#include <vector>
namespace stream_order {
/// temporal_pre_point's choice, the scene's camera already known: a draw into the scene starts a run of it again; the first
/// draw after one is the candidate, and it is the point when it is the one learnt (a frame before had it there). Point is
/// {program, format, width, height} with ==; a Point of program 0 is none. here_of() gives the draw's own Point; it is asked
/// only when the answer depends on it (once a run's candidate is another than the learnt one, no later draw of the run can be
/// the point), so a caller that has to look the program up (the dry pass) does it for the first draw after each run
template <class Point, class HereOf>
bool pre_point_step(Point& candidate, const Point& learnt, bool into_scene, HereOf&& here_of) {
if (into_scene) {
candidate = Point{};
return false;
}
if (candidate.program && !(candidate == learnt)) return false;
const Point here = here_of();
if (!candidate.program) candidate = here;
return here == learnt && candidate == here;
}
/// Entries the scene's draws noted (skinned draws, stored positions), each with its place in the stream: in stream order
/// again, and each one's key named again by which of its model's entries it is (the key's low 8 bits, dlss_actor::key; the
/// model is the rest). Entries already in stream order (the replay thread alone) are left as they are: their names are what
/// they were given. pos_of(e): its place; key_of(e): a reference to its key
template <class T, class Pos, class Key>
void renumber(std::vector<T>& v, Pos&& pos_of, Key&& key_of) {
const auto before = [&](const T& a, const T& b) { return pos_of(a) < pos_of(b); };
if (std::is_sorted(v.begin(), v.end(), before)) return;
std::stable_sort(v.begin(), v.end(), before);
std::unordered_map<uint64_t, int> seen;
for (T& e : v) {
uint64_t& k = key_of(e);
const int nth = seen[k >> 8]++;
k = (k & ~0xffull) | (uint64_t)std::min(nth, 255);
}
}
/// Where a command buffer's segments start: indices into points (the dry pass's points, every 64 draws and dispatches, each
/// with its work count, in order), for about `per` draws and dispatches a segment over the work before `until` (the tail's
/// start, or the command buffer's end), at most `most` segments. The first segment starts at the command buffer's start and
/// is not listed. work_of(points[i]): the work count after point i
template <class P, class Work>
std::vector<size_t> cuts(const std::vector<P>& points, int64_t until, int64_t per, int most, Work&& work_of) {
std::vector<size_t> at;
const int n = (int)std::clamp<int64_t>(per > 0 ? until / per : 1, 1, std::max(most, 1));
size_t from = 0;
for (int k = 1; k < n; k++) {
const int64_t want = until * k / n;
size_t i = from;
while (i < points.size() && work_of(points[i]) < want) i++;
if (i >= points.size() || work_of(points[i]) >= until) break;
if (!at.empty() && at.back() == i) continue;
at.push_back(i);
from = i + 1;
}
return at;
}
/// The segments' order where it has to be kept: segment k notes that it is recorded (done), and the tail waits until every
/// segment before it is (wait_before). One a command buffer
class InOrder {
public:
explicit InOrder(size_t n) : done_(n, false) {}
void done(size_t k) {
{
std::lock_guard lk(mu_);
if (k < done_.size()) done_[k] = true;
}
cv_.notify_all();
}
void wait_before(size_t k) {
std::unique_lock lk(mu_);
cv_.wait(lk, [&] {
for (size_t i = 0; i < k && i < done_.size(); i++)
if (!done_[i]) return false;
return true;
});
}
private:
std::mutex mu_;
std::condition_variable cv_;
std::vector<bool> done_;
};
} // namespace stream_order

104
gpu/vk/tail_early.h Normal file
View File

@@ -0,0 +1,104 @@
// (Triforce) Start next frame early: the early tail (vk_replay.cpp "the early tail"; the Graphics tab's row, RX_VK_TAIL_EARLY
// over it; after NX2navi's tail_early, NX2NAVI_TAIL_EARLY, by epicmeme546). Its rules apart from Vulkan, so that
// tests/test_tail_early.py runs them, and the production functions that use them, over recorded command streams.
//
// OoT's Presentation thread polls its frame's event (nvnEventGetValue, host/hos_thread.c present_poll) before it presents and
// lets the next frame be built. The event's word reaches the game's memory when the GPU has run the signal and, where its page
// has a VRAM mirror, the write-back at the end of the signal's recording (vk_memory.cpp memory_end_submission): after
// everything the GPU runs of that recording, the frame's tail too. The tail is what follows the scene: the gameplay upscale's
// dispatch (the game's network, or DLSS, DLAA, FSR 3.1 or FSR 4 in its place), the interface drawn over the picture, the
// recording's end. Nothing of it is what the game reads next: it writes images (the picture, the upscalers' own), which the
// game never reads back, and the present's blit is a recording of its own after it.
//
// With the row on, the replay cuts its recording at the gameplay upscale's dispatch (a part's end as maybe_split makes it: the
// uniform updates before it, the write-backs, a full barrier, the part to the queue) and records the rest into one command
// buffer, without further splits (the tail). Its event signals are held, not written in it. At the recording's end the tail
// is judged by what its commands did (verdict, below). One that passes has its held words written in its recording's upload
// command buffer, after the tail's uploads and before the tail's commands: on the GPU after every write-back of the work
// before the tail (the part before it is ahead in the queue and ends with a full barrier; the words' writes begin with one),
// and before any of the tail's commands. The game's wait ends there, while the GPU runs the tail. One that fails has them
// written at its end, before its write-backs, as their words always reached the game's memory (the game's wait ends where it
// did before).
//
// What a tail must not have done to pass, each a rule (verdict), counted from its first command to its recording's end:
// WRITES a binding written by the GPU (into a mirror, which a write-back copies to the game's memory after the tail, or into
// guest memory itself). The tail's inline uniform updates are not counted: they are the stream's own writes of
// uniform blocks, written at its recording's end as ever (flush_ub_updates), and the game does not read them.
// GUEST a binding given guest memory itself (a pool with no mirror, a mirror piece left in guest memory): the GPU would read
// the game's memory while the game writes its next frame there.
// PROGRAMS a program of the game's that reaches guest memory past its bindings: through the page table (global memory) or a
// texel buffer (a view of a pool's import), read or written.
// BAKE a texture's write-back into the game's memory at its recording's end (vk_texture.cpp texture_end_recording).
// EVENT an event's word outside the pools, or not 4-aligned: the early write has nowhere to go.
// CUT the tail did not stay one command buffer to its recording's end (cut short by something that submits it).
// What it reads otherwise is the replay's: mirrors (synced in its uploads, ahead of the words), its own stream buffers (written
// by the CPU at the replay), images. Counter reports are the CPU's at the replay already (vk_replay.cpp NOP_REPORT_COUNTER),
// and the waits between the game's queues are ordered on the CPU from the signals' values (ev_values), which the replay keeps
// as before.
#pragma once
#include <cstddef>
#include <cstdint>
#include <vector>
namespace tail_early {
/// What a tail's commands did, from its first to its recording's end (vk_replay.cpp tail_seen)
struct Seen {
uint64_t written = 0; // bindings the GPU writes (WRITES)
uint64_t guest = 0; // bindings given guest memory itself (GUEST)
uint64_t programs = 0; // the game's programs that reach guest memory past their bindings (PROGRAMS)
bool bake = false; // a texture written back into guest memory at the recording's end (BAKE)
bool event_outside = false; // a held event's word with no buffer for the early write (EVENT)
bool cut = false; // not one command buffer to its recording's end (CUT)
};
enum Why { EARLY = 0, CUT, WRITES, GUEST, PROGRAMS, BAKE, EVENT, WHY_N };
inline const char* why_name(Why w) {
static const char* const n[WHY_N] = {"early", "cut short", "a written binding", "guest memory bound", "a program past its bindings",
"a texture's write-back", "an event outside the pools"};
return w >= 0 && w < WHY_N ? n[w] : "?";
}
/// EARLY: the tail goes after its held words; else the first rule it fails, and the words are written at its end
inline Why verdict(const Seen& s) {
if (s.cut) return CUT;
if (s.written) return WRITES;
if (s.guest) return GUEST;
if (s.programs) return PROGRAMS;
if (s.bake) return BAKE;
if (s.event_outside) return EVENT;
return EARLY;
}
/// The tail's event signals, held in their order. A signal of mode ADD adds to the last value held for its word, else to
/// what the game's memory has (vk_replay.cpp NOP_SIGNAL_EVENT)
struct Signal {
uint64_t gpu;
uint32_t v;
};
class Held {
public:
void clear() { list_.clear(); }
void add(uint64_t gpu, uint32_t v) { list_.push_back({gpu, v}); }
bool last(uint64_t gpu, uint32_t* v) const {
for (size_t i = list_.size(); i--;)
if (list_[i].gpu == gpu) return *v = list_[i].v, true;
return false;
}
const std::vector<Signal>& all() const { return list_; }
bool empty() const { return list_.empty(); }
private:
std::vector<Signal> list_;
};
/// The counts of what the tails did, for the log's line (vk_replay.cpp tail_stats)
struct Stats {
uint64_t tails = 0, signals = 0, moved = 0; // (tails judged; their signals held; those written before their tail)
uint64_t why[WHY_N] = {};
void judged(Why w, size_t held) {
tails++, why[w]++, signals += held;
if (w == EARLY) moved += held;
}
};
} // namespace tail_early

60
gpu/vk/tex_bc1.h Normal file
View File

@@ -0,0 +1,60 @@
// (Triforce) Opaque ASTC textures kept as BC1 instead of BC3: half the video memory, the same texels (vk_texture.cpp).
// After NX2navi's astc_bc1 (its 4c5855ce: the ASTC textures of a tour 2,568 instead of 3,104 MiB), written anew for this
// renderer. tests/test_tex_bc1.py
//
// A decoded ASTC texture the renderer keeps compressed is encoded on the GPU (host_shaders/bc_encoder_comp_spv.h). A BC3
// block is eight bytes of alpha and eight of colour, and the colour half is laid out as a BC1 block: two 5:6:5 colours and
// sixteen 2-bit indices. BC3 always reads it as four colours (c0, c1, 2/3 c0 + 1/3 c1, 1/3 c0 + 2/3 c1). BC1 reads the same
// eight bytes that way only when c0 > c1; with c0 <= c1 it reads three colours (the third their mean) and index 3 as black,
// transparent in the RGBA format. So a colour half is the same picture as a BC1 block exactly when colour_half_same says so.
// The encoder's EncodeColor(texels, false), which writes BC3's colour half, already orders its colours c0 > c1 and writes
// all sixteen indices 0 when they are equal: every colour half it makes passes. With its push constant bc3 = 0 the encoder
// writes EncodeColor(texels, any texel's alpha under 0.5) alone, eight bytes a block: for texels that are all opaque, the
// same call with the same arguments as BC3's colour half. And BC3's alpha half for such a block is (255, 255, indices 0),
// which decodes to 255, what the BC1 RGB format gives every texel. Which textures are all opaque: astc_opaque.h.
//
// A row on the Graphics tab, Off unless set, because what a test cannot show is whether the GPU decoder agrees with
// astc_opaque.h on every block of the game's textures, and the image changes in vk_texture.cpp have not run in the game.
#pragma once
#include <cstdint>
#include <cstdlib>
namespace tex_bc1 {
/// Whether this run keeps opaque ASTC textures as BC1: RX_VK_TEX_BC1 (0: never, anything else: yes) over the Graphics
/// tab's row as the options file had it at the start (0 Off, 1 On)
inline bool wanted(const char* env, int row) {
if (env && *env) return std::atoi(env) != 0;
return row == 1;
}
/// Whether a BC3 block's colour half (its two colours and its indices, as stored) decodes to the same sixteen colours when
/// the same eight bytes are a BC1 block
constexpr bool colour_half_same(uint16_t c0, uint16_t c1, uint32_t indices) {
if (c0 > c1) return true;
if (c0 < c1) return false;
// (equal: BC1's first three colours are that colour, as all four of BC3's are; its fourth is black)
return ((indices & (indices >> 1)) & 0x55555555u) == 0;
}
/// The bytes a BC1 image's level chain takes: eight a block of 4 x 4 texels (BC3: sixteen)
constexpr uint64_t bytes(uint32_t width, uint32_t height, uint32_t levels, uint32_t layers) {
uint64_t blocks = 0;
for (uint32_t l = 0; l < levels; l++) {
const uint32_t w = (width >> l) ? (width >> l) : 1, h = (height >> l) ? (height >> l) : 1;
blocks += (uint64_t)((w + 3) / 4) * ((h + 3) / 4);
}
return blocks * 8 * layers;
}
/// The Vulkan formats by their numbers (vk_texture.cpp asserts them against the headers): the image's, and a sampled
/// view's for the game's linear or sRGB ASTC format. The RGB formats of BC1: alpha is one whatever the block holds
constexpr int kBc1 = 131, kBc1Srgb = 132, kBc3 = 137, kBc3Srgb = 138;
constexpr int image_format(bool bc1) { return bc1 ? kBc1 : kBc3; }
constexpr int view_format(bool bc1, bool srgb) { return bc1 ? (srgb ? kBc1Srgb : kBc1) : (srgb ? kBc3Srgb : kBc3); }
/// Whether a copy on the GPU between two kept textures may be a copy of their blocks: only between images of one format.
/// A BC1 block is eight bytes and a BC3 block sixteen, so the other pairs must not reach vkCmdCopyImage
constexpr bool block_copy_ok(bool src_bc1, bool dst_bc1) { return src_bc1 == dst_bc1; }
} // namespace tex_bc1

Some files were not shown because too many files have changed in this diff Show More