Triforce v0.2.0

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
ruckus7382sfgsgdf
2026-11-10 19:32:48 -06:00
commit 34b3683441
1730 changed files with 554853 additions and 0 deletions

152
gpu/vk/VULKAN.md Normal file
View File

@@ -0,0 +1,152 @@
# The Vulkan back end (mkw_vk.dll)
> This is the back end's development log from mkw-recomp, kept for its reasons and not updated since. Names and
> defaults have moved: the DLL is `hookshot_vk.dll` (`hookshot_vk.so` on Linux) and the default renderer, with
> `RX_BACKEND` only to pick another; the image and upload code of `vk_image.cpp` and `vk_upload.cpp` is in
> `vk_texture.cpp`; a draw whose pipeline is not ready waits with `RX_VK_PIPE_SKIP=0` (there is no `RX_VK_PIPE_WAIT`);
> `RX_VK_DUMP` takes a present number and writes PNGs; `RX_VK_FRAMEGEN` only turns frame generation off (the Insert
> menu turns it on); the texture budget `RX_VK_VRAM_MB` defaults to 70% of what the driver allows (at most 6 GB),
> `RX_VK_TEX_MAX_MB` to 1536 and `RX_VK_MIRROR_MB` to a share of the card; and `eden/` carries Hookshot's changes.
> The current switches are in [docs/development.md](../../docs/development.md#switches-for-testing-and-debugging).
`RX_BACKEND=vulkan` renders through `mkw_vk.dll` instead of the WebGPU renderer in the executable
(`gpu/render`). `host/rx_backend.c` owns the `rx_*` names the host calls. It loads the DLL, passes it the host
functions (`vk_api.h` `MkwVkHost`) and routes the calls to the table the DLL returns (`MkwVkRenderer`). If the DLL
or Vulkan is missing, the WebGPU renderer stays in place.
Why a DLL of its own: it carries Eden's shader recompiler (`gpu/vk/eden`, copied unmodified from Eglantine, with its
SPIR-V back end and NVN2 task/mesh support) and sirit (`gpu/vk/sirit`, v1.0.5 plus Eden's mesh shader ops patch).
Their C++ namespaces (`Shader::`, `Common::`) are those of the WebGPU renderer's suyu copy, so the two cannot be
linked into one binary.
Why Vulkan, after WebGPU:
- **Mesh shaders.** MKW draws high-detail geometry with NVN2 task/mesh programs, about 60 draws a frame. WebGPU has
no mesh stage (the WebGPU renderer drops them: pop-in, low LODs). `VK_EXT_mesh_shader` runs them as they are.
- **Guest memory in place.** `VK_EXT_external_memory_host` imports the game's pools as buffers. GPU writes land in
guest memory and CPU writes are seen by the GPU, with no mirrored buffer cache, write-back syncs or upload
streams (the WebGPU renderer's biggest costs). Hot read-only ranges can later be mirrored to device memory, as
totk-pc-recomp does.
- **SPIR-V straight from the IR.** No WGSL, no naga, and none of the WebGPU workarounds the WGSL back end carries
(texture swizzle and border emulation, d16 quantization, storage format casts, vertex pulling).
- **No validation or usage tracking on every call**, and bindless descriptors like NVN's own pools.
Reference: totk-pc-recomp (MIT, `work/ext/totk-pc-recomp`, local clone, read only). It has an NVN-on-Vulkan back end
for TOTK: pools imported as host memory, a recorder thread, graphics pipeline libraries, and a uniform snapshot.
## Status
- Milestone 1 (done, 4649fa5): the device (Vulkan 1.3; the RTX 4060 Ti reports mesh shaders, host memory import and
graphics pipeline libraries), the window's swapchain, and presents (a clear colour). The replay is a stub that
completes submissions.
- Milestone 2 (in progress): the replay draws.
1. Guest memory (done): each NVN pool imported as host memory with a buffer over it (`vk_memory.cpp`). MKW has
~2,400 pools at the title, under NVIDIA's ~4,096 allocation limit; merging adjacent pools into one import is
the fix if that grows. GPU address -> (VkBuffer, offset, device address).
2. NVN state (done): the record parser and state of `gpu/render/rx_nvn.cpp` carried over (`vk_replay.cpp`);
submits recorded into command buffers from a ring and signalled on a timeline semaphore (the NVN fences);
render targets as images by storage address (VMA, GENERAL layout, full barriers); colour / depth-stencil
clears; the stream's memory writes on the GPU timeline; presents blit the window texture's image to the
swapchain. Fixed on the way: the present waited on a timeline value another thread had counted but not yet
submitted, which blocks a single queue for good.
Open: with draws and dispatches not made, the game stops at the title after ~80 frames (its job system waits;
no fence wait is outstanding): most likely a compute result it reads back. Draws and dispatches come next.
3. Shaders: an `Environment` for Eden's recompiler like the WebGPU renderer's `NvnEnvironment`
(`gpu/render/rx_shader.cpp`: NVN2 control section, code at +0x50, cbuf reads, texture types from the NVN
texture pool), then the SPIR-V back end; bindings in Eden's layout (`backend/bindings.h`, as its
renderer_vulkan does).
4. Textures: block linear and ASTC decode on the CPU (suyu's `video_core/textures` decoders, copied into the
DLL), BC formats uploaded as they are; render targets as GPU images found by address.
5. Pipelines: dynamic rendering, NVN state -> VkPipeline, push descriptors.
6. Present: the window texture's image blitted to the swapchain.
Note: with guest memory shared, the stream's memory writes (UPDATE_UB, COPY_B2B, SIGNAL_EVENT, REPORT_COUNTER)
must happen on the GPU timeline (vkCmdUpdateBuffer, vkCmdCopyBuffer, timestamp queries), not on the CPU at replay
time as the WebGPU renderer does (its buffers are copies). RENDER_COND maps to VK_EXT_conditional_rendering.
- Milestone 3: compute, indirect draws/dispatches, mesh/task draws, copies and clears, counters and queries.
- Milestone 4: performance: device-local mirrors of hot ranges, pipeline libraries and a pipeline cache, a recorder
thread. Then Vulkan becomes the default.
## Status (Nov 2026): it renders
The replay draws, dispatches, clears, copies and presents. A frame dumped with
`RX_VK_DUMP=<file.ppm> RX_VK_DUMP_AT=<present>` is the game's map screen, right way up and in colour, with its
textures crisp (compare `RX_VK_DUMP_TEX=<dir>`, which writes the decoded textures themselves).
What it took, in the order it blocked a picture:
- **Pools were imported eagerly** on the first submit (~2,100 NVN pools, tens of seconds). They are imported on
first use now (vk_memory.cpp).
- **`NVN_FORMAT_RGBA8` (37) was missing** from the table, so the 4K window texture had no `VkImage` at all. The
RGBA8/RGB8/sRGB family is in vk_formats.cpp now.
- **The window texture is written through a view in another view format** (RGBA8 while the texture says RGBA8), so
the present path keyed past the image the game actually rendered into: `rendered_at()` finds it by storage, and
`present()` blits that (vk_image.cpp, vk_replay.cpp).
- **Transfers were not replayed.** `COPY_T2T/T2B/B2T` and `CLEAR_TEX` now use totk-pc-recomp's implementations
(`sub_resolve`, `buffer_side`, `copy_buffer_image`, `copy_t2t`, `clear_tex`) in vk_xfer.cpp.
- **Textures were decoded into garbage.** The RTX 4060 Ti does not sample ASTC (`textureCompressionASTC_LDR` is
NVIDIA's gap), and MKW's map, terrain and character art are ASTC, so every one is decoded on upload. The first
version hand-rolled the block-linear unswizzle and used totk-pc-recomp's decoder: the result had the right colours
in the right places but every detail shredded into block-sized noise (text unreadable, a dither lattice over flat
areas). Dumping a decoded texture (`RX_VK_DUMP_TEX=<dir>`) showed the corruption was already there, before any
draw. The fix is to stop guessing: `gpu/vk/vk_upload.cpp` now calls suyu's own `Tegra::Texture::UnswizzleTexture`
and `Tegra::Texture::ASTC::DecompressSerial` (`gpu/third_party/suyu/video_core/textures/`), the exact pair the
working WebGPU back end decodes with in rx_texture.cpp. Those two files are built into `mkw_vk.dll` now
(nx2/build.py `vk_sources()`, with suyu's root on the include path). The dumped map texture is pixel-crisp
afterwards, "WORLD MAP" and the course names legible.
- **Backface culling threw away every triangle.** The viewport is flipped with a negative height to turn NVN's
+y-up NDC into Vulkan's y-down, which already reverses the screen-space winding, so NVN's front face maps
straight across; telling Vulkan the opposite (as the code did, "the flipped viewport mirrors the winding") culled
the scene and left a black frame. `RX_VK_NOCULL` was what proved it: 199,300 distinct colours instead of 1.
- **Image cache is budgeted**: least-recently-used images are retired past `RX_VK_VRAM_MB` (2048 default) and
destroyed a few frames later, when recorded work cannot still read them (vk_image.cpp `sweep_images`). Texture
churn had taken the cache to 583 images and climbing.
## The crash at `mkw_vk.dll+0xeb45e`, solved
Not a race. The DLL's include path had suyu's root (`gpu/third_party/suyu`, added for its texture decoders) before
`gpu/vk/eden`, so `#include "shader_recompiler/shader_info.h"` found suyu's copy, whose `Shader::Info` lacks Eden's
`nvn_mesh_stage` and `uses_quad_shuffles`. Objects compiled before that change (Eden's recompiler) and after it
(`vk_draw.cpp`) disagreed on the layout of `Info`, and the descriptor walk read a garbage vector at `this+0x238`. The
object cache leaves `-I` folders out of its keys, which is why nothing rebuilt to show it. nx2/build.py now puts
Eden first and carries `-DMKWVK_INCLUDES=2` so the keys change with the order. (The "descriptor list is not host
memory" guards added while chasing it refused every draw and are gone.)
## The renderer after the rewrite (Nov 1 evening)
Designed after totk-pc-recomp's runtime/vk_*.c (MIT) and Eden's renderer_vulkan, both read in full for it:
- **Textures** (`vk_texture.cpp`): one `VkImage` per texture storage with every level and layer, keyed by storage
address, size, levels, layout and format class (bytes per block for colour, the family for BC / ASTC, the format
for depth): a texture rendered as RGBA8 and sampled as RGBA8_SRGB is one image (`MUTABLE_FORMAT`). Views per use:
sampled (the shader's type: 1D/2D/3D/cube/arrays, NVN swizzles, X formats' alpha one), attachments (one level and
layer; 3D slices through `2D_ARRAY_COMPATIBLE`), storage. Uploaded at first use and again when the CPU wrote the
storage (page write generations, `vk_memory.cpp`, from the host's soft write watch); not once the GPU wrote the
image, unless the texture was finalized since. ASTC is decoded on the GPU by Eden's `astc_decoder.comp`
(`host_shaders/`, with the Switch 2 tiling), straight from the block-linear storage; other formats are unswizzled
on the CPU into the staging memory. Huge textures leave out top levels (`RX_VK_TEX_MAX_DIM` 2048,
`RX_VK_TEX_MAX_MB` 512: the terrain arrays are 8192 x 8192 x 68 ASTC, 24 GB as RGBA8). New images are cleared.
An LRU budget (`RX_VK_VRAM_MB`, 4096, less when `VK_EXT_memory_budget` says the process is near its share).
Images are VRAM only (`requiredFlags`): VMA's fallback to system memory is what ate the user's RAM once.
- **Uploads never end a pass**: each recording has a second command buffer, submitted right before it, for uploads,
new images' layout changes and mirror syncs.
- **Passes** stay open while the targets stay; clears run inside them (`vkCmdClearAttachments`, cut by the scissor,
a full-screen triangle for channel masks); a target the fragment shader does not write keeps its texels (write
mask 0); duplicate attachments are left unattached; draws without targets get the viewport's area.
- **Descriptors**: one set in Eden's order, pushed (`VK_KHR_push_descriptor`) when it fits; storage images, texel
buffers, custom border colours, min/max samplers. Handles resolve once a recording (`handle_hits`).
- **Pipelines**: whatever Vulkan 1.3 and `VK_EXT_extended_dynamic_state3` take as dynamic is dynamic; graphics
programs are translated and pipelines built on worker threads (`vk_jobs.cpp`). A draw whose program or pipeline
is not ready is left out when its targets hold an earlier frame's picture and waits otherwise (TOTK's
`TOTK_PIPE_SKIP` rule; `RX_VK_PIPE_WAIT=1` always waits). The pipeline cache persists (`mkw_vk_pipelines.bin`).
- **Uniform updates** (`UPDATE_UB`) go to guest memory at the end of their recording; draws after one bind a copy
of the range with it applied (`ub_overlay`).
- **The VRAM mirror** (`vk_memory.cpp`, TOTK's): pools bound as buffers get a device-local copy at the next
submission; CPU-written pages are copied in at each submission's first use, GPU writes go back to guest memory at
its end. `RX_VK_MIRROR=0` turns it off, `RX_VK_MIRROR_MB` (3072) is its budget.
- **Mesh shading**: `NOP_DRAW_MESH` records NVN2's DrawMeshTasks*; programs whose code writes the internal stage
buffer are the task / mesh stages (marked in their headers as Eden's GraphicsEnvironment does). Unverified.
- **Frame interpolation** (`vk_framegen.cpp`, `RX_VK_FRAMEGEN=1`): the WebGPU renderer's, ported to GLSL
(`shaders/`, compiled with glslang): made frames between the game's, shown evenly by a display thread.
- **Headless**: without a window presents render offscreen; `RX_VK_SHOTS=<dir>` writes every `RX_VK_SHOT_EVERY`-th
as a PNG. `work/run-vk.ps1` runs it (refuses while Siege runs, stops past `-MaxGB` of working set);
`work/vksum.py <run>` summarises the log.

16
gpu/vk/astc_bc_policy.h Normal file
View File

@@ -0,0 +1,16 @@
// Which cards keep every large decoded ASTC texture as BC3, the finer blocks (UI, characters) too (vk_texture.cpp).
// tests/test_astc_bc_policy.py
#pragma once
#include <cstdint>
namespace astc_bc_policy {
// The VRAM the driver allows the process, in bytes: under 8.5 GB, every large ASTC texture as BC3 (a quarter of RGBA8).
// Issue 519 set it under 6 GB (an RTX 3050 Laptop, 3.3 GB allowed: 1.75 GB of decoded ASTC in Goron City, paged, 4 fps).
// 8 GB cards are allowed ~7-7.6 GB and fill up the same way: an RX 7700S (7.1 GB allowed) held 2.5 GB of decoded ASTC in
// Hyrule Castle Town's market with the VRAM at the limit; as BC3, 1.3 GB. 10 GB cards and up keep the finer ones as RGBA8
inline bool all_by_vram(uint64_t allowed) {
return allowed && allowed < (8704ull << 20);
}
} // namespace astc_bc_policy

33
gpu/vk/bc_levels.h Normal file
View File

@@ -0,0 +1,33 @@
/* Block-compressed levels whose block grid is not level 0's halved (vk_texture.cpp, AMD). */
#pragma once
#include <algorithm>
#include <cstdint>
namespace mkwvk::bc_levels {
/// The 4 x 4 blocks across `texels`
inline uint32_t blocks(uint32_t texels) {
return (texels + 3) / 4;
}
/// The first level (from 1) whose block grid is not level 0's halved, or `levels` when every one is: 1365 texels are
/// 342 blocks, but level 2 (341 texels) is 86 blocks, not 342 >> 2 = 85
inline uint32_t first_uneven(uint32_t width, uint32_t height, uint32_t levels) {
for (uint32_t l = 1; l < levels; l++)
if (blocks(std::max(1u, width >> l)) != std::max(1u, blocks(width) >> l) ||
blocks(std::max(1u, height >> l)) != std::max(1u, blocks(height) >> l))
return l;
return levels;
}
/// The nearest size from `texels` up whose blocks halve evenly down to its levels of 16 texels or more:
/// 1365 -> 1536, 683 -> 768; 768 and 1536 stay
inline uint32_t even_size(uint32_t texels, uint32_t levels) {
uint32_t k = 0;
while (k < levels && (texels >> k) >= 16) k++;
const uint32_t m = 4u << (k ? k - 1 : 0);
return (texels + m - 1) / m * m;
}
} // namespace mkwvk::bc_levels

View File

@@ -0,0 +1,120 @@
// SPDX-FileCopyrightText: Copyright 2026 suyu Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: 2014 Jannik Vogel <email@jannikvogel.de>
// SPDX-License-Identifier: CC0-1.0
#pragma once
#include <bit>
#include <cstddef>
#include <new>
#include <type_traits>
namespace Common {
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr T AlignUp(T value_, size_t size) {
using U = typename std::make_unsigned_t<T>;
auto value{static_cast<U>(value_)};
auto mod{static_cast<T>(value % size)};
value -= mod;
return static_cast<T>(mod == T{0} ? value : value + size);
}
template <typename T>
requires std::is_unsigned_v<T>
[[nodiscard]] constexpr T AlignUpLog2(T value, size_t align_log2) {
return static_cast<T>((value + ((1ULL << align_log2) - 1)) >> align_log2 << align_log2);
}
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr T AlignDown(T value_, size_t size) {
using U = typename std::make_unsigned_t<T>;
const auto value{static_cast<U>(value_)};
return static_cast<T>(value - value % size);
}
template <typename T>
requires std::is_unsigned_v<T>
[[nodiscard]] constexpr bool IsWordAligned(T value) {
return (value & 0b11) == 0;
}
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr bool IsAligned(T value, size_t alignment) noexcept {
using U = typename std::make_unsigned_t<T>;
const U mask = U(alignment - 1);
return (value & mask) == 0;
}
template <typename T, typename U>
requires std::is_integral_v<T>
[[nodiscard]] constexpr T DivideUp(T x, U y) {
return (x + (y - 1)) / y;
}
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr T LeastSignificantOneBit(T x) {
return x & ~(x - 1);
}
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr T ResetLeastSignificantOneBit(T x) {
return x & (x - 1);
}
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr bool IsPowerOfTwo(T x) {
return x > 0 && ResetLeastSignificantOneBit(x) == 0;
}
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr T FloorPowerOfTwo(T x) {
return T{1} << (sizeof(T) * 8 - std::countl_zero(x) - 1);
}
template <typename T, size_t Align = 16>
class AlignmentAllocator {
public:
using value_type = T;
using size_type = size_t;
using difference_type = ptrdiff_t;
using propagate_on_container_copy_assignment = std::true_type;
using propagate_on_container_move_assignment = std::true_type;
using propagate_on_container_swap = std::true_type;
using is_always_equal = std::false_type;
constexpr AlignmentAllocator() noexcept = default;
template <typename T2>
constexpr AlignmentAllocator(const AlignmentAllocator<T2, Align>&) noexcept {}
[[nodiscard]] T* allocate(size_type n) {
return static_cast<T*>(::operator new(n * sizeof(T), std::align_val_t{Align}));
}
void deallocate(T* p, size_type n) {
::operator delete(p, n * sizeof(T), std::align_val_t{Align});
}
template <typename T2>
struct rebind {
using other = AlignmentAllocator<T2, Align>;
};
template <typename T2, size_t Align2>
constexpr bool operator==(const AlignmentAllocator<T2, Align2>&) const noexcept {
return std::is_same_v<T, T2> && Align == Align2;
}
};
} // namespace Common

View File

@@ -0,0 +1,40 @@
// Rexcomp replacement for suyu's common/assert.h: same macros; a failed assertion prints and carries on, as suyu's
// soft asserts do in release builds.
#pragma once
#include <cstdio>
namespace Common {
inline void AssertFail(const char* file, int line, const char* what) {
std::fprintf(stderr, "[shader] assertion failed at %s:%d: %s\n", file, line, what);
}
} // namespace Common
// each assertion reports its first 3 failures
#define ASSERT_MSG(_a_, ...) \
do { \
if (!(_a_)) [[unlikely]] { \
static int assert_reports_; \
if (assert_reports_++ < 3) ::Common::AssertFail(__FILE__, __LINE__, #_a_); \
} \
} while (0)
#define ASSERT(_a_) ASSERT_MSG(_a_, "")
#define UNREACHABLE_MSG(...) \
do { \
::Common::AssertFail(__FILE__, __LINE__, "unreachable"); \
__builtin_unreachable(); \
} while (0)
#define UNREACHABLE() UNREACHABLE_MSG("")
#define DEBUG_ASSERT(_a_) ((void)0)
#define DEBUG_ASSERT_MSG(_a_, ...) ((void)0)
#define UNIMPLEMENTED() ASSERT(false && "Unimplemented!")
#define UNIMPLEMENTED_MSG(...) ASSERT_MSG(false, __VA_ARGS__)
#define UNIMPLEMENTED_IF(cond) ASSERT((!(cond)) && "Unimplemented!")
#define UNIMPLEMENTED_IF_MSG(cond, ...) ASSERT_MSG(!(cond), __VA_ARGS__)
#define ASSERT_OR_EXECUTE(_a_, _b_) \
do { \
ASSERT(_a_); \
if (!(_a_)) [[unlikely]] { \
_b_ \
} \
} while (0)
#define ASSERT_OR_EXECUTE_MSG(_a_, _b_, ...) ASSERT_OR_EXECUTE(_a_, _b_)

View File

@@ -0,0 +1,175 @@
// SPDX-FileCopyrightText: Copyright 2026 suyu Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: 2014 Tony Wasserka
// SPDX-FileCopyrightText: 2014 Dolphin Emulator Project
// SPDX-License-Identifier: BSD-3-Clause AND GPL-2.0-or-later
#pragma once
#include <cstddef>
#include <limits>
#include <type_traits>
#include <climits>
#include "common/swap.h"
/*
* Abstract bitfield class
*
* Allows endianness-independent access to individual bitfields within some raw
* integer value. The assembly generated by this class is identical to the
* usage of raw bitfields, so it's a perfectly fine replacement.
*
* For BitField<X,Y,Z>, X is the distance of the bitfield to the LSB of the
* raw value, Y is the length in bits of the bitfield. Z is an integer type
* which determines the sign of the bitfield. Z must have the same size as the
* raw integer.
*
*
* General usage:
*
* Create a new union with the raw integer value as a member.
* Then for each bitfield you want to expose, add a BitField member
* in the union. The template parameters are the bit offset and the number
* of desired bits.
*
* Changes in the bitfield members will then get reflected in the raw integer
* value and vice-versa.
*
*
* Sample usage:
*
* union SomeRegister
* {
* u32 hex;
*
* BitField<0,7,u32> first_seven_bits; // unsigned
* BitField<7,8,u32> next_eight_bits; // unsigned
* BitField<3,15,s32> some_signed_fields; // signed
* };
*
* This is equivalent to the little-endian specific code:
*
* union SomeRegister
* {
* u32 hex;
*
* struct
* {
* u32 first_seven_bits : 7;
* u32 next_eight_bits : 8;
* };
* struct
* {
* u32 : 3; // padding
* s32 some_signed_fields : 15;
* };
* };
*
*
* Caveats:
*
* 1)
* BitField provides automatic casting from and to the storage type where
* appropriate. However, when using non-typesafe functions like printf, an
* explicit cast must be performed on the BitField object to make sure it gets
* passed correctly, e.g.:
* printf("Value: %d", (s32)some_register.some_signed_fields);
*
* 2)
* Not really a caveat, but potentially irritating: This class is used in some
* packed structures that do not guarantee proper alignment. Therefore we have
* to use #pragma pack here not to pack the members of the class, but instead
* to break GCC's assumption that the members of the class are aligned on
* sizeof(StorageType).
* TODO(neobrain): Confirm that this is a proper fix and not just masking
* symptoms.
*/
#pragma pack(1)
template <std::size_t Position, std::size_t Bits, typename T, typename EndianTag = LETag>
requires std::is_trivially_copyable_v<T>
struct BitField {
private:
// UnderlyingType is T for non-enum types and the underlying type of T if
// T is an enumeration. Note that T is wrapped within an enable_if in the
// former case to workaround compile errors which arise when using
// std::underlying_type<T>::type directly.
using UnderlyingType = typename std::conditional_t<std::is_enum_v<T>, std::underlying_type<T>, std::enable_if<true, T>>::type;
// We store the value as the unsigned type to avoid undefined behaviour on value shifting
using StorageType = std::make_unsigned_t<UnderlyingType>;
using StorageTypeWithEndian = typename AddEndian<StorageType, EndianTag>::type;
public:
/// Constants to allow limited introspection of fields if needed
static constexpr std::size_t position = Position;
static constexpr std::size_t bits = Bits;
static constexpr StorageType mask = (StorageType(~0) >> (CHAR_BIT * sizeof(T) - bits)) << position;
/**
* Formats a value by masking and shifting it according to the field parameters. A value
* containing several bitfields can be assembled by formatting each of their values and ORing
* the results together.
*/
[[nodiscard]] static constexpr StorageType FormatValue(const T& value) {
return (StorageType(value) << position) & mask;
}
/// @brief Extracts a value from the passed storage. In most situations prefer use the member functions
/// (such as Value() or operator T), but this can be used to extract a value from a bitfield
/// union in a constexpr context.
[[nodiscard]] static constexpr T ExtractValue(const StorageType& storage) {
if constexpr (std::numeric_limits<UnderlyingType>::is_signed) {
std::size_t shift = CHAR_BIT * sizeof(T) - bits;
return T(UnderlyingType(storage << (shift - position)) >> shift);
} else {
return T((storage & mask) >> position);
}
}
// This constructor and assignment operator might be considered ambiguous:
// Would they initialize the storage or just the bitfield?
// Hence, delete them. Use the Assign method to set bitfield values!
BitField(T val) = delete;
BitField& operator=(T val) = delete;
constexpr BitField() noexcept = default;
constexpr BitField(const BitField&) noexcept = default;
constexpr BitField& operator=(const BitField&) noexcept = default;
constexpr BitField(BitField&&) noexcept = default;
constexpr BitField& operator=(BitField&&) noexcept = default;
constexpr void Assign(const T value) {
storage = StorageType((storage & ~mask) | FormatValue(value));
}
[[nodiscard]] constexpr T Value() const {
return ExtractValue(storage);
}
[[nodiscard]] constexpr operator T() const {
return Value();
}
[[nodiscard]] constexpr explicit operator bool() const {
return Value() != 0;
}
private:
StorageTypeWithEndian storage;
static_assert(bits + position <= CHAR_BIT * sizeof(T), "Bitfield out of range");
// And, you know, just in case people specify something stupid like bits=position=0x80000000
static_assert(position < CHAR_BIT * sizeof(T), "Invalid position");
static_assert(bits <= CHAR_BIT * sizeof(T), "Invalid number of bits");
static_assert(bits > 0, "Invalid number of bits");
};
#pragma pack()
template <std::size_t Position, std::size_t Bits, typename T>
using BitFieldBE = BitField<Position, Bits, T, BETag>;
template <std::size_t Position, std::size_t Bits, typename T, typename EndianTag = LETag>
inline auto format_as(BitField<Position, Bits, T, EndianTag> bitfield) {
return bitfield.Value();
}

View File

@@ -0,0 +1,57 @@
// SPDX-FileCopyrightText: Copyright 2026 suyu Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2018 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <bit>
#include <climits>
#include <cstddef>
#include <type_traits>
#include "common/common_types.h"
namespace Common {
/// Gets the size of a specified type T in bits.
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr std::size_t BitSize() {
return std::size_t(sizeof(T) * CHAR_BIT);
}
template<typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr u32 MostSignificantBit(const T value) {
return u32(sizeof(T) * CHAR_BIT - 1 - std::countl_zero(value));
}
template<typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr T Log2Floor(const T value) {
return T(MostSignificantBit<T>(value));
}
template<typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr T Log2Ceil(const T value) {
const T log2_f = Log2Floor<T>(value);
return T(log2_f + T((value ^ (T(1ULL) << log2_f)) != T(0ULL)));
}
template <typename T>
requires std::is_integral_v<T>
[[nodiscard]] T NextPow2(T value) {
return T(1ULL << (sizeof(T) * CHAR_BIT - std::countl_zero(value - 1U)));
}
template <size_t bit_index, typename T>
requires std::is_integral_v<T>
[[nodiscard]] constexpr bool Bit(const T value) {
static_assert(bit_index < BitSize<T>(), "bit_index must be smaller than size of T");
return ((value >> bit_index) & T(1)) == T(1);
}
} // namespace Common

View File

@@ -0,0 +1,111 @@
// SPDX-FileCopyrightText: Copyright 2026 suyu Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2019 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <array>
#include <iterator>
#include <utility>
#include <cstdlib> // for exit
#include "common/common_types.h"
#ifndef __cpp_lib_to_underlying
namespace std {
// https://en.cppreference.com/cpp/utility/to_underlying
template<class T>
requires std::is_enum_v<T>
constexpr std::underlying_type_t<T> to_underlying(T e) noexcept {
return std::underlying_type_t<T>(e);
}
}
#endif
/// Textually concatenates two tokens. The double-expansion is required by the C preprocessor.
#define CONCAT2(x, y) DO_CONCAT2(x, y)
#define DO_CONCAT2(x, y) x##y
/// Helper macros to insert unused bytes or words to properly align structs. These values will be
/// zero-initialized.
#define INSERT_PADDING_BYTES(num_bytes) \
[[maybe_unused]] std::array<u8, num_bytes> CONCAT2(pad, __LINE__) {}
#define INSERT_PADDING_WORDS(num_words) \
[[maybe_unused]] std::array<u32, num_words> CONCAT2(pad, __LINE__) {}
/// These are similar to the INSERT_PADDING_* macros but do not zero-initialize the contents.
/// This keeps the structure trivial to construct.
#define INSERT_PADDING_BYTES_NOINIT(num_bytes) \
[[maybe_unused]] std::array<u8, num_bytes> CONCAT2(pad, __LINE__)
#define INSERT_PADDING_WORDS_NOINIT(num_words) \
[[maybe_unused]] std::array<u32, num_words> CONCAT2(pad, __LINE__)
#ifdef _MSC_VER
# define locale_t _locale_t // Locale Cross-Compatibility
#endif // _MSC_VER
#define DECLARE_ENUM_FLAG_OPERATORS(type) \
[[nodiscard]] constexpr type operator|(type a, type b) noexcept { \
return type(std::to_underlying(a) | std::to_underlying(b)); \
} \
[[nodiscard]] constexpr type operator&(type a, type b) noexcept { \
return type(std::to_underlying(a) & std::to_underlying(b)); \
} \
[[nodiscard]] constexpr type operator^(type a, type b) noexcept { \
return type(std::to_underlying(a) ^ std::to_underlying(b)); \
} \
[[nodiscard]] constexpr type operator<<(type a, type b) noexcept { \
return type(std::to_underlying(a) << std::to_underlying(b)); \
} \
[[nodiscard]] constexpr type operator>>(type a, type b) noexcept { \
return type(std::to_underlying(a) >> std::to_underlying(b)); \
} \
constexpr type operator|=(type& a, type b) noexcept { \
return a = a | b; \
} \
constexpr type operator&=(type& a, type b) noexcept { \
return a = a & b; \
} \
constexpr type operator^=(type& a, type b) noexcept { \
return a = a ^ b; \
} \
constexpr type operator<<=(type& a, type b) noexcept { \
return a = a << b; \
} \
constexpr type operator>>=(type& a, type b) noexcept { \
return a = a >> b; \
} \
[[nodiscard]] constexpr type operator~(type key) noexcept { \
return type(~std::to_underlying(key)); \
} \
[[nodiscard]] constexpr bool True(type key) noexcept { \
return std::to_underlying(key) != 0; \
} \
[[nodiscard]] constexpr bool False(type key) noexcept { \
return std::to_underlying(key) == 0; \
}
#define YUZU_NON_COPYABLE(cls) \
cls(const cls&) = delete; \
cls& operator=(const cls&) = delete
#define YUZU_NON_MOVEABLE(cls) \
cls(cls&&) = delete; \
cls& operator=(cls&&) = delete
// suyu-branded aliases
#define SUYU_NON_COPYABLE(cls) YUZU_NON_COPYABLE(cls)
#define SUYU_NON_MOVEABLE(cls) YUZU_NON_MOVEABLE(cls)
namespace Common {
[[nodiscard]] constexpr u32 MakeMagic(char a, char b, char c, char d) {
return u32(a) | u32(b) << 8 | u32(c) << 16 | u32(d) << 24;
}
[[nodiscard]] constexpr u64 MakeMagic(char a, char b, char c, char d, char e, char f, char g, char h) {
return u64(a) << 0 | u64(b) << 8 | u64(c) << 16 | u64(d) << 24 | u64(e) << 32 | u64(f) << 40 | u64(g) << 48 | u64(h) << 56;
}
} // namespace Common

View File

@@ -0,0 +1,56 @@
// SPDX-FileCopyrightText: Copyright 2025 suyu Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: 2012 Gekko Emulator
// SPDX-FileContributor: ShizZy <shizzy247@gmail.com>
// SPDX-License-Identifier: GPL-2.0-or-later
/**
* Copyright (C) 2005-2012 Gekko Emulator
*
* @file common_types.h
* @author ShizZy <shizzy247@gmail.com>
* @date 2012-02-11
* @brief Common types used throughout the project
*
* @section LICENSE
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation; either version 2 of
* the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details at
* http://www.gnu.org/copyleft/gpl.html
*
* Official project repository can be found at:
* http://code.google.com/p/gekko-gc-emu/
*/
#pragma once
#include <array>
#include <cstdint>
using u8 = std::uint8_t; ///< 8-bit unsigned byte
using u16 = std::uint16_t; ///< 16-bit unsigned short
using u32 = std::uint32_t; ///< 32-bit unsigned word
using u64 = std::uint64_t; ///< 64-bit unsigned int
using s8 = std::int8_t; ///< 8-bit signed byte
using s16 = std::int16_t; ///< 16-bit signed short
using s32 = std::int32_t; ///< 32-bit signed word
using s64 = std::int64_t; ///< 64-bit signed int
using f32 = float; ///< 32-bit floating point
using f64 = double; ///< 64-bit floating point
using VAddr = u64; ///< Represents a pointer in the userspace virtual address space.
using DAddr = u64; ///< Represents a pointer in the device specific virtual address space.
using PAddr = u64; ///< Represents a pointer in the ARM11 physical address space.
using GPUVAddr = u64; ///< Represents a pointer in the GPU virtual address space.
using u128 = std::array<std::uint64_t, 2>;
static_assert(sizeof(u128) == 16, "u128 must be 128 bits wide");

View File

@@ -0,0 +1,15 @@
// mkw-recomp: Eden's Common::unordered_map (boost's flat map there) as std::unordered_map: the API it uses
#pragma once
#include <functional>
#include <memory>
#include <unordered_map>
#include <utility>
#include "common/container_hash.h"
namespace Common {
template <class Key, class T, class Hash = std::hash<Key>, class Pred = std::equal_to<Key>,
class Allocator = std::allocator<std::pair<const Key, T>>>
using unordered_map = std::unordered_map<Key, T, Hash, Pred, Allocator>;
}

View File

@@ -0,0 +1,13 @@
// mkw-recomp: Eden's Common::unordered_set (boost's flat set there) as std::unordered_set: the API it uses
#pragma once
#include <functional>
#include <memory>
#include <unordered_set>
#include "common/container_hash.h"
namespace Common {
template <class Key, class Hash = std::hash<Key>, class Pred = std::equal_to<Key>, class Allocator = std::allocator<Key>>
using unordered_set = std::unordered_set<Key, Hash, Pred, Allocator>;
}

View File

@@ -0,0 +1,152 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: 2005-2014 Daniel James
// SPDX-FileCopyrightText: 2016 Austin Appleby
// SPDX-License-Identifier: BSL-1.0
#pragma once
#include <array>
#include <climits>
#include <cstdint>
#include <functional>
#include <limits>
#include <tuple>
#include <type_traits>
#include <utility>
#include <vector>
namespace Common {
/// The default hash of Common::unordered_map/set. std::hash of an integer or a pointer is a
/// byte-wise FNV-1a on MSVC, and boost's flat containers mix a hash that does not say it
/// avalanches anyway: pass those keys through (the GPU thread's page maps looked keys up
/// tens of thousands of times a second with it). Other keys keep std::hash.
template <class Key>
struct FastHash : std::hash<Key> {};
template <class Key>
requires std::is_integral_v<Key>
struct FastHash<Key> {
std::size_t operator()(Key key) const noexcept {
return static_cast<std::size_t>(key);
}
};
template <class T>
struct FastHash<T*> {
std::size_t operator()(T* pointer) const noexcept {
return reinterpret_cast<std::size_t>(pointer);
}
};
namespace detail {
template <typename T>
requires std::is_unsigned_v<T>
inline std::size_t HashValue(T val) {
const unsigned int size_t_bits = std::numeric_limits<std::size_t>::digits;
const unsigned int length =
(std::numeric_limits<T>::digits - 1) / static_cast<unsigned int>(size_t_bits);
std::size_t seed = 0;
for (unsigned int i = length * size_t_bits; i > 0; i -= size_t_bits) {
seed ^= static_cast<size_t>(val >> i) + (seed << 6) + (seed >> 2);
}
seed ^= static_cast<size_t>(val) + (seed << 6) + (seed >> 2);
return seed;
}
template <size_t Bits>
struct HashCombineImpl {
template <typename T>
static inline T fn(T seed, T value) {
seed ^= value + 0x9e3779b9 + (seed << 6) + (seed >> 2);
return seed;
}
};
template <>
struct HashCombineImpl<64> {
static inline std::uint64_t fn(std::uint64_t h, std::uint64_t k) {
const std::uint64_t m = (std::uint64_t(0xc6a4a793) << 32) + 0x5bd1e995;
const int r = 47;
k *= m;
k ^= k >> r;
k *= m;
h ^= k;
h *= m;
// Completely arbitrary number, to prevent 0's
// from hashing to 0.
h += 0xe6546b64;
return h;
}
};
} // namespace detail
template <typename T>
requires std::is_unsigned_v<T>
inline void HashCombine(std::size_t& seed, const T& v) {
seed = detail::HashCombineImpl<sizeof(std::size_t) * CHAR_BIT>::fn(seed, detail::HashValue(v));
}
template <typename T>
requires (!std::is_unsigned_v<T>)
inline void HashCombine(std::size_t& seed, const T& v) {
seed = detail::HashCombineImpl<sizeof(std::size_t) * CHAR_BIT>::fn(seed, std::hash<T>{}(v));
}
template <typename It>
inline std::size_t HashRange(It first, It last) {
std::size_t seed = 0;
for (; first != last; ++first) {
HashCombine<typename std::iterator_traits<It>::value_type>(seed, *first);
}
return seed;
}
template <typename T, size_t Size>
std::size_t HashValue(const std::array<T, Size>& v) {
return HashRange(v.cbegin(), v.cend());
}
template <typename T, typename Allocator>
std::size_t HashValue(const std::vector<T, Allocator>& v) {
return HashRange(v.cbegin(), v.cend());
}
} // namespace Common
namespace std {
template <typename... Args>
struct hash<std::tuple<Args...>> {
std::size_t operator()(const std::tuple<Args...>& t) const noexcept {
std::size_t seed = 0;
std::apply([&seed](const Args&... args) { (Common::HashCombine(seed, args), ...); }, t);
return seed;
}
};
template <class A, class B>
struct hash<std::pair<A, B>> {
std::size_t operator()(const std::pair<A, B>& p) const noexcept {
std::size_t seed = 0;
Common::HashCombine(seed, p.first);
Common::HashCombine(seed, p.second);
return seed;
}
};
} // namespace std

View File

@@ -0,0 +1,25 @@
// SPDX-FileCopyrightText: Copyright 2020 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <cstddef>
#include <type_traits>
namespace Common {
/// Ceiled integer division.
template <typename N, typename D>
requires std::is_integral_v<N> && std::is_unsigned_v<D>
[[nodiscard]] constexpr N DivCeil(N number, D divisor) {
return static_cast<N>((static_cast<D>(number) + divisor - 1) / divisor);
}
/// Ceiled integer division with logarithmic divisor in base 2
template <typename N, typename D>
requires std::is_integral_v<N> && std::is_unsigned_v<D>
[[nodiscard]] constexpr N DivCeilLog2(N value, D alignment_log2) {
return static_cast<N>((static_cast<D>(value) + (D(1) << alignment_log2) - 1) >> alignment_log2);
}
} // namespace Common

View File

@@ -0,0 +1,15 @@
// Rexcomp replacement for suyu's common/logging.h: log lines go nowhere unless GPU_SHADER_LOG is defined.
#pragma once
#include <fmt/format.h> // suyu code reaches fmt through this header
#ifdef GPU_SHADER_LOG
#include <cstdio>
#define LOG_GENERIC_(level, cls, ...) std::fprintf(stderr, "[shader %s] %s\n", level, fmt::format(__VA_ARGS__).c_str())
#else
#define LOG_GENERIC_(level, cls, ...) ((void)0)
#endif
#define LOG_TRACE(cls, ...) ((void)0)
#define LOG_DEBUG(cls, ...) ((void)0)
#define LOG_INFO(cls, ...) LOG_GENERIC_("info", cls, __VA_ARGS__)
#define LOG_WARNING(cls, ...) LOG_GENERIC_("warning", cls, __VA_ARGS__)
#define LOG_ERROR(cls, ...) LOG_GENERIC_("error", cls, __VA_ARGS__)
#define LOG_CRITICAL(cls, ...) LOG_GENERIC_("critical", cls, __VA_ARGS__)

View File

@@ -0,0 +1,100 @@
// SPDX-FileCopyrightText: Copyright 2026 suyu Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: 2013 Dolphin Emulator Project
// SPDX-FileCopyrightText: 2014 Citra Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <algorithm>
#include <cstdlib>
#include <type_traits>
namespace Common {
template <class T>
struct Rectangle {
T left{};
T top{};
T right{};
T bottom{};
constexpr Rectangle() = default;
constexpr Rectangle(T width, T height) : right(width), bottom(height) {}
constexpr Rectangle(T left_, T top_, T right_, T bottom_)
: left(left_), top(top_), right(right_), bottom(bottom_) {}
[[nodiscard]] constexpr T Left() const {
return left;
}
[[nodiscard]] constexpr T Top() const {
return top;
}
[[nodiscard]] constexpr T Right() const {
return right;
}
[[nodiscard]] constexpr T Bottom() const {
return bottom;
}
[[nodiscard]] constexpr bool IsEmpty() const {
return (GetWidth() <= 0) || (GetHeight() <= 0);
}
[[nodiscard]] constexpr T GetWidth() const {
if constexpr (std::is_floating_point_v<T>) {
return std::abs(right - left);
} else {
return static_cast<T>(std::abs(static_cast<std::make_signed_t<T>>(right - left)));
}
}
[[nodiscard]] constexpr T GetHeight() const {
if constexpr (std::is_floating_point_v<T>) {
return std::abs(bottom - top);
} else {
return static_cast<T>(std::abs(static_cast<std::make_signed_t<T>>(bottom - top)));
}
}
[[nodiscard]] constexpr Rectangle<T> TranslateX(const T x) const {
return Rectangle{left + x, top, right + x, bottom};
}
[[nodiscard]] constexpr Rectangle<T> TranslateY(const T y) const {
return Rectangle{left, top + y, right, bottom + y};
}
[[nodiscard]] constexpr Rectangle<T> Scale(const float s) const {
return Rectangle{left, top, static_cast<T>(static_cast<float>(left + GetWidth()) * s),
static_cast<T>(static_cast<float>(top + GetHeight()) * s)};
}
[[nodiscard]] constexpr bool operator==(const Rectangle<T>& rhs) const {
return (left == rhs.left) && (top == rhs.top) && (right == rhs.right) &&
(bottom == rhs.bottom);
}
[[nodiscard]] constexpr bool operator!=(const Rectangle<T>& rhs) const {
return !operator==(rhs);
}
[[nodiscard]] constexpr bool Intersect(const Rectangle<T>& with, Rectangle<T>* result) const {
result->left = (std::max)(left, with.left);
result->top = (std::max)(top, with.top);
result->right = (std::min)(right, with.right);
result->bottom = (std::min)(bottom, with.bottom);
return !result->IsEmpty();
}
};
template <typename T>
Rectangle(T, T, T, T) -> Rectangle<T>;
} // namespace Common

View File

@@ -0,0 +1,82 @@
// SPDX-FileCopyrightText: 2014 Citra Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <utility>
#include "common/common_funcs.h"
namespace detail {
template <class F>
class ScopeGuard {
YUZU_NON_COPYABLE(ScopeGuard);
private:
F f;
bool active;
public:
constexpr ScopeGuard(F f_) : f(std::move(f_)), active(true) {}
constexpr ~ScopeGuard() {
if (active) {
f();
}
}
constexpr void Cancel() {
active = false;
}
constexpr ScopeGuard(ScopeGuard&& rhs) : f(std::move(rhs.f)), active(rhs.active) {
rhs.Cancel();
}
ScopeGuard& operator=(ScopeGuard&& rhs) = delete;
};
template <class F>
constexpr ScopeGuard<F> MakeScopeGuard(F f) {
return ScopeGuard<F>(std::move(f));
}
enum class ScopeGuardOnExit {};
template <typename F>
constexpr ScopeGuard<F> operator+(ScopeGuardOnExit, F&& f) {
return ScopeGuard<F>(std::forward<F>(f));
}
} // namespace detail
#define CONCATENATE_IMPL(s1, s2) s1##s2
#define CONCATENATE(s1, s2) CONCATENATE_IMPL(s1, s2)
#ifdef __COUNTER__
#define ANONYMOUS_VARIABLE(pref) CONCATENATE(pref, __COUNTER__)
#else
#define ANONYMOUS_VARIABLE(pref) CONCATENATE(pref, __LINE__)
#endif
/**
* This macro is similar to SCOPE_EXIT, except the object is caller managed. This is intended to be
* used when the caller might want to cancel the ScopeExit.
*/
#define SCOPE_GUARD detail::ScopeGuardOnExit() + [&]()
/**
* This macro allows you to conveniently specify a block of code that will run on scope exit. Handy
* for doing ad-hoc clean-up tasks in a function with multiple returns.
*
* Example usage:
* \code
* const int saved_val = g_foo;
* g_foo = 55;
* SCOPE_EXIT{ g_foo = saved_val; };
*
* if (Bar()) {
* return 0;
* } else {
* return 20;
* }
* \endcode
*/
#define SCOPE_EXIT auto ANONYMOUS_VARIABLE(SCOPE_EXIT_STATE_) = SCOPE_GUARD

View File

@@ -0,0 +1,38 @@
// mkw-recomp: the settings Eden's shader recompiler reads (gpu/vk/eden), as constants. The Vulkan backend
// renders at the game's own resolution, so resolution scaling is off.
#pragma once
#include "common/common_types.h"
namespace Settings {
template <typename T>
struct Constant {
T value{};
T GetValue() const {
return value;
}
operator T() const {
return value;
}
};
struct ResolutionScalingInfo {
u32 up_scale{1};
u32 down_shift{0};
f32 up_factor{1.0f};
f32 down_factor{1.0f};
bool active{false};
bool downscale{false};
};
struct Values {
bool disable_shader_loop_safety_checks{false};
Constant<bool> fix_bloom_effects{};
bool renderer_debug{false};
ResolutionScalingInfo resolution_info{};
};
inline Values values{};
} // namespace Settings

664
gpu/vk/compat/common/swap.h Normal file
View File

@@ -0,0 +1,664 @@
// SPDX-FileCopyrightText: Copyright 2026 suyu Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: 2012 PPSSPP Project
// SPDX-FileCopyrightText: 2012 Dolphin Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
// Official git repository and contact information can be found at
// https://github.com/hrydgard/ppsspp and http://www.ppsspp.org/.
#pragma once
#include <cstdlib>
#include <cstring>
#include <type_traits>
#include <bit>
#include "common/common_types.h"
namespace Common {
#if defined(__Bitrig__) || defined(__OpenBSD__)
// We'll redefine swap16, swap32, swap64 as inline functions
// but OpenBSD is like "wow I bring my own stuff"
// It would be nice if we could use them without C++ namespace shenanigans
// But alas :)
#undef swap16
#undef swap32
#undef swap64
#endif
#if defined(_MSC_VER)
[[nodiscard]] inline u16 swap16(u16 data) noexcept {
return _byteswap_ushort(data);
}
[[nodiscard]] inline u32 swap32(u32 data) noexcept {
return _byteswap_ulong(data);
}
[[nodiscard]] inline u64 swap64(u64 data) noexcept {
return _byteswap_uint64(data);
}
#elif defined(__clang__) || defined(__GNUC__)
[[nodiscard]] inline u16 swap16(u16 data) noexcept {
return __builtin_bswap16(data);
}
[[nodiscard]] inline u32 swap32(u32 data) noexcept {
return __builtin_bswap32(data);
}
[[nodiscard]] inline u64 swap64(u64 data) noexcept {
return __builtin_bswap64(data);
}
#else
// Generic implementation - compiler will optimise these into their respective
// __builtin_byteswapXX() and such; if not, the compiler is stupid and we probably
// have bigger problems to worry about :)
[[nodiscard]] inline u16 swap16(u16 data) noexcept {
return (data >> 8) | (data << 8);
}
[[nodiscard]] inline u32 swap32(u32 data) noexcept {
return ((data & 0xFF000000U) >> 24) | ((data & 0x00FF0000U) >> 8) |
((data & 0x0000FF00U) << 8) | ((data & 0x000000FFU) << 24);
}
[[nodiscard]] inline u64 swap64(u64 data) noexcept {
return ((data & 0xFF00000000000000ULL) >> 56) | ((data & 0x00FF000000000000ULL) >> 40) |
((data & 0x0000FF0000000000ULL) >> 24) | ((data & 0x000000FF00000000ULL) >> 8) |
((data & 0x00000000FF000000ULL) << 8) | ((data & 0x0000000000FF0000ULL) << 24) |
((data & 0x000000000000FF00ULL) << 40) | ((data & 0x00000000000000FFULL) << 56);
}
#endif
[[nodiscard]] inline float swapf(float f) noexcept {
static_assert(sizeof(u32) == sizeof(float), "float must be the same size as uint32_t.");
u32 value;
std::memcpy(&value, &f, sizeof(u32));
value = swap32(value);
std::memcpy(&f, &value, sizeof(u32));
return f;
}
[[nodiscard]] inline double swapd(double f) noexcept {
static_assert(sizeof(u64) == sizeof(double), "double must be the same size as uint64_t.");
u64 value;
std::memcpy(&value, &f, sizeof(u64));
value = swap64(value);
std::memcpy(&f, &value, sizeof(u64));
return f;
}
} // Namespace Common
template <typename T, typename F>
struct SwapStructT {
using SwappedT = SwapStructT;
protected:
T value;
static T swap(T v) {
return F::swap(v);
}
public:
T swap() const {
return swap(value);
}
SwapStructT() = default;
SwapStructT(const T& v) : value(swap(v)) {}
template <typename S>
SwappedT& operator=(const S& source) {
value = swap(T(source));
return *this;
}
operator s8() const {
return s8(swap());
}
operator u8() const {
return u8(swap());
}
operator s16() const {
return s16(swap());
}
operator u16() const {
return u16(swap());
}
operator s32() const {
return s32(swap());
}
operator u32() const {
return u32(swap());
}
operator s64() const {
return s64(swap());
}
operator u64() const {
return u64(swap());
}
operator float() const {
return float(swap());
}
operator double() const {
return double(swap());
}
// +v
SwappedT operator+() const {
return +swap();
}
// -v
SwappedT operator-() const {
return -swap();
}
// v / 5
SwappedT operator/(const SwappedT& i) const {
return swap() / i.swap();
}
template <typename S>
SwappedT operator/(const S& i) const {
return swap() / i;
}
// v * 5
SwappedT operator*(const SwappedT& i) const {
return swap() * i.swap();
}
template <typename S>
SwappedT operator*(const S& i) const {
return swap() * i;
}
// v + 5
SwappedT operator+(const SwappedT& i) const {
return swap() + i.swap();
}
template <typename S>
SwappedT operator+(const S& i) const {
return swap() + T(i);
}
// v - 5
SwappedT operator-(const SwappedT& i) const {
return swap() - i.swap();
}
template <typename S>
SwappedT operator-(const S& i) const {
return swap() - T(i);
}
// v += 5
SwappedT& operator+=(const SwappedT& i) {
value = swap(swap() + i.swap());
return *this;
}
template <typename S>
SwappedT& operator+=(const S& i) {
value = swap(swap() + T(i));
return *this;
}
// v -= 5
SwappedT& operator-=(const SwappedT& i) {
value = swap(swap() - i.swap());
return *this;
}
template <typename S>
SwappedT& operator-=(const S& i) {
value = swap(swap() - T(i));
return *this;
}
// ++v
SwappedT& operator++() {
value = swap(swap() + 1);
return *this;
}
// --v
SwappedT& operator--() {
value = swap(swap() - 1);
return *this;
}
// v++
SwappedT operator++(int) {
SwappedT old = *this;
value = swap(swap() + 1);
return old;
}
// v--
SwappedT operator--(int) {
SwappedT old = *this;
value = swap(swap() - 1);
return old;
}
// Comparison
// v == i
bool operator==(const SwappedT& i) const {
return swap() == i.swap();
}
template <typename S>
bool operator==(const S& i) const {
return swap() == i;
}
// v != i
bool operator!=(const SwappedT& i) const {
return swap() != i.swap();
}
template <typename S>
bool operator!=(const S& i) const {
return swap() != i;
}
// v > i
bool operator>(const SwappedT& i) const {
return swap() > i.swap();
}
template <typename S>
bool operator>(const S& i) const {
return swap() > i;
}
// v < i
bool operator<(const SwappedT& i) const {
return swap() < i.swap();
}
template <typename S>
bool operator<(const S& i) const {
return swap() < i;
}
// v >= i
bool operator>=(const SwappedT& i) const {
return swap() >= i.swap();
}
template <typename S>
bool operator>=(const S& i) const {
return swap() >= i;
}
// v <= i
bool operator<=(const SwappedT& i) const {
return swap() <= i.swap();
}
template <typename S>
bool operator<=(const S& i) const {
return swap() <= i;
}
// logical
SwappedT operator!() const {
return !swap();
}
// bitmath
SwappedT operator~() const {
return ~swap();
}
SwappedT operator&(const SwappedT& b) const {
return swap() & b.swap();
}
template <typename S>
SwappedT operator&(const S& b) const {
return swap() & b;
}
SwappedT& operator&=(const SwappedT& b) {
value = swap(swap() & b.swap());
return *this;
}
template <typename S>
SwappedT& operator&=(const S b) {
value = swap(swap() & b);
return *this;
}
SwappedT operator|(const SwappedT& b) const {
return swap() | b.swap();
}
template <typename S>
SwappedT operator|(const S& b) const {
return swap() | b;
}
SwappedT& operator|=(const SwappedT& b) {
value = swap(swap() | b.swap());
return *this;
}
template <typename S>
SwappedT& operator|=(const S& b) {
value = swap(swap() | b);
return *this;
}
SwappedT operator^(const SwappedT& b) const {
return swap() ^ b.swap();
}
template <typename S>
SwappedT operator^(const S& b) const {
return swap() ^ b;
}
SwappedT& operator^=(const SwappedT& b) {
value = swap(swap() ^ b.swap());
return *this;
}
template <typename S>
SwappedT& operator^=(const S& b) {
value = swap(swap() ^ b);
return *this;
}
template <typename S>
SwappedT operator<<(const S& b) const {
return swap() << b;
}
template <typename S>
SwappedT& operator<<=(const S& b) const {
value = swap(swap() << b);
return *this;
}
template <typename S>
SwappedT operator>>(const S& b) const {
return swap() >> b;
}
template <typename S>
SwappedT& operator>>=(const S& b) const {
value = swap(swap() >> b);
return *this;
}
// Member
/** todo **/
// Arithmetic
template <typename S, typename T2, typename F2>
friend S operator+(const S& p, const SwappedT v);
template <typename S, typename T2, typename F2>
friend S operator-(const S& p, const SwappedT v);
template <typename S, typename T2, typename F2>
friend S operator/(const S& p, const SwappedT v);
template <typename S, typename T2, typename F2>
friend S operator*(const S& p, const SwappedT v);
template <typename S, typename T2, typename F2>
friend S operator%(const S& p, const SwappedT v);
// Arithmetic + assignments
template <typename S, typename T2, typename F2>
friend S operator+=(const S& p, const SwappedT v);
template <typename S, typename T2, typename F2>
friend S operator-=(const S& p, const SwappedT v);
// Bitmath
template <typename S, typename T2, typename F2>
friend S operator&(const S& p, const SwappedT v);
// Comparison
template <typename S, typename T2, typename F2>
friend bool operator<(const S& p, const SwappedT v);
template <typename S, typename T2, typename F2>
friend bool operator>(const S& p, const SwappedT v);
template <typename S, typename T2, typename F2>
friend bool operator<=(const S& p, const SwappedT v);
template <typename S, typename T2, typename F2>
friend bool operator>=(const S& p, const SwappedT v);
template <typename S, typename T2, typename F2>
friend bool operator!=(const S& p, const SwappedT v);
template <typename S, typename T2, typename F2>
friend bool operator==(const S& p, const SwappedT v);
};
// Arithmetic
template <typename S, typename T, typename F>
S operator+(const S& i, const SwapStructT<T, F> v) {
return i + v.swap();
}
template <typename S, typename T, typename F>
S operator-(const S& i, const SwapStructT<T, F> v) {
return i - v.swap();
}
template <typename S, typename T, typename F>
S operator/(const S& i, const SwapStructT<T, F> v) {
return i / v.swap();
}
template <typename S, typename T, typename F>
S operator*(const S& i, const SwapStructT<T, F> v) {
return i * v.swap();
}
template <typename S, typename T, typename F>
S operator%(const S& i, const SwapStructT<T, F> v) {
return i % v.swap();
}
// Arithmetic + assignments
template <typename S, typename T, typename F>
S& operator+=(S& i, const SwapStructT<T, F> v) {
i += v.swap();
return i;
}
template <typename S, typename T, typename F>
S& operator-=(S& i, const SwapStructT<T, F> v) {
i -= v.swap();
return i;
}
// Logical
template <typename S, typename T, typename F>
S operator&(const S& i, const SwapStructT<T, F> v) {
return i & v.swap();
}
// Comparison
template <typename S, typename T, typename F>
bool operator<(const S& p, const SwapStructT<T, F> v) {
return p < v.swap();
}
template <typename S, typename T, typename F>
bool operator>(const S& p, const SwapStructT<T, F> v) {
return p > v.swap();
}
template <typename S, typename T, typename F>
bool operator<=(const S& p, const SwapStructT<T, F> v) {
return p <= v.swap();
}
template <typename S, typename T, typename F>
bool operator>=(const S& p, const SwapStructT<T, F> v) {
return p >= v.swap();
}
template <typename S, typename T, typename F>
bool operator!=(const S& p, const SwapStructT<T, F> v) {
return p != v.swap();
}
template <typename S, typename T, typename F>
bool operator==(const S& p, const SwapStructT<T, F> v) {
return p == v.swap();
}
template <typename T>
struct Swap64T {
static T swap(T x) {
return T(Common::swap64(x));
}
};
template <typename T>
struct Swap32T {
static T swap(T x) {
return T(Common::swap32(x));
}
};
template <typename T>
struct Swap16T {
static T swap(T x) {
return T(Common::swap16(x));
}
};
template <typename T>
struct SwapFloatT {
static T swap(T x) {
return T(Common::swapf(x));
}
};
template <typename T>
struct SwapDoubleT {
static T swap(T x) {
return T(Common::swapd(x));
}
};
template <typename T>
struct SwapEnumT {
static_assert(std::is_enum_v<T>);
using base = std::underlying_type_t<T>;
public:
SwapEnumT() = default;
SwapEnumT(const T& v) : value(swap(v)) {}
SwapEnumT& operator=(const T& v) {
value = swap(v);
return *this;
}
operator T() const {
return swap(value);
}
explicit operator base() const {
return base(swap(value));
}
protected:
T value{};
using swap_t = std::conditional_t<
std::is_same_v<base, u16>, Swap16T<u16>, std::conditional_t<
std::is_same_v<base, s16>, Swap16T<s16>, std::conditional_t<
std::is_same_v<base, u32>, Swap32T<u32>, std::conditional_t<
std::is_same_v<base, s32>, Swap32T<s32>, std::conditional_t<
std::is_same_v<base, u64>, Swap64T<u64>, std::conditional_t<
std::is_same_v<base, s64>, Swap64T<s64>, void>>>>>>;
static T swap(T x) {
return T(swap_t::swap(base(x)));
}
};
struct SwapTag {}; // Use the different endianness from the system
struct KeepTag {}; // Use the same endianness as the system
template <typename T, typename Tag>
struct AddEndian;
// KeepTag specializations
template <typename T>
struct AddEndian<T, KeepTag> {
using type = T;
};
// SwapTag specializations
template <>
struct AddEndian<u8, SwapTag> {
using type = u8;
};
template <>
struct AddEndian<u16, SwapTag> {
using type = SwapStructT<u16, Swap16T<u16>>;
};
template <>
struct AddEndian<u32, SwapTag> {
using type = SwapStructT<u32, Swap32T<u32>>;
};
template <>
struct AddEndian<u64, SwapTag> {
using type = SwapStructT<u64, Swap64T<u64>>;
};
template <>
struct AddEndian<s8, SwapTag> {
using type = s8;
};
template <>
struct AddEndian<s16, SwapTag> {
using type = SwapStructT<s16, Swap16T<s16>>;
};
template <>
struct AddEndian<s32, SwapTag> {
using type = SwapStructT<s32, Swap32T<s32>>;
};
template <>
struct AddEndian<s64, SwapTag> {
using type = SwapStructT<s64, Swap64T<s64>>;
};
template <>
struct AddEndian<float, SwapTag> {
using type = SwapStructT<float, SwapFloatT<float>>;
};
template <>
struct AddEndian<double, SwapTag> {
using type = SwapStructT<double, SwapDoubleT<double>>;
};
template <typename T>
struct AddEndian<T, SwapTag> {
static_assert(std::is_enum_v<T>);
using type = SwapEnumT<T>;
};
// Alias LETag/BETag as KeepTag/SwapTag depending on the system
using LETag = std::conditional_t<std::endian::native == std::endian::little, KeepTag, SwapTag>;
using BETag = std::conditional_t<std::endian::native == std::endian::big, KeepTag, SwapTag>;
// Aliases for LE types
using u16_le = AddEndian<u16, LETag>::type;
using u32_le = AddEndian<u32, LETag>::type;
using u64_le = AddEndian<u64, LETag>::type;
using s16_le = AddEndian<s16, LETag>::type;
using s32_le = AddEndian<s32, LETag>::type;
using s64_le = AddEndian<s64, LETag>::type;
template <typename T>
using enum_le = std::enable_if_t<std::is_enum_v<T>, typename AddEndian<T, LETag>::type>;
using float_le = AddEndian<float, LETag>::type;
using double_le = AddEndian<double, LETag>::type;
// Aliases for BE types
using u16_be = AddEndian<u16, BETag>::type;
using u32_be = AddEndian<u32, BETag>::type;
using u64_be = AddEndian<u64, BETag>::type;
using s16_be = AddEndian<s16, BETag>::type;
using s32_be = AddEndian<s32, BETag>::type;
using s64_be = AddEndian<s64, BETag>::type;
template <typename T>
using enum_be = std::enable_if_t<std::is_enum_v<T>, typename AddEndian<T, BETag>::type>;
using float_be = AddEndian<float, BETag>::type;
using double_be = AddEndian<double, BETag>::type;

56
gpu/vk/dlss_actor.h Normal file
View File

@@ -0,0 +1,56 @@
// (hookshot) NVIDIA DLSS's actor motion (vk_dlss.cpp): what is worked out on the CPU for a skinned draw drawn again, kept
// apart so tests/test_dlss_actor.cpp checks it without a GPU. Header only, no Vulkan.
#pragma once
#include <algorithm>
#include <cstdint>
#include <cstring>
#include <vector>
namespace dlss_actor {
/// A draw's name across frames: its model (vertex and index buffers, count, base vertex, program) and which of the frame's
/// draws of that model it is (the same model drawn twice: the n-th draw of it is matched with the n-th the frame before)
inline uint64_t key(uint64_t vbuf, uint64_t ibuf, int count, int base_vertex, uint64_t program, int nth) {
uint64_t h = 0x5bd1e995;
for (uint64_t v : {vbuf, ibuf, (uint64_t)(uint32_t)count, (uint64_t)(uint32_t)base_vertex, program}) {
h ^= v + 0x9e3779b97f4a7c15ull + (h << 6) + (h >> 2);
h *= 0xff51afd7ed558ccdull;
}
return (h & ~0xffull) | (uint64_t)std::clamp(nth, 0, 255);
}
inline bool same_model(uint64_t a, uint64_t b) { return a >> 8 == b >> 8; }
/// shaders/dlss_actor.vert's push constants (the fragment stage's the same)
struct Push {
uint64_t vertices; // (the vertex stream's device address)
uint64_t indices; // (the index buffer's, 4-aligned down: first says where the draw's indices start in it)
uint64_t origins[2]; // (the draw's origin, xyz: this frame's and the frame before's, copied into the pose buffers)
uint64_t bones[2]; // (its palettes: 3 x 4 row-major matrices, 12 floats a bone)
uint32_t nbones;
uint32_t stride; // (in words)
uint32_t count; // (the vertices in the stream: an index past them draws nothing)
uint32_t first; // (in indices, from the address)
uint32_t itype; // (0: 8-bit, 1: 16-bit, 2: 32-bit indices)
int32_t base_vertex;
uint32_t pos, sets; // (bytes in a vertex: the position; the joint and weight sets, 1 or 2)
uint32_t joints[2], weights[2];
uint32_t formats; // (kHalfPosition, kByteJoints << set)
uint32_t tolerance; // (depth steps from the scene's within which a pixel is the draw's)
};
static_assert(sizeof(Push) == 6 * 8 + 14 * 4 && sizeof(Push) <= 128, "dlss_actor.vert's push constants");
/// Where a draw's indices are for the shader: the address 4-aligned down, and its first index from there
inline void index_start(uint64_t address, int itype, uint64_t* aligned, uint32_t* first) {
*aligned = address & ~3ull;
*first = (uint32_t)((address & 3) >> (itype < 0 ? 0 : itype > 2 ? 2 : itype));
}
/// Push::formats: the position is RGBA16F (else RGB32F); a set's joints are RGBA8UI (else RGBA16UI)
constexpr uint32_t kHalfPosition = 1, kByteJoints = 2;
/// The cameras' buffer, 36 words: this frame's camera (as drawn, jittered), the frame before's (its jitter taken off), this
/// frame's jitter in clip space, the render size
constexpr uint32_t kHeadWords = 36;
} // namespace dlss_actor

87
gpu/vk/dlss_checks.h Normal file
View File

@@ -0,0 +1,87 @@
/* (hookshot) Two small checks of NVIDIA DLSS's (gpu/vk/vk_dlss.cpp), as plain functions: which scene sizes DLSS can take
for an output, and a Linux path (UTF-8) for NGX's wide strings. No Vulkan, no NGX: tests/test_dlss_checks.cpp
checks them. */
#pragma once
#include <cstdint>
#include <string>
#include <string_view>
namespace mkwvk::dlss_checks {
/// Whether a scene of w x h can be DLSS's input for an output of ow x oh: the output 1 to 3 times its size on each axis
/// (DLAA at 1, Ultra Performance at 3: 512x288 for 1536x864, 683x384 for 2048x1152). False while the output is not known
/// (0). The scene's camera and the actor motion take only these, and the upscale draws only these.
inline bool scene_size_fits(uint32_t w, uint32_t h, uint32_t ow, uint32_t oh) {
return w && h && ow && oh && w <= ow && h <= oh && (uint64_t)w * 3 >= ow && (uint64_t)h * 3 >= oh;
}
/// Whether s is all ASCII: NGX's Linux library takes only such paths (vk_dlss.cpp wide())
inline bool ascii(std::string_view s) {
for (const char c : s)
if ((unsigned char)c >= 0x80) return false;
return true;
}
/// UTF-8 bytes (a Linux path) as wide characters: UTF-32 where wchar_t has 32 bits (Linux), UTF-16 where it has 16. Decoded
/// here, not through the process's locale. Bytes that are not well-formed UTF-8 (a stray continuation byte, a cut sequence,
/// an overlong form, a surrogate, past U+10FFFF) become U+FFFD, one for each maximal ill-formed part (Unicode's practice);
/// *bad is set when that happened. U+FFFD rather than the raw byte as a character: the raw byte would name another, real
/// path (0xE9 alone would become "é", which NGX writes back as C3 A9), U+FFFD names none.
inline std::wstring utf8_to_wide(std::string_view s, bool* bad = nullptr) {
std::wstring out;
out.reserve(s.size());
const auto put = [&](uint32_t cp) {
if (sizeof(wchar_t) == 2 && cp > 0xFFFF) {
cp -= 0x10000;
out += (wchar_t)(0xD800 + (cp >> 10));
out += (wchar_t)(0xDC00 + (cp & 0x3FF));
} else {
out += (wchar_t)cp;
}
};
const auto replace = [&] {
put(0xFFFD);
if (bad) *bad = true;
};
size_t i = 0;
while (i < s.size()) {
const unsigned char b = (unsigned char)s[i];
if (b < 0x80) {
put(b), i++;
continue;
}
// (the lead byte: how many continuation bytes follow, and the first one's range, Unicode's table 3-7)
int need = 0;
uint32_t cp = 0;
unsigned char lo = 0x80, hi = 0xBF;
if (b >= 0xC2 && b <= 0xDF) {
need = 1, cp = b & 0x1F;
} else if (b >= 0xE0 && b <= 0xEF) {
need = 2, cp = b & 0x0F;
if (b == 0xE0) lo = 0xA0; // (no overlongs)
if (b == 0xED) hi = 0x9F; // (no surrogates)
} else if (b >= 0xF0 && b <= 0xF4) {
need = 3, cp = b & 0x07;
if (b == 0xF0) lo = 0x90; // (no overlongs)
if (b == 0xF4) hi = 0x8F; // (nothing past U+10FFFF)
} else { // (a continuation byte alone, C0, C1, F5 to FF)
replace(), i++;
continue;
}
size_t j = i + 1;
int got = 0;
for (; got < need && j < s.size(); got++, j++) {
const unsigned char c = (unsigned char)s[j];
if (c < lo || c > hi) break;
cp = (cp << 6) | (c & 0x3F);
lo = 0x80, hi = 0xBF;
}
if (got < need) replace(); // (the lead and the good bytes after it: one U+FFFD; the bad byte is read again)
else put(cp);
i = j;
}
return out;
}
} // namespace mkwvk::dlss_checks

103
gpu/vk/dlss_math.h Normal file
View File

@@ -0,0 +1,103 @@
/* (hookshot) NVIDIA DLSS's camera math, as plain functions: the jitter's sequence (host/rx_backend.c, in C) and the motion
vectors' reprojection (gpu/vk/vk_dlss.cpp). No Vulkan, no NGX: tests/test_dlss_math.cpp checks them. */
#pragma once
#ifdef __cplusplus
#include <cmath>
#include <utility>
#endif
/* The radical inverse of index in base: the Halton sequence's index-th value, in [0, 1) */
static inline float dlss_halton(int index, int base) {
float f = 1.0f, r = 0.0f;
for (int i = index; i > 0; i /= base) {
f /= (float)base;
r += f * (float)(i % base);
}
return r;
}
/* The frame's sub-pixel jitter, in pixels, each in [-0.5, 0.5): the Halton (2, 3) sequence over `phases` frames (from 1: its
first value, 0, would put every phase's start on the pixel's corner) */
static inline void dlss_jitter_pixels(unsigned frame, int phases, float out[2]) {
const int i = (int)(frame % (unsigned)(phases > 0 ? phases : 1)) + 1;
out[0] = dlss_halton(i, 2) - 0.5f;
out[1] = dlss_halton(i, 3) - 0.5f;
}
/* The frame's jitter for a projection matrix p (row major, as the game fills its views' uniform buffers: nx2/oot_jitter.py),
in clip space (what p[0][2] and p[1][2] get) into out, for a scene of w x h pixels; 0 (none) when p is not one of the
picture's: not perspective (its w row is not 0 0 -1 0: a shadow map's, the HUD's) or not of 16:9 or the window's ratio
wide (0: 16:9 only), or when there is no scene size (DLSS not drawing) */
static inline int dlss_jitter_clip(const float p[16], float wide, unsigned frame, int phases, float w, float h, float out[2]) {
if (!(w >= 1.0f) || !(h >= 1.0f)) return 0;
if (p[12] != 0.0f || p[13] != 0.0f || p[14] != -1.0f || p[15] != 0.0f || !(p[0] > 0.0f)) return 0;
const float r = p[5] / p[0], d = r - 16.0f / 9.0f, e = wide > 0.0f ? r - wide : 1.0f;
if (!((d <= 0.01f && d >= -0.01f) || (e <= 0.01f && e >= -0.01f))) return 0;
float px[2];
dlss_jitter_pixels(frame, phases, px);
out[0] = 2.0f * px[0] / w, out[1] = 2.0f * px[1] / h;
return 1;
}
/* How many frames the jitter's sequence takes: NVIDIA's guide, 8 times the share of pixels DLSS fills in, (output / render)
squared, so each output pixel is covered about 8 times; 8 at least (DLAA), 64 at most (a long sequence is seen less of
before the camera moves) */
static inline int dlss_jitter_phases(unsigned render_w, unsigned render_h, unsigned out_w, unsigned out_h) {
if (!render_w || !render_h) return 8;
const double area = (double)out_w * (double)out_h / ((double)render_w * (double)render_h);
const int n = (int)(8.0 * area + 0.5);
return n < 8 ? 8 : n > 64 ? 64 : n;
}
#ifdef __cplusplus
namespace mkwvk::dlss_math {
/// This frame's clip space to the frame before's: previous * inverse(current), both row major (clip = M * position); false
/// when current is singular or the result is not finite
inline bool reprojection(const float* current, const float* previous, float out[16]) {
double a[4][8];
for (int r = 0; r < 4; r++)
for (int c = 0; c < 8; c++) a[r][c] = c < 4 ? current[r * 4 + c] : (c - 4 == r ? 1.0 : 0.0);
for (int c = 0; c < 4; c++) { // (Gauss-Jordan with partial pivoting)
int p = c;
for (int r = c + 1; r < 4; r++)
if (std::fabs(a[r][c]) > std::fabs(a[p][c])) p = r;
if (std::fabs(a[p][c]) < 1e-12) return false;
if (p != c)
for (int k = 0; k < 8; k++) std::swap(a[p][k], a[c][k]);
const double inv = 1.0 / a[c][c];
for (int k = 0; k < 8; k++) a[c][k] *= inv;
for (int r = 0; r < 4; r++) {
if (r == c || a[r][c] == 0.0) continue;
const double f = a[r][c];
for (int k = 0; k < 8; k++) a[r][k] -= f * a[c][k];
}
}
for (int r = 0; r < 4; r++)
for (int c = 0; c < 4; c++) {
double v = 0;
for (int k = 0; k < 4; k++) v += previous[r * 4 + k] * a[k][c + 4];
if (!std::isfinite(v)) return false;
out[r * 4 + c] = (float)v;
}
return true;
}
/// A view-projection's jitter taken off (row major): a jitter moves clip x and y by jitter * w, so jitter times the w row
/// comes off the first two rows
inline void unjitter(float m[16], const float jitter[2]) {
for (int c = 0; c < 4; c++) {
m[0 * 4 + c] -= jitter[0] * m[3 * 4 + c];
m[1 * 4 + c] -= jitter[1] * m[3 * 4 + c];
}
}
/// A jitter in clip space as DLSS takes it: in render pixels, x right and y down (clip y is up)
inline void jitter_to_pixels(const float clip[2], unsigned width, unsigned height, float out[2]) {
out[0] = clip[0] * 0.5f * (float)width;
out[1] = -clip[1] * 0.5f * (float)height;
}
} // namespace mkwvk::dlss_math
#endif

View File

@@ -0,0 +1,20 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "common/common_types.h"
namespace Shader::Backend {
struct Bindings {
u32 unified{};
u32 uniform_buffer{};
u32 storage_buffer{};
u32 texture{};
u32 image{};
u32 texture_scaling_index{};
u32 image_scaling_index{};
};
} // namespace Shader::Backend

View File

@@ -0,0 +1,764 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <algorithm>
#include <span>
#include <tuple>
#include <type_traits>
#include <utility>
#include <vector>
#include "common/settings.h"
#include "shader_recompiler/backend/spirv/emit_spirv.h"
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
#include "shader_recompiler/frontend/ir/basic_block.h"
#include "shader_recompiler/frontend/ir/program.h"
#include "shader_recompiler/backend/spirv/mma_layout.h" // (hookshot)
namespace Shader::Backend::SPIRV {
namespace {
template <class Func>
struct FuncTraits {};
template <class ReturnType_, class... Args>
struct FuncTraits<ReturnType_ (*)(Args...)> {
using ReturnType = ReturnType_;
static constexpr size_t NUM_ARGS = sizeof...(Args);
template <size_t I>
using ArgType = std::tuple_element_t<I, std::tuple<Args...>>;
};
#ifdef _MSC_VER
#pragma warning(push)
#pragma warning(disable : 4702) // Ignore unreachable code warning
#endif
template <auto func, typename... Args>
void SetDefinition(EmitContext& ctx, IR::Inst* inst, Args... args) {
inst->SetDefinition<Id>(func(ctx, std::forward<Args>(args)...));
}
#ifdef _MSC_VER
#pragma warning(pop)
#endif
template <typename ArgType>
ArgType Arg(EmitContext& ctx, const IR::Value& arg) {
if constexpr (std::is_same_v<ArgType, Id>) {
return ctx.Def(arg);
} else if constexpr (std::is_same_v<ArgType, const IR::Value&>) {
return arg;
} else if constexpr (std::is_same_v<ArgType, u32>) {
return arg.U32();
} else if constexpr (std::is_same_v<ArgType, IR::Attribute>) {
return arg.Attribute();
} else if constexpr (std::is_same_v<ArgType, IR::Patch>) {
return arg.Patch();
} else if constexpr (std::is_same_v<ArgType, IR::Reg>) {
return arg.Reg();
}
}
template <auto func, bool is_first_arg_inst, size_t... I>
void Invoke(EmitContext& ctx, IR::Inst* inst, std::index_sequence<I...>) {
using Traits = FuncTraits<decltype(func)>;
if constexpr (std::is_same_v<typename Traits::ReturnType, Id>) {
if constexpr (is_first_arg_inst) {
SetDefinition<func>(
ctx, inst, inst,
Arg<typename Traits::template ArgType<I + 2>>(ctx, inst->Arg(I))...);
} else {
SetDefinition<func>(
ctx, inst, Arg<typename Traits::template ArgType<I + 1>>(ctx, inst->Arg(I))...);
}
} else {
if constexpr (is_first_arg_inst) {
func(ctx, inst, Arg<typename Traits::template ArgType<I + 2>>(ctx, inst->Arg(I))...);
} else {
func(ctx, Arg<typename Traits::template ArgType<I + 1>>(ctx, inst->Arg(I))...);
}
}
}
template <auto func>
void Invoke(EmitContext& ctx, IR::Inst* inst) {
using Traits = FuncTraits<decltype(func)>;
static_assert(Traits::NUM_ARGS >= 1, "Insufficient arguments");
if constexpr (Traits::NUM_ARGS == 1) {
Invoke<func, false>(ctx, inst, std::make_index_sequence<0>{});
} else {
using FirstArgType = typename Traits::template ArgType<1>;
static constexpr bool is_first_arg_inst = std::is_same_v<FirstArgType, IR::Inst*>;
using Indices = std::make_index_sequence<Traits::NUM_ARGS - (is_first_arg_inst ? 2 : 1)>;
Invoke<func, is_first_arg_inst>(ctx, inst, Indices{});
}
}
void EmitInstOf(EmitContext& ctx, IR::Inst* inst) {
switch (inst->GetOpcode()) {
#define OPCODE(name, result_type, ...) \
case IR::Opcode::name: \
return Invoke<&Emit##name>(ctx, inst);
#include "shader_recompiler/frontend/ir/opcodes.inc"
#undef OPCODE
}
throw LogicError("Invalid opcode {}", inst->GetOpcode());
}
void EmitInst(EmitContext& ctx, IR::Inst* inst) {
// (hookshot) the emitters that are left unimplemented say only "SPIR-V Instruction": name the IR op
try {
EmitInstOf(ctx, inst);
} catch (const NotImplementedException& e) {
throw NotImplementedException("{} ({})", e.what(), IR::NameOf(inst->GetOpcode()));
}
}
Id TypeId(const EmitContext& ctx, IR::Type type) {
switch (type) {
case IR::Type::U1:
return ctx.U1;
case IR::Type::U32:
return ctx.U32[1];
default:
throw NotImplementedException("Phi node type {}", type);
}
}
void Traverse(EmitContext& ctx, IR::Program& program) {
IR::Block* current_block{};
// (EvilOOT, RX_VK_BLOCK_PROBE=1 with RX_VK_MISS_LOG=1 and RX_VK_MISS_LOG_PROG=<key>: each block's entries counted, by
// invocation, in the miss log's words 128 + the block's index in program.blocks: IR::DumpProgram's "Block $n")
static const bool probe_env{std::getenv("RX_VK_BLOCK_PROBE") != nullptr};
const bool probe{probe_env && Sirit::ValidId(ctx.probe_log)};
const auto block_index{[&](const IR::Block* b) -> u32 {
const auto it{std::find(program.blocks.begin(), program.blocks.end(), b)};
return static_cast<u32>(it - program.blocks.begin());
}};
for (const IR::AbstractSyntaxNode& node : program.syntax_list) {
switch (node.type) {
case IR::AbstractSyntaxNode::Type::Block: {
const Id label{node.data.block->Definition<Id>()};
if (current_block) {
ctx.OpBranch(label);
}
current_block = node.data.block;
ctx.AddLabel(label);
bool probed{!probe};
for (IR::Inst& inst : node.data.block->Instructions()) {
if (!probed && inst.GetOpcode() != IR::Opcode::Phi) { // (after the phis: they must lead the block)
probed = true;
const u32 at{128U + std::min<u32>(block_index(node.data.block), 4000U)};
const Id counter{ctx.OpAccessChain(ctx.storage_types.U32.element, ctx.probe_log, ctx.u32_zero_value,
ctx.Const(at))};
ctx.OpAtomicIAdd(ctx.U32[1], counter, ctx.Const(static_cast<u32>(spv::Scope::Device)), ctx.u32_zero_value,
ctx.Const(1U));
}
EmitInst(ctx, &inst);
}
break;
}
case IR::AbstractSyntaxNode::Type::If: {
const Id if_label{node.data.if_node.body->Definition<Id>()};
const Id endif_label{node.data.if_node.merge->Definition<Id>()};
ctx.OpSelectionMerge(endif_label, spv::SelectionControlMask::MaskNone);
ctx.OpBranchConditional(ctx.Def(node.data.if_node.cond), if_label, endif_label);
break;
}
case IR::AbstractSyntaxNode::Type::Loop: {
const Id body_label{node.data.loop.body->Definition<Id>()};
const Id continue_label{node.data.loop.continue_block->Definition<Id>()};
const Id endloop_label{node.data.loop.merge->Definition<Id>()};
ctx.OpLoopMerge(endloop_label, continue_label, spv::LoopControlMask::MaskNone);
ctx.OpBranch(body_label);
break;
}
case IR::AbstractSyntaxNode::Type::Break: {
const Id break_label{node.data.break_node.merge->Definition<Id>()};
const Id skip_label{node.data.break_node.skip->Definition<Id>()};
ctx.OpBranchConditional(ctx.Def(node.data.break_node.cond), break_label, skip_label);
break;
}
case IR::AbstractSyntaxNode::Type::EndIf:
if (current_block) {
ctx.OpBranch(node.data.end_if.merge->Definition<Id>());
}
break;
case IR::AbstractSyntaxNode::Type::Repeat: {
Id cond{ctx.Def(node.data.repeat.cond)};
if (!Settings::values.disable_shader_loop_safety_checks) {
const Id pointer_type{ctx.TypePointer(spv::StorageClass::Private, ctx.U32[1])};
const Id safety_counter{ctx.AddGlobalVariable(
pointer_type, spv::StorageClass::Private, ctx.Const(0x2000u))};
if (ctx.profile.supported_spirv >= 0x00010400) {
ctx.interfaces.push_back(safety_counter);
}
const Id old_counter{ctx.OpLoad(ctx.U32[1], safety_counter)};
const Id new_counter{ctx.OpISub(ctx.U32[1], old_counter, ctx.Const(1u))};
ctx.OpStore(safety_counter, new_counter);
const Id safety_cond{
ctx.OpSGreaterThanEqual(ctx.U1, new_counter, ctx.u32_zero_value)};
cond = ctx.OpLogicalAnd(ctx.U1, cond, safety_cond);
}
const Id loop_header_label{node.data.repeat.loop_header->Definition<Id>()};
const Id merge_label{node.data.repeat.merge->Definition<Id>()};
ctx.OpBranchConditional(cond, loop_header_label, merge_label);
break;
}
case IR::AbstractSyntaxNode::Type::Return:
ctx.OpReturn();
break;
case IR::AbstractSyntaxNode::Type::Unreachable:
if (ctx.stage == Stage::Mesh && !ctx.mesh_fallback) {
// (hookshot) an indirect branch to none of its table's targets: the mesh shader leaves with no outputs.
// RADV makes OpUnreachable a return that leaves its output counts undefined, and GS_ALLOC_REQ then asked
// for a garbage count and hung the GPU (issue 1000). Marker for OpSetMeshOutputsEXT 0 0
// (PatchMeshInstructions); the epilogue writes the outputs, so none were written yet
ctx.OpStore(ctx.u32_zero_value, ctx.u32_zero_value);
ctx.mesh_zero_outputs_marker = true;
ctx.OpReturn();
} else {
ctx.OpUnreachable();
}
break;
}
if (node.type != IR::AbstractSyntaxNode::Type::Block) {
current_block = nullptr;
}
}
}
Id DefineMain(EmitContext& ctx, IR::Program& program) {
const Id void_function{ctx.TypeFunction(ctx.void_id)};
const Id main{ctx.OpFunction(ctx.void_id, spv::FunctionControlMask::MaskNone, void_function)};
for (IR::Block* const block : program.blocks) {
block->SetDefinition(ctx.OpLabel());
}
Traverse(ctx, program);
ctx.OpFunctionEnd();
return main;
}
spv::ExecutionMode ExecutionMode(TessPrimitive primitive) {
switch (primitive) {
case TessPrimitive::Isolines:
return spv::ExecutionMode::Isolines;
case TessPrimitive::Triangles:
return spv::ExecutionMode::Triangles;
case TessPrimitive::Quads:
return spv::ExecutionMode::Quads;
}
throw InvalidArgument("Tessellation primitive {}", primitive);
}
spv::ExecutionMode ExecutionMode(TessSpacing spacing) {
switch (spacing) {
case TessSpacing::Equal:
return spv::ExecutionMode::SpacingEqual;
case TessSpacing::FractionalOdd:
return spv::ExecutionMode::SpacingFractionalOdd;
case TessSpacing::FractionalEven:
return spv::ExecutionMode::SpacingFractionalEven;
}
throw InvalidArgument("Tessellation spacing {}", spacing);
}
void DefineEntryPoint(const IR::Program& program, EmitContext& ctx, Id main) {
const std::span interfaces(ctx.interfaces.data(), ctx.interfaces.size());
if (ctx.mesh_fallback && (program.stage == Stage::Task || program.stage == Stage::Mesh)) {
// (hookshot, issue 640: on a GPU without mesh shaders, a task or mesh program is a compute shader)
ctx.AddExecutionMode(main, spv::ExecutionMode::LocalSize, program.workgroup_size[0], 1U, 1U);
ctx.AddEntryPoint(spv::ExecutionModel::GLCompute, main, "main", interfaces);
return;
}
spv::ExecutionModel execution_model{};
switch (program.stage) {
case Stage::Compute: {
const std::array<u32, 3> workgroup_size{program.workgroup_size};
execution_model = spv::ExecutionModel::GLCompute;
ctx.AddExecutionMode(main, spv::ExecutionMode::LocalSize, workgroup_size[0],
workgroup_size[1], workgroup_size[2]);
break;
}
case Stage::VertexB:
execution_model = spv::ExecutionModel::Vertex;
break;
case Stage::Task:
execution_model = spv::ExecutionModel::TaskEXT;
ctx.AddExecutionMode(main, spv::ExecutionMode::LocalSize, program.workgroup_size[0], 1U,
1U);
break;
case Stage::Mesh:
execution_model = spv::ExecutionModel::MeshEXT;
ctx.AddExecutionMode(main, spv::ExecutionMode::LocalSize, program.workgroup_size[0], 1U,
1U);
ctx.AddExecutionMode(main, spv::ExecutionMode::OutputVertices, program.mesh.max_vertices);
ctx.AddExecutionMode(main, spv::ExecutionMode::OutputPrimitivesEXT,
program.mesh.max_primitives);
ctx.AddExecutionMode(main, program.mesh.Corners() == 1 ? spv::ExecutionMode::OutputPoints
: program.mesh.Corners() == 2 ? spv::ExecutionMode::OutputLinesEXT
: spv::ExecutionMode::OutputTrianglesEXT);
break;
case Stage::TessellationControl:
execution_model = spv::ExecutionModel::TessellationControl;
ctx.AddCapability(spv::Capability::Tessellation);
ctx.AddExecutionMode(main, spv::ExecutionMode::OutputVertices, program.invocations);
break;
case Stage::TessellationEval:
execution_model = spv::ExecutionModel::TessellationEvaluation;
ctx.AddCapability(spv::Capability::Tessellation);
ctx.AddExecutionMode(main, ExecutionMode(ctx.runtime_info.tess_primitive));
ctx.AddExecutionMode(main, ExecutionMode(ctx.runtime_info.tess_spacing));
ctx.AddExecutionMode(main, ctx.runtime_info.tess_clockwise
? spv::ExecutionMode::VertexOrderCw
: spv::ExecutionMode::VertexOrderCcw);
break;
case Stage::Geometry:
execution_model = spv::ExecutionModel::Geometry;
ctx.AddCapability(spv::Capability::Geometry);
if (ctx.profile.support_geometry_streams) {
ctx.AddCapability(spv::Capability::GeometryStreams);
}
switch (ctx.runtime_info.input_topology) {
case InputTopology::Points:
ctx.AddExecutionMode(main, spv::ExecutionMode::InputPoints);
break;
case InputTopology::Lines:
ctx.AddExecutionMode(main, spv::ExecutionMode::InputLines);
break;
case InputTopology::LinesAdjacency:
ctx.AddExecutionMode(main, spv::ExecutionMode::InputLinesAdjacency);
break;
case InputTopology::Triangles:
ctx.AddExecutionMode(main, spv::ExecutionMode::Triangles);
break;
case InputTopology::TrianglesAdjacency:
ctx.AddExecutionMode(main, spv::ExecutionMode::InputTrianglesAdjacency);
break;
}
switch (program.output_topology) {
case OutputTopology::PointList:
ctx.AddExecutionMode(main, spv::ExecutionMode::OutputPoints);
break;
case OutputTopology::LineStrip:
ctx.AddExecutionMode(main, spv::ExecutionMode::OutputLineStrip);
break;
case OutputTopology::TriangleStrip:
ctx.AddExecutionMode(main, spv::ExecutionMode::OutputTriangleStrip);
break;
}
if (program.info.stores[IR::Attribute::PointSize]) {
ctx.AddCapability(spv::Capability::GeometryPointSize);
}
ctx.AddExecutionMode(main, spv::ExecutionMode::OutputVertices, program.output_vertices);
ctx.AddExecutionMode(main, spv::ExecutionMode::Invocations, program.invocations);
if (program.is_geometry_passthrough) {
if (ctx.profile.support_geometry_shader_passthrough) {
ctx.AddExtension("SPV_NV_geometry_shader_passthrough");
ctx.AddCapability(spv::Capability::GeometryShaderPassthroughNV);
} else {
LOG_WARNING(Shader_SPIRV, "Geometry shader passthrough used with no support");
}
}
break;
case Stage::Fragment:
execution_model = spv::ExecutionModel::Fragment;
if (ctx.profile.lower_left_origin_mode) {
ctx.AddExecutionMode(main, spv::ExecutionMode::OriginLowerLeft);
} else {
ctx.AddExecutionMode(main, spv::ExecutionMode::OriginUpperLeft);
}
if (program.info.stores_frag_depth) {
ctx.AddExecutionMode(main, spv::ExecutionMode::DepthReplacing);
}
if (ctx.runtime_info.force_early_z) {
ctx.AddExecutionMode(main, spv::ExecutionMode::EarlyFragmentTests);
}
if (ctx.profile.support_shader_quad_control && program.info.uses_quad_shuffles) {
ctx.AddExtension("SPV_KHR_quad_control");
ctx.AddCapability(spv::Capability::QuadControlKHR);
ctx.AddExecutionMode(main, spv::ExecutionMode::RequireFullQuadsKHR);
}
break;
default:
throw NotImplementedException("Stage {}", program.stage);
}
ctx.AddEntryPoint(execution_model, main, "main", interfaces);
}
void SetupDenormControl(const Profile& profile, const IR::Program& program, EmitContext& ctx,
Id main_func) {
const Info& info{program.info};
if (info.uses_fp32_denorms_flush && info.uses_fp32_denorms_preserve) {
LOG_DEBUG(Shader_SPIRV, "Fp32 denorm flush and preserve on the same shader");
} else if (info.uses_fp32_denorms_flush) {
if (profile.support_fp32_denorm_flush && !profile.has_broken_fp32_denorm_flush) {
ctx.AddCapability(spv::Capability::DenormFlushToZero);
ctx.AddExecutionMode(main_func, spv::ExecutionMode::DenormFlushToZero, 32U);
} else {
// Drivers will most likely flush denorms by default, no need to warn
}
} else if (info.uses_fp32_denorms_preserve) {
if (profile.support_fp32_denorm_preserve) {
ctx.AddCapability(spv::Capability::DenormPreserve);
ctx.AddExecutionMode(main_func, spv::ExecutionMode::DenormPreserve, 32U);
} else {
LOG_DEBUG(Shader_SPIRV, "Fp32 denorm preserve used in shader without host support");
}
}
if (!profile.support_separate_denorm_behavior || profile.has_broken_fp16_float_controls) {
// No separate denorm behavior
return;
}
if (info.uses_fp16_denorms_flush && info.uses_fp16_denorms_preserve) {
LOG_DEBUG(Shader_SPIRV, "Fp16 denorm flush and preserve on the same shader");
} else if (info.uses_fp16_denorms_flush) {
if (profile.support_fp16_denorm_flush) {
ctx.AddCapability(spv::Capability::DenormFlushToZero);
ctx.AddExecutionMode(main_func, spv::ExecutionMode::DenormFlushToZero, 16U);
} else {
// Same as fp32, no need to warn as most drivers will flush by default
}
} else if (info.uses_fp16_denorms_preserve) {
if (profile.support_fp16_denorm_preserve) {
ctx.AddCapability(spv::Capability::DenormPreserve);
ctx.AddExecutionMode(main_func, spv::ExecutionMode::DenormPreserve, 16U);
} else {
LOG_DEBUG(Shader_SPIRV, "Fp16 denorm preserve used in shader without host support");
}
}
}
void SetupSignedNanCapabilities(const Profile& profile, const IR::Program& program,
EmitContext& ctx, Id main_func) {
if (profile.has_broken_fp16_float_controls && program.info.uses_fp16) {
return;
}
if (program.info.uses_fp16 && profile.support_fp16_signed_zero_nan_preserve) {
ctx.AddCapability(spv::Capability::SignedZeroInfNanPreserve);
ctx.AddExecutionMode(main_func, spv::ExecutionMode::SignedZeroInfNanPreserve, 16U);
}
if (profile.support_fp32_signed_zero_nan_preserve) {
ctx.AddCapability(spv::Capability::SignedZeroInfNanPreserve);
ctx.AddExecutionMode(main_func, spv::ExecutionMode::SignedZeroInfNanPreserve, 32U);
}
if (program.info.uses_fp64 && profile.support_fp64_signed_zero_nan_preserve) {
ctx.AddCapability(spv::Capability::SignedZeroInfNanPreserve);
ctx.AddExecutionMode(main_func, spv::ExecutionMode::SignedZeroInfNanPreserve, 64U);
}
}
void SetupTransformFeedbackCapabilities(EmitContext& ctx, Id main_func) {
if (ctx.runtime_info.xfb_count == 0) {
return;
}
ctx.AddCapability(spv::Capability::TransformFeedback);
ctx.AddExecutionMode(main_func, spv::ExecutionMode::Xfb);
}
void SetupCapabilities(const Profile& profile, const Info& info, EmitContext& ctx) {
if (info.uses_sampled_1d) {
ctx.AddCapability(spv::Capability::Sampled1D);
}
if (info.uses_image_1d) {
ctx.AddCapability(spv::Capability::Image1D);
}
if (info.uses_sparse_residency) {
ctx.AddCapability(spv::Capability::SparseResidency);
}
if (info.uses_demote_to_helper_invocation && profile.support_demote_to_helper_invocation) {
if (profile.supported_spirv < 0x00010600) {
ctx.AddExtension("SPV_EXT_demote_to_helper_invocation");
}
ctx.AddCapability(spv::Capability::DemoteToHelperInvocation);
}
if (info.stores[IR::Attribute::ViewportIndex] && profile.support_multi_viewport) {
ctx.AddCapability(spv::Capability::MultiViewport);
}
if (info.stores[IR::Attribute::ViewportMask] && profile.support_viewport_mask) {
ctx.AddExtension("SPV_NV_viewport_array2");
ctx.AddCapability(spv::Capability::ShaderViewportMaskNV);
}
if (info.stores[IR::Attribute::Layer] || info.stores[IR::Attribute::ViewportIndex]) {
if (profile.support_viewport_index_layer_non_geometry && ctx.stage != Stage::Geometry) {
ctx.AddExtension("SPV_EXT_shader_viewport_index_layer");
ctx.AddCapability(spv::Capability::ShaderViewportIndexLayerEXT);
}
}
if ((!profile.support_vertex_instance_id &&
(info.loads[IR::Attribute::InstanceId] || info.loads[IR::Attribute::VertexId])) ||
info.loads[IR::Attribute::BaseInstance] || info.loads[IR::Attribute::BaseVertex] ||
info.loads[IR::Attribute::DrawID] || (ctx.wants_mesh_first && !ctx.mesh_fallback)) {
ctx.AddExtension("SPV_KHR_shader_draw_parameters");
ctx.AddCapability(spv::Capability::DrawParameters);
}
if ((info.uses_subgroup_vote || info.uses_subgroup_invocation_id ||
info.uses_subgroup_shuffles || info.uses_subgroup_mask) &&
profile.support_vote && profile.SupportsSubgroupStage(ctx.stage)) {
ctx.AddCapability(spv::Capability::GroupNonUniformBallot);
ctx.AddCapability(spv::Capability::GroupNonUniformShuffle);
if (info.uses_subgroup_shuffles && profile.support_shuffle_relative) {
ctx.AddCapability(spv::Capability::GroupNonUniformShuffleRelative);
}
if (!profile.warp_size_potentially_larger_than_guest) {
// vote ops are only used when not taking the long path
ctx.AddCapability(spv::Capability::GroupNonUniformVote);
}
}
if (info.uses_cooperative_matrix && !mma_software) {
ctx.AddExtension("SPV_KHR_cooperative_matrix");
ctx.AddCapability(spv::Capability::CooperativeMatrixKHR);
ctx.AddExtension("SPV_KHR_vulkan_memory_model");
ctx.AddCapability(spv::Capability::VulkanMemoryModel);
ctx.AddCapability(spv::Capability::VulkanMemoryModelDeviceScope);
ctx.SetMemoryModel(spv::MemoryModel::VulkanKHR);
}
if (info.uses_cooperative_matrix && mma_software) { // (hookshot: DefineSoftwareMatrixFunction's)
ctx.AddCapability(spv::Capability::GroupNonUniform);
ctx.AddCapability(spv::Capability::GroupNonUniformShuffle);
}
if (info.uses_quad_shuffles) {
if (profile.support_quad_shuffles) {
ctx.AddCapability(spv::Capability::GroupNonUniformQuad);
}
ctx.AddCapability(spv::Capability::GroupNonUniformShuffle);
}
if (info.uses_int64_bit_atomics && profile.support_int64_atomics) {
ctx.AddCapability(spv::Capability::Int64Atomics);
}
if (info.uses_typeless_image_reads && profile.support_typeless_image_loads) {
ctx.AddCapability(spv::Capability::StorageImageReadWithoutFormat);
}
if (info.uses_typeless_image_writes) {
ctx.AddCapability(spv::Capability::StorageImageWriteWithoutFormat);
}
if (info.uses_image_buffers) {
ctx.AddCapability(spv::Capability::ImageBuffer);
}
if (info.uses_sample_id) {
ctx.AddCapability(spv::Capability::SampleRateShading);
}
if (info.uses_derivatives) {
ctx.AddCapability(spv::Capability::DerivativeControl);
}
// TODO: Track this usage
ctx.AddCapability(spv::Capability::ImageGatherExtended);
ctx.AddCapability(spv::Capability::ImageQuery);
ctx.AddCapability(spv::Capability::SampledBuffer);
if (!ctx.non_uniform_ids.empty()) {
if (ctx.profile.supported_spirv < 0x00010500)
ctx.AddExtension("SPV_EXT_descriptor_indexing");
ctx.AddCapability(spv::Capability::ShaderNonUniform);
if (ctx.uses_nonuniform_sampled_image) {
ctx.AddCapability(spv::Capability::SampledImageArrayNonUniformIndexing);
}
if (ctx.uses_nonuniform_storage_image) {
ctx.AddCapability(spv::Capability::StorageImageArrayNonUniformIndexing);
}
if (ctx.uses_nonuniform_uniform_texel_buffer) {
ctx.AddCapability(spv::Capability::UniformTexelBufferArrayNonUniformIndexing);
}
if (ctx.uses_nonuniform_storage_texel_buffer) {
ctx.AddCapability(spv::Capability::StorageTexelBufferArrayNonUniformIndexing);
}
}
}
void PatchPhiNodes(IR::Program& program, EmitContext& ctx) {
// Flatten all leading PHIs from each block into a vector
std::vector<IR::Inst*> phi_instructions;
for (IR::Block* block : program.blocks) {
for (auto it = block->begin(); it != block->end(); ++it) {
if (it->GetOpcode() != IR::Opcode::Phi)
break;
phi_instructions.push_back(&*it);
}
}
if (phi_instructions.empty()) {
return; // nothing to patch
}
// Start "before" first PHI; advance on phi_arg == 0
size_t phi_index = static_cast<size_t>(-1);
ctx.PatchDeferredPhi([&](size_t phi_arg, Id parent) -> std::pair<Id, Id> {
if (phi_arg == 0) {
++phi_index;
}
IR::Inst* phi = phi_instructions[phi_index];
return { ctx.Def(phi->Arg(phi_arg)), parent };
});
}
void RewriteOpcodes(std::vector<u32>& code, std::span<const std::pair<u32, spv::Op>> rewrites) {
if (rewrites.empty()) {
return;
}
size_t offset = 5;
while (offset + 2 < code.size()) {
const u32 word_count{code[offset] >> 16};
const auto opcode{static_cast<spv::Op>(code[offset] & 0xFFFFu)};
if (word_count == 0) {
return;
}
if (opcode == spv::Op::OpGroupNonUniformShuffleXor ||
opcode == spv::Op::OpGroupNonUniformQuadBroadcast) {
const auto it{std::ranges::find(rewrites, code[offset + 2],
&std::pair<u32, spv::Op>::first)};
if (it != rewrites.end()) {
code[offset] = (code[offset] & 0xFFFF0000u) | static_cast<u32>(it->second);
}
}
offset += word_count;
}
}
// Replaces the OpStore markers of the mesh instructions Sirit cannot emit: OpStore(vertices,
// primitives) becomes OpSetMeshOutputsEXT, and OpStore(x, y), OpStore(z, payload), OpReturn
// becomes the OpEmitMeshTasksEXT terminator.
void PatchMeshInstructions(std::vector<u32>& code, const EmitContext& ctx) {
const bool set_outputs{Sirit::ValidId(ctx.mesh_outputs_marker[0])};
const bool emit_tasks{Sirit::ValidId(ctx.mesh_tasks_marker[0])};
const bool zero_outputs{ctx.mesh_zero_outputs_marker};
if (!set_outputs && !emit_tasks && !zero_outputs) {
return;
}
const auto store_at = [&](size_t offset, Id pointer, Id object) {
return offset + 2 < code.size() &&
code[offset] == ((3U << 16) | static_cast<u32>(spv::Op::OpStore)) &&
code[offset + 1] == pointer.value && code[offset + 2] == object.value;
};
const auto& tasks{ctx.mesh_tasks_marker};
size_t offset = 5;
bool patched{};
while (offset < code.size()) {
const u32 word_count{code[offset] >> 16};
if (word_count == 0) {
break;
}
if (set_outputs &&
store_at(offset, ctx.mesh_outputs_marker[0], ctx.mesh_outputs_marker[1])) {
code[offset] = (3U << 16) | static_cast<u32>(spv::Op::OpSetMeshOutputsEXT);
patched = true;
} else if (zero_outputs && store_at(offset, ctx.u32_zero_value, ctx.u32_zero_value)) {
code[offset] = (3U << 16) | static_cast<u32>(spv::Op::OpSetMeshOutputsEXT);
patched = true;
} else if (emit_tasks && store_at(offset, tasks[0], tasks[1]) &&
store_at(offset + 3, tasks[2], tasks[3]) && offset + 6 < code.size() &&
code[offset + 6] == ((1U << 16) | static_cast<u32>(spv::Op::OpReturn))) {
const std::array<u32, 5> emit{(5U << 16) | static_cast<u32>(spv::Op::OpEmitMeshTasksEXT),
tasks[0].value, tasks[1].value, tasks[2].value,
tasks[3].value};
code.erase(code.begin() + offset, code.begin() + offset + 7);
code.insert(code.begin() + offset, emit.begin(), emit.end());
patched = true;
offset += emit.size();
continue;
}
offset += word_count;
}
if (!patched) {
throw LogicError("Mesh instruction markers not found");
}
}
} // Anonymous namespace
std::vector<u32> EmitSPIRV(const Profile& profile, const RuntimeInfo& runtime_info, IR::Program& program, Bindings& bindings) {
EmitContext ctx{profile, runtime_info, program, bindings};
if (program.info.uses_cooperative_matrix && mma_software) DefineSoftwareMatrixFunction(ctx);
Id main{};
if (runtime_info.mesh_fallback_vertex) { // (hookshot: the fallback's generated vertex stage, which reads the records)
const Id void_function{ctx.TypeFunction(ctx.void_id)};
main = ctx.OpFunction(ctx.void_id, spv::FunctionControlMask::MaskNone, void_function);
ctx.AddLabel();
EmitMeshFallbackVertexBody(ctx);
ctx.OpReturn();
ctx.OpFunctionEnd();
} else {
main = DefineMain(ctx, program);
}
DefineEntryPoint(program, ctx, main);
if (profile.support_float_controls) {
ctx.AddExtension("SPV_KHR_float_controls");
SetupDenormControl(profile, program, ctx, main);
SetupSignedNanCapabilities(profile, program, ctx, main);
}
SetupCapabilities(profile, program.info, ctx);
SetupTransformFeedbackCapabilities(ctx, main);
PatchPhiNodes(program, ctx);
std::vector<u32> code{ctx.Assemble()};
RewriteOpcodes(code, ctx.opcode_rewrites);
PatchMeshInstructions(code, ctx);
return code;
}
Id EmitPhi(EmitContext& ctx, IR::Inst* inst) {
const size_t num_args{inst->NumArgs()};
boost::container::small_vector<Id, 32> blocks;
blocks.reserve(num_args);
for (size_t index = 0; index < num_args; ++index) {
blocks.push_back(inst->PhiBlock(index)->Definition<Id>());
}
// The type of a phi instruction is stored in its flags
const Id result_type{TypeId(ctx, inst->Flags<IR::Type>())};
return ctx.DeferredOpPhi(result_type, std::span(blocks.data(), blocks.size()));
}
void EmitVoid(EmitContext&) {}
Id EmitIdentity(EmitContext& ctx, const IR::Value& value) {
const Id id{ctx.Def(value)};
if (!Sirit::ValidId(id)) {
throw NotImplementedException("Forward identity declaration");
}
return id;
}
Id EmitConditionRef(EmitContext& ctx, const IR::Value& value) {
const Id id{ctx.Def(value)};
if (!Sirit::ValidId(id)) {
throw NotImplementedException("Forward identity declaration");
}
return id;
}
void EmitReference(EmitContext&) {}
void EmitPhiMove(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitGetZeroFromOp(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitGetSignFromOp(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitGetCarryFromOp(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitGetOverflowFromOp(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitGetSparseFromOp(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitGetInBoundsFromOp(EmitContext&) {
throw LogicError("Unreachable instruction");
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,72 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <vector>
#include "common/common_types.h"
#include "shader_recompiler/backend/bindings.h"
#include "shader_recompiler/frontend/ir/program.h"
#include "shader_recompiler/profile.h"
#include "shader_recompiler/runtime_info.h"
namespace Shader::Backend::SPIRV {
constexpr u32 NUM_TEXTURE_SCALING_WORDS = 4;
constexpr u32 NUM_IMAGE_SCALING_WORDS = 2;
constexpr u32 NUM_TEXTURE_AND_IMAGE_SCALING_WORDS =
NUM_TEXTURE_SCALING_WORDS + NUM_IMAGE_SCALING_WORDS;
struct RescalingLayout {
alignas(16) std::array<u32, NUM_TEXTURE_SCALING_WORDS> rescaling_textures;
alignas(16) std::array<u32, NUM_IMAGE_SCALING_WORDS> rescaling_images;
u32 down_factor;
};
struct RenderAreaLayout {
std::array<f32, 4> render_area;
};
constexpr u32 RESCALING_LAYOUT_WORDS_OFFSET = offsetof(RescalingLayout, rescaling_textures);
constexpr u32 RESCALING_LAYOUT_DOWN_FACTOR_OFFSET = offsetof(RescalingLayout, down_factor);
constexpr u32 RENDERAREA_LAYOUT_OFFSET = offsetof(RenderAreaLayout, render_area);
// (hookshot) guest memory's page table, a u64 device address pushed to every compute program that reads global memory, after the blocks
// above. It was a constant in the SPIR-V, and the address changes from run to run: the drivers' pipeline caches never hit
constexpr u32 GUEST_PAGES_PUSH_OFFSET = sizeof(RescalingLayout) + sizeof(RenderAreaLayout);
// (hookshot) a task program's first workgroup: NVN draws tasks [first, first + count), Vulkan from 0. The address of the draw's first
// (a u32, 16 bytes per draw of an indirect draw, read at + DrawIndex * 16) is pushed here
constexpr u32 MESH_FIRST_PUSH_OFFSET = GUEST_PAGES_PUSH_OFFSET + 8;
// (hookshot) the mesh shader fallback's buffers, after it: the task payloads and the vertex records (u64 device addresses).
// Paths without the fallback never push them.
constexpr u32 MESH_FALLBACK_PUSH_OFFSET = MESH_FIRST_PUSH_OFFSET + 8;
constexpr u32 MESH_FALLBACK_PAYLOAD_PUSH_OFFSET = MESH_FALLBACK_PUSH_OFFSET;
constexpr u32 MESH_FALLBACK_RECORDS_PUSH_OFFSET = MESH_FALLBACK_PUSH_OFFSET + 8;
constexpr u32 MESH_FALLBACK_INDIRECT_INPUT_PUSH_OFFSET = MESH_FALLBACK_PUSH_OFFSET + 16;
constexpr u32 MESH_FALLBACK_PUSH_END = MESH_FALLBACK_PUSH_OFFSET + 24;
// (hookshot) the fallback's mesh workgroups a draw can run: each a slot, a word of the payload buffer saying whether its task
// started it; then each task workgroup's payload. It was 1024, a record block each, and Ocarina's floor grass (433 tasks of
// 32 slots) lost 13 of 14 meshlets. At 16384, Hyrule Field's far trees (3cb76827/4f0c0b40: 721 tasks of 32) lost the tasks
// past 512, a different set each frame as the culling's list order changes: they flickered. 65535: Vulkan's least
// maxComputeWorkGroupCount, which the mesh dispatch can reach; the payload buffer is ~6.5 MB a recording thread at 800 words.
constexpr u32 MESH_FALLBACK_CAPACITY = 65535;
// (hookshot) the records buffer: a VkDrawIndirectCommand the vertex stage's draw reads (its vertex count the records written),
// the allocator's word, then the records. A mesh workgroup takes records for its primitives' corners only, so a draw covers
// what was drawn, not every slot's maximum
constexpr u32 MESH_FALLBACK_DRAW_WORD = 0; // (vertexCount; instanceCount at word 1, set to 1 by the host)
constexpr u32 MESH_FALLBACK_ALLOC_WORD = 4; // (records taken, past the end too: those that do not fit are not drawn)
// (a VkDispatchIndirectCommand for the mesh workgroups a task program started (y and z 1, set by the host): each started task
// thread takes an index of it and writes its slot there in the payload buffer's map, so the mesh dispatch runs only those. With
// a count from the GPU (DrawMeshTasksIndirect, the floor grass) the host knew none, and dispatched no mesh workgroup)
constexpr u32 MESH_FALLBACK_DISPATCH_WORD = 5;
constexpr u32 MESH_FALLBACK_HEADER_WORDS = 8;
constexpr u32 MESH_FALLBACK_RECORD_WORDS = 16u << 20; // (64 MB: ~47k grass primitives of 31 words are 17 MB)
[[nodiscard]] std::vector<u32> EmitSPIRV(const Profile& profile, const RuntimeInfo& runtime_info, IR::Program& program, Bindings& bindings);
[[nodiscard]] inline std::vector<u32> EmitSPIRV(const Profile& profile, IR::Program& program) {
Bindings binding;
return EmitSPIRV(profile, {}, program, binding);
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,625 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <bit>
#include "shader_recompiler/backend/spirv/emit_spirv.h"
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
namespace {
Id SharedPointer(EmitContext& ctx, Id offset, u32 index_offset = 0) {
const Id shift_id{ctx.Const(2U)};
Id index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)};
if (index_offset > 0) {
index = ctx.OpIAdd(ctx.U32[1], index, ctx.Const(index_offset));
}
return ctx.uses_explicit_workgroup_layout
? ctx.OpAccessChain(ctx.shared_u32, ctx.shared_memory_u32, ctx.u32_zero_value, index)
: ctx.OpAccessChain(ctx.shared_u32, ctx.shared_memory_u32, index);
}
Id StorageIndex(EmitContext& ctx, const IR::Value& offset, size_t element_size) {
if (offset.IsImmediate()) {
const u32 imm_offset{static_cast<u32>(offset.U32() / element_size)};
return ctx.Const(imm_offset);
}
const u32 shift{static_cast<u32>(std::countr_zero(element_size))};
const Id index{ctx.Def(offset)};
if (shift == 0) {
return index;
}
const Id shift_id{ctx.Const(shift)};
return ctx.OpShiftRightLogical(ctx.U32[1], index, shift_id);
}
Id StoragePointer(EmitContext& ctx, const StorageTypeDefinition& type_def,
Id StorageDefinitions::*member_ptr, const IR::Value& binding,
const IR::Value& offset, size_t element_size) {
if (!binding.IsImmediate()) {
throw NotImplementedException("Dynamic storage buffer indexing");
}
const Id ssbo{ctx.ssbos[binding.U32()].*member_ptr};
const Id index{StorageIndex(ctx, offset, element_size)};
return ctx.OpAccessChain(type_def.element, ssbo, ctx.u32_zero_value, index);
}
std::pair<Id, Id> AtomicArgs(EmitContext& ctx) {
const Id scope{ctx.Const(static_cast<u32>(spv::Scope::Device))};
const Id semantics{ctx.u32_zero_value};
return {scope, semantics};
}
Id SharedAtomicU32(EmitContext& ctx, Id offset, Id value,
Id (Sirit::Module::*atomic_func)(Id, Id, Id, Id, Id)) {
const Id pointer{SharedPointer(ctx, offset)};
const auto [scope, semantics]{AtomicArgs(ctx)};
return (ctx.*atomic_func)(ctx.U32[1], pointer, scope, semantics, value);
}
Id StorageAtomicU32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset, Id value,
Id (Sirit::Module::*atomic_func)(Id, Id, Id, Id, Id)) {
const Id pointer{StoragePointer(ctx, ctx.storage_types.U32, &StorageDefinitions::U32, binding,
offset, sizeof(u32))};
const auto [scope, semantics]{AtomicArgs(ctx)};
return (ctx.*atomic_func)(ctx.U32[1], pointer, scope, semantics, value);
}
Id StorageAtomicU64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset, Id value,
Id (Sirit::Module::*atomic_func)(Id, Id, Id, Id, Id),
Id (Sirit::Module::*non_atomic_func)(Id, Id, Id)) {
if (!ctx.profile.support_descriptor_aliasing) {
LOG_WARNING(Shader_SPIRV, "Descriptor aliasing not supported, this cannot be atomic.");
return ctx.ConstantNull(ctx.U64);
}
if (ctx.profile.support_int64_atomics) {
const Id pointer{StoragePointer(ctx, ctx.storage_types.U64, &StorageDefinitions::U64,
binding, offset, sizeof(u64))};
const auto [scope, semantics]{AtomicArgs(ctx)};
return (ctx.*atomic_func)(ctx.U64, pointer, scope, semantics, value);
}
LOG_WARNING(Shader_SPIRV, "Int64 atomics not supported, fallback to non-atomic");
const Id pointer{StoragePointer(ctx, ctx.storage_types.U32x2, &StorageDefinitions::U32x2,
binding, offset, sizeof(u32[2]))};
const Id original_value{ctx.OpBitcast(ctx.U64, ctx.OpLoad(ctx.U32[2], pointer))};
const Id result{(ctx.*non_atomic_func)(ctx.U64, value, original_value)};
ctx.OpStore(pointer, ctx.OpBitcast(ctx.U32[2], result));
return original_value;
}
Id StorageAtomicU32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset, Id value,
Id (Sirit::Module::*non_atomic_func)(Id, Id, Id)) {
if (!ctx.profile.support_descriptor_aliasing) {
LOG_WARNING(Shader_SPIRV, "Descriptor aliasing not supported, this cannot be atomic.");
return ctx.ConstantNull(ctx.U32[2]);
}
LOG_WARNING(Shader_SPIRV, "Int64 atomics not supported, fallback to non-atomic");
const Id pointer{StoragePointer(ctx, ctx.storage_types.U32x2, &StorageDefinitions::U32x2,
binding, offset, sizeof(u32[2]))};
const Id original_value{ctx.OpLoad(ctx.U32[2], pointer)};
const Id result{(ctx.*non_atomic_func)(ctx.U32[2], value, original_value)};
ctx.OpStore(pointer, result);
return original_value;
}
} // Anonymous namespace
Id EmitSharedAtomicIAdd32(EmitContext& ctx, Id offset, Id value) {
return SharedAtomicU32(ctx, offset, value, &Sirit::Module::OpAtomicIAdd);
}
Id EmitSharedAtomicSMin32(EmitContext& ctx, Id offset, Id value) {
return SharedAtomicU32(ctx, offset, value, &Sirit::Module::OpAtomicSMin);
}
Id EmitSharedAtomicUMin32(EmitContext& ctx, Id offset, Id value) {
return SharedAtomicU32(ctx, offset, value, &Sirit::Module::OpAtomicUMin);
}
Id EmitSharedAtomicSMax32(EmitContext& ctx, Id offset, Id value) {
return SharedAtomicU32(ctx, offset, value, &Sirit::Module::OpAtomicSMax);
}
Id EmitSharedAtomicUMax32(EmitContext& ctx, Id offset, Id value) {
return SharedAtomicU32(ctx, offset, value, &Sirit::Module::OpAtomicUMax);
}
Id EmitSharedAtomicInc32(EmitContext& ctx, Id offset, Id value) {
const Id shift_id{ctx.Const(2U)};
const Id index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)};
return ctx.OpFunctionCall(ctx.U32[1], ctx.increment_cas_shared, index, value);
}
Id EmitSharedAtomicDec32(EmitContext& ctx, Id offset, Id value) {
const Id shift_id{ctx.Const(2U)};
const Id index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)};
return ctx.OpFunctionCall(ctx.U32[1], ctx.decrement_cas_shared, index, value);
}
Id EmitSharedAtomicAnd32(EmitContext& ctx, Id offset, Id value) {
return SharedAtomicU32(ctx, offset, value, &Sirit::Module::OpAtomicAnd);
}
Id EmitSharedAtomicOr32(EmitContext& ctx, Id offset, Id value) {
return SharedAtomicU32(ctx, offset, value, &Sirit::Module::OpAtomicOr);
}
Id EmitSharedAtomicXor32(EmitContext& ctx, Id offset, Id value) {
return SharedAtomicU32(ctx, offset, value, &Sirit::Module::OpAtomicXor);
}
Id EmitSharedAtomicExchange32(EmitContext& ctx, Id offset, Id value) {
return SharedAtomicU32(ctx, offset, value, &Sirit::Module::OpAtomicExchange);
}
Id EmitSharedAtomicExchange64(EmitContext& ctx, Id offset, Id value) {
if (ctx.profile.support_shared_int64_atomics && ctx.uses_explicit_workgroup_layout) {
const Id shift_id{ctx.Const(3U)};
const Id index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)};
const Id pointer{
ctx.OpAccessChain(ctx.shared_u64, ctx.shared_memory_u64, ctx.u32_zero_value, index)};
const auto [scope, semantics]{AtomicArgs(ctx)};
return ctx.OpAtomicExchange(ctx.U64, pointer, scope, semantics, value);
}
LOG_WARNING(Shader_SPIRV, "Int64 atomics not supported, fallback to non-atomic");
const Id pointer_1{SharedPointer(ctx, offset, 0)};
const Id pointer_2{SharedPointer(ctx, offset, 1)};
const Id value_1{ctx.OpLoad(ctx.U32[1], pointer_1)};
const Id value_2{ctx.OpLoad(ctx.U32[1], pointer_2)};
const Id new_vector{ctx.OpBitcast(ctx.U32[2], value)};
ctx.OpStore(pointer_1, ctx.OpCompositeExtract(ctx.U32[1], new_vector, 0U));
ctx.OpStore(pointer_2, ctx.OpCompositeExtract(ctx.U32[1], new_vector, 1U));
return ctx.OpBitcast(ctx.U64, ctx.OpCompositeConstruct(ctx.U32[2], value_1, value_2));
}
Id EmitSharedAtomicExchange32x2(EmitContext& ctx, Id offset, Id value) {
LOG_WARNING(Shader_SPIRV, "Int64 atomics not supported, fallback to non-atomic");
const Id pointer_1{SharedPointer(ctx, offset, 0)};
const Id pointer_2{SharedPointer(ctx, offset, 1)};
const Id value_1{ctx.OpLoad(ctx.U32[1], pointer_1)};
const Id value_2{ctx.OpLoad(ctx.U32[1], pointer_2)};
const Id new_vector{ctx.OpBitcast(ctx.U32[2], value)};
ctx.OpStore(pointer_1, ctx.OpCompositeExtract(ctx.U32[1], new_vector, 0U));
ctx.OpStore(pointer_2, ctx.OpCompositeExtract(ctx.U32[1], new_vector, 1U));
return ctx.OpCompositeConstruct(ctx.U32[2], value_1, value_2);
}
Id EmitStorageAtomicIAdd32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32(ctx, binding, offset, value, &Sirit::Module::OpAtomicIAdd);
}
Id EmitStorageAtomicSMin32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32(ctx, binding, offset, value, &Sirit::Module::OpAtomicSMin);
}
Id EmitStorageAtomicUMin32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32(ctx, binding, offset, value, &Sirit::Module::OpAtomicUMin);
}
Id EmitStorageAtomicSMax32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32(ctx, binding, offset, value, &Sirit::Module::OpAtomicSMax);
}
Id EmitStorageAtomicUMax32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32(ctx, binding, offset, value, &Sirit::Module::OpAtomicUMax);
}
Id EmitStorageAtomicInc32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
const Id ssbo{ctx.ssbos[binding.U32()].U32};
const Id base_index{StorageIndex(ctx, offset, sizeof(u32))};
return ctx.OpFunctionCall(ctx.U32[1], ctx.increment_cas_ssbo, base_index, value, ssbo);
}
Id EmitStorageAtomicDec32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
const Id ssbo{ctx.ssbos[binding.U32()].U32};
const Id base_index{StorageIndex(ctx, offset, sizeof(u32))};
return ctx.OpFunctionCall(ctx.U32[1], ctx.decrement_cas_ssbo, base_index, value, ssbo);
}
Id EmitStorageAtomicAnd32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32(ctx, binding, offset, value, &Sirit::Module::OpAtomicAnd);
}
Id EmitStorageAtomicOr32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32(ctx, binding, offset, value, &Sirit::Module::OpAtomicOr);
}
Id EmitStorageAtomicXor32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32(ctx, binding, offset, value, &Sirit::Module::OpAtomicXor);
}
Id EmitStorageAtomicExchange32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32(ctx, binding, offset, value, &Sirit::Module::OpAtomicExchange);
}
Id EmitStorageAtomicCompareExchange32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id comparator, Id value) { // (hookshot: ATOMG.CAS)
const Id pointer{StoragePointer(ctx, ctx.storage_types.U32, &StorageDefinitions::U32, binding,
offset, sizeof(u32))};
const auto [scope, semantics]{AtomicArgs(ctx)};
return ctx.OpAtomicCompareExchange(ctx.U32[1], pointer, scope, semantics, semantics, value, comparator);
}
Id EmitStorageAtomicIAdd64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU64(ctx, binding, offset, value, &Sirit::Module::OpAtomicIAdd,
&Sirit::Module::OpIAdd);
}
Id EmitStorageAtomicSMin64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU64(ctx, binding, offset, value, &Sirit::Module::OpAtomicSMin,
&Sirit::Module::OpSMin);
}
Id EmitStorageAtomicUMin64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU64(ctx, binding, offset, value, &Sirit::Module::OpAtomicUMin,
&Sirit::Module::OpUMin);
}
Id EmitStorageAtomicSMax64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU64(ctx, binding, offset, value, &Sirit::Module::OpAtomicSMax,
&Sirit::Module::OpSMax);
}
Id EmitStorageAtomicUMax64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU64(ctx, binding, offset, value, &Sirit::Module::OpAtomicUMax,
&Sirit::Module::OpUMax);
}
Id EmitStorageAtomicAnd64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU64(ctx, binding, offset, value, &Sirit::Module::OpAtomicAnd,
&Sirit::Module::OpBitwiseAnd);
}
Id EmitStorageAtomicOr64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU64(ctx, binding, offset, value, &Sirit::Module::OpAtomicOr,
&Sirit::Module::OpBitwiseOr);
}
Id EmitStorageAtomicXor64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU64(ctx, binding, offset, value, &Sirit::Module::OpAtomicXor,
&Sirit::Module::OpBitwiseXor);
}
Id EmitStorageAtomicExchange64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
if (ctx.profile.support_int64_atomics) {
const Id pointer{StoragePointer(ctx, ctx.storage_types.U64, &StorageDefinitions::U64,
binding, offset, sizeof(u64))};
const auto [scope, semantics]{AtomicArgs(ctx)};
return ctx.OpAtomicExchange(ctx.U64, pointer, scope, semantics, value);
}
LOG_WARNING(Shader_SPIRV, "Int64 atomics not supported, fallback to non-atomic");
const Id pointer{StoragePointer(ctx, ctx.storage_types.U32x2, &StorageDefinitions::U32x2,
binding, offset, sizeof(u32[2]))};
const Id original{ctx.OpBitcast(ctx.U64, ctx.OpLoad(ctx.U32[2], pointer))};
ctx.OpStore(pointer, value);
return original;
}
Id EmitStorageAtomicIAdd32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32x2(ctx, binding, offset, value, &Sirit::Module::OpIAdd);
}
Id EmitStorageAtomicSMin32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32x2(ctx, binding, offset, value, &Sirit::Module::OpSMin);
}
Id EmitStorageAtomicUMin32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32x2(ctx, binding, offset, value, &Sirit::Module::OpUMin);
}
Id EmitStorageAtomicSMax32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32x2(ctx, binding, offset, value, &Sirit::Module::OpSMax);
}
Id EmitStorageAtomicUMax32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32x2(ctx, binding, offset, value, &Sirit::Module::OpUMax);
}
Id EmitStorageAtomicAnd32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32x2(ctx, binding, offset, value, &Sirit::Module::OpBitwiseAnd);
}
Id EmitStorageAtomicOr32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32x2(ctx, binding, offset, value, &Sirit::Module::OpBitwiseOr);
}
Id EmitStorageAtomicXor32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
return StorageAtomicU32x2(ctx, binding, offset, value, &Sirit::Module::OpBitwiseXor);
}
Id EmitStorageAtomicExchange32x2(EmitContext& ctx, const IR::Value& binding,
const IR::Value& offset, Id value) {
LOG_WARNING(Shader_SPIRV, "Int64 atomics not supported, fallback to non-atomic");
const Id pointer{StoragePointer(ctx, ctx.storage_types.U32x2, &StorageDefinitions::U32x2,
binding, offset, sizeof(u32[2]))};
const Id original{ctx.OpLoad(ctx.U32[2], pointer)};
ctx.OpStore(pointer, value);
return original;
}
Id EmitStorageAtomicAddF32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
const Id ssbo{ctx.ssbos[binding.U32()].U32};
const Id base_index{StorageIndex(ctx, offset, sizeof(u32))};
return ctx.OpFunctionCall(ctx.F32[1], ctx.f32_add_cas, base_index, value, ssbo);
}
Id EmitStorageAtomicAddF16x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
const Id ssbo{ctx.ssbos[binding.U32()].U32};
const Id base_index{StorageIndex(ctx, offset, sizeof(u32))};
const Id result{ctx.OpFunctionCall(ctx.F16[2], ctx.f16x2_add_cas, base_index, value, ssbo)};
return ctx.OpBitcast(ctx.U32[1], result);
}
Id EmitStorageAtomicAddF32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
const Id ssbo{ctx.ssbos[binding.U32()].U32};
const Id base_index{StorageIndex(ctx, offset, sizeof(u32))};
const Id result{ctx.OpFunctionCall(ctx.F32[2], ctx.f32x2_add_cas, base_index, value, ssbo)};
return ctx.OpPackHalf2x16(ctx.U32[1], result);
}
Id EmitStorageAtomicMinF16x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
const Id ssbo{ctx.ssbos[binding.U32()].U32};
const Id base_index{StorageIndex(ctx, offset, sizeof(u32))};
const Id result{ctx.OpFunctionCall(ctx.F16[2], ctx.f16x2_min_cas, base_index, value, ssbo)};
return ctx.OpBitcast(ctx.U32[1], result);
}
Id EmitStorageAtomicMinF32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
const Id ssbo{ctx.ssbos[binding.U32()].U32};
const Id base_index{StorageIndex(ctx, offset, sizeof(u32))};
const Id result{ctx.OpFunctionCall(ctx.F32[2], ctx.f32x2_min_cas, base_index, value, ssbo)};
return ctx.OpPackHalf2x16(ctx.U32[1], result);
}
Id EmitStorageAtomicMaxF16x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
const Id ssbo{ctx.ssbos[binding.U32()].U32};
const Id base_index{StorageIndex(ctx, offset, sizeof(u32))};
const Id result{ctx.OpFunctionCall(ctx.F16[2], ctx.f16x2_max_cas, base_index, value, ssbo)};
return ctx.OpBitcast(ctx.U32[1], result);
}
Id EmitStorageAtomicMaxF32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
const Id ssbo{ctx.ssbos[binding.U32()].U32};
const Id base_index{StorageIndex(ctx, offset, sizeof(u32))};
const Id result{ctx.OpFunctionCall(ctx.F32[2], ctx.f32x2_max_cas, base_index, value, ssbo)};
return ctx.OpPackHalf2x16(ctx.U32[1], result);
}
Id EmitGlobalAtomicIAdd32(EmitContext& ctx, Id address, Id value) { // (hookshot: the fallback)
if (!Sirit::ValidId(ctx.atomic_global_func_u32[0])) {
throw NotImplementedException("SPIR-V Instruction");
}
return ctx.OpFunctionCall(ctx.U32[1], ctx.atomic_global_func_u32[0], address, value);
}
Id EmitGlobalAtomicSMin32(EmitContext& ctx, Id address, Id value) { // (hookshot: the fallback)
if (!Sirit::ValidId(ctx.atomic_global_func_u32[1])) {
throw NotImplementedException("SPIR-V Instruction");
}
return ctx.OpFunctionCall(ctx.U32[1], ctx.atomic_global_func_u32[1], address, value);
}
Id EmitGlobalAtomicUMin32(EmitContext& ctx, Id address, Id value) { // (hookshot: the fallback)
if (!Sirit::ValidId(ctx.atomic_global_func_u32[2])) {
throw NotImplementedException("SPIR-V Instruction");
}
return ctx.OpFunctionCall(ctx.U32[1], ctx.atomic_global_func_u32[2], address, value);
}
Id EmitGlobalAtomicSMax32(EmitContext& ctx, Id address, Id value) { // (hookshot: the fallback)
if (!Sirit::ValidId(ctx.atomic_global_func_u32[3])) {
throw NotImplementedException("SPIR-V Instruction");
}
return ctx.OpFunctionCall(ctx.U32[1], ctx.atomic_global_func_u32[3], address, value);
}
Id EmitGlobalAtomicUMax32(EmitContext& ctx, Id address, Id value) { // (hookshot: the fallback)
if (!Sirit::ValidId(ctx.atomic_global_func_u32[4])) {
throw NotImplementedException("SPIR-V Instruction");
}
return ctx.OpFunctionCall(ctx.U32[1], ctx.atomic_global_func_u32[4], address, value);
}
Id EmitGlobalAtomicInc32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicDec32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicAnd32(EmitContext& ctx, Id address, Id value) { // (hookshot: the fallback)
if (!Sirit::ValidId(ctx.atomic_global_func_u32[5])) {
throw NotImplementedException("SPIR-V Instruction");
}
return ctx.OpFunctionCall(ctx.U32[1], ctx.atomic_global_func_u32[5], address, value);
}
Id EmitGlobalAtomicOr32(EmitContext& ctx, Id address, Id value) { // (hookshot: the fallback)
if (!Sirit::ValidId(ctx.atomic_global_func_u32[6])) {
throw NotImplementedException("SPIR-V Instruction");
}
return ctx.OpFunctionCall(ctx.U32[1], ctx.atomic_global_func_u32[6], address, value);
}
Id EmitGlobalAtomicXor32(EmitContext& ctx, Id address, Id value) { // (hookshot: the fallback)
if (!Sirit::ValidId(ctx.atomic_global_func_u32[7])) {
throw NotImplementedException("SPIR-V Instruction");
}
return ctx.OpFunctionCall(ctx.U32[1], ctx.atomic_global_func_u32[7], address, value);
}
Id EmitGlobalAtomicExchange32(EmitContext& ctx, Id address, Id value) { // (hookshot: the fallback)
if (!Sirit::ValidId(ctx.atomic_global_func_u32[8])) {
throw NotImplementedException("SPIR-V Instruction");
}
return ctx.OpFunctionCall(ctx.U32[1], ctx.atomic_global_func_u32[8], address, value);
}
Id EmitGlobalAtomicCompareExchange32(EmitContext& ctx, Id address, Id comparator, Id value) { // (hookshot: the fallback)
if (!Sirit::ValidId(ctx.atomic_global_cas_func_u32)) {
throw NotImplementedException("SPIR-V Instruction");
}
return ctx.OpFunctionCall(ctx.U32[1], ctx.atomic_global_cas_func_u32, address, comparator, value);
}
Id EmitGlobalAtomicIAdd64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicSMin64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicUMin64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicSMax64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicUMax64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicInc64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicDec64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicAnd64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicOr64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicXor64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicExchange64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicIAdd32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicSMin32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicUMin32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicSMax32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicUMax32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicInc32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicDec32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicAnd32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicOr32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicXor32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicExchange32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicAddF32(EmitContext& ctx, Id address, Id value) { // (hookshot: the fallback)
if (!Sirit::ValidId(ctx.atomic_global_add_f32_func)) {
throw NotImplementedException("SPIR-V Instruction");
}
return ctx.OpFunctionCall(ctx.F32[1], ctx.atomic_global_add_f32_func, address, value);
}
Id EmitGlobalAtomicAddF16x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicAddF32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicMinF16x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicMinF32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicMaxF16x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitGlobalAtomicMaxF32x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,45 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
namespace {
void MemoryBarrier(EmitContext& ctx, spv::Scope scope) {
const auto semantics{
spv::MemorySemanticsMask::AcquireRelease | spv::MemorySemanticsMask::UniformMemory |
spv::MemorySemanticsMask::WorkgroupMemory | spv::MemorySemanticsMask::AtomicCounterMemory |
spv::MemorySemanticsMask::ImageMemory};
ctx.OpMemoryBarrier(ctx.Const(static_cast<u32>(scope)), ctx.Const(static_cast<u32>(semantics)));
}
} // Anonymous namespace
void EmitBarrier(EmitContext& ctx) {
const auto execution{spv::Scope::Workgroup};
const auto memory{spv::Scope::Workgroup};
const auto memory_semantics{spv::MemorySemanticsMask::AcquireRelease |
spv::MemorySemanticsMask::WorkgroupMemory};
ctx.OpControlBarrier(ctx.Const(static_cast<u32>(execution)),
ctx.Const(static_cast<u32>(memory)),
ctx.Const(static_cast<u32>(memory_semantics)));
}
// (hookshot) The subgroup's invocations meet, and their workgroup (shared) memory accesses before it are visible to the ones after:
// what the Switch's in-order warp gives a warp's stores and LDSM's reads of them without a BAR
void EmitSubgroupBarrier(EmitContext& ctx) {
const auto scope{spv::Scope::Subgroup};
const auto memory_semantics{spv::MemorySemanticsMask::AcquireRelease | spv::MemorySemanticsMask::WorkgroupMemory};
ctx.OpControlBarrier(ctx.Const(static_cast<u32>(scope)), ctx.Const(static_cast<u32>(scope)),
ctx.Const(static_cast<u32>(memory_semantics)));
}
void EmitWorkgroupMemoryBarrier(EmitContext& ctx) {
MemoryBarrier(ctx, spv::Scope::Workgroup);
}
void EmitDeviceMemoryBarrier(EmitContext& ctx) {
MemoryBarrier(ctx, spv::Scope::Device);
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,65 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
void EmitBitCastU16F16(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBitCastU32F32(EmitContext& ctx, Id value) {
return ctx.OpBitcast(ctx.U32[1], value);
}
void EmitBitCastU64F64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitBitCastF16U16(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBitCastF32U32(EmitContext& ctx, Id value) {
return ctx.OpBitcast(ctx.F32[1], value);
}
void EmitBitCastF64U64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitPackUint2x32(EmitContext& ctx, Id value) {
return ctx.OpBitcast(ctx.U64, value);
}
Id EmitUnpackUint2x32(EmitContext& ctx, Id value) {
return ctx.OpBitcast(ctx.U32[2], value);
}
Id EmitPackFloat2x16(EmitContext& ctx, Id value) {
return ctx.OpBitcast(ctx.U32[1], value);
}
Id EmitUnpackFloat2x16(EmitContext& ctx, Id value) {
return ctx.OpBitcast(ctx.F16[2], value);
}
Id EmitPackHalf2x16(EmitContext& ctx, Id value) {
return ctx.OpPackHalf2x16(ctx.U32[1], value);
}
Id EmitUnpackHalf2x16(EmitContext& ctx, Id value) {
return ctx.OpUnpackHalf2x16(ctx.F32[2], value);
}
Id EmitPackDouble2x32(EmitContext& ctx, Id value) {
return ctx.OpBitcast(ctx.F64[1], value);
}
Id EmitUnpackDouble2x32(EmitContext& ctx, Id value) {
return ctx.OpBitcast(ctx.U32[2], value);
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,153 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
Id EmitCompositeConstructU32x2(EmitContext& ctx, Id e1, Id e2) {
return ctx.OpCompositeConstruct(ctx.U32[2], e1, e2);
}
Id EmitCompositeConstructU32x3(EmitContext& ctx, Id e1, Id e2, Id e3) {
return ctx.OpCompositeConstruct(ctx.U32[3], e1, e2, e3);
}
Id EmitCompositeConstructU32x4(EmitContext& ctx, Id e1, Id e2, Id e3, Id e4) {
return ctx.OpCompositeConstruct(ctx.U32[4], e1, e2, e3, e4);
}
Id EmitCompositeExtractU32x2(EmitContext& ctx, Id composite, u32 index) {
return ctx.OpCompositeExtract(ctx.U32[1], composite, index);
}
Id EmitCompositeExtractU32x3(EmitContext& ctx, Id composite, u32 index) {
return ctx.OpCompositeExtract(ctx.U32[1], composite, index);
}
Id EmitCompositeExtractU32x4(EmitContext& ctx, Id composite, u32 index) {
return ctx.OpCompositeExtract(ctx.U32[1], composite, index);
}
Id EmitCompositeInsertU32x2(EmitContext& ctx, Id composite, Id object, u32 index) {
return ctx.OpCompositeInsert(ctx.U32[2], object, composite, index);
}
Id EmitCompositeInsertU32x3(EmitContext& ctx, Id composite, Id object, u32 index) {
return ctx.OpCompositeInsert(ctx.U32[3], object, composite, index);
}
Id EmitCompositeInsertU32x4(EmitContext& ctx, Id composite, Id object, u32 index) {
return ctx.OpCompositeInsert(ctx.U32[4], object, composite, index);
}
Id EmitCompositeConstructF16x2(EmitContext& ctx, Id e1, Id e2) {
return ctx.OpCompositeConstruct(ctx.F16[2], e1, e2);
}
Id EmitCompositeConstructF16x3(EmitContext& ctx, Id e1, Id e2, Id e3) {
return ctx.OpCompositeConstruct(ctx.F16[3], e1, e2, e3);
}
Id EmitCompositeConstructF16x4(EmitContext& ctx, Id e1, Id e2, Id e3, Id e4) {
return ctx.OpCompositeConstruct(ctx.F16[4], e1, e2, e3, e4);
}
Id EmitCompositeExtractF16x2(EmitContext& ctx, Id composite, u32 index) {
return ctx.OpCompositeExtract(ctx.F16[1], composite, index);
}
Id EmitCompositeExtractF16x3(EmitContext& ctx, Id composite, u32 index) {
return ctx.OpCompositeExtract(ctx.F16[1], composite, index);
}
Id EmitCompositeExtractF16x4(EmitContext& ctx, Id composite, u32 index) {
return ctx.OpCompositeExtract(ctx.F16[1], composite, index);
}
Id EmitCompositeInsertF16x2(EmitContext& ctx, Id composite, Id object, u32 index) {
return ctx.OpCompositeInsert(ctx.F16[2], object, composite, index);
}
Id EmitCompositeInsertF16x3(EmitContext& ctx, Id composite, Id object, u32 index) {
return ctx.OpCompositeInsert(ctx.F16[3], object, composite, index);
}
Id EmitCompositeInsertF16x4(EmitContext& ctx, Id composite, Id object, u32 index) {
return ctx.OpCompositeInsert(ctx.F16[4], object, composite, index);
}
Id EmitCompositeConstructF32x2(EmitContext& ctx, Id e1, Id e2) {
return ctx.OpCompositeConstruct(ctx.F32[2], e1, e2);
}
Id EmitCompositeConstructF32x3(EmitContext& ctx, Id e1, Id e2, Id e3) {
return ctx.OpCompositeConstruct(ctx.F32[3], e1, e2, e3);
}
Id EmitCompositeConstructF32x4(EmitContext& ctx, Id e1, Id e2, Id e3, Id e4) {
return ctx.OpCompositeConstruct(ctx.F32[4], e1, e2, e3, e4);
}
Id EmitCompositeExtractF32x2(EmitContext& ctx, Id composite, u32 index) {
return ctx.OpCompositeExtract(ctx.F32[1], composite, index);
}
Id EmitCompositeExtractF32x3(EmitContext& ctx, Id composite, u32 index) {
return ctx.OpCompositeExtract(ctx.F32[1], composite, index);
}
Id EmitCompositeExtractF32x4(EmitContext& ctx, Id composite, u32 index) {
return ctx.OpCompositeExtract(ctx.F32[1], composite, index);
}
Id EmitCompositeInsertF32x2(EmitContext& ctx, Id composite, Id object, u32 index) {
return ctx.OpCompositeInsert(ctx.F32[2], object, composite, index);
}
Id EmitCompositeInsertF32x3(EmitContext& ctx, Id composite, Id object, u32 index) {
return ctx.OpCompositeInsert(ctx.F32[3], object, composite, index);
}
Id EmitCompositeInsertF32x4(EmitContext& ctx, Id composite, Id object, u32 index) {
return ctx.OpCompositeInsert(ctx.F32[4], object, composite, index);
}
void EmitCompositeConstructF64x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitCompositeConstructF64x3(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitCompositeConstructF64x4(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitCompositeExtractF64x2(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitCompositeExtractF64x3(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitCompositeExtractF64x4(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitCompositeInsertF64x2(EmitContext& ctx, Id composite, Id object, u32 index) {
return ctx.OpCompositeInsert(ctx.F64[2], object, composite, index);
}
Id EmitCompositeInsertF64x3(EmitContext& ctx, Id composite, Id object, u32 index) {
return ctx.OpCompositeInsert(ctx.F64[3], object, composite, index);
}
Id EmitCompositeInsertF64x4(EmitContext& ctx, Id composite, Id object, u32 index) {
return ctx.OpCompositeInsert(ctx.F64[4], object, composite, index);
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,660 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <bit>
#include <tuple>
#include <utility>
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/mma_layout.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
namespace {
template <typename... Args>
Id AttrPointer(EmitContext& ctx, Id pointer_type, Id vertex, Id base, Args&&... args) {
switch (ctx.stage) {
case Stage::TessellationControl:
case Stage::TessellationEval:
case Stage::Geometry:
return ctx.OpAccessChain(pointer_type, base, vertex, std::forward<Args>(args)...);
default:
return ctx.OpAccessChain(pointer_type, base, std::forward<Args>(args)...);
}
}
template <typename... Args>
Id OutputAccessChain(EmitContext& ctx, Id result_type, Id base, Args&&... args) {
if (ctx.stage == Stage::TessellationControl) {
const Id invocation_id{ctx.OpLoad(ctx.U32[1], ctx.invocation_id)};
return ctx.OpAccessChain(result_type, base, invocation_id, std::forward<Args>(args)...);
} else {
return ctx.OpAccessChain(result_type, base, std::forward<Args>(args)...);
}
}
struct OutAttr {
OutAttr(Id pointer_) : pointer{pointer_} {}
OutAttr(Id pointer_, Id type_) : pointer{pointer_}, type{type_} {}
Id pointer{};
Id type{};
};
std::optional<OutAttr> OutputAttrPointer(EmitContext& ctx, IR::Attribute attr) {
if (IR::IsGeneric(attr)) {
const u32 index{IR::GenericAttributeIndex(attr)};
const u32 element{IR::GenericAttributeElement(attr)};
const GenericElementInfo& info{ctx.output_generics.at(index).at(element)};
if (info.num_components == 1) {
return info.id;
} else {
const u32 index_element{element - info.first_element};
const Id index_id{ctx.Const(index_element)};
return OutputAccessChain(ctx, ctx.output_f32, info.id, index_id);
}
}
switch (attr) {
case IR::Attribute::PointSize:
return ctx.output_point_size;
case IR::Attribute::PositionX:
case IR::Attribute::PositionY:
case IR::Attribute::PositionZ:
case IR::Attribute::PositionW: {
const u32 element{static_cast<u32>(attr) % 4};
const Id element_id{ctx.Const(element)};
return OutputAccessChain(ctx, ctx.output_f32, ctx.output_position, element_id);
}
case IR::Attribute::ClipDistance0:
case IR::Attribute::ClipDistance1:
case IR::Attribute::ClipDistance2:
case IR::Attribute::ClipDistance3:
case IR::Attribute::ClipDistance4:
case IR::Attribute::ClipDistance5:
case IR::Attribute::ClipDistance6:
case IR::Attribute::ClipDistance7: {
const u32 base{static_cast<u32>(IR::Attribute::ClipDistance0)};
const u32 index{static_cast<u32>(attr) - base};
if (index >= ctx.profile.max_user_clip_distances) {
LOG_WARNING(Shader, "Ignoring clip distance store {} >= {} supported", index,
ctx.profile.max_user_clip_distances);
return std::nullopt;
}
const Id clip_num{ctx.Const(index)};
return OutputAccessChain(ctx, ctx.output_f32, ctx.clip_distances, clip_num);
}
case IR::Attribute::Layer:
if (ctx.profile.support_viewport_index_layer_non_geometry ||
ctx.stage == Shader::Stage::Geometry) {
return OutAttr{ctx.layer, ctx.U32[1]};
}
return std::nullopt;
case IR::Attribute::ViewportIndex:
if (!ctx.profile.support_multi_viewport) {
LOG_WARNING(Shader, "Ignoring viewport index store on non-supporting driver");
return std::nullopt;
}
if (ctx.profile.support_viewport_index_layer_non_geometry ||
ctx.stage == Shader::Stage::Geometry) {
return OutAttr{ctx.viewport_index, ctx.U32[1]};
}
return std::nullopt;
case IR::Attribute::ViewportMask:
if (!ctx.profile.support_viewport_mask) {
return std::nullopt;
}
return OutAttr{ctx.OpAccessChain(ctx.output_u32, ctx.viewport_mask, ctx.u32_zero_value),
ctx.U32[1]};
default:
throw NotImplementedException("Read attribute {}", attr);
}
}
Id GetCbuf(EmitContext& ctx, Id result_type, Id UniformDefinitions::*member_ptr, u32 element_size,
const IR::Value& binding, const IR::Value& offset, const Id indirect_func) {
Id buffer_offset;
const Id uniform_type{ctx.uniform_types.*member_ptr};
if (offset.IsImmediate()) {
// Hardware been proved to read the aligned offset (e.g. LDC.U32 at 6 will read offset 4)
const Id imm_offset{ctx.Const(offset.U32() / element_size)};
buffer_offset = imm_offset;
} else if (element_size > 1) {
const u32 log2_element_size{static_cast<u32>(std::countr_zero(element_size))};
const Id shift{ctx.Const(log2_element_size)};
buffer_offset = ctx.OpShiftRightLogical(ctx.U32[1], ctx.Def(offset), shift);
} else {
buffer_offset = ctx.Def(offset);
}
if (!binding.IsImmediate()) {
return ctx.OpFunctionCall(result_type, indirect_func, ctx.Def(binding), buffer_offset);
}
const Id cbuf{ctx.cbufs[binding.U32()].*member_ptr};
const Id access_chain{ctx.OpAccessChain(uniform_type, cbuf, ctx.u32_zero_value, buffer_offset)};
const Id val = ctx.OpLoad(result_type, access_chain);
if (offset.IsImmediate() || !ctx.profile.has_broken_robust) {
return val;
}
const Id cond = ctx.OpULessThanEqual(ctx.TypeBool(), buffer_offset, ctx.Const(0xFFFFu));
const Id zero = ctx.ConstantNull(result_type);
return ctx.OpSelect(result_type, cond, val, zero);
}
Id GetCbufU32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
return GetCbuf(ctx, ctx.U32[1], &UniformDefinitions::U32, sizeof(u32), binding, offset,
ctx.load_const_func_u32);
}
Id GetCbufU32x4(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
return GetCbuf(ctx, ctx.U32[4], &UniformDefinitions::U32x4, sizeof(u32[4]), binding, offset,
ctx.load_const_func_u32x4);
}
Id GetCbufElement(EmitContext& ctx, Id vector, const IR::Value& offset, u32 index_offset) {
if (offset.IsImmediate()) {
const u32 element{(offset.U32() / 4) % 4 + index_offset};
return ctx.OpCompositeExtract(ctx.U32[1], vector, element);
}
const Id shift{ctx.OpShiftRightLogical(ctx.U32[1], ctx.Def(offset), ctx.Const(2u))};
Id element{ctx.OpBitwiseAnd(ctx.U32[1], shift, ctx.Const(3u))};
if (index_offset > 0) {
element = ctx.OpIAdd(ctx.U32[1], element, ctx.Const(index_offset));
}
return ctx.OpVectorExtractDynamic(ctx.U32[1], vector, element);
}
} // Anonymous namespace
void EmitGetRegister(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitSetRegister(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitGetPred(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitSetPred(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitSetGotoVariable(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitGetGotoVariable(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitSetIndirectBranchVariable(EmitContext&) {
throw LogicError("Unreachable instruction");
}
void EmitGetIndirectBranchVariable(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitGetCbufU8(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_descriptor_aliasing && ctx.profile.support_int8 &&
ctx.profile.support_uniform_and_storage_buffer_8bit) {
const Id load{GetCbuf(ctx, ctx.U8, &UniformDefinitions::U8, sizeof(u8), binding, offset,
ctx.load_const_func_u8)};
return ctx.OpUConvert(ctx.U32[1], load);
}
Id element{};
if (ctx.profile.support_descriptor_aliasing) {
element = GetCbufU32(ctx, binding, offset);
} else {
const Id vector{GetCbufU32x4(ctx, binding, offset)};
element = GetCbufElement(ctx, vector, offset, 0u);
}
const Id bit_offset{ctx.BitOffset8(offset)};
return ctx.OpBitFieldUExtract(ctx.U32[1], element, bit_offset, ctx.Const(8u));
}
Id EmitGetCbufS8(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_descriptor_aliasing && ctx.profile.support_int8 &&
ctx.profile.support_uniform_and_storage_buffer_8bit) {
const Id load{GetCbuf(ctx, ctx.S8, &UniformDefinitions::S8, sizeof(s8), binding, offset,
ctx.load_const_func_u8)};
return ctx.OpSConvert(ctx.U32[1], load);
}
Id element{};
if (ctx.profile.support_descriptor_aliasing) {
element = GetCbufU32(ctx, binding, offset);
} else {
const Id vector{GetCbufU32x4(ctx, binding, offset)};
element = GetCbufElement(ctx, vector, offset, 0u);
}
const Id bit_offset{ctx.BitOffset8(offset)};
return ctx.OpBitFieldSExtract(ctx.U32[1], element, bit_offset, ctx.Const(8u));
}
Id EmitGetCbufU16(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_descriptor_aliasing && ctx.profile.support_int16 &&
ctx.profile.support_uniform_and_storage_buffer_16bit) {
const Id load{GetCbuf(ctx, ctx.U16, &UniformDefinitions::U16, sizeof(u16), binding, offset,
ctx.load_const_func_u16)};
return ctx.OpUConvert(ctx.U32[1], load);
}
Id element{};
if (ctx.profile.support_descriptor_aliasing) {
element = GetCbufU32(ctx, binding, offset);
} else {
const Id vector{GetCbufU32x4(ctx, binding, offset)};
element = GetCbufElement(ctx, vector, offset, 0u);
}
const Id bit_offset{ctx.BitOffset16(offset)};
return ctx.OpBitFieldUExtract(ctx.U32[1], element, bit_offset, ctx.Const(16u));
}
Id EmitGetCbufS16(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_descriptor_aliasing && ctx.profile.support_int16 &&
ctx.profile.support_uniform_and_storage_buffer_16bit) {
const Id load{GetCbuf(ctx, ctx.S16, &UniformDefinitions::S16, sizeof(s16), binding, offset,
ctx.load_const_func_u16)};
return ctx.OpSConvert(ctx.U32[1], load);
}
Id element{};
if (ctx.profile.support_descriptor_aliasing) {
element = GetCbufU32(ctx, binding, offset);
} else {
const Id vector{GetCbufU32x4(ctx, binding, offset)};
element = GetCbufElement(ctx, vector, offset, 0u);
}
const Id bit_offset{ctx.BitOffset16(offset)};
return ctx.OpBitFieldSExtract(ctx.U32[1], element, bit_offset, ctx.Const(16u));
}
Id EmitGetCbufU32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_descriptor_aliasing) {
return GetCbufU32(ctx, binding, offset);
} else {
const Id vector{GetCbufU32x4(ctx, binding, offset)};
return GetCbufElement(ctx, vector, offset, 0u);
}
}
Id EmitGetCbufF32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_descriptor_aliasing) {
return GetCbuf(ctx, ctx.F32[1], &UniformDefinitions::F32, sizeof(f32), binding, offset,
ctx.load_const_func_f32);
} else {
const Id vector{GetCbufU32x4(ctx, binding, offset)};
return ctx.OpBitcast(ctx.F32[1], GetCbufElement(ctx, vector, offset, 0u));
}
}
Id EmitGetCbufU32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_descriptor_aliasing) {
return GetCbuf(ctx, ctx.U32[2], &UniformDefinitions::U32x2, sizeof(u32[2]), binding, offset,
ctx.load_const_func_u32x2);
} else {
const Id vector{GetCbufU32x4(ctx, binding, offset)};
return ctx.OpCompositeConstruct(ctx.U32[2], GetCbufElement(ctx, vector, offset, 0u),
GetCbufElement(ctx, vector, offset, 1u));
}
}
Id EmitGetAttribute(EmitContext& ctx, IR::Attribute attr, Id vertex) {
const u32 element{static_cast<u32>(attr) % 4};
if (IR::IsGeneric(attr)) {
const u32 index{IR::GenericAttributeIndex(attr)};
if (ctx.stage == Stage::Fragment && ctx.runtime_info.point_coord_generics.test(index)) {
// (hookshot) the point sprite's coordinate in place of the generic (RuntimeInfo::point_coord_generics)
if (element < 2) {
return ctx.OpLoad(ctx.F32[1],
ctx.OpAccessChain(ctx.input_f32, ctx.point_coord, ctx.Const(element)));
}
return ctx.Const(element == 3 ? 1.0f : 0.0f);
}
const auto& generic{ctx.input_generics.at(index)};
if (!ValidId(generic.id)) {
// Attribute is disabled or varying component is not written
return ctx.Const(element == 3 ? 1.0f : 0.0f);
}
const Id pointer{
AttrPointer(ctx, generic.pointer_type, vertex, generic.id, ctx.Const(element))};
const Id value{ctx.OpLoad(generic.component_type, pointer)};
return [&ctx, generic, value]() {
switch (generic.load_op) {
case InputGenericLoadOp::Bitcast:
return ctx.OpBitcast(ctx.F32[1], value);
case InputGenericLoadOp::SToF:
return ctx.OpConvertSToF(ctx.F32[1], value);
case InputGenericLoadOp::UToF:
return ctx.OpConvertUToF(ctx.F32[1], value);
default:
return value;
};
}();
}
switch (attr) {
case IR::Attribute::PrimitiveId:
return ctx.OpBitcast(ctx.F32[1], ctx.OpLoad(ctx.U32[1], ctx.primitive_id));
case IR::Attribute::Layer:
return ctx.OpBitcast(ctx.F32[1], ctx.OpLoad(ctx.U32[1], ctx.layer));
case IR::Attribute::PositionX:
case IR::Attribute::PositionY:
case IR::Attribute::PositionZ:
case IR::Attribute::PositionW:
return ctx.OpLoad(
ctx.F32[1],
ctx.need_input_position_indirect
? AttrPointer(ctx, ctx.input_f32, vertex, ctx.input_position, ctx.u32_zero_value,
ctx.Const(element))
: AttrPointer(ctx, ctx.input_f32, vertex, ctx.input_position, ctx.Const(element)));
case IR::Attribute::InstanceId:
if (ctx.profile.support_vertex_instance_id) {
return ctx.OpBitcast(ctx.F32[1], ctx.OpLoad(ctx.U32[1], ctx.instance_id));
} else {
const Id index{ctx.OpLoad(ctx.U32[1], ctx.instance_index)};
const Id base{ctx.OpLoad(ctx.U32[1], ctx.base_instance)};
return ctx.OpBitcast(ctx.F32[1], ctx.OpISub(ctx.U32[1], index, base));
}
case IR::Attribute::VertexId:
if (ctx.profile.support_vertex_instance_id) {
return ctx.OpBitcast(ctx.F32[1], ctx.OpLoad(ctx.U32[1], ctx.vertex_id));
} else {
return ctx.OpBitcast(ctx.F32[1], ctx.OpLoad(ctx.U32[1], ctx.vertex_index));
}
case IR::Attribute::BaseInstance:
return ctx.OpBitcast(ctx.F32[1], ctx.OpLoad(ctx.U32[1], ctx.base_instance));
case IR::Attribute::BaseVertex:
return ctx.OpBitcast(ctx.F32[1], ctx.OpLoad(ctx.U32[1], ctx.base_vertex));
case IR::Attribute::DrawID:
return ctx.OpBitcast(ctx.F32[1], ctx.OpLoad(ctx.U32[1], ctx.draw_index));
case IR::Attribute::FrontFace:
return ctx.OpSelect(ctx.F32[1], ctx.OpLoad(ctx.U1, ctx.front_face),
ctx.OpBitcast(ctx.F32[1], ctx.Const((std::numeric_limits<u32>::max)())),
ctx.f32_zero_value);
case IR::Attribute::PointSpriteS:
return ctx.OpLoad(ctx.F32[1],
ctx.OpAccessChain(ctx.input_f32, ctx.point_coord, ctx.u32_zero_value));
case IR::Attribute::PointSpriteT:
return ctx.OpLoad(ctx.F32[1],
ctx.OpAccessChain(ctx.input_f32, ctx.point_coord, ctx.Const(1U)));
case IR::Attribute::TessellationEvaluationPointU:
return ctx.OpLoad(ctx.F32[1],
ctx.OpAccessChain(ctx.input_f32, ctx.tess_coord, ctx.u32_zero_value));
case IR::Attribute::TessellationEvaluationPointV:
return ctx.OpLoad(ctx.F32[1],
ctx.OpAccessChain(ctx.input_f32, ctx.tess_coord, ctx.Const(1U)));
default:
throw NotImplementedException("Read attribute {}", attr);
}
}
Id EmitGetAttributeU32(EmitContext& ctx, IR::Attribute attr, Id) {
switch (attr) {
case IR::Attribute::PrimitiveId:
return ctx.OpLoad(ctx.U32[1], ctx.primitive_id);
case IR::Attribute::InstanceId:
if (ctx.profile.support_vertex_instance_id) {
return ctx.OpLoad(ctx.U32[1], ctx.instance_id);
} else {
const Id index{ctx.OpLoad(ctx.U32[1], ctx.instance_index)};
const Id base{ctx.OpLoad(ctx.U32[1], ctx.base_instance)};
return ctx.OpISub(ctx.U32[1], index, base);
}
case IR::Attribute::VertexId:
if (ctx.profile.support_vertex_instance_id) {
return ctx.OpLoad(ctx.U32[1], ctx.vertex_id);
} else {
return ctx.OpLoad(ctx.U32[1], ctx.vertex_index);
}
case IR::Attribute::BaseInstance:
return ctx.OpLoad(ctx.U32[1], ctx.base_instance);
case IR::Attribute::BaseVertex:
return ctx.OpLoad(ctx.U32[1], ctx.base_vertex);
case IR::Attribute::DrawID:
return ctx.OpLoad(ctx.U32[1], ctx.draw_index);
default:
throw NotImplementedException("Read U32 attribute {}", attr);
}
}
void EmitSetAttribute(EmitContext& ctx, IR::Attribute attr, Id value, [[maybe_unused]] Id vertex) {
const std::optional<OutAttr> output{OutputAttrPointer(ctx, attr)};
if (!output) {
return;
}
if (Sirit::ValidId(output->type)) {
value = ctx.OpBitcast(output->type, value);
}
static constexpr IR::Attribute cd0 = IR::Attribute::ClipDistance0;
static constexpr IR::Attribute cd7 = IR::Attribute::ClipDistance7;
if (attr >= cd0 && attr <= cd7) {
const u32 idx = (u32) attr - (u32) cd0;
clip_distance_written.set(idx);
}
ctx.OpStore(output->pointer, value);
}
Id EmitGetAttributeIndexed(EmitContext& ctx, Id offset, Id vertex) {
switch (ctx.stage) {
case Stage::TessellationControl:
case Stage::TessellationEval:
case Stage::Geometry:
return ctx.OpFunctionCall(ctx.F32[1], ctx.indexed_load_func, offset, vertex);
default:
return ctx.OpFunctionCall(ctx.F32[1], ctx.indexed_load_func, offset);
}
}
void EmitSetAttributeIndexed(EmitContext& ctx, Id offset, Id value, [[maybe_unused]] Id vertex) {
ctx.OpFunctionCall(ctx.void_id, ctx.indexed_store_func, offset, value);
}
Id EmitGetPatch(EmitContext& ctx, IR::Patch patch) {
if (!IR::IsGeneric(patch)) {
throw NotImplementedException("Non-generic patch load");
}
const u32 index{IR::GenericPatchIndex(patch)};
const Id element{ctx.Const(IR::GenericPatchElement(patch))};
const Id type{ctx.stage == Stage::TessellationControl ? ctx.output_f32 : ctx.input_f32};
const Id pointer{ctx.OpAccessChain(type, ctx.patches.at(index), element)};
return ctx.OpLoad(ctx.F32[1], pointer);
}
void EmitSetPatch(EmitContext& ctx, IR::Patch patch, Id value) {
const Id pointer{[&] {
if (IR::IsGeneric(patch)) {
const u32 index{IR::GenericPatchIndex(patch)};
const Id element{ctx.Const(IR::GenericPatchElement(patch))};
return ctx.OpAccessChain(ctx.output_f32, ctx.patches.at(index), element);
}
switch (patch) {
case IR::Patch::TessellationLodLeft:
case IR::Patch::TessellationLodRight:
case IR::Patch::TessellationLodTop:
case IR::Patch::TessellationLodBottom: {
const u32 index{static_cast<u32>(patch) - u32(IR::Patch::TessellationLodLeft)};
const Id index_id{ctx.Const(index)};
return ctx.OpAccessChain(ctx.output_f32, ctx.output_tess_level_outer, index_id);
}
case IR::Patch::TessellationLodInteriorU:
return ctx.OpAccessChain(ctx.output_f32, ctx.output_tess_level_inner,
ctx.u32_zero_value);
case IR::Patch::TessellationLodInteriorV:
return ctx.OpAccessChain(ctx.output_f32, ctx.output_tess_level_inner, ctx.Const(1u));
default:
throw NotImplementedException("Patch {}", patch);
}
}()};
ctx.OpStore(pointer, value);
}
void EmitSetFragColor(EmitContext& ctx, u32 index, u32 component, Id value) {
const Id component_id{ctx.Const(component)};
const AttributeType type{ctx.runtime_info.color_output_types[index]};
if (type == AttributeType::Float) {
const Id pointer{ctx.OpAccessChain(ctx.output_f32, ctx.frag_color.at(index), component_id)};
if (frag_nan_guard) { // (amd-fix, mma_layout.h) NaN -> 0, the rest clamped to +-1e30
const Id clamped{ctx.OpNMin(ctx.F32[1], ctx.OpNMax(ctx.F32[1], value, ctx.Const(-1e30f)), ctx.Const(1e30f))};
value = ctx.OpSelect(ctx.F32[1], ctx.OpIsNan(ctx.U1, value), ctx.Const(0.0f), clamped);
}
ctx.OpStore(pointer, value);
} else if (type == AttributeType::UnsignedInt) {
const Id pointer{ctx.OpAccessChain(ctx.output_u32, ctx.frag_color.at(index), component_id)};
ctx.OpStore(pointer, ctx.OpBitcast(ctx.U32[1], value));
} else if (type == AttributeType::SignedInt) {
const Id output_s32{ctx.TypePointer(spv::StorageClass::Output, ctx.S32[1])};
const Id pointer{ctx.OpAccessChain(output_s32, ctx.frag_color.at(index), component_id)};
ctx.OpStore(pointer, ctx.OpBitcast(ctx.S32[1], value));
} else {
// Disabled or unknown, treat as float
const Id pointer{ctx.OpAccessChain(ctx.output_f32, ctx.frag_color.at(index), component_id)};
ctx.OpStore(pointer, value);
}
}
void EmitSetSampleMask(EmitContext& ctx, Id value) {
const Id pointer{ctx.OpAccessChain(ctx.output_u32, ctx.sample_mask, ctx.u32_zero_value)};
ctx.OpStore(pointer, value);
}
void EmitSetFragDepth(EmitContext& ctx, Id value) {
if (!ctx.runtime_info.convert_depth_mode || ctx.profile.support_native_ndc) {
ctx.OpStore(ctx.frag_depth, value);
return;
}
const Id unit{ctx.Const(0.5f)};
const Id new_depth{ctx.OpFma(ctx.F32[1], value, unit, unit)};
ctx.OpStore(ctx.frag_depth, new_depth);
}
void EmitGetZFlag(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitGetSFlag(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitGetCFlag(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitGetOFlag(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitSetZFlag(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitSetSFlag(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitSetCFlag(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
void EmitSetOFlag(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitWorkgroupId(EmitContext& ctx) {
const Id id{ctx.OpLoad(ctx.U32[3], ctx.workgroup_id)};
if (ctx.mesh_fallback && ctx.stage == Stage::Mesh && ctx.runtime_info.mesh_fallback_task) {
// (hookshot) a fallback mesh workgroup is a slot (its task's map gives it, EmitMeshPrologue): its id is its index in its
// task, as a task launches them from 0
const Id slot{Sirit::ValidId(ctx.mesh_fallback_slot) ? ctx.mesh_fallback_slot
: ctx.OpCompositeExtract(ctx.U32[1], id, 0U)};
const Id x{ctx.OpUMod(ctx.U32[1], slot, ctx.Const(ctx.runtime_info.mesh_fallback_task_threads))};
return ctx.OpCompositeConstruct(ctx.U32[3], x, ctx.u32_zero_value, ctx.u32_zero_value);
}
return id;
}
Id EmitLocalInvocationId(EmitContext& ctx) {
return ctx.OpLoad(ctx.U32[3], ctx.local_invocation_id);
}
Id EmitInvocationId(EmitContext& ctx) {
return ctx.OpLoad(ctx.U32[1], ctx.invocation_id);
}
Id EmitInvocationInfo(EmitContext& ctx) {
switch (ctx.stage) {
case Stage::TessellationControl:
case Stage::TessellationEval:
return ctx.OpShiftLeftLogical(ctx.U32[1], ctx.OpLoad(ctx.U32[1], ctx.patch_vertices_in), ctx.Const(16u));
case Stage::Geometry:
return ctx.OpShiftLeftLogical(ctx.U32[1], ctx.Const(InputTopologyVertices::vertices(ctx.runtime_info.input_topology)), ctx.Const(16u));
default:
LOG_WARNING(Shader, "(STUBBED) called");
return ctx.Const(0x00ff0000u);
}
}
Id EmitSampleId(EmitContext& ctx) {
return ctx.OpLoad(ctx.U32[1], ctx.sample_id);
}
Id EmitIsHelperInvocation(EmitContext& ctx) {
return ctx.OpLoad(ctx.U1, ctx.is_helper_invocation);
}
Id EmitSR_WScaleFactorXY(EmitContext& ctx) {
LOG_WARNING(Shader, "(STUBBED) called");
return ctx.Const(0x00ff0000u);
}
Id EmitSR_WScaleFactorZ(EmitContext& ctx) {
LOG_WARNING(Shader, "(STUBBED) called");
return ctx.Const(0x00ff0000u);
}
Id EmitYDirection(EmitContext& ctx) {
return ctx.Const(ctx.runtime_info.y_negate ? -1.0f : 1.0f);
}
Id EmitResolutionDownFactor(EmitContext& ctx) {
if (ctx.profile.unified_descriptor_binding) {
const Id pointer_type{ctx.TypePointer(spv::StorageClass::PushConstant, ctx.F32[1])};
const Id index{ctx.Const(ctx.rescaling_downfactor_member_index)};
const Id pointer{ctx.OpAccessChain(pointer_type, ctx.rescaling_push_constants, index)};
return ctx.OpLoad(ctx.F32[1], pointer);
} else {
const Id composite{ctx.OpLoad(ctx.F32[4], ctx.rescaling_uniform_constant)};
return ctx.OpCompositeExtract(ctx.F32[1], composite, 2u);
}
}
Id EmitRenderArea(EmitContext& ctx) {
if (ctx.profile.unified_descriptor_binding) {
const Id pointer_type{ctx.TypePointer(spv::StorageClass::PushConstant, ctx.F32[4])};
const Id index{ctx.Const(ctx.render_are_member_index)};
const Id pointer{ctx.OpAccessChain(pointer_type, ctx.render_area_push_constant, index)};
return ctx.OpLoad(ctx.F32[4], pointer);
} else {
throw NotImplementedException("SPIR-V Instruction");
}
}
Id EmitLoadLocal(EmitContext& ctx, Id word_offset) {
const Id pointer{
ctx.OpAccessChain(ctx.private_u32, ctx.local_memory, word_offset, ctx.Const(0U))};
return ctx.OpLoad(ctx.U32[1], pointer);
}
void EmitWriteLocal(EmitContext& ctx, Id word_offset, Id value) {
const Id pointer{
ctx.OpAccessChain(ctx.private_u32, ctx.local_memory, word_offset, ctx.Const(0U))};
ctx.OpStore(pointer, value);
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,27 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
void EmitJoin(EmitContext&) {
throw NotImplementedException("Join shouldn't be emitted");
}
void EmitDemoteToHelperInvocation(EmitContext& ctx) {
if (ctx.profile.support_demote_to_helper_invocation) {
ctx.OpDemoteToHelperInvocationEXT();
} else {
const Id kill_label{ctx.OpLabel()};
const Id impossible_label{ctx.OpLabel()};
ctx.OpSelectionMerge(impossible_label, spv::SelectionControlMask::MaskNone);
ctx.OpBranchConditional(ctx.true_value, kill_label, impossible_label);
ctx.AddLabel(kill_label);
ctx.OpKill();
ctx.AddLabel(impossible_label);
}
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,326 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
namespace {
Id ExtractU16(EmitContext& ctx, Id value) {
if (ctx.profile.support_int16) {
return ctx.OpUConvert(ctx.U16, value);
} else {
return ctx.OpBitFieldUExtract(ctx.U32[1], value, ctx.u32_zero_value, ctx.Const(16u));
}
}
Id ExtractS16(EmitContext& ctx, Id value) {
if (ctx.profile.support_int16) {
return ctx.OpSConvert(ctx.S16, value);
} else {
return ctx.OpBitFieldSExtract(ctx.U32[1], value, ctx.u32_zero_value, ctx.Const(16u));
}
}
Id ExtractU8(EmitContext& ctx, Id value) {
if (ctx.profile.support_int8) {
return ctx.OpUConvert(ctx.U8, value);
} else {
return ctx.OpBitFieldUExtract(ctx.U32[1], value, ctx.u32_zero_value, ctx.Const(8u));
}
}
Id ExtractS8(EmitContext& ctx, Id value) {
if (ctx.profile.support_int8) {
return ctx.OpSConvert(ctx.S8, value);
} else {
return ctx.OpBitFieldSExtract(ctx.U32[1], value, ctx.u32_zero_value, ctx.Const(8u));
}
}
} // Anonymous namespace
Id EmitConvertS16F16(EmitContext& ctx, Id value) {
if (ctx.profile.support_int16) {
return ctx.OpSConvert(ctx.U32[1], ctx.OpConvertFToS(ctx.U16, value));
} else {
return ExtractS16(ctx, ctx.OpConvertFToS(ctx.U32[1], value));
}
}
Id EmitConvertS16F32(EmitContext& ctx, Id value) {
if (ctx.profile.support_int16) {
return ctx.OpSConvert(ctx.U32[1], ctx.OpConvertFToS(ctx.U16, value));
} else {
return ExtractS16(ctx, ctx.OpConvertFToS(ctx.U32[1], value));
}
}
Id EmitConvertS16F64(EmitContext& ctx, Id value) {
if (ctx.profile.support_int16) {
return ctx.OpSConvert(ctx.U32[1], ctx.OpConvertFToS(ctx.U16, value));
} else {
return ExtractS16(ctx, ctx.OpConvertFToS(ctx.U32[1], value));
}
}
Id EmitConvertS32F16(EmitContext& ctx, Id value) {
return ctx.OpConvertFToS(ctx.U32[1], value);
}
Id EmitConvertS32F32(EmitContext& ctx, Id value) {
if (ctx.profile.has_broken_signed_operations) {
return ctx.OpBitcast(ctx.U32[1], ctx.OpConvertFToS(ctx.S32[1], value));
} else {
return ctx.OpConvertFToS(ctx.U32[1], value);
}
}
Id EmitConvertS32F64(EmitContext& ctx, Id value) {
return ctx.OpConvertFToS(ctx.U32[1], value);
}
Id EmitConvertS64F16(EmitContext& ctx, Id value) {
return ctx.OpConvertFToS(ctx.U64, value);
}
Id EmitConvertS64F32(EmitContext& ctx, Id value) {
return ctx.OpConvertFToS(ctx.U64, value);
}
Id EmitConvertS64F64(EmitContext& ctx, Id value) {
return ctx.OpConvertFToS(ctx.U64, value);
}
Id EmitConvertU16F16(EmitContext& ctx, Id value) {
if (ctx.profile.support_int16) {
return ctx.OpUConvert(ctx.U32[1], ctx.OpConvertFToU(ctx.U16, value));
} else {
return ExtractU16(ctx, ctx.OpConvertFToU(ctx.U32[1], value));
}
}
Id EmitConvertU16F32(EmitContext& ctx, Id value) {
if (ctx.profile.support_int16) {
return ctx.OpUConvert(ctx.U32[1], ctx.OpConvertFToU(ctx.U16, value));
} else {
return ExtractU16(ctx, ctx.OpConvertFToU(ctx.U32[1], value));
}
}
Id EmitConvertU16F64(EmitContext& ctx, Id value) {
if (ctx.profile.support_int16) {
return ctx.OpUConvert(ctx.U32[1], ctx.OpConvertFToU(ctx.U16, value));
} else {
return ExtractU16(ctx, ctx.OpConvertFToU(ctx.U32[1], value));
}
}
Id EmitConvertU32F16(EmitContext& ctx, Id value) {
return ctx.OpConvertFToU(ctx.U32[1], value);
}
Id EmitConvertU32F32(EmitContext& ctx, Id value) {
return ctx.OpConvertFToU(ctx.U32[1], value);
}
Id EmitConvertU32F64(EmitContext& ctx, Id value) {
return ctx.OpConvertFToU(ctx.U32[1], value);
}
Id EmitConvertU64F16(EmitContext& ctx, Id value) {
return ctx.OpConvertFToU(ctx.U64, value);
}
Id EmitConvertU64F32(EmitContext& ctx, Id value) {
return ctx.OpConvertFToU(ctx.U64, value);
}
Id EmitConvertU64F64(EmitContext& ctx, Id value) {
return ctx.OpConvertFToU(ctx.U64, value);
}
Id EmitConvertU64U32(EmitContext& ctx, Id value) {
return ctx.OpUConvert(ctx.U64, value);
}
Id EmitConvertU32U64(EmitContext& ctx, Id value) {
return ctx.OpUConvert(ctx.U32[1], value);
}
Id EmitConvertF16F32(EmitContext& ctx, Id value) {
#ifdef __ANDROID__
return ctx.OpFConvert(ctx.F16[1], value);
#else
const auto result = ctx.OpFConvert(ctx.F16[1], value);
const auto isOverflowing = ctx.OpIsNan(ctx.U1, result);
return ctx.OpSelect(ctx.F16[1], isOverflowing, ctx.Constant(ctx.F16[1], 0), result);
#endif
}
Id EmitConvertF32F16(EmitContext& ctx, Id value) {
return ctx.OpFConvert(ctx.F32[1], value);
}
Id EmitConvertF32F64(EmitContext& ctx, Id value) {
return ctx.OpFConvert(ctx.F32[1], value);
}
Id EmitConvertF64F32(EmitContext& ctx, Id value) {
return ctx.OpFConvert(ctx.F64[1], value);
}
Id EmitConvertF16S8(EmitContext& ctx, Id value) {
return ctx.OpConvertSToF(ctx.F16[1], ExtractS8(ctx, value));
}
Id EmitConvertF16S16(EmitContext& ctx, Id value) {
return ctx.OpConvertSToF(ctx.F16[1], ExtractS16(ctx, value));
}
Id EmitConvertF16S32(EmitContext& ctx, Id value) {
return ctx.OpConvertSToF(ctx.F16[1], value);
}
Id EmitConvertF16S64(EmitContext& ctx, Id value) {
return ctx.OpConvertSToF(ctx.F16[1], value);
}
Id EmitConvertF16U8(EmitContext& ctx, Id value) {
return ctx.OpConvertUToF(ctx.F16[1], ExtractU8(ctx, value));
}
Id EmitConvertF16U16(EmitContext& ctx, Id value) {
return ctx.OpConvertUToF(ctx.F16[1], ExtractU16(ctx, value));
}
Id EmitConvertF16U32(EmitContext& ctx, Id value) {
return ctx.OpConvertUToF(ctx.F16[1], value);
}
Id EmitConvertF16U64(EmitContext& ctx, Id value) {
return ctx.OpConvertUToF(ctx.F16[1], value);
}
Id EmitConvertF32S8(EmitContext& ctx, Id value) {
return ctx.OpConvertSToF(ctx.F32[1], ExtractS8(ctx, value));
}
Id EmitConvertF32S16(EmitContext& ctx, Id value) {
return ctx.OpConvertSToF(ctx.F32[1], ExtractS16(ctx, value));
}
Id EmitConvertF32S32(EmitContext& ctx, Id value) {
if (ctx.profile.has_broken_signed_operations) {
value = ctx.OpBitcast(ctx.S32[1], value);
}
return ctx.OpConvertSToF(ctx.F32[1], value);
}
Id EmitConvertF32S64(EmitContext& ctx, Id value) {
return ctx.OpConvertSToF(ctx.F32[1], value);
}
Id EmitConvertF32U8(EmitContext& ctx, Id value) {
return ctx.OpConvertUToF(ctx.F32[1], ExtractU8(ctx, value));
}
Id EmitConvertF32U16(EmitContext& ctx, Id value) {
return ctx.OpConvertUToF(ctx.F32[1], ExtractU16(ctx, value));
}
Id EmitConvertF32U32(EmitContext& ctx, Id value) {
return ctx.OpConvertUToF(ctx.F32[1], value);
}
Id EmitConvertF32U64(EmitContext& ctx, Id value) {
return ctx.OpConvertUToF(ctx.F32[1], value);
}
Id EmitConvertF64S8(EmitContext& ctx, Id value) {
return ctx.OpConvertSToF(ctx.F64[1], ExtractS8(ctx, value));
}
Id EmitConvertF64S16(EmitContext& ctx, Id value) {
return ctx.OpConvertSToF(ctx.F64[1], ExtractS16(ctx, value));
}
Id EmitConvertF64S32(EmitContext& ctx, Id value) {
if (ctx.profile.has_broken_signed_operations) {
value = ctx.OpBitcast(ctx.S32[1], value);
}
return ctx.OpConvertSToF(ctx.F64[1], value);
}
Id EmitConvertF64S64(EmitContext& ctx, Id value) {
return ctx.OpConvertSToF(ctx.F64[1], value);
}
Id EmitConvertF64U8(EmitContext& ctx, Id value) {
return ctx.OpConvertUToF(ctx.F64[1], ExtractU8(ctx, value));
}
Id EmitConvertF64U16(EmitContext& ctx, Id value) {
return ctx.OpConvertUToF(ctx.F64[1], ExtractU16(ctx, value));
}
Id EmitConvertF64U32(EmitContext& ctx, Id value) {
return ctx.OpConvertUToF(ctx.F64[1], value);
}
Id EmitConvertF64U64(EmitContext& ctx, Id value) {
return ctx.OpConvertUToF(ctx.F64[1], value);
}
Id EmitConvertU16U32(EmitContext& ctx, Id value) {
if (ctx.profile.support_int16) {
return ctx.OpUConvert(ctx.U16, value);
} else {
return ctx.OpBitFieldUExtract(ctx.U32[1], value, ctx.u32_zero_value, ctx.Const(16u));
}
}
Id EmitConvertU32U16(EmitContext& ctx, Id value) {
if (ctx.profile.support_int16) {
return ctx.OpUConvert(ctx.U32[1], value);
} else {
return ExtractU16(ctx, value);
}
}
Id EmitConvertU8U32(EmitContext& ctx, Id value) {
if (ctx.profile.support_int8) {
return ctx.OpUConvert(ctx.U8, value);
} else {
return ExtractU8(ctx, value);
}
}
Id EmitConvertU32U8(EmitContext& ctx, Id value) {
if (ctx.profile.support_int8) {
return ctx.OpUConvert(ctx.U32[1], value);
} else {
return ctx.OpBitFieldUExtract(ctx.U32[1], value, ctx.u32_zero_value, ctx.Const(8u));
}
}
// in signed
Id EmitConvertS32S8(EmitContext& ctx, Id value) {
if (ctx.profile.support_int8) {
return ctx.OpSConvert(ctx.U32[1], value);
} else {
return ctx.OpBitFieldSExtract(ctx.U32[1], value, ctx.u32_zero_value, ctx.Const(8u));
}
}
Id EmitConvertS32S16(EmitContext& ctx, Id value) {
if (ctx.profile.support_int16) {
return ctx.OpSConvert(ctx.U32[1], value);
} else {
return ctx.OpBitFieldSExtract(ctx.U32[1], value, ctx.u32_zero_value, ctx.Const(16u));
}
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,396 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
#include "shader_recompiler/frontend/ir/modifiers.h"
namespace Shader::Backend::SPIRV {
namespace {
Id Decorate(EmitContext& ctx, IR::Inst* inst, Id op) {
const auto flags{inst->Flags<IR::FpControl>()};
if (flags.no_contraction) {
ctx.Decorate(op, spv::Decoration::NoContraction);
}
return op;
}
// (amd-fix, Lichii #432/#437) The guest's FMNMX returns the operand that is not NaN and saturation turns NaN into 0.
// FMin/FMax/FClamp leave NaN undefined: NVIDIA happens to return the other operand, AMD returns the NaN, and in OoT's
// bloom chain it fills the screen (the yellow-white haze). NMin/NMax return the operand that is not NaN.
Id Clamp(EmitContext& ctx, Id type, Id value, Id zero, Id one) {
return ctx.OpNMin(type, ctx.OpNMax(type, value, zero), one);
}
Id FPOrdNotEqual(EmitContext& ctx, Id lhs, Id rhs) {
if (ctx.profile.ignore_nan_fp_comparisons) {
const Id comp{ctx.OpFOrdEqual(ctx.U1, lhs, rhs)};
const Id lhs_not_nan{ctx.OpLogicalNot(ctx.U1, ctx.OpIsNan(ctx.U1, lhs))};
const Id rhs_not_nan{ctx.OpLogicalNot(ctx.U1, ctx.OpIsNan(ctx.U1, rhs))};
return ctx.OpLogicalAnd(ctx.U1, ctx.OpLogicalAnd(ctx.U1, comp, lhs_not_nan), rhs_not_nan);
} else {
return ctx.OpFOrdNotEqual(ctx.U1, lhs, rhs);
}
}
Id FPUnordCompare(Id (EmitContext::*comp_func)(Id, Id, Id), EmitContext& ctx, Id lhs, Id rhs) {
if (ctx.profile.ignore_nan_fp_comparisons) {
const Id lhs_nan{ctx.OpIsNan(ctx.U1, lhs)};
const Id rhs_nan{ctx.OpIsNan(ctx.U1, rhs)};
const Id comp{(ctx.*comp_func)(ctx.U1, lhs, rhs)};
return ctx.OpLogicalOr(ctx.U1, ctx.OpLogicalOr(ctx.U1, comp, lhs_nan), rhs_nan);
} else {
return (ctx.*comp_func)(ctx.U1, lhs, rhs);
}
}
} // Anonymous namespace
Id EmitFPAbs16(EmitContext& ctx, Id value) {
return ctx.OpFAbs(ctx.F16[1], value);
}
Id EmitFPAbs32(EmitContext& ctx, Id value) {
return ctx.OpFAbs(ctx.F32[1], value);
}
Id EmitFPAbs64(EmitContext& ctx, Id value) {
return ctx.OpFAbs(ctx.F64[1], value);
}
Id EmitFPAdd16(EmitContext& ctx, IR::Inst* inst, Id a, Id b) {
return Decorate(ctx, inst, ctx.OpFAdd(ctx.F16[1], a, b));
}
Id EmitFPAdd32(EmitContext& ctx, IR::Inst* inst, Id a, Id b) {
return Decorate(ctx, inst, ctx.OpFAdd(ctx.F32[1], a, b));
}
Id EmitFPAdd64(EmitContext& ctx, IR::Inst* inst, Id a, Id b) {
return Decorate(ctx, inst, ctx.OpFAdd(ctx.F64[1], a, b));
}
Id EmitFPFma16(EmitContext& ctx, IR::Inst* inst, Id a, Id b, Id c) {
return Decorate(ctx, inst, ctx.OpFma(ctx.F16[1], a, b, c));
}
Id EmitFPFma32(EmitContext& ctx, IR::Inst* inst, Id a, Id b, Id c) {
return Decorate(ctx, inst, ctx.OpFma(ctx.F32[1], a, b, c));
}
Id EmitFPFma64(EmitContext& ctx, IR::Inst* inst, Id a, Id b, Id c) {
return Decorate(ctx, inst, ctx.OpFma(ctx.F64[1], a, b, c));
}
Id EmitFPMax32(EmitContext& ctx, Id a, Id b) {
return ctx.OpNMax(ctx.F32[1], a, b);
}
Id EmitFPMax64(EmitContext& ctx, Id a, Id b) {
return ctx.OpNMax(ctx.F64[1], a, b);
}
Id EmitFPMin32(EmitContext& ctx, Id a, Id b) {
return ctx.OpNMin(ctx.F32[1], a, b);
}
Id EmitFPMin64(EmitContext& ctx, Id a, Id b) {
return ctx.OpNMin(ctx.F64[1], a, b);
}
Id EmitFPMul16(EmitContext& ctx, IR::Inst* inst, Id a, Id b) {
return Decorate(ctx, inst, ctx.OpFMul(ctx.F16[1], a, b));
}
Id EmitFPMul32(EmitContext& ctx, IR::Inst* inst, Id a, Id b) {
return Decorate(ctx, inst, ctx.OpFMul(ctx.F32[1], a, b));
}
Id EmitFPMul64(EmitContext& ctx, IR::Inst* inst, Id a, Id b) {
return Decorate(ctx, inst, ctx.OpFMul(ctx.F64[1], a, b));
}
Id EmitFPNeg16(EmitContext& ctx, Id value) {
return ctx.OpFNegate(ctx.F16[1], value);
}
Id EmitFPNeg32(EmitContext& ctx, Id value) {
return ctx.OpFNegate(ctx.F32[1], value);
}
Id EmitFPNeg64(EmitContext& ctx, Id value) {
return ctx.OpFNegate(ctx.F64[1], value);
}
Id EmitFPSin(EmitContext& ctx, Id value) {
return ctx.OpSin(ctx.F32[1], value);
}
Id EmitFPCos(EmitContext& ctx, Id value) {
return ctx.OpCos(ctx.F32[1], value);
}
Id EmitFPExp2(EmitContext& ctx, Id value) {
return ctx.OpExp2(ctx.F32[1], value);
}
Id EmitFPLog2(EmitContext& ctx, Id value) {
return ctx.OpLog2(ctx.F32[1], value);
}
Id EmitFPRecip32(EmitContext& ctx, Id value) {
return ctx.OpFDiv(ctx.F32[1], ctx.Const(1.0f), value);
}
Id EmitFPRecip64(EmitContext& ctx, Id value) {
return ctx.OpFDiv(ctx.F64[1], ctx.Constant(ctx.F64[1], 1.0f), value);
}
Id EmitFPRecipSqrt32(EmitContext& ctx, Id value) {
return ctx.OpInverseSqrt(ctx.F32[1], value);
}
Id EmitFPRecipSqrt64(EmitContext& ctx, Id value) {
return ctx.OpInverseSqrt(ctx.F64[1], value);
}
Id EmitFPSqrt(EmitContext& ctx, Id value) {
return ctx.OpSqrt(ctx.F32[1], value);
}
Id EmitFPSaturate16(EmitContext& ctx, Id value) {
const Id zero{ctx.Constant(ctx.F16[1], u16{0})};
const Id one{ctx.Constant(ctx.F16[1], u16{0x3c00})};
return Clamp(ctx, ctx.F16[1], value, zero, one);
}
Id EmitFPSaturate32(EmitContext& ctx, Id value) {
const Id zero{ctx.Const(f32{0.0f})};
const Id one{ctx.Const(f32{1.0f})};
return Clamp(ctx, ctx.F32[1], value, zero, one);
}
Id EmitFPSaturate64(EmitContext& ctx, Id value) {
const Id zero{ctx.Constant(ctx.F64[1], f64{0.0})};
const Id one{ctx.Constant(ctx.F64[1], f64{1.0})};
return Clamp(ctx, ctx.F64[1], value, zero, one);
}
Id EmitFPClamp16(EmitContext& ctx, Id value, Id min_value, Id max_value) {
return Clamp(ctx, ctx.F16[1], value, min_value, max_value);
}
Id EmitFPClamp32(EmitContext& ctx, Id value, Id min_value, Id max_value) {
return Clamp(ctx, ctx.F32[1], value, min_value, max_value);
}
Id EmitFPClamp64(EmitContext& ctx, Id value, Id min_value, Id max_value) {
return Clamp(ctx, ctx.F64[1], value, min_value, max_value);
}
Id EmitFPRoundEven16(EmitContext& ctx, Id value) {
return ctx.OpRoundEven(ctx.F16[1], value);
}
Id EmitFPRoundEven32(EmitContext& ctx, Id value) {
return ctx.OpRoundEven(ctx.F32[1], value);
}
Id EmitFPRoundEven64(EmitContext& ctx, Id value) {
return ctx.OpRoundEven(ctx.F64[1], value);
}
Id EmitFPFloor16(EmitContext& ctx, Id value) {
return ctx.OpFloor(ctx.F16[1], value);
}
Id EmitFPFloor32(EmitContext& ctx, Id value) {
return ctx.OpFloor(ctx.F32[1], value);
}
Id EmitFPFloor64(EmitContext& ctx, Id value) {
return ctx.OpFloor(ctx.F64[1], value);
}
Id EmitFPCeil16(EmitContext& ctx, Id value) {
return ctx.OpCeil(ctx.F16[1], value);
}
Id EmitFPCeil32(EmitContext& ctx, Id value) {
return ctx.OpCeil(ctx.F32[1], value);
}
Id EmitFPCeil64(EmitContext& ctx, Id value) {
return ctx.OpCeil(ctx.F64[1], value);
}
Id EmitFPTrunc16(EmitContext& ctx, Id value) {
return ctx.OpTrunc(ctx.F16[1], value);
}
Id EmitFPTrunc32(EmitContext& ctx, Id value) {
return ctx.OpTrunc(ctx.F32[1], value);
}
Id EmitFPTrunc64(EmitContext& ctx, Id value) {
return ctx.OpTrunc(ctx.F64[1], value);
}
Id EmitFPOrdEqual16(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdEqual(ctx.U1, lhs, rhs);
}
Id EmitFPOrdEqual32(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdEqual(ctx.U1, lhs, rhs);
}
Id EmitFPOrdEqual64(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdEqual(ctx.U1, lhs, rhs);
}
Id EmitFPUnordEqual16(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordEqual, ctx, lhs, rhs);
}
Id EmitFPUnordEqual32(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordEqual, ctx, lhs, rhs);
}
Id EmitFPUnordEqual64(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordEqual, ctx, lhs, rhs);
}
Id EmitFPOrdNotEqual16(EmitContext& ctx, Id lhs, Id rhs) {
return FPOrdNotEqual(ctx, lhs, rhs);
}
Id EmitFPOrdNotEqual32(EmitContext& ctx, Id lhs, Id rhs) {
return FPOrdNotEqual(ctx, lhs, rhs);
}
Id EmitFPOrdNotEqual64(EmitContext& ctx, Id lhs, Id rhs) {
return FPOrdNotEqual(ctx, lhs, rhs);
}
Id EmitFPUnordNotEqual16(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFUnordNotEqual(ctx.U1, lhs, rhs);
}
Id EmitFPUnordNotEqual32(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFUnordNotEqual(ctx.U1, lhs, rhs);
}
Id EmitFPUnordNotEqual64(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFUnordNotEqual(ctx.U1, lhs, rhs);
}
Id EmitFPOrdLessThan16(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdLessThan(ctx.U1, lhs, rhs);
}
Id EmitFPOrdLessThan32(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdLessThan(ctx.U1, lhs, rhs);
}
Id EmitFPOrdLessThan64(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdLessThan(ctx.U1, lhs, rhs);
}
Id EmitFPUnordLessThan16(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordLessThan, ctx, lhs, rhs);
}
Id EmitFPUnordLessThan32(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordLessThan, ctx, lhs, rhs);
}
Id EmitFPUnordLessThan64(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordLessThan, ctx, lhs, rhs);
}
Id EmitFPOrdGreaterThan16(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdGreaterThan(ctx.U1, lhs, rhs);
}
Id EmitFPOrdGreaterThan32(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdGreaterThan(ctx.U1, lhs, rhs);
}
Id EmitFPOrdGreaterThan64(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdGreaterThan(ctx.U1, lhs, rhs);
}
Id EmitFPUnordGreaterThan16(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordGreaterThan, ctx, lhs, rhs);
}
Id EmitFPUnordGreaterThan32(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordGreaterThan, ctx, lhs, rhs);
}
Id EmitFPUnordGreaterThan64(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordGreaterThan, ctx, lhs, rhs);
}
Id EmitFPOrdLessThanEqual16(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdLessThanEqual(ctx.U1, lhs, rhs);
}
Id EmitFPOrdLessThanEqual32(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdLessThanEqual(ctx.U1, lhs, rhs);
}
Id EmitFPOrdLessThanEqual64(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdLessThanEqual(ctx.U1, lhs, rhs);
}
Id EmitFPUnordLessThanEqual16(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordLessThanEqual, ctx, lhs, rhs);
}
Id EmitFPUnordLessThanEqual32(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordLessThanEqual, ctx, lhs, rhs);
}
Id EmitFPUnordLessThanEqual64(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordLessThanEqual, ctx, lhs, rhs);
}
Id EmitFPOrdGreaterThanEqual16(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdGreaterThanEqual(ctx.U1, lhs, rhs);
}
Id EmitFPOrdGreaterThanEqual32(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdGreaterThanEqual(ctx.U1, lhs, rhs);
}
Id EmitFPOrdGreaterThanEqual64(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpFOrdGreaterThanEqual(ctx.U1, lhs, rhs);
}
Id EmitFPUnordGreaterThanEqual16(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordGreaterThanEqual, ctx, lhs, rhs);
}
Id EmitFPUnordGreaterThanEqual32(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordGreaterThanEqual, ctx, lhs, rhs);
}
Id EmitFPUnordGreaterThanEqual64(EmitContext& ctx, Id lhs, Id rhs) {
return FPUnordCompare(&EmitContext::OpFUnordGreaterThanEqual, ctx, lhs, rhs);
}
Id EmitFPIsNan16(EmitContext& ctx, Id value) {
return ctx.OpIsNan(ctx.U1, value);
}
Id EmitFPIsNan32(EmitContext& ctx, Id value) {
return ctx.OpIsNan(ctx.U1, value);
}
Id EmitFPIsNan64(EmitContext& ctx, Id value) {
return ctx.OpIsNan(ctx.U1, value);
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,838 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <boost/container/static_vector.hpp>
#include "common/settings.h"
#include "shader_recompiler/backend/spirv/emit_spirv.h"
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
#include "shader_recompiler/frontend/ir/modifiers.h"
namespace Shader::Backend::SPIRV {
namespace {
enum class NonUniformKind {
SampledImage,
StorageImage,
UniformTexelBuffer,
StorageTexelBuffer,
};
[[nodiscard]] bool IsNonUniformSupported(const Profile& profile, NonUniformKind kind) noexcept {
switch (kind) {
case NonUniformKind::SampledImage:
return profile.support_sampled_image_array_nonuniform_indexing;
case NonUniformKind::StorageImage:
return profile.support_storage_image_array_nonuniform_indexing;
case NonUniformKind::UniformTexelBuffer:
return profile.support_uniform_texel_buffer_array_nonuniform_indexing;
case NonUniformKind::StorageTexelBuffer:
return profile.support_storage_texel_buffer_array_nonuniform_indexing;
}
return false;
}
void DecorateNonUniform(EmitContext& ctx, Id object) {
if (ctx.non_uniform_ids.contains(object.value)) {
return;
}
ctx.Decorate(object, spv::Decoration::NonUniform);
ctx.non_uniform_ids.insert(object.value);
}
[[nodiscard]] bool MarkNonUniform(EmitContext& ctx, Id idx, const IR::Value& index,
NonUniformKind kind) {
if (index.IsImmediate() || !IsNonUniformSupported(ctx.profile, kind)) {
return false;
}
DecorateNonUniform(ctx, idx);
switch (kind) {
case NonUniformKind::SampledImage:
ctx.uses_nonuniform_sampled_image = true;
break;
case NonUniformKind::StorageImage:
ctx.uses_nonuniform_storage_image = true;
break;
case NonUniformKind::UniformTexelBuffer:
ctx.uses_nonuniform_uniform_texel_buffer = true;
break;
case NonUniformKind::StorageTexelBuffer:
ctx.uses_nonuniform_storage_texel_buffer = true;
break;
}
return true;
}
class ImageOperands {
public:
[[maybe_unused]] static constexpr bool ImageSampleOffsetAllowed = false;
[[maybe_unused]] static constexpr bool ImageGatherOffsetAllowed = true;
[[maybe_unused]] static constexpr bool ImageFetchOffsetAllowed = false;
[[maybe_unused]] static constexpr bool ImageGradientOffsetAllowed = false;
explicit ImageOperands(EmitContext& ctx, bool has_bias, bool has_lod, bool has_lod_clamp,
Id lod, const IR::Value& offset) {
if (has_bias) {
const Id bias{has_lod_clamp ? ctx.OpCompositeExtract(ctx.F32[1], lod, 0) : lod};
Add(spv::ImageOperandsMask::Bias, bias);
}
if (has_lod) {
const Id lod_value{has_lod_clamp ? ctx.OpCompositeExtract(ctx.F32[1], lod, 0) : lod};
Add(spv::ImageOperandsMask::Lod, lod_value);
}
AddOffset(ctx, offset, ImageSampleOffsetAllowed);
if (has_lod_clamp) {
const Id lod_clamp{has_bias ? ctx.OpCompositeExtract(ctx.F32[1], lod, 1) : lod};
Add(spv::ImageOperandsMask::MinLod, lod_clamp);
}
}
explicit ImageOperands(EmitContext& ctx, const IR::Value& offset, const IR::Value& offset2) {
if (offset2.IsEmpty()) {
AddOffset(ctx, offset, ImageGatherOffsetAllowed);
return;
}
const std::array values{offset.InstRecursive(), offset2.InstRecursive()};
if (!values[0]->AreAllArgsImmediates() || !values[1]->AreAllArgsImmediates()) {
LOG_WARNING(Shader_SPIRV, "Not all arguments in PTP are immediate, ignoring");
return;
}
const IR::Opcode opcode{values[0]->GetOpcode()};
if (opcode != values[1]->GetOpcode() || opcode != IR::Opcode::CompositeConstructU32x4) {
throw LogicError("Invalid PTP arguments");
}
auto read{[&](unsigned int a, unsigned int b) { return static_cast<s32>(values[a]->Arg(b).U32()); }};
const Id offsets{ctx.ConstantComposite(
ctx.TypeArray(ctx.S32[2], ctx.Const(4U)), ctx.SConst(read(0, 0), read(0, 1)),
ctx.SConst(read(0, 2), read(0, 3)), ctx.SConst(read(1, 0), read(1, 1)),
ctx.SConst(read(1, 2), read(1, 3)))};
Add(spv::ImageOperandsMask::ConstOffsets, offsets);
}
explicit ImageOperands(Id lod, Id ms) {
if (Sirit::ValidId(lod)) {
Add(spv::ImageOperandsMask::Lod, lod);
}
if (Sirit::ValidId(ms)) {
Add(spv::ImageOperandsMask::Sample, ms);
}
}
explicit ImageOperands(EmitContext& ctx, bool has_lod_clamp, Id derivatives,
u32 num_derivatives, const IR::Value& offset, Id lod_clamp) {
if (!Sirit::ValidId(derivatives)) {
throw LogicError("Derivatives must be present");
}
boost::container::static_vector<Id, 3> deriv_x_accum;
boost::container::static_vector<Id, 3> deriv_y_accum;
for (u32 i = 0; i < num_derivatives; ++i) {
deriv_x_accum.push_back(ctx.OpCompositeExtract(ctx.F32[1], derivatives, i * 2));
deriv_y_accum.push_back(ctx.OpCompositeExtract(ctx.F32[1], derivatives, i * 2 + 1));
}
const Id derivatives_X{ctx.OpCompositeConstruct(
ctx.F32[num_derivatives], std::span{deriv_x_accum.data(), deriv_x_accum.size()})};
const Id derivatives_Y{ctx.OpCompositeConstruct(
ctx.F32[num_derivatives], std::span{deriv_y_accum.data(), deriv_y_accum.size()})};
Add(spv::ImageOperandsMask::Grad, derivatives_X, derivatives_Y);
AddOffset(ctx, offset, ImageGradientOffsetAllowed);
if (has_lod_clamp) {
Add(spv::ImageOperandsMask::MinLod, lod_clamp);
}
}
explicit ImageOperands(EmitContext& ctx, bool has_lod_clamp, Id derivatives_1, Id derivatives_2,
const IR::Value& offset, Id lod_clamp) {
if (!Sirit::ValidId(derivatives_1) || !Sirit::ValidId(derivatives_2)) {
throw LogicError("Derivatives must be present");
}
boost::container::static_vector<Id, 3> deriv_1_accum{
ctx.OpCompositeExtract(ctx.F32[1], derivatives_1, 0),
ctx.OpCompositeExtract(ctx.F32[1], derivatives_1, 2),
ctx.OpCompositeExtract(ctx.F32[1], derivatives_2, 0),
};
boost::container::static_vector<Id, 3> deriv_2_accum{
ctx.OpCompositeExtract(ctx.F32[1], derivatives_1, 1),
ctx.OpCompositeExtract(ctx.F32[1], derivatives_1, 3),
ctx.OpCompositeExtract(ctx.F32[1], derivatives_2, 1),
};
const Id derivatives_id1{ctx.OpCompositeConstruct(
ctx.F32[3], std::span{deriv_1_accum.data(), deriv_1_accum.size()})};
const Id derivatives_id2{ctx.OpCompositeConstruct(
ctx.F32[3], std::span{deriv_2_accum.data(), deriv_2_accum.size()})};
Add(spv::ImageOperandsMask::Grad, derivatives_id1, derivatives_id2);
AddOffset(ctx, offset, ImageGradientOffsetAllowed);
if (has_lod_clamp) {
Add(spv::ImageOperandsMask::MinLod, lod_clamp);
}
}
std::span<const Id> Span() const noexcept {
return std::span{operands.data(), operands.size()};
}
std::optional<spv::ImageOperandsMask> MaskOptional() const noexcept {
return mask != spv::ImageOperandsMask{} ? std::make_optional(mask) : std::nullopt;
}
spv::ImageOperandsMask Mask() const noexcept {
return mask;
}
private:
void AddOffset(EmitContext& ctx, const IR::Value& offset, bool runtime_offset_allowed) {
if (offset.IsEmpty()) {
return;
}
if (offset.IsImmediate()) {
Add(spv::ImageOperandsMask::ConstOffset, ctx.SConst(static_cast<s32>(offset.U32())));
return;
}
IR::Inst* const inst{offset.InstRecursive()};
if (inst->AreAllArgsImmediates()) {
switch (inst->GetOpcode()) {
case IR::Opcode::CompositeConstructU32x2:
Add(spv::ImageOperandsMask::ConstOffset,
ctx.SConst(static_cast<s32>(inst->Arg(0).U32()),
static_cast<s32>(inst->Arg(1).U32())));
return;
case IR::Opcode::CompositeConstructU32x3:
Add(spv::ImageOperandsMask::ConstOffset,
ctx.SConst(static_cast<s32>(inst->Arg(0).U32()),
static_cast<s32>(inst->Arg(1).U32()),
static_cast<s32>(inst->Arg(2).U32())));
return;
case IR::Opcode::CompositeConstructU32x4:
Add(spv::ImageOperandsMask::ConstOffset,
ctx.SConst(static_cast<s32>(inst->Arg(0).U32()),
static_cast<s32>(inst->Arg(1).U32()),
static_cast<s32>(inst->Arg(2).U32()),
static_cast<s32>(inst->Arg(3).U32())));
return;
default:
break;
}
}
if (runtime_offset_allowed) {
Add(spv::ImageOperandsMask::Offset, ctx.Def(offset));
}
}
void Add(spv::ImageOperandsMask new_mask, Id value) {
mask = static_cast<spv::ImageOperandsMask>(static_cast<unsigned>(mask) |
static_cast<unsigned>(new_mask));
operands.push_back(value);
}
void Add(spv::ImageOperandsMask new_mask, Id value_1, Id value_2) {
mask = static_cast<spv::ImageOperandsMask>(static_cast<unsigned>(mask) |
static_cast<unsigned>(new_mask));
operands.push_back(value_1);
operands.push_back(value_2);
}
boost::container::static_vector<Id, 4> operands;
spv::ImageOperandsMask mask{};
};
Id Texture(EmitContext& ctx, IR::TextureInstInfo info, [[maybe_unused]] const IR::Value& index) {
const TextureDefinition& def{ctx.textures.at(info.descriptor_index)};
if (def.count > 1) {
auto const idx = index.IsImmediate() ? ctx.Const(index.U32()) : ctx.Def(index);
const bool non_uniform{MarkNonUniform(ctx, idx, index, NonUniformKind::SampledImage)};
const Id pointer{ctx.OpAccessChain(def.pointer_type, def.id, idx)};
const Id object{ctx.OpLoad(def.sampled_type, pointer)};
if (non_uniform) {
DecorateNonUniform(ctx, pointer);
DecorateNonUniform(ctx, object);
}
return object;
} else {
return ctx.OpLoad(def.sampled_type, def.id);
}
}
Id TextureImage(EmitContext& ctx, IR::TextureInstInfo info, const IR::Value& index) {
if (info.type == TextureType::Buffer) {
const TextureBufferDefinition& def{ctx.texture_buffers.at(info.descriptor_index)};
if (def.count > 1) {
const Id idx{index.IsImmediate() ? ctx.Const(index.U32()) : ctx.Def(index)};
const bool non_uniform{
MarkNonUniform(ctx, idx, index, NonUniformKind::UniformTexelBuffer)};
const Id ptr{ctx.OpAccessChain(ctx.image_buffer_type, def.id, idx)};
const Id object{ctx.OpLoad(ctx.image_buffer_type, ptr)};
if (non_uniform) {
DecorateNonUniform(ctx, ptr);
DecorateNonUniform(ctx, object);
}
return object;
}
return ctx.OpLoad(ctx.image_buffer_type, def.id);
} else {
const TextureDefinition& def{ctx.textures.at(info.descriptor_index)};
if (def.count > 1) {
auto const idx = index.IsImmediate() ? ctx.Const(index.U32()) : ctx.Def(index);
const bool non_uniform{MarkNonUniform(ctx, idx, index, NonUniformKind::SampledImage)};
const Id ptr = ctx.OpAccessChain(def.pointer_type, def.id, idx);
const Id object = ctx.OpLoad(def.sampled_type, ptr);
const Id image = ctx.OpImage(def.image_type, object);
if (non_uniform) {
DecorateNonUniform(ctx, ptr);
DecorateNonUniform(ctx, object);
DecorateNonUniform(ctx, image);
}
return image;
}
return ctx.OpImage(def.image_type, ctx.OpLoad(def.sampled_type, def.id));
}
}
std::pair<Id, bool> Image(EmitContext& ctx, const IR::Value& index, IR::TextureInstInfo info) {
if (info.type == TextureType::Buffer) {
const ImageBufferDefinition def{ctx.image_buffers.at(info.descriptor_index)};
if (def.count > 1) {
const Id idx{index.IsImmediate() ? ctx.Const(index.U32()) : ctx.Def(index)};
const bool non_uniform{
MarkNonUniform(ctx, idx, index, NonUniformKind::StorageTexelBuffer)};
const Id ptr{ctx.OpAccessChain(def.pointer_type, def.id, idx)};
const Id image{ctx.OpLoad(def.image_type, ptr)};
if (non_uniform) {
DecorateNonUniform(ctx, ptr);
DecorateNonUniform(ctx, image);
}
return {image, def.is_integer};
}
return {ctx.OpLoad(def.image_type, def.id), def.is_integer};
} else {
const ImageDefinition def{ctx.images.at(info.descriptor_index)};
if (def.count > 1) {
const Id idx{index.IsImmediate() ? ctx.Const(index.U32()) : ctx.Def(index)};
const bool non_uniform{MarkNonUniform(ctx, idx, index, NonUniformKind::StorageImage)};
const Id ptr{ctx.OpAccessChain(def.pointer_type, def.id, idx)};
const Id image{ctx.OpLoad(def.image_type, ptr)};
if (non_uniform) {
DecorateNonUniform(ctx, ptr);
DecorateNonUniform(ctx, image);
}
return {image, def.is_integer};
}
return {ctx.OpLoad(def.image_type, def.id), def.is_integer};
}
}
bool IsTextureMsaa(EmitContext& ctx, const IR::TextureInstInfo& info) {
if (info.type == TextureType::Buffer) {
return false;
}
return ctx.textures.at(info.descriptor_index).is_multisample;
}
bool IsTextureInteger(EmitContext& ctx, const IR::TextureInstInfo& info) {
if (info.type == TextureType::Buffer) {
return false;
}
return ctx.textures.at(info.descriptor_index).is_integer;
}
Id Decorate(EmitContext& ctx, IR::Inst* inst, Id sample) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
if (info.relaxed_precision != 0) {
ctx.Decorate(sample, spv::Decoration::RelaxedPrecision);
}
return sample;
}
template <typename MethodPtrType, typename... Args>
Id Emit(MethodPtrType sparse_ptr, MethodPtrType non_sparse_ptr, EmitContext& ctx, IR::Inst* inst,
Id result_type, Args&&... args) {
IR::Inst* const sparse{inst->GetAssociatedPseudoOperation(IR::Opcode::GetSparseFromOp)};
if (!sparse) {
return Decorate(ctx, inst, (ctx.*non_sparse_ptr)(result_type, std::forward<Args>(args)...));
}
const Id struct_type{ctx.TypeStruct(ctx.U32[1], result_type)};
const Id sample{(ctx.*sparse_ptr)(struct_type, std::forward<Args>(args)...)};
const Id resident_code{ctx.OpCompositeExtract(ctx.U32[1], sample, 0U)};
sparse->SetDefinition(ctx.OpImageSparseTexelsResident(ctx.U1, resident_code));
sparse->Invalidate();
Decorate(ctx, inst, sample);
return ctx.OpCompositeExtract(result_type, sample, 1U);
}
bool IsDynamicPtp(IR::Inst* inst, const IR::Value& offset, const IR::Value& offset2) {
if (offset2.IsEmpty() || offset.IsEmpty() ||
inst->GetAssociatedPseudoOperation(IR::Opcode::GetSparseFromOp)) {
return false;
}
const auto constant{[](const IR::Value& value) {
return value.IsImmediate() || value.InstRecursive()->AreAllArgsImmediates();
}};
return !constant(offset) || !constant(offset2);
}
// Per-tap offsets the IR could not fold to constants (ConstOffsets must be constant): one gather
// per tap with a runtime Offset, keeping the i0,j0 texel (w) of each footprint, which is the
// texel a per-tap gather returns for that tap.
template <typename GatherFn>
Id GatherDynamicPtp(EmitContext& ctx, bool is_integer, Id result_type, const IR::Value& offset,
const IR::Value& offset2, GatherFn&& gather) {
const std::array packed{ctx.Def(offset), ctx.Def(offset2)};
const Id component_type{is_integer ? ctx.U32[1] : ctx.F32[1]};
std::array<Id, 4> texels;
for (u32 tap = 0; tap < 4; ++tap) {
const Id vector{packed[tap / 2]};
const u32 first{(tap % 2) * 2};
const Id x{ctx.OpBitcast(ctx.S32[1], ctx.OpCompositeExtract(ctx.U32[1], vector, first))};
const Id y{ctx.OpBitcast(ctx.S32[1], ctx.OpCompositeExtract(ctx.U32[1], vector, first + 1))};
const Id tap_offset{ctx.OpCompositeConstruct(ctx.S32[2], x, y)};
texels[tap] = ctx.OpCompositeExtract(component_type, gather(tap_offset), 3U);
}
return ctx.OpCompositeConstruct(result_type, std::span<const Id>{texels.data(), texels.size()});
}
Id IsScaled(EmitContext& ctx, const IR::Value& index, Id member_index, u32 base_index) {
const Id push_constant_u32{ctx.TypePointer(spv::StorageClass::PushConstant, ctx.U32[1])};
Id bit{};
if (index.IsImmediate()) {
// Use BitwiseAnd instead of BitfieldExtract for better codegen on Nvidia OpenGL.
// LOP32I.NZ is used to set the predicate rather than BFE+ISETP.
const u32 index_value{index.U32() + base_index};
const Id word_index{ctx.Const(index_value / 32)};
const Id bit_index_mask{ctx.Const(1u << (index_value % 32))};
const Id pointer{ctx.OpAccessChain(push_constant_u32, ctx.rescaling_push_constants,
member_index, word_index)};
const Id word{ctx.OpLoad(ctx.U32[1], pointer)};
bit = ctx.OpBitwiseAnd(ctx.U32[1], word, bit_index_mask);
} else {
Id index_value{ctx.Def(index)};
if (base_index != 0) {
index_value = ctx.OpIAdd(ctx.U32[1], index_value, ctx.Const(base_index));
}
const Id word_index{ctx.OpShiftRightLogical(ctx.U32[1], index_value, ctx.Const(5u))};
const Id bit_index{ctx.OpBitwiseAnd(ctx.U32[1], index_value, ctx.Const(31u))};
const Id pointer{ctx.OpAccessChain(push_constant_u32, ctx.rescaling_push_constants,
member_index, word_index)};
const Id word{ctx.OpLoad(ctx.U32[1], pointer)};
const Id bit_index_mask{ctx.OpShiftLeftLogical(ctx.U32[1], ctx.Const(1u), bit_index)};
bit = ctx.OpBitwiseAnd(ctx.U32[1], word, bit_index_mask);
}
return ctx.OpINotEqual(ctx.U1, bit, ctx.u32_zero_value);
}
Id BitTest(EmitContext& ctx, Id mask, Id bit) {
const Id shifted{ctx.OpShiftRightLogical(ctx.U32[1], mask, bit)};
const Id bit_value{ctx.OpBitwiseAnd(ctx.U32[1], shifted, ctx.Const(1u))};
return ctx.OpINotEqual(ctx.U1, bit_value, ctx.u32_zero_value);
}
Id ImageGatherSubpixelOffset(EmitContext& ctx, const IR::TextureInstInfo& info, Id texture,
Id coords) {
// Apply a subpixel offset of 1/512 the texel size of the texture to ensure same rounding on
// AMD hardware as on Maxwell or other Nvidia architectures.
const auto calculate_coords{[&](size_t dim) {
const Id nudge{ctx.Const(0x1p-9f)};
const Id image_size{ctx.OpImageQuerySizeLod(ctx.U32[dim], texture, ctx.u32_zero_value)};
Id offset{dim == 2 ? ctx.ConstantComposite(ctx.F32[dim], nudge, nudge)
: ctx.ConstantComposite(ctx.F32[dim], nudge, nudge, ctx.f32_zero_value)};
offset = ctx.OpFDiv(ctx.F32[dim], offset, ctx.OpConvertUToF(ctx.F32[dim], image_size));
return ctx.OpFAdd(ctx.F32[dim], coords, offset);
}};
switch (info.type) {
case TextureType::Color2D:
case TextureType::Color2DRect:
return calculate_coords(2);
case TextureType::ColorArray2D:
case TextureType::ColorCube:
return calculate_coords(3);
default:
return coords;
}
}
void AddOffsetToCoordinates(EmitContext& ctx, const IR::TextureInstInfo& info, Id& coords,
Id offset) {
if (!Sirit::ValidId(offset)) {
return;
}
Id result_type{};
switch (info.type) {
case TextureType::Buffer:
case TextureType::Color1D: {
result_type = ctx.U32[1];
break;
}
case TextureType::ColorArray1D:
offset = ctx.OpCompositeConstruct(ctx.U32[2], offset, ctx.u32_zero_value);
[[fallthrough]];
case TextureType::Color2D:
case TextureType::Color2DRect: {
result_type = ctx.U32[2];
break;
}
case TextureType::ColorArray2D:
offset = ctx.OpCompositeConstruct(ctx.U32[3], ctx.OpCompositeExtract(ctx.U32[1], offset, 0),
ctx.OpCompositeExtract(ctx.U32[1], offset, 1),
ctx.u32_zero_value);
[[fallthrough]];
case TextureType::Color3D: {
result_type = ctx.U32[3];
break;
}
case TextureType::ColorCube:
case TextureType::ColorArrayCube:
return;
}
coords = ctx.OpIAdd(result_type, coords, offset);
}
} // Anonymous namespace
Id EmitBindlessImageSampleImplicitLod(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBindlessImageSampleExplicitLod(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBindlessImageSampleDrefImplicitLod(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBindlessImageSampleDrefExplicitLod(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBindlessImageGather(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBindlessImageGatherDref(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBindlessImageFetch(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBindlessImageQueryDimensions(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBindlessImageQueryLod(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBindlessImageGradient(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBindlessImageRead(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBindlessImageWrite(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBoundImageSampleImplicitLod(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBoundImageSampleExplicitLod(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBoundImageSampleDrefImplicitLod(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBoundImageSampleDrefExplicitLod(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBoundImageGather(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBoundImageGatherDref(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBoundImageFetch(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBoundImageQueryDimensions(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBoundImageQueryLod(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBoundImageGradient(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBoundImageRead(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitBoundImageWrite(EmitContext&) {
throw LogicError("Unreachable instruction");
}
Id EmitImageSampleImplicitLod(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id bias_lc, const IR::Value& offset) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
const bool is_integer{IsTextureInteger(ctx, info)};
const Id result_type{is_integer ? ctx.U32[4] : ctx.F32[4]};
Id color;
if (ctx.stage == Stage::Fragment) {
const ImageOperands operands(ctx, info.has_bias != 0, false, info.has_lod_clamp != 0,
bias_lc, offset);
color = Emit(&EmitContext::OpImageSparseSampleImplicitLod,
&EmitContext::OpImageSampleImplicitLod, ctx, inst, result_type,
Texture(ctx, info, index), coords, operands.MaskOptional(), operands.Span());
} else {
// We can't use implicit lods on non-fragment stages on SPIR-V. Maxwell hardware behaves as
// if the lod was explicitly zero. This may change on Turing with implicit compute
// derivatives
const Id lod{ctx.Const(0.0f)};
const ImageOperands operands(ctx, false, true, info.has_lod_clamp != 0, lod, offset);
color = Emit(&EmitContext::OpImageSparseSampleExplicitLod,
&EmitContext::OpImageSampleExplicitLod, ctx, inst, result_type,
Texture(ctx, info, index), coords, operands.Mask(), operands.Span());
}
return is_integer ? ctx.OpBitcast(ctx.F32[4], color) : color;
}
Id EmitImageSampleExplicitLod(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id lod, const IR::Value& offset) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
const bool is_integer{IsTextureInteger(ctx, info)};
const Id result_type{is_integer ? ctx.U32[4] : ctx.F32[4]};
const ImageOperands operands(ctx, false, true, false, lod, offset);
Id result = Emit(&EmitContext::OpImageSparseSampleExplicitLod,
&EmitContext::OpImageSampleExplicitLod, ctx, inst, result_type,
Texture(ctx, info, index), coords, operands.Mask(), operands.Span());
#ifdef __ANDROID__
if (!is_integer && Settings::values.fix_bloom_effects.GetValue()) {
result = ctx.OpVectorTimesScalar(ctx.F32[4], result, ctx.Const(0.98f));
}
#endif
return is_integer ? ctx.OpBitcast(ctx.F32[4], result) : result;
}
Id EmitImageSampleDrefImplicitLod(EmitContext& ctx, IR::Inst* inst, const IR::Value& index,
Id coords, Id dref, Id bias_lc, const IR::Value& offset) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
if (ctx.stage == Stage::Fragment) {
const ImageOperands operands(ctx, info.has_bias != 0, false, info.has_lod_clamp != 0,
bias_lc, offset);
return Emit(&EmitContext::OpImageSparseSampleDrefImplicitLod,
&EmitContext::OpImageSampleDrefImplicitLod, ctx, inst, ctx.F32[1],
Texture(ctx, info, index), coords, dref, operands.MaskOptional(),
operands.Span());
} else {
// Implicit lods in compute behave on hardware as if sampling from LOD 0.
// This check is to ensure all drivers behave this way.
const Id lod{ctx.Const(0.0f)};
const ImageOperands operands(ctx, false, true, false, lod, offset);
return Emit(&EmitContext::OpImageSparseSampleDrefExplicitLod,
&EmitContext::OpImageSampleDrefExplicitLod, ctx, inst, ctx.F32[1],
Texture(ctx, info, index), coords, dref, operands.Mask(), operands.Span());
}
}
Id EmitImageSampleDrefExplicitLod(EmitContext& ctx, IR::Inst* inst, const IR::Value& index,
Id coords, Id dref, Id lod, const IR::Value& offset) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
const ImageOperands operands(ctx, false, true, false, lod, offset);
return Emit(&EmitContext::OpImageSparseSampleDrefExplicitLod,
&EmitContext::OpImageSampleDrefExplicitLod, ctx, inst, ctx.F32[1],
Texture(ctx, info, index), coords, dref, operands.Mask(), operands.Span());
}
Id EmitImageGather(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
const IR::Value& offset, const IR::Value& offset2) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
const bool is_integer{IsTextureInteger(ctx, info)};
const Id result_type{is_integer ? ctx.U32[4] : ctx.F32[4]};
if (ctx.profile.need_gather_subpixel_offset) {
coords = ImageGatherSubpixelOffset(ctx, info, TextureImage(ctx, info, index), coords);
}
if (IsDynamicPtp(inst, offset, offset2)) {
const Id texture{Texture(ctx, info, index)};
const Id component{ctx.Const(info.gather_component)};
const Id color{GatherDynamicPtp(ctx, is_integer, result_type, offset, offset2, [&](Id tap_offset) {
return Decorate(ctx, inst,
ctx.OpImageGather(result_type, texture, coords, component,
spv::ImageOperandsMask::Offset,
std::span<const Id>{&tap_offset, 1}));
})};
return is_integer ? ctx.OpBitcast(ctx.F32[4], color) : color;
}
const ImageOperands operands(ctx, offset, offset2);
const Id color{Emit(&EmitContext::OpImageSparseGather, &EmitContext::OpImageGather, ctx, inst,
result_type, Texture(ctx, info, index), coords,
ctx.Const(info.gather_component), operands.MaskOptional(),
operands.Span())};
return is_integer ? ctx.OpBitcast(ctx.F32[4], color) : color;
}
Id EmitImageGatherDref(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
const IR::Value& offset, const IR::Value& offset2, Id dref) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
const bool is_integer{IsTextureInteger(ctx, info)};
const Id result_type{is_integer ? ctx.U32[4] : ctx.F32[4]};
if (ctx.profile.need_gather_subpixel_offset) {
coords = ImageGatherSubpixelOffset(ctx, info, TextureImage(ctx, info, index), coords);
}
if (IsDynamicPtp(inst, offset, offset2)) {
const Id texture{Texture(ctx, info, index)};
const Id color{GatherDynamicPtp(ctx, is_integer, result_type, offset, offset2, [&](Id tap_offset) {
return Decorate(ctx, inst,
ctx.OpImageDrefGather(result_type, texture, coords, dref,
spv::ImageOperandsMask::Offset,
std::span<const Id>{&tap_offset, 1}));
})};
return is_integer ? ctx.OpBitcast(ctx.F32[4], color) : color;
}
const ImageOperands operands(ctx, offset, offset2);
const Id color{Emit(&EmitContext::OpImageSparseDrefGather, &EmitContext::OpImageDrefGather,
ctx, inst, result_type, Texture(ctx, info, index), coords, dref,
operands.MaskOptional(), operands.Span())};
return is_integer ? ctx.OpBitcast(ctx.F32[4], color) : color;
}
Id EmitImageFetch(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords, Id offset,
Id lod, Id ms) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
const bool is_integer{IsTextureInteger(ctx, info)};
const Id result_type{is_integer ? ctx.U32[4] : ctx.F32[4]};
AddOffsetToCoordinates(ctx, info, coords, offset);
if (info.type == TextureType::Buffer) {
lod = Id{};
}
if (Sirit::ValidId(ms)) {
// This image is multisampled, lod must be implicit
lod = Id{};
}
const ImageOperands operands(lod, ms);
const Id color{Emit(&EmitContext::OpImageSparseFetch, &EmitContext::OpImageFetch, ctx, inst,
result_type, TextureImage(ctx, info, index), coords,
operands.MaskOptional(), operands.Span())};
return is_integer ? ctx.OpBitcast(ctx.F32[4], color) : color;
}
Id EmitImageQueryDimensions(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id lod,
const IR::Value& skip_mips_val) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
const Id image{TextureImage(ctx, info, index)};
const Id zero{ctx.u32_zero_value};
const bool skip_mips{skip_mips_val.U1()};
const auto mips{[&] { return skip_mips ? zero : ctx.OpImageQueryLevels(ctx.U32[1], image); }};
const bool is_msaa{IsTextureMsaa(ctx, info)};
const bool uses_lod{!is_msaa && info.type != TextureType::Buffer};
const auto query{[&](Id type) {
return uses_lod ? ctx.OpImageQuerySizeLod(type, image, lod)
: ctx.OpImageQuerySize(type, image);
}};
switch (info.type) {
case TextureType::Color1D:
return ctx.OpCompositeConstruct(ctx.U32[4], query(ctx.U32[1]), zero, zero, mips());
case TextureType::ColorArray1D:
case TextureType::Color2D:
case TextureType::ColorCube:
case TextureType::Color2DRect:
return ctx.OpCompositeConstruct(ctx.U32[4], query(ctx.U32[2]), zero, mips());
case TextureType::ColorArray2D:
case TextureType::Color3D:
case TextureType::ColorArrayCube:
return ctx.OpCompositeConstruct(ctx.U32[4], query(ctx.U32[3]), mips());
case TextureType::Buffer:
return ctx.OpCompositeConstruct(ctx.U32[4], query(ctx.U32[1]), zero, zero, mips());
}
throw LogicError("Unspecified image type {}", info.type.Value());
}
Id EmitImageQueryLod(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
const Id zero{ctx.f32_zero_value};
const Id sampler{Texture(ctx, info, index)};
return ctx.OpCompositeConstruct(ctx.F32[4], ctx.OpImageQueryLod(ctx.F32[2], sampler, coords),
zero, zero);
}
Id EmitImageGradient(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id derivatives, const IR::Value& offset, Id lod_clamp) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
const bool is_integer{IsTextureInteger(ctx, info)};
const Id result_type{is_integer ? ctx.U32[4] : ctx.F32[4]};
const auto operands = info.num_derivatives == 3
? ImageOperands(ctx, info.has_lod_clamp != 0, derivatives,
ctx.Def(offset), {}, lod_clamp)
: ImageOperands(ctx, info.has_lod_clamp != 0, derivatives,
info.num_derivatives, offset, lod_clamp);
const Id color{Emit(&EmitContext::OpImageSparseSampleExplicitLod,
&EmitContext::OpImageSampleExplicitLod, ctx, inst, result_type,
Texture(ctx, info, index), coords, operands.Mask(), operands.Span())};
return is_integer ? ctx.OpBitcast(ctx.F32[4], color) : color;
}
Id EmitImageRead(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
if (info.image_format == ImageFormat::Typeless && !ctx.profile.support_typeless_image_loads) {
LOG_WARNING(Shader_SPIRV, "Typeless image read not supported by host");
return ctx.ConstantNull(ctx.U32[4]);
}
const auto [image, is_integer] = Image(ctx, index, info);
const Id result_type{is_integer ? ctx.U32[4] : ctx.F32[4]};
Id color{Emit(&EmitContext::OpImageSparseRead, &EmitContext::OpImageRead, ctx, inst,
result_type, image, coords, std::nullopt, std::span<const Id>{})};
if (!is_integer) {
color = ctx.OpBitcast(ctx.U32[4], color);
}
return color;
}
void EmitImageWrite(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords, Id color) {
const auto info{inst->Flags<IR::TextureInstInfo>()};
const auto [image, is_integer] = Image(ctx, index, info);
if (!is_integer) {
color = ctx.OpBitcast(ctx.F32[4], color);
}
ctx.OpImageWrite(image, coords, color);
}
Id EmitIsTextureScaled(EmitContext& ctx, const IR::Value& index) {
if (ctx.profile.unified_descriptor_binding) {
const Id member_index{ctx.Const(ctx.rescaling_textures_member_index)};
return IsScaled(ctx, index, member_index, ctx.texture_rescaling_index);
} else {
const Id composite{ctx.OpLoad(ctx.F32[4], ctx.rescaling_uniform_constant)};
const Id mask_f32{ctx.OpCompositeExtract(ctx.F32[1], composite, 0u)};
const Id mask{ctx.OpBitcast(ctx.U32[1], mask_f32)};
return BitTest(ctx, mask, ctx.Def(index));
}
}
Id EmitIsImageScaled(EmitContext& ctx, const IR::Value& index) {
if (ctx.profile.unified_descriptor_binding) {
const Id member_index{ctx.Const(ctx.rescaling_images_member_index)};
return IsScaled(ctx, index, member_index, ctx.image_rescaling_index);
} else {
const Id composite{ctx.OpLoad(ctx.F32[4], ctx.rescaling_uniform_constant)};
const Id mask_f32{ctx.OpCompositeExtract(ctx.F32[1], composite, 1u)};
const Id mask{ctx.OpBitcast(ctx.U32[1], mask_f32)};
return BitTest(ctx, mask, ctx.Def(index));
}
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,183 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
#include "shader_recompiler/frontend/ir/modifiers.h"
namespace Shader::Backend::SPIRV {
namespace {
Id Image(EmitContext& ctx, IR::TextureInstInfo info) {
if (info.type == TextureType::Buffer) {
const ImageBufferDefinition def{ctx.image_buffers.at(info.descriptor_index)};
return def.id;
} else {
const ImageDefinition def{ctx.images.at(info.descriptor_index)};
return def.id;
}
}
std::pair<Id, Id> AtomicArgs(EmitContext& ctx) {
const Id scope{ctx.Const(static_cast<u32>(spv::Scope::Device))};
const Id semantics{ctx.u32_zero_value};
return {scope, semantics};
}
Id ImageAtomicU32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords, Id value,
Id (Sirit::Module::*atomic_func)(Id, Id, Id, Id, Id)) {
if (!index.IsImmediate() || index.U32() != 0) {
// TODO: handle layers
throw NotImplementedException("Image indexing");
}
const auto info{inst->Flags<IR::TextureInstInfo>()};
const Id image{Image(ctx, info)};
const Id pointer{ctx.OpImageTexelPointer(ctx.image_u32, image, coords, ctx.Const(0U))};
const auto [scope, semantics]{AtomicArgs(ctx)};
return (ctx.*atomic_func)(ctx.U32[1], pointer, scope, semantics, value);
}
} // Anonymous namespace
Id EmitImageAtomicIAdd32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value) {
return ImageAtomicU32(ctx, inst, index, coords, value, &Sirit::Module::OpAtomicIAdd);
}
Id EmitImageAtomicSMin32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value) {
return ImageAtomicU32(ctx, inst, index, coords, value, &Sirit::Module::OpAtomicSMin);
}
Id EmitImageAtomicUMin32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value) {
return ImageAtomicU32(ctx, inst, index, coords, value, &Sirit::Module::OpAtomicUMin);
}
Id EmitImageAtomicSMax32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value) {
return ImageAtomicU32(ctx, inst, index, coords, value, &Sirit::Module::OpAtomicSMax);
}
Id EmitImageAtomicUMax32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value) {
return ImageAtomicU32(ctx, inst, index, coords, value, &Sirit::Module::OpAtomicUMax);
}
Id EmitImageAtomicInc32(EmitContext&, IR::Inst*, const IR::Value&, Id, Id) {
// TODO: This is not yet implemented
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitImageAtomicDec32(EmitContext&, IR::Inst*, const IR::Value&, Id, Id) {
// TODO: This is not yet implemented
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitImageAtomicAnd32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value) {
return ImageAtomicU32(ctx, inst, index, coords, value, &Sirit::Module::OpAtomicAnd);
}
Id EmitImageAtomicOr32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value) {
return ImageAtomicU32(ctx, inst, index, coords, value, &Sirit::Module::OpAtomicOr);
}
Id EmitImageAtomicXor32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value) {
return ImageAtomicU32(ctx, inst, index, coords, value, &Sirit::Module::OpAtomicXor);
}
Id EmitImageAtomicExchange32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value) {
return ImageAtomicU32(ctx, inst, index, coords, value, &Sirit::Module::OpAtomicExchange);
}
Id EmitBindlessImageAtomicIAdd32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBindlessImageAtomicSMin32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBindlessImageAtomicUMin32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBindlessImageAtomicSMax32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBindlessImageAtomicUMax32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBindlessImageAtomicInc32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBindlessImageAtomicDec32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBindlessImageAtomicAnd32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBindlessImageAtomicOr32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBindlessImageAtomicXor32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBindlessImageAtomicExchange32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBoundImageAtomicIAdd32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBoundImageAtomicSMin32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBoundImageAtomicUMin32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBoundImageAtomicSMax32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBoundImageAtomicUMax32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBoundImageAtomicInc32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBoundImageAtomicDec32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBoundImageAtomicAnd32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBoundImageAtomicOr32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBoundImageAtomicXor32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitBoundImageAtomicExchange32(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,647 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <sirit/sirit.h>
#include "common/common_types.h"
namespace Shader::IR {
enum class Attribute : u64;
enum class Patch : u64;
class Inst;
class Value;
} // namespace Shader::IR
namespace Shader::Backend::SPIRV {
using Sirit::Id;
class EmitContext;
// Microinstruction emitters
Id EmitPhi(EmitContext& ctx, IR::Inst* inst);
void EmitVoid(EmitContext& ctx);
Id EmitIdentity(EmitContext& ctx, const IR::Value& value);
Id EmitConditionRef(EmitContext& ctx, const IR::Value& value);
void EmitReference(EmitContext&);
void EmitPhiMove(EmitContext&);
void EmitJoin(EmitContext& ctx);
void EmitDemoteToHelperInvocation(EmitContext& ctx);
void EmitBarrier(EmitContext& ctx);
void EmitSubgroupBarrier(EmitContext& ctx);
void EmitWorkgroupMemoryBarrier(EmitContext& ctx);
void EmitDeviceMemoryBarrier(EmitContext& ctx);
void EmitPrologue(EmitContext& ctx);
void EmitEpilogue(EmitContext& ctx);
void EmitEmitVertex(EmitContext& ctx, const IR::Value& stream);
void EmitEndPrimitive(EmitContext& ctx, const IR::Value& stream);
void EmitGetRegister(EmitContext& ctx);
void EmitSetRegister(EmitContext& ctx);
void EmitGetPred(EmitContext& ctx);
void EmitSetPred(EmitContext& ctx);
void EmitSetGotoVariable(EmitContext& ctx);
void EmitGetGotoVariable(EmitContext& ctx);
void EmitSetIndirectBranchVariable(EmitContext& ctx);
void EmitGetIndirectBranchVariable(EmitContext& ctx);
Id EmitGetCbufU8(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitGetCbufS8(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitGetCbufU16(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitGetCbufS16(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitGetCbufU32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitGetCbufF32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitGetCbufU32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitGetAttribute(EmitContext& ctx, IR::Attribute attr, Id vertex);
Id EmitGetAttributeU32(EmitContext& ctx, IR::Attribute attr, Id vertex);
void EmitSetAttribute(EmitContext& ctx, IR::Attribute attr, Id value, Id vertex);
Id EmitGetAttributeIndexed(EmitContext& ctx, Id offset, Id vertex);
void EmitSetAttributeIndexed(EmitContext& ctx, Id offset, Id value, Id vertex);
Id EmitGetPatch(EmitContext& ctx, IR::Patch patch);
void EmitSetPatch(EmitContext& ctx, IR::Patch patch, Id value);
Id EmitIsbeRead(EmitContext& ctx, Id address);
void EmitIsbeWriteAttribute(EmitContext& ctx, Id address, Id value);
void EmitIsbeWriteMapU8(EmitContext& ctx, Id address, Id value);
void EmitIsbeWriteMapU32(EmitContext& ctx, const IR::Value& address, Id value);
void EmitMeshPrologue(EmitContext& ctx);
void EmitMeshEpilogue(EmitContext& ctx);
// (hookshot) a GPU without mesh shaders: the generated vertex stage that reads the mesh epilogue's records
void EmitMeshFallbackVertexBody(EmitContext& ctx);
void EmitSetFragColor(EmitContext& ctx, u32 index, u32 component, Id value);
void EmitSetSampleMask(EmitContext& ctx, Id value);
void EmitSetFragDepth(EmitContext& ctx, Id value);
void EmitGetZFlag(EmitContext& ctx);
void EmitGetSFlag(EmitContext& ctx);
void EmitGetCFlag(EmitContext& ctx);
void EmitGetOFlag(EmitContext& ctx);
void EmitSetZFlag(EmitContext& ctx);
void EmitSetSFlag(EmitContext& ctx);
void EmitSetCFlag(EmitContext& ctx);
void EmitSetOFlag(EmitContext& ctx);
Id EmitWorkgroupId(EmitContext& ctx);
Id EmitLocalInvocationId(EmitContext& ctx);
Id EmitInvocationId(EmitContext& ctx);
Id EmitInvocationInfo(EmitContext& ctx);
Id EmitSampleId(EmitContext& ctx);
Id EmitIsHelperInvocation(EmitContext& ctx);
Id EmitSR_WScaleFactorXY(EmitContext& ctx);
Id EmitSR_WScaleFactorZ(EmitContext& ctx);
Id EmitYDirection(EmitContext& ctx);
Id EmitResolutionDownFactor(EmitContext& ctx);
Id EmitRenderArea(EmitContext& ctx);
Id EmitLoadLocal(EmitContext& ctx, Id word_offset);
void EmitWriteLocal(EmitContext& ctx, Id word_offset, Id value);
Id EmitUndefU1(EmitContext& ctx);
Id EmitUndefU8(EmitContext& ctx);
Id EmitUndefU16(EmitContext& ctx);
Id EmitUndefU32(EmitContext& ctx);
Id EmitUndefU64(EmitContext& ctx);
Id EmitLoadGlobalU8(EmitContext& ctx, Id address);
Id EmitLoadGlobalS8(EmitContext& ctx, Id address);
Id EmitLoadGlobalU16(EmitContext& ctx, Id address);
Id EmitLoadGlobalS16(EmitContext& ctx, Id address);
Id EmitLoadGlobal32(EmitContext& ctx, Id address);
Id EmitLoadGlobal64(EmitContext& ctx, Id address);
Id EmitLoadGlobal128(EmitContext& ctx, Id address);
void EmitWriteGlobalU8(EmitContext& ctx, Id address, Id value);
void EmitWriteGlobalS8(EmitContext& ctx, Id address, Id value);
void EmitWriteGlobalU16(EmitContext& ctx, Id address, Id value);
void EmitWriteGlobalS16(EmitContext& ctx, Id address, Id value);
void EmitWriteGlobal32(EmitContext& ctx, Id address, Id value);
void EmitWriteGlobal64(EmitContext& ctx, Id address, Id value);
void EmitWriteGlobal128(EmitContext& ctx, Id address, Id value);
Id EmitLoadStorageU8(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitLoadStorageS8(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitLoadStorageU16(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitLoadStorageS16(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitLoadStorage32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitLoadStorage64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
Id EmitLoadStorage128(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset);
void EmitWriteStorageU8(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
void EmitWriteStorageS8(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
void EmitWriteStorageU16(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
void EmitWriteStorageS16(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
void EmitWriteStorage32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
void EmitWriteStorage64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
void EmitWriteStorage128(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitLoadSharedU8(EmitContext& ctx, Id offset);
Id EmitLoadSharedS8(EmitContext& ctx, Id offset);
Id EmitLoadSharedU16(EmitContext& ctx, Id offset);
Id EmitLoadSharedS16(EmitContext& ctx, Id offset);
Id EmitLoadSharedU32(EmitContext& ctx, Id offset);
Id EmitLoadSharedU64(EmitContext& ctx, Id offset);
Id EmitLoadSharedU128(EmitContext& ctx, Id offset);
void EmitWriteSharedU8(EmitContext& ctx, Id offset, Id value);
void EmitWriteSharedU16(EmitContext& ctx, Id offset, Id value);
void EmitWriteSharedU32(EmitContext& ctx, Id offset, Id value);
void EmitWriteSharedU64(EmitContext& ctx, Id offset, Id value);
void EmitWriteSharedU128(EmitContext& ctx, Id offset, Id value);
Id EmitCompositeConstructU32x2(EmitContext& ctx, Id e1, Id e2);
Id EmitCompositeConstructU32x3(EmitContext& ctx, Id e1, Id e2, Id e3);
Id EmitCompositeConstructU32x4(EmitContext& ctx, Id e1, Id e2, Id e3, Id e4);
Id EmitCompositeExtractU32x2(EmitContext& ctx, Id composite, u32 index);
Id EmitCompositeExtractU32x3(EmitContext& ctx, Id composite, u32 index);
Id EmitCompositeExtractU32x4(EmitContext& ctx, Id composite, u32 index);
Id EmitCompositeInsertU32x2(EmitContext& ctx, Id composite, Id object, u32 index);
Id EmitCompositeInsertU32x3(EmitContext& ctx, Id composite, Id object, u32 index);
Id EmitCompositeInsertU32x4(EmitContext& ctx, Id composite, Id object, u32 index);
Id EmitCompositeConstructF16x2(EmitContext& ctx, Id e1, Id e2);
Id EmitCompositeConstructF16x3(EmitContext& ctx, Id e1, Id e2, Id e3);
Id EmitCompositeConstructF16x4(EmitContext& ctx, Id e1, Id e2, Id e3, Id e4);
Id EmitCompositeExtractF16x2(EmitContext& ctx, Id composite, u32 index);
Id EmitCompositeExtractF16x3(EmitContext& ctx, Id composite, u32 index);
Id EmitCompositeExtractF16x4(EmitContext& ctx, Id composite, u32 index);
Id EmitCompositeInsertF16x2(EmitContext& ctx, Id composite, Id object, u32 index);
Id EmitCompositeInsertF16x3(EmitContext& ctx, Id composite, Id object, u32 index);
Id EmitCompositeInsertF16x4(EmitContext& ctx, Id composite, Id object, u32 index);
Id EmitCompositeConstructF32x2(EmitContext& ctx, Id e1, Id e2);
Id EmitCompositeConstructF32x3(EmitContext& ctx, Id e1, Id e2, Id e3);
Id EmitCompositeConstructF32x4(EmitContext& ctx, Id e1, Id e2, Id e3, Id e4);
Id EmitCompositeExtractF32x2(EmitContext& ctx, Id composite, u32 index);
Id EmitCompositeExtractF32x3(EmitContext& ctx, Id composite, u32 index);
Id EmitCompositeExtractF32x4(EmitContext& ctx, Id composite, u32 index);
Id EmitCompositeInsertF32x2(EmitContext& ctx, Id composite, Id object, u32 index);
Id EmitCompositeInsertF32x3(EmitContext& ctx, Id composite, Id object, u32 index);
Id EmitCompositeInsertF32x4(EmitContext& ctx, Id composite, Id object, u32 index);
void EmitCompositeConstructF64x2(EmitContext& ctx);
void EmitCompositeConstructF64x3(EmitContext& ctx);
void EmitCompositeConstructF64x4(EmitContext& ctx);
void EmitCompositeExtractF64x2(EmitContext& ctx);
void EmitCompositeExtractF64x3(EmitContext& ctx);
void EmitCompositeExtractF64x4(EmitContext& ctx);
Id EmitCompositeInsertF64x2(EmitContext& ctx, Id composite, Id object, u32 index);
Id EmitCompositeInsertF64x3(EmitContext& ctx, Id composite, Id object, u32 index);
Id EmitCompositeInsertF64x4(EmitContext& ctx, Id composite, Id object, u32 index);
Id EmitSelectU1(EmitContext& ctx, Id cond, Id true_value, Id false_value);
Id EmitSelectU8(EmitContext& ctx, Id cond, Id true_value, Id false_value);
Id EmitSelectU16(EmitContext& ctx, Id cond, Id true_value, Id false_value);
Id EmitSelectU32(EmitContext& ctx, Id cond, Id true_value, Id false_value);
Id EmitSelectU64(EmitContext& ctx, Id cond, Id true_value, Id false_value);
Id EmitSelectF16(EmitContext& ctx, Id cond, Id true_value, Id false_value);
Id EmitSelectF32(EmitContext& ctx, Id cond, Id true_value, Id false_value);
Id EmitSelectF64(EmitContext& ctx, Id cond, Id true_value, Id false_value);
void EmitBitCastU16F16(EmitContext& ctx);
Id EmitBitCastU32F32(EmitContext& ctx, Id value);
void EmitBitCastU64F64(EmitContext& ctx);
void EmitBitCastF16U16(EmitContext&);
Id EmitBitCastF32U32(EmitContext& ctx, Id value);
void EmitBitCastF64U64(EmitContext& ctx);
Id EmitPackUint2x32(EmitContext& ctx, Id value);
Id EmitUnpackUint2x32(EmitContext& ctx, Id value);
Id EmitPackFloat2x16(EmitContext& ctx, Id value);
Id EmitUnpackFloat2x16(EmitContext& ctx, Id value);
Id EmitPackHalf2x16(EmitContext& ctx, Id value);
Id EmitUnpackHalf2x16(EmitContext& ctx, Id value);
Id EmitPackDouble2x32(EmitContext& ctx, Id value);
Id EmitUnpackDouble2x32(EmitContext& ctx, Id value);
void EmitGetZeroFromOp(EmitContext& ctx);
void EmitGetSignFromOp(EmitContext& ctx);
void EmitGetCarryFromOp(EmitContext& ctx);
void EmitGetOverflowFromOp(EmitContext& ctx);
void EmitGetSparseFromOp(EmitContext& ctx);
void EmitGetInBoundsFromOp(EmitContext& ctx);
Id EmitFPAbs16(EmitContext& ctx, Id value);
Id EmitFPAbs32(EmitContext& ctx, Id value);
Id EmitFPAbs64(EmitContext& ctx, Id value);
Id EmitFPAdd16(EmitContext& ctx, IR::Inst* inst, Id a, Id b);
Id EmitFPAdd32(EmitContext& ctx, IR::Inst* inst, Id a, Id b);
Id EmitFPAdd64(EmitContext& ctx, IR::Inst* inst, Id a, Id b);
Id EmitFPFma16(EmitContext& ctx, IR::Inst* inst, Id a, Id b, Id c);
Id EmitFPFma32(EmitContext& ctx, IR::Inst* inst, Id a, Id b, Id c);
Id EmitFPFma64(EmitContext& ctx, IR::Inst* inst, Id a, Id b, Id c);
Id EmitFPMax32(EmitContext& ctx, Id a, Id b);
Id EmitFPMax64(EmitContext& ctx, Id a, Id b);
Id EmitFPMin32(EmitContext& ctx, Id a, Id b);
Id EmitFPMin64(EmitContext& ctx, Id a, Id b);
Id EmitFPMul16(EmitContext& ctx, IR::Inst* inst, Id a, Id b);
Id EmitFPMul32(EmitContext& ctx, IR::Inst* inst, Id a, Id b);
Id EmitFPMul64(EmitContext& ctx, IR::Inst* inst, Id a, Id b);
Id EmitFPNeg16(EmitContext& ctx, Id value);
Id EmitFPNeg32(EmitContext& ctx, Id value);
Id EmitFPNeg64(EmitContext& ctx, Id value);
Id EmitFPSin(EmitContext& ctx, Id value);
Id EmitFPCos(EmitContext& ctx, Id value);
Id EmitFPExp2(EmitContext& ctx, Id value);
Id EmitFPLog2(EmitContext& ctx, Id value);
Id EmitFPRecip32(EmitContext& ctx, Id value);
Id EmitFPRecip64(EmitContext& ctx, Id value);
Id EmitFPRecipSqrt32(EmitContext& ctx, Id value);
Id EmitFPRecipSqrt64(EmitContext& ctx, Id value);
Id EmitFPSqrt(EmitContext& ctx, Id value);
Id EmitFPSaturate16(EmitContext& ctx, Id value);
Id EmitFPSaturate32(EmitContext& ctx, Id value);
Id EmitFPSaturate64(EmitContext& ctx, Id value);
Id EmitFPClamp16(EmitContext& ctx, Id value, Id min_value, Id max_value);
Id EmitFPClamp32(EmitContext& ctx, Id value, Id min_value, Id max_value);
Id EmitFPClamp64(EmitContext& ctx, Id value, Id min_value, Id max_value);
Id EmitFPRoundEven16(EmitContext& ctx, Id value);
Id EmitFPRoundEven32(EmitContext& ctx, Id value);
Id EmitFPRoundEven64(EmitContext& ctx, Id value);
Id EmitFPFloor16(EmitContext& ctx, Id value);
Id EmitFPFloor32(EmitContext& ctx, Id value);
Id EmitFPFloor64(EmitContext& ctx, Id value);
Id EmitFPCeil16(EmitContext& ctx, Id value);
Id EmitFPCeil32(EmitContext& ctx, Id value);
Id EmitFPCeil64(EmitContext& ctx, Id value);
Id EmitFPTrunc16(EmitContext& ctx, Id value);
Id EmitFPTrunc32(EmitContext& ctx, Id value);
Id EmitFPTrunc64(EmitContext& ctx, Id value);
Id EmitFPOrdEqual16(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdEqual32(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdEqual64(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordEqual16(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordEqual32(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordEqual64(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdNotEqual16(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdNotEqual32(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdNotEqual64(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordNotEqual16(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordNotEqual32(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordNotEqual64(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdLessThan16(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdLessThan32(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdLessThan64(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordLessThan16(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordLessThan32(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordLessThan64(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdGreaterThan16(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdGreaterThan32(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdGreaterThan64(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordGreaterThan16(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordGreaterThan32(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordGreaterThan64(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdLessThanEqual16(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdLessThanEqual32(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdLessThanEqual64(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordLessThanEqual16(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordLessThanEqual32(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordLessThanEqual64(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdGreaterThanEqual16(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdGreaterThanEqual32(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPOrdGreaterThanEqual64(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordGreaterThanEqual16(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordGreaterThanEqual32(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPUnordGreaterThanEqual64(EmitContext& ctx, Id lhs, Id rhs);
Id EmitFPIsNan16(EmitContext& ctx, Id value);
Id EmitFPIsNan32(EmitContext& ctx, Id value);
Id EmitFPIsNan64(EmitContext& ctx, Id value);
Id EmitIAdd32(EmitContext& ctx, IR::Inst* inst, Id a, Id b);
Id EmitIAdd64(EmitContext& ctx, Id a, Id b);
Id EmitISub32(EmitContext& ctx, Id a, Id b);
Id EmitISub64(EmitContext& ctx, Id a, Id b);
Id EmitIMul32(EmitContext& ctx, Id a, Id b);
Id EmitSDiv32(EmitContext& ctx, Id a, Id b);
Id EmitUDiv32(EmitContext& ctx, Id a, Id b);
Id EmitINeg32(EmitContext& ctx, Id value);
Id EmitINeg64(EmitContext& ctx, Id value);
Id EmitIAbs32(EmitContext& ctx, Id value);
Id EmitIAbs64(EmitContext& ctx, Id value);
Id EmitShiftLeftLogical32(EmitContext& ctx, Id base, Id shift);
Id EmitShiftLeftLogical64(EmitContext& ctx, Id base, Id shift);
Id EmitShiftRightLogical32(EmitContext& ctx, Id base, Id shift);
Id EmitShiftRightLogical64(EmitContext& ctx, Id base, Id shift);
Id EmitShiftRightArithmetic32(EmitContext& ctx, Id base, Id shift);
Id EmitShiftRightArithmetic64(EmitContext& ctx, Id base, Id shift);
Id EmitBitwiseAnd32(EmitContext& ctx, IR::Inst* inst, Id a, Id b);
Id EmitBitwiseOr32(EmitContext& ctx, IR::Inst* inst, Id a, Id b);
Id EmitBitwiseXor32(EmitContext& ctx, IR::Inst* inst, Id a, Id b);
Id EmitBitFieldInsert(EmitContext& ctx, Id base, Id insert, Id offset, Id count);
Id EmitBitFieldSExtract(EmitContext& ctx, IR::Inst* inst, Id base, Id offset, Id count);
Id EmitBitFieldUExtract(EmitContext& ctx, IR::Inst* inst, Id base, Id offset, Id count);
Id EmitBitReverse32(EmitContext& ctx, Id value);
Id EmitBitCount32(EmitContext& ctx, Id value);
Id EmitBitwiseNot32(EmitContext& ctx, Id value);
Id EmitFindSMsb32(EmitContext& ctx, Id value);
Id EmitFindUMsb32(EmitContext& ctx, Id value);
Id EmitSMin32(EmitContext& ctx, Id a, Id b);
Id EmitUMin32(EmitContext& ctx, Id a, Id b);
Id EmitSMax32(EmitContext& ctx, Id a, Id b);
Id EmitUMax32(EmitContext& ctx, Id a, Id b);
Id EmitSClamp32(EmitContext& ctx, IR::Inst* inst, Id value, Id min, Id max);
Id EmitUClamp32(EmitContext& ctx, IR::Inst* inst, Id value, Id min, Id max);
Id EmitSLessThan(EmitContext& ctx, Id lhs, Id rhs);
Id EmitULessThan(EmitContext& ctx, Id lhs, Id rhs);
Id EmitIEqual(EmitContext& ctx, Id lhs, Id rhs);
Id EmitSLessThanEqual(EmitContext& ctx, Id lhs, Id rhs);
Id EmitULessThanEqual(EmitContext& ctx, Id lhs, Id rhs);
Id EmitSGreaterThan(EmitContext& ctx, Id lhs, Id rhs);
Id EmitUGreaterThan(EmitContext& ctx, Id lhs, Id rhs);
Id EmitINotEqual(EmitContext& ctx, Id lhs, Id rhs);
Id EmitSGreaterThanEqual(EmitContext& ctx, Id lhs, Id rhs);
Id EmitUGreaterThanEqual(EmitContext& ctx, Id lhs, Id rhs);
Id EmitSharedAtomicIAdd32(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitSharedAtomicSMin32(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitSharedAtomicUMin32(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitSharedAtomicSMax32(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitSharedAtomicUMax32(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitSharedAtomicInc32(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitSharedAtomicDec32(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitSharedAtomicAnd32(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitSharedAtomicOr32(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitSharedAtomicXor32(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitSharedAtomicExchange32(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitSharedAtomicExchange64(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitSharedAtomicExchange32x2(EmitContext& ctx, Id pointer_offset, Id value);
Id EmitStorageAtomicIAdd32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicSMin32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicUMin32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicSMax32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicUMax32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicInc32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicDec32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicAnd32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicOr32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicXor32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicExchange32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicCompareExchange32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id comparator, Id value);
Id EmitStorageAtomicIAdd64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicSMin64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicUMin64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicSMax64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicUMax64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicAnd64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicOr64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicXor64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicExchange64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicIAdd32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicSMin32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicUMin32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicSMax32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicUMax32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicAnd32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicOr32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicXor32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicExchange32x2(EmitContext& ctx, const IR::Value& binding,
const IR::Value& offset, Id value);
Id EmitStorageAtomicAddF32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicAddF16x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicAddF32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicMinF16x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicMinF32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicMaxF16x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitStorageAtomicMaxF32x2(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value);
Id EmitGlobalAtomicIAdd32(EmitContext& ctx, Id address, Id value);
Id EmitGlobalAtomicSMin32(EmitContext& ctx, Id address, Id value);
Id EmitGlobalAtomicUMin32(EmitContext& ctx, Id address, Id value);
Id EmitGlobalAtomicSMax32(EmitContext& ctx, Id address, Id value);
Id EmitGlobalAtomicUMax32(EmitContext& ctx, Id address, Id value);
Id EmitGlobalAtomicInc32(EmitContext& ctx);
Id EmitGlobalAtomicDec32(EmitContext& ctx);
Id EmitGlobalAtomicAnd32(EmitContext& ctx, Id address, Id value);
Id EmitGlobalAtomicOr32(EmitContext& ctx, Id address, Id value);
Id EmitGlobalAtomicXor32(EmitContext& ctx, Id address, Id value);
Id EmitGlobalAtomicExchange32(EmitContext& ctx, Id address, Id value);
Id EmitGlobalAtomicCompareExchange32(EmitContext& ctx, Id address, Id comparator, Id value);
Id EmitGlobalAtomicIAdd64(EmitContext& ctx);
Id EmitGlobalAtomicSMin64(EmitContext& ctx);
Id EmitGlobalAtomicUMin64(EmitContext& ctx);
Id EmitGlobalAtomicSMax64(EmitContext& ctx);
Id EmitGlobalAtomicUMax64(EmitContext& ctx);
Id EmitGlobalAtomicInc64(EmitContext& ctx);
Id EmitGlobalAtomicDec64(EmitContext& ctx);
Id EmitGlobalAtomicAnd64(EmitContext& ctx);
Id EmitGlobalAtomicOr64(EmitContext& ctx);
Id EmitGlobalAtomicXor64(EmitContext& ctx);
Id EmitGlobalAtomicExchange64(EmitContext& ctx);
Id EmitGlobalAtomicIAdd32x2(EmitContext& ctx);
Id EmitGlobalAtomicSMin32x2(EmitContext& ctx);
Id EmitGlobalAtomicUMin32x2(EmitContext& ctx);
Id EmitGlobalAtomicSMax32x2(EmitContext& ctx);
Id EmitGlobalAtomicUMax32x2(EmitContext& ctx);
Id EmitGlobalAtomicInc32x2(EmitContext& ctx);
Id EmitGlobalAtomicDec32x2(EmitContext& ctx);
Id EmitGlobalAtomicAnd32x2(EmitContext& ctx);
Id EmitGlobalAtomicOr32x2(EmitContext& ctx);
Id EmitGlobalAtomicXor32x2(EmitContext& ctx);
Id EmitGlobalAtomicExchange32x2(EmitContext& ctx);
Id EmitGlobalAtomicAddF32(EmitContext& ctx, Id address, Id value);
Id EmitGlobalAtomicAddF16x2(EmitContext& ctx);
Id EmitGlobalAtomicAddF32x2(EmitContext& ctx);
Id EmitGlobalAtomicMinF16x2(EmitContext& ctx);
Id EmitGlobalAtomicMinF32x2(EmitContext& ctx);
Id EmitGlobalAtomicMaxF16x2(EmitContext& ctx);
Id EmitGlobalAtomicMaxF32x2(EmitContext& ctx);
Id EmitLogicalOr(EmitContext& ctx, Id a, Id b);
Id EmitLogicalAnd(EmitContext& ctx, Id a, Id b);
Id EmitLogicalXor(EmitContext& ctx, Id a, Id b);
Id EmitLogicalNot(EmitContext& ctx, Id value);
Id EmitConvertS16F16(EmitContext& ctx, Id value);
Id EmitConvertS16F32(EmitContext& ctx, Id value);
Id EmitConvertS16F64(EmitContext& ctx, Id value);
Id EmitConvertS32F16(EmitContext& ctx, Id value);
Id EmitConvertS32F32(EmitContext& ctx, Id value);
Id EmitConvertS32F64(EmitContext& ctx, Id value);
Id EmitConvertS64F16(EmitContext& ctx, Id value);
Id EmitConvertS64F32(EmitContext& ctx, Id value);
Id EmitConvertS64F64(EmitContext& ctx, Id value);
Id EmitConvertU16F16(EmitContext& ctx, Id value);
Id EmitConvertU16F32(EmitContext& ctx, Id value);
Id EmitConvertU16F64(EmitContext& ctx, Id value);
Id EmitConvertU32F16(EmitContext& ctx, Id value);
Id EmitConvertU32F32(EmitContext& ctx, Id value);
Id EmitConvertU32F64(EmitContext& ctx, Id value);
Id EmitConvertU64F16(EmitContext& ctx, Id value);
Id EmitConvertU64F32(EmitContext& ctx, Id value);
Id EmitConvertU64F64(EmitContext& ctx, Id value);
Id EmitConvertU64U32(EmitContext& ctx, Id value);
Id EmitConvertU32U64(EmitContext& ctx, Id value);
Id EmitConvertF16F32(EmitContext& ctx, Id value);
Id EmitConvertF32F16(EmitContext& ctx, Id value);
Id EmitConvertF32F64(EmitContext& ctx, Id value);
Id EmitConvertF64F32(EmitContext& ctx, Id value);
Id EmitConvertF16S8(EmitContext& ctx, Id value);
Id EmitConvertF16S16(EmitContext& ctx, Id value);
Id EmitConvertF16S32(EmitContext& ctx, Id value);
Id EmitConvertF16S64(EmitContext& ctx, Id value);
Id EmitConvertF16U8(EmitContext& ctx, Id value);
Id EmitConvertF16U16(EmitContext& ctx, Id value);
Id EmitConvertF16U32(EmitContext& ctx, Id value);
Id EmitConvertF16U64(EmitContext& ctx, Id value);
Id EmitConvertF32S8(EmitContext& ctx, Id value);
Id EmitConvertF32S16(EmitContext& ctx, Id value);
Id EmitConvertF32S32(EmitContext& ctx, Id value);
Id EmitConvertF32S64(EmitContext& ctx, Id value);
Id EmitConvertF32U8(EmitContext& ctx, Id value);
Id EmitConvertF32U16(EmitContext& ctx, Id value);
Id EmitConvertF32U32(EmitContext& ctx, Id value);
Id EmitConvertF32U64(EmitContext& ctx, Id value);
Id EmitConvertF64S8(EmitContext& ctx, Id value);
Id EmitConvertF64S16(EmitContext& ctx, Id value);
Id EmitConvertF64S32(EmitContext& ctx, Id value);
Id EmitConvertF64S64(EmitContext& ctx, Id value);
Id EmitConvertF64U8(EmitContext& ctx, Id value);
Id EmitConvertF64U16(EmitContext& ctx, Id value);
Id EmitConvertF64U32(EmitContext& ctx, Id value);
Id EmitConvertF64U64(EmitContext& ctx, Id value);
Id EmitConvertU16U32(EmitContext& ctx, Id value);
Id EmitConvertU32U16(EmitContext& ctx, Id value);
Id EmitConvertU8U32(EmitContext& ctx, Id value);
Id EmitConvertU32U8(EmitContext& ctx, Id value);
Id EmitConvertS32S8(EmitContext& ctx, Id value);
Id EmitConvertS32S16(EmitContext& ctx, Id value);
Id EmitBindlessImageSampleImplicitLod(EmitContext&);
Id EmitBindlessImageSampleExplicitLod(EmitContext&);
Id EmitBindlessImageSampleDrefImplicitLod(EmitContext&);
Id EmitBindlessImageSampleDrefExplicitLod(EmitContext&);
Id EmitBindlessImageGather(EmitContext&);
Id EmitBindlessImageGatherDref(EmitContext&);
Id EmitBindlessImageFetch(EmitContext&);
Id EmitBindlessImageQueryDimensions(EmitContext&);
Id EmitBindlessImageQueryLod(EmitContext&);
Id EmitBindlessImageGradient(EmitContext&);
Id EmitBindlessImageRead(EmitContext&);
Id EmitBindlessImageWrite(EmitContext&);
Id EmitBoundImageSampleImplicitLod(EmitContext&);
Id EmitBoundImageSampleExplicitLod(EmitContext&);
Id EmitBoundImageSampleDrefImplicitLod(EmitContext&);
Id EmitBoundImageSampleDrefExplicitLod(EmitContext&);
Id EmitBoundImageGather(EmitContext&);
Id EmitBoundImageGatherDref(EmitContext&);
Id EmitBoundImageFetch(EmitContext&);
Id EmitBoundImageQueryDimensions(EmitContext&);
Id EmitBoundImageQueryLod(EmitContext&);
Id EmitBoundImageGradient(EmitContext&);
Id EmitBoundImageRead(EmitContext&);
Id EmitBoundImageWrite(EmitContext&);
Id EmitImageSampleImplicitLod(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id bias_lc, const IR::Value& offset);
Id EmitImageSampleExplicitLod(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id lod, const IR::Value& offset);
Id EmitImageSampleDrefImplicitLod(EmitContext& ctx, IR::Inst* inst, const IR::Value& index,
Id coords, Id dref, Id bias_lc, const IR::Value& offset);
Id EmitImageSampleDrefExplicitLod(EmitContext& ctx, IR::Inst* inst, const IR::Value& index,
Id coords, Id dref, Id lod, const IR::Value& offset);
Id EmitImageGather(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
const IR::Value& offset, const IR::Value& offset2);
Id EmitImageGatherDref(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
const IR::Value& offset, const IR::Value& offset2, Id dref);
Id EmitImageFetch(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords, Id offset,
Id lod, Id ms);
Id EmitImageQueryDimensions(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id lod,
const IR::Value& skip_mips);
Id EmitImageQueryLod(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords);
Id EmitImageGradient(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id derivatives, const IR::Value& offset, Id lod_clamp);
Id EmitImageRead(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords);
void EmitImageWrite(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords, Id color);
Id EmitIsTextureScaled(EmitContext& ctx, const IR::Value& index);
Id EmitIsImageScaled(EmitContext& ctx, const IR::Value& index);
Id EmitBindlessImageAtomicIAdd32(EmitContext&);
Id EmitBindlessImageAtomicSMin32(EmitContext&);
Id EmitBindlessImageAtomicUMin32(EmitContext&);
Id EmitBindlessImageAtomicSMax32(EmitContext&);
Id EmitBindlessImageAtomicUMax32(EmitContext&);
Id EmitBindlessImageAtomicInc32(EmitContext&);
Id EmitBindlessImageAtomicDec32(EmitContext&);
Id EmitBindlessImageAtomicAnd32(EmitContext&);
Id EmitBindlessImageAtomicOr32(EmitContext&);
Id EmitBindlessImageAtomicXor32(EmitContext&);
Id EmitBindlessImageAtomicExchange32(EmitContext&);
Id EmitBoundImageAtomicIAdd32(EmitContext&);
Id EmitBoundImageAtomicSMin32(EmitContext&);
Id EmitBoundImageAtomicUMin32(EmitContext&);
Id EmitBoundImageAtomicSMax32(EmitContext&);
Id EmitBoundImageAtomicUMax32(EmitContext&);
Id EmitBoundImageAtomicInc32(EmitContext&);
Id EmitBoundImageAtomicDec32(EmitContext&);
Id EmitBoundImageAtomicAnd32(EmitContext&);
Id EmitBoundImageAtomicOr32(EmitContext&);
Id EmitBoundImageAtomicXor32(EmitContext&);
Id EmitBoundImageAtomicExchange32(EmitContext&);
Id EmitImageAtomicIAdd32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value);
Id EmitImageAtomicSMin32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value);
Id EmitImageAtomicUMin32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value);
Id EmitImageAtomicSMax32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value);
Id EmitImageAtomicUMax32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value);
Id EmitImageAtomicInc32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value);
Id EmitImageAtomicDec32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value);
Id EmitImageAtomicAnd32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value);
Id EmitImageAtomicOr32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value);
Id EmitImageAtomicXor32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value);
Id EmitImageAtomicExchange32(EmitContext& ctx, IR::Inst* inst, const IR::Value& index, Id coords,
Id value);
Id EmitLaneId(EmitContext& ctx);
Id EmitVoteAll(EmitContext& ctx, Id pred);
Id EmitVoteAny(EmitContext& ctx, Id pred);
Id EmitVoteEqual(EmitContext& ctx, Id pred);
Id EmitSubgroupBallot(EmitContext& ctx, Id pred);
Id EmitSubgroupEqMask(EmitContext& ctx);
Id EmitSubgroupLtMask(EmitContext& ctx);
Id EmitSubgroupLeMask(EmitContext& ctx);
Id EmitSubgroupGtMask(EmitContext& ctx);
Id EmitSubgroupGeMask(EmitContext& ctx);
Id EmitShuffleIndex(EmitContext& ctx, IR::Inst* inst, Id value, Id index, Id clamp,
Id segmentation_mask);
Id EmitShuffleUp(EmitContext& ctx, IR::Inst* inst, Id value, Id index, Id clamp,
Id segmentation_mask);
Id EmitShuffleDown(EmitContext& ctx, IR::Inst* inst, Id value, Id index, Id clamp,
Id segmentation_mask);
Id EmitShuffleButterfly(EmitContext& ctx, IR::Inst* inst, Id value, Id index, Id clamp,
Id segmentation_mask);
Id EmitQuadBroadcast(EmitContext& ctx, Id value, Id lane);
Id EmitMatrixMulAddF16(EmitContext& ctx, Id a, Id b, Id c);
void DefineSoftwareMatrixFunction(EmitContext& ctx);
Id EmitQuadSwap(EmitContext& ctx, Id value, Id direction);
Id EmitFSwizzleAdd(EmitContext& ctx, Id op_a, Id op_b, Id swizzle);
Id EmitDPdxFine(EmitContext& ctx, Id op_a);
Id EmitDPdyFine(EmitContext& ctx, Id op_a);
Id EmitDPdxCoarse(EmitContext& ctx, Id op_a);
Id EmitDPdyCoarse(EmitContext& ctx, Id op_a);
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,284 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
namespace {
void SetZeroFlag(EmitContext& ctx, IR::Inst* inst, Id result) {
IR::Inst* const zero{inst->GetAssociatedPseudoOperation(IR::Opcode::GetZeroFromOp)};
if (!zero) {
return;
}
zero->SetDefinition(ctx.OpIEqual(ctx.U1, result, ctx.u32_zero_value));
zero->Invalidate();
}
void SetSignFlag(EmitContext& ctx, IR::Inst* inst, Id result) {
IR::Inst* const sign{inst->GetAssociatedPseudoOperation(IR::Opcode::GetSignFromOp)};
if (!sign) {
return;
}
sign->SetDefinition(ctx.OpSLessThan(ctx.U1, result, ctx.u32_zero_value));
sign->Invalidate();
}
} // Anonymous namespace
Id EmitIAdd32(EmitContext& ctx, IR::Inst* inst, Id a, Id b) {
Id result{};
if (IR::Inst* const carry{inst->GetAssociatedPseudoOperation(IR::Opcode::GetCarryFromOp)}) {
const Id carry_type{ctx.TypeStruct(ctx.U32[1], ctx.U32[1])};
const Id carry_result{ctx.OpIAddCarry(carry_type, a, b)};
result = ctx.OpCompositeExtract(ctx.U32[1], carry_result, 0U);
const Id carry_value{ctx.OpCompositeExtract(ctx.U32[1], carry_result, 1U)};
carry->SetDefinition(ctx.OpINotEqual(ctx.U1, carry_value, ctx.u32_zero_value));
carry->Invalidate();
} else {
result = ctx.OpIAdd(ctx.U32[1], a, b);
}
SetZeroFlag(ctx, inst, result);
SetSignFlag(ctx, inst, result);
if (IR::Inst * overflow{inst->GetAssociatedPseudoOperation(IR::Opcode::GetOverflowFromOp)}) {
// https://stackoverflow.com/questions/55468823/how-to-detect-integer-overflow-in-c
constexpr u32 s32_max{static_cast<u32>((std::numeric_limits<s32>::max)())};
const Id is_positive{ctx.OpSGreaterThanEqual(ctx.U1, a, ctx.u32_zero_value)};
const Id sub_a{ctx.OpISub(ctx.U32[1], ctx.Const(s32_max), a)};
const Id positive_test{ctx.OpSGreaterThan(ctx.U1, b, sub_a)};
const Id negative_test{ctx.OpSLessThan(ctx.U1, b, sub_a)};
const Id carry_flag{ctx.OpSelect(ctx.U1, is_positive, positive_test, negative_test)};
overflow->SetDefinition(carry_flag);
overflow->Invalidate();
}
return result;
}
Id EmitIAdd64(EmitContext& ctx, Id a, Id b) {
return ctx.OpIAdd(ctx.U64, a, b);
}
Id EmitISub32(EmitContext& ctx, Id a, Id b) {
return ctx.OpISub(ctx.U32[1], a, b);
}
Id EmitISub64(EmitContext& ctx, Id a, Id b) {
return ctx.OpISub(ctx.U64, a, b);
}
Id EmitIMul32(EmitContext& ctx, Id a, Id b) {
return ctx.OpIMul(ctx.U32[1], a, b);
}
Id EmitSDiv32(EmitContext& ctx, Id a, Id b) {
return ctx.OpSDiv(ctx.U32[1], a, b);
}
Id EmitUDiv32(EmitContext& ctx, Id a, Id b) {
return ctx.OpUDiv(ctx.U32[1], a, b);
}
Id EmitINeg32(EmitContext& ctx, Id value) {
return ctx.OpSNegate(ctx.U32[1], value);
}
Id EmitINeg64(EmitContext& ctx, Id value) {
return ctx.OpSNegate(ctx.U64, value);
}
Id EmitIAbs32(EmitContext& ctx, Id value) {
return ctx.OpSAbs(ctx.U32[1], value);
}
Id EmitIAbs64(EmitContext& ctx, Id value) {
return ctx.OpSAbs(ctx.U64, value);
}
Id EmitShiftLeftLogical32(EmitContext& ctx, Id base, Id shift) {
return ctx.OpShiftLeftLogical(ctx.U32[1], base, shift);
}
Id EmitShiftLeftLogical64(EmitContext& ctx, Id base, Id shift) {
return ctx.OpShiftLeftLogical(ctx.U64, base, shift);
}
Id EmitShiftRightLogical32(EmitContext& ctx, Id base, Id shift) {
return ctx.OpShiftRightLogical(ctx.U32[1], base, shift);
}
Id EmitShiftRightLogical64(EmitContext& ctx, Id base, Id shift) {
return ctx.OpShiftRightLogical(ctx.U64, base, shift);
}
Id EmitShiftRightArithmetic32(EmitContext& ctx, Id base, Id shift) {
return ctx.OpShiftRightArithmetic(ctx.U32[1], base, shift);
}
Id EmitShiftRightArithmetic64(EmitContext& ctx, Id base, Id shift) {
return ctx.OpShiftRightArithmetic(ctx.U64, base, shift);
}
Id EmitBitwiseAnd32(EmitContext& ctx, IR::Inst* inst, Id a, Id b) {
const Id result{ctx.OpBitwiseAnd(ctx.U32[1], a, b)};
SetZeroFlag(ctx, inst, result);
SetSignFlag(ctx, inst, result);
return result;
}
Id EmitBitwiseOr32(EmitContext& ctx, IR::Inst* inst, Id a, Id b) {
const Id result{ctx.OpBitwiseOr(ctx.U32[1], a, b)};
SetZeroFlag(ctx, inst, result);
SetSignFlag(ctx, inst, result);
return result;
}
Id EmitBitwiseXor32(EmitContext& ctx, IR::Inst* inst, Id a, Id b) {
const Id result{ctx.OpBitwiseXor(ctx.U32[1], a, b)};
SetZeroFlag(ctx, inst, result);
SetSignFlag(ctx, inst, result);
return result;
}
Id EmitBitFieldInsert(EmitContext& ctx, Id base, Id insert, Id offset, Id count) {
return ctx.OpBitFieldInsert(ctx.U32[1], base, insert, offset, count);
}
Id EmitBitFieldSExtract(EmitContext& ctx, IR::Inst* inst, Id base, Id offset, Id count) {
const Id result{ctx.OpBitFieldSExtract(ctx.U32[1], base, offset, count)};
SetZeroFlag(ctx, inst, result);
SetSignFlag(ctx, inst, result);
return result;
}
Id EmitBitFieldUExtract(EmitContext& ctx, IR::Inst* inst, Id base, Id offset, Id count) {
const Id result{ctx.OpBitFieldUExtract(ctx.U32[1], base, offset, count)};
SetZeroFlag(ctx, inst, result);
SetSignFlag(ctx, inst, result);
return result;
}
Id EmitBitReverse32(EmitContext& ctx, Id value) {
return ctx.OpBitReverse(ctx.U32[1], value);
}
Id EmitBitCount32(EmitContext& ctx, Id value) {
return ctx.OpBitCount(ctx.U32[1], value);
}
Id EmitBitwiseNot32(EmitContext& ctx, Id value) {
return ctx.OpNot(ctx.U32[1], value);
}
Id EmitFindSMsb32(EmitContext& ctx, Id value) {
return ctx.OpFindSMsb(ctx.U32[1], value);
}
Id EmitFindUMsb32(EmitContext& ctx, Id value) {
return ctx.OpFindUMsb(ctx.U32[1], value);
}
Id EmitSMin32(EmitContext& ctx, Id a, Id b) {
const bool is_broken{ctx.profile.has_broken_signed_operations};
if (is_broken) {
a = ctx.OpBitcast(ctx.S32[1], a);
b = ctx.OpBitcast(ctx.S32[1], b);
}
const Id result{ctx.OpSMin(ctx.U32[1], a, b)};
return is_broken ? ctx.OpBitcast(ctx.U32[1], result) : result;
}
Id EmitUMin32(EmitContext& ctx, Id a, Id b) {
return ctx.OpUMin(ctx.U32[1], a, b);
}
Id EmitSMax32(EmitContext& ctx, Id a, Id b) {
const bool is_broken{ctx.profile.has_broken_signed_operations};
if (is_broken) {
a = ctx.OpBitcast(ctx.S32[1], a);
b = ctx.OpBitcast(ctx.S32[1], b);
}
const Id result{ctx.OpSMax(ctx.U32[1], a, b)};
return is_broken ? ctx.OpBitcast(ctx.U32[1], result) : result;
}
Id EmitUMax32(EmitContext& ctx, Id a, Id b) {
return ctx.OpUMax(ctx.U32[1], a, b);
}
Id EmitSClamp32(EmitContext& ctx, IR::Inst* inst, Id value, Id min, Id max) {
Id result{};
if (ctx.profile.has_broken_signed_operations || ctx.profile.has_broken_spirv_clamp) {
value = ctx.OpBitcast(ctx.S32[1], value);
min = ctx.OpBitcast(ctx.S32[1], min);
max = ctx.OpBitcast(ctx.S32[1], max);
if (ctx.profile.has_broken_spirv_clamp) {
result = ctx.OpSMax(ctx.S32[1], ctx.OpSMin(ctx.S32[1], value, max), min);
} else {
result = ctx.OpSClamp(ctx.S32[1], value, min, max);
}
result = ctx.OpBitcast(ctx.U32[1], result);
} else {
result = ctx.OpSClamp(ctx.U32[1], value, min, max);
}
SetZeroFlag(ctx, inst, result);
SetSignFlag(ctx, inst, result);
return result;
}
Id EmitUClamp32(EmitContext& ctx, IR::Inst* inst, Id value, Id min, Id max) {
Id result{};
if (ctx.profile.has_broken_spirv_clamp) {
result = ctx.OpUMax(ctx.U32[1], ctx.OpUMin(ctx.U32[1], value, max), min);
} else {
result = ctx.OpUClamp(ctx.U32[1], value, min, max);
}
SetZeroFlag(ctx, inst, result);
SetSignFlag(ctx, inst, result);
return result;
}
Id EmitSLessThan(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpSLessThan(ctx.U1, lhs, rhs);
}
Id EmitULessThan(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpULessThan(ctx.U1, lhs, rhs);
}
Id EmitIEqual(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpIEqual(ctx.U1, lhs, rhs);
}
Id EmitSLessThanEqual(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpSLessThanEqual(ctx.U1, lhs, rhs);
}
Id EmitULessThanEqual(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpULessThanEqual(ctx.U1, lhs, rhs);
}
Id EmitSGreaterThan(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpSGreaterThan(ctx.U1, lhs, rhs);
}
Id EmitUGreaterThan(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpUGreaterThan(ctx.U1, lhs, rhs);
}
Id EmitINotEqual(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpINotEqual(ctx.U1, lhs, rhs);
}
Id EmitSGreaterThanEqual(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpSGreaterThanEqual(ctx.U1, lhs, rhs);
}
Id EmitUGreaterThanEqual(EmitContext& ctx, Id lhs, Id rhs) {
return ctx.OpUGreaterThanEqual(ctx.U1, lhs, rhs);
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,25 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
Id EmitLogicalOr(EmitContext& ctx, Id a, Id b) {
return ctx.OpLogicalOr(ctx.U1, a, b);
}
Id EmitLogicalAnd(EmitContext& ctx, Id a, Id b) {
return ctx.OpLogicalAnd(ctx.U1, a, b);
}
Id EmitLogicalXor(EmitContext& ctx, Id a, Id b) {
return ctx.OpLogicalNotEqual(ctx.U1, a, b);
}
Id EmitLogicalNot(EmitContext& ctx, Id value) {
return ctx.OpLogicalNot(ctx.U1, value);
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,343 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <bit>
#include "shader_recompiler/backend/spirv/emit_spirv.h"
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
namespace {
Id StorageIndex(EmitContext& ctx, const IR::Value& offset, size_t element_size,
u32 index_offset = 0) {
if (offset.IsImmediate()) {
const u32 imm_offset{static_cast<u32>(offset.U32() / element_size) + index_offset};
return ctx.Const(imm_offset);
}
const u32 shift{static_cast<u32>(std::countr_zero(element_size))};
Id index{ctx.Def(offset)};
if (shift != 0) {
const Id shift_id{ctx.Const(shift)};
index = ctx.OpShiftRightLogical(ctx.U32[1], index, shift_id);
}
if (index_offset != 0) {
index = ctx.OpIAdd(ctx.U32[1], index, ctx.Const(index_offset));
}
return index;
}
Id StoragePointer(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
const StorageTypeDefinition& type_def, size_t element_size,
Id StorageDefinitions::*member_ptr, u32 index_offset = 0) {
if (!binding.IsImmediate()) {
throw NotImplementedException("Dynamic storage buffer indexing");
}
const Id ssbo{ctx.ssbos[binding.U32()].*member_ptr};
const Id index{StorageIndex(ctx, offset, element_size, index_offset)};
return ctx.OpAccessChain(type_def.element, ssbo, ctx.u32_zero_value, index);
}
Id LoadStorage(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset, Id result_type,
const StorageTypeDefinition& type_def, size_t element_size,
Id StorageDefinitions::*member_ptr, u32 index_offset = 0) {
const Id pointer{
StoragePointer(ctx, binding, offset, type_def, element_size, member_ptr, index_offset)};
return ctx.OpLoad(result_type, pointer);
}
Id LoadStorage32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
u32 index_offset = 0) {
return LoadStorage(ctx, binding, offset, ctx.U32[1], ctx.storage_types.U32, sizeof(u32),
&StorageDefinitions::U32, index_offset);
}
void WriteStorage(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset, Id value,
const StorageTypeDefinition& type_def, size_t element_size,
Id StorageDefinitions::*member_ptr, u32 index_offset = 0) {
const Id pointer{
StoragePointer(ctx, binding, offset, type_def, element_size, member_ptr, index_offset)};
ctx.OpStore(pointer, value);
}
void WriteStorage32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset, Id value,
u32 index_offset = 0) {
WriteStorage(ctx, binding, offset, value, ctx.storage_types.U32, sizeof(u32),
&StorageDefinitions::U32, index_offset);
}
void WriteStorageByCasLoop(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value, Id bit_offset, Id bit_count) {
const Id pointer{StoragePointer(ctx, binding, offset, ctx.storage_types.U32, sizeof(u32),
&StorageDefinitions::U32)};
ctx.OpFunctionCall(ctx.TypeVoid(), ctx.write_storage_cas_loop_func, pointer, value, bit_offset,
bit_count);
}
} // Anonymous namespace
// (hookshot) 8- and 16-bit loads through a pointer no storage buffer was found for: the aligned 32-bit word by the same
// search as LoadGlobal32, and the byte or half of it the address names (the console's loads are naturally aligned). They
// threw "not implemented": 30 of OoT's mesh programs (LoadGlobalS16) were left untranslated on every card, issue 351
namespace {
Id LoadGlobalNarrow(EmitContext& ctx, Id address, u32 bits, bool is_signed) {
if (!ctx.profile.support_int64) {
LOG_WARNING(Shader_SPIRV, "Int64 not supported, ignoring memory operation");
return ctx.Const(0u);
}
const Id aligned{ctx.OpBitwiseAnd(ctx.U64, address, ctx.Constant(ctx.U64, u64{~u64{3}}))};
const Id word{ctx.OpFunctionCall(ctx.U32[1], ctx.load_global_func_u32, aligned)};
const Id low{ctx.OpUConvert(ctx.U32[1], ctx.OpBitwiseAnd(ctx.U64, address, ctx.Constant(ctx.U64, u64{3})))};
const Id shift{ctx.OpShiftLeftLogical(ctx.U32[1], low, ctx.Const(3u))};
return is_signed ? ctx.OpBitFieldSExtract(ctx.U32[1], word, shift, ctx.Const(bits))
: ctx.OpBitFieldUExtract(ctx.U32[1], word, shift, ctx.Const(bits));
}
} // Anonymous namespace
Id EmitLoadGlobalU8(EmitContext& ctx, Id address) {
return LoadGlobalNarrow(ctx, address, 8, false);
}
Id EmitLoadGlobalS8(EmitContext& ctx, Id address) {
return LoadGlobalNarrow(ctx, address, 8, true);
}
Id EmitLoadGlobalU16(EmitContext& ctx, Id address) {
return LoadGlobalNarrow(ctx, address, 16, false);
}
Id EmitLoadGlobalS16(EmitContext& ctx, Id address) {
return LoadGlobalNarrow(ctx, address, 16, true);
}
Id EmitLoadGlobal32(EmitContext& ctx, Id address) {
if (ctx.profile.support_int64) {
return ctx.OpFunctionCall(ctx.U32[1], ctx.load_global_func_u32, address);
}
LOG_WARNING(Shader_SPIRV, "Int64 not supported, ignoring memory operation");
return ctx.Const(0u);
}
Id EmitLoadGlobal64(EmitContext& ctx, Id address) {
if (ctx.profile.support_int64) {
return ctx.OpFunctionCall(ctx.U32[2], ctx.load_global_func_u32x2, address);
}
LOG_WARNING(Shader_SPIRV, "Int64 not supported, ignoring memory operation");
return ctx.Const(0u, 0u);
}
Id EmitLoadGlobal128(EmitContext& ctx, Id address) {
if (ctx.profile.support_int64) {
return ctx.OpFunctionCall(ctx.U32[4], ctx.load_global_func_u32x4, address);
}
LOG_WARNING(Shader_SPIRV, "Int64 not supported, ignoring memory operation");
return ctx.Const(0u, 0u, 0u, 0u);
}
// (hookshot) 8- and 16-bit stores through a pointer no storage buffer was found for: the aligned 32-bit word by the same
// search as LoadGlobal32, changed in place with a compare-exchange loop (spirv_emit_context.cpp). They threw "not
// implemented", the same gap as the narrow loads (issue 192)
namespace {
void WriteGlobalNarrow(EmitContext& ctx, Id address, Id value, u32 bits) {
if (!ctx.profile.support_int64) {
LOG_WARNING(Shader_SPIRV, "Int64 not supported, ignoring memory operation");
return;
}
const Id low{ctx.OpUConvert(ctx.U32[1], ctx.OpBitwiseAnd(ctx.U64, address, ctx.Constant(ctx.U64, u64{3})))};
const Id bit_offset{ctx.OpShiftLeftLogical(ctx.U32[1], low, ctx.Const(3u))};
ctx.OpFunctionCall(ctx.TypeVoid(), ctx.write_global_cas_loop_func, address, value, bit_offset, ctx.Const(bits));
}
} // Anonymous namespace
void EmitWriteGlobalU8(EmitContext& ctx, Id address, Id value) {
WriteGlobalNarrow(ctx, address, value, 8);
}
void EmitWriteGlobalS8(EmitContext& ctx, Id address, Id value) {
WriteGlobalNarrow(ctx, address, value, 8);
}
void EmitWriteGlobalU16(EmitContext& ctx, Id address, Id value) {
WriteGlobalNarrow(ctx, address, value, 16);
}
void EmitWriteGlobalS16(EmitContext& ctx, Id address, Id value) {
WriteGlobalNarrow(ctx, address, value, 16);
}
void EmitWriteGlobal32(EmitContext& ctx, Id address, Id value) {
if (ctx.profile.support_int64) {
ctx.OpFunctionCall(ctx.void_id, ctx.write_global_func_u32, address, value);
return;
}
LOG_WARNING(Shader_SPIRV, "Int64 not supported, ignoring memory operation");
}
void EmitWriteGlobal64(EmitContext& ctx, Id address, Id value) {
if (ctx.profile.support_int64) {
ctx.OpFunctionCall(ctx.void_id, ctx.write_global_func_u32x2, address, value);
return;
}
LOG_WARNING(Shader_SPIRV, "Int64 not supported, ignoring memory operation");
}
void EmitWriteGlobal128(EmitContext& ctx, Id address, Id value) {
if (ctx.profile.support_int64) {
ctx.OpFunctionCall(ctx.void_id, ctx.write_global_func_u32x4, address, value);
return;
}
LOG_WARNING(Shader_SPIRV, "Int64 not supported, ignoring memory operation");
}
Id EmitLoadStorageU8(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_int8 && ctx.profile.support_storage_buffer_8bit &&
ctx.profile.support_descriptor_aliasing) {
return ctx.OpUConvert(ctx.U32[1],
LoadStorage(ctx, binding, offset, ctx.U8, ctx.storage_types.U8,
sizeof(u8), &StorageDefinitions::U8));
} else {
return ctx.OpBitFieldUExtract(ctx.U32[1], LoadStorage32(ctx, binding, offset),
ctx.BitOffset8(offset), ctx.Const(8u));
}
}
Id EmitLoadStorageS8(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_int8 && ctx.profile.support_storage_buffer_8bit &&
ctx.profile.support_descriptor_aliasing) {
return ctx.OpSConvert(ctx.U32[1],
LoadStorage(ctx, binding, offset, ctx.S8, ctx.storage_types.S8,
sizeof(s8), &StorageDefinitions::S8));
} else {
return ctx.OpBitFieldSExtract(ctx.U32[1], LoadStorage32(ctx, binding, offset),
ctx.BitOffset8(offset), ctx.Const(8u));
}
}
Id EmitLoadStorageU16(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_int16 && ctx.profile.support_storage_buffer_16bit &&
ctx.profile.support_descriptor_aliasing) {
return ctx.OpUConvert(ctx.U32[1],
LoadStorage(ctx, binding, offset, ctx.U16, ctx.storage_types.U16,
sizeof(u16), &StorageDefinitions::U16));
} else {
return ctx.OpBitFieldUExtract(ctx.U32[1], LoadStorage32(ctx, binding, offset),
ctx.BitOffset16(offset), ctx.Const(16u));
}
}
Id EmitLoadStorageS16(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_int16 && ctx.profile.support_storage_buffer_16bit &&
ctx.profile.support_descriptor_aliasing) {
return ctx.OpSConvert(ctx.U32[1],
LoadStorage(ctx, binding, offset, ctx.S16, ctx.storage_types.S16,
sizeof(s16), &StorageDefinitions::S16));
} else {
return ctx.OpBitFieldSExtract(ctx.U32[1], LoadStorage32(ctx, binding, offset),
ctx.BitOffset16(offset), ctx.Const(16u));
}
}
Id EmitLoadStorage32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
return LoadStorage32(ctx, binding, offset);
}
Id EmitLoadStorage64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_descriptor_aliasing) {
return LoadStorage(ctx, binding, offset, ctx.U32[2], ctx.storage_types.U32x2,
sizeof(u32[2]), &StorageDefinitions::U32x2);
} else {
return ctx.OpCompositeConstruct(ctx.U32[2], LoadStorage32(ctx, binding, offset, 0),
LoadStorage32(ctx, binding, offset, 1));
}
}
Id EmitLoadStorage128(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset) {
if (ctx.profile.support_descriptor_aliasing) {
return LoadStorage(ctx, binding, offset, ctx.U32[4], ctx.storage_types.U32x4,
sizeof(u32[4]), &StorageDefinitions::U32x4);
} else {
return ctx.OpCompositeConstruct(ctx.U32[4], LoadStorage32(ctx, binding, offset, 0),
LoadStorage32(ctx, binding, offset, 1),
LoadStorage32(ctx, binding, offset, 2),
LoadStorage32(ctx, binding, offset, 3));
}
}
void EmitWriteStorageU8(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
if (ctx.profile.support_int8 && ctx.profile.support_storage_buffer_8bit &&
ctx.profile.support_descriptor_aliasing) {
WriteStorage(ctx, binding, offset, ctx.OpSConvert(ctx.U8, value), ctx.storage_types.U8,
sizeof(u8), &StorageDefinitions::U8);
} else {
WriteStorageByCasLoop(ctx, binding, offset, value, ctx.BitOffset8(offset), ctx.Const(8u));
}
}
void EmitWriteStorageS8(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
if (ctx.profile.support_int8 && ctx.profile.support_storage_buffer_8bit &&
ctx.profile.support_descriptor_aliasing) {
WriteStorage(ctx, binding, offset, ctx.OpSConvert(ctx.S8, value), ctx.storage_types.S8,
sizeof(s8), &StorageDefinitions::S8);
} else {
WriteStorageByCasLoop(ctx, binding, offset, value, ctx.BitOffset8(offset), ctx.Const(8u));
}
}
void EmitWriteStorageU16(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
if (ctx.profile.support_int16 && ctx.profile.support_storage_buffer_16bit &&
ctx.profile.support_descriptor_aliasing) {
WriteStorage(ctx, binding, offset, ctx.OpSConvert(ctx.U16, value), ctx.storage_types.U16,
sizeof(u16), &StorageDefinitions::U16);
} else {
WriteStorageByCasLoop(ctx, binding, offset, value, ctx.BitOffset16(offset), ctx.Const(16u));
}
}
void EmitWriteStorageS16(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
if (ctx.profile.support_int16 && ctx.profile.support_storage_buffer_16bit &&
ctx.profile.support_descriptor_aliasing) {
WriteStorage(ctx, binding, offset, ctx.OpSConvert(ctx.S16, value), ctx.storage_types.S16,
sizeof(s16), &StorageDefinitions::S16);
} else {
WriteStorageByCasLoop(ctx, binding, offset, value, ctx.BitOffset16(offset), ctx.Const(16u));
}
}
void EmitWriteStorage32(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
WriteStorage32(ctx, binding, offset, value);
}
void EmitWriteStorage64(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
if (ctx.profile.support_descriptor_aliasing) {
WriteStorage(ctx, binding, offset, value, ctx.storage_types.U32x2, sizeof(u32[2]),
&StorageDefinitions::U32x2);
} else {
for (u32 index = 0; index < 2; ++index) {
const Id element{ctx.OpCompositeExtract(ctx.U32[1], value, index)};
WriteStorage32(ctx, binding, offset, element, index);
}
}
}
void EmitWriteStorage128(EmitContext& ctx, const IR::Value& binding, const IR::Value& offset,
Id value) {
if (ctx.profile.support_descriptor_aliasing) {
WriteStorage(ctx, binding, offset, value, ctx.storage_types.U32x4, sizeof(u32[4]),
&StorageDefinitions::U32x4);
} else {
for (u32 index = 0; index < 4; ++index) {
const Id element{ctx.OpCompositeExtract(ctx.U32[1], value, index)};
WriteStorage32(ctx, binding, offset, element, index);
}
}
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,656 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// Switch 2 (NVN) task and mesh programs. They talk to the hardware through the stage buffer
// (ISBE): a task program writes the number of mesh workgroups at byte 4 of its output region and
// its payload after it; each mesh workgroup reads that region (its own index at byte 0) and
// writes its vertices and primitives to an attribute region in groups of 32, one 32-word line per
// attribute, plus the primitive count and 8-bit vertex indices to a map region. Here the task's
// output region is the task payload, the mesh output regions live in workgroup memory, and the
// epilogue (reached by every invocation, see structured_control_flow.cpp) hands them to Vulkan.
#include <algorithm>
#include <cstdlib>
#include <span>
#include "shader_recompiler/backend/spirv/emit_spirv.h"
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
namespace {
void Barrier(EmitContext& ctx) {
ctx.OpControlBarrier(ctx.Const(static_cast<u32>(spv::Scope::Workgroup)),
ctx.Const(static_cast<u32>(spv::Scope::Workgroup)),
ctx.Const(static_cast<u32>(spv::MemorySemanticsMask::AcquireRelease |
spv::MemorySemanticsMask::WorkgroupMemory)));
}
Id LocalInvocation(EmitContext& ctx) {
return ctx.OpCompositeExtract(ctx.U32[1], ctx.OpLoad(ctx.U32[3], ctx.local_invocation_id), 0U);
}
Id Clamped(EmitContext& ctx, Id index, u32 count) {
return ctx.OpUMin(ctx.U32[1], index, ctx.Const(count - 1));
}
Id PushU64(EmitContext& ctx, u32 member);
Id PushU32(EmitContext& ctx, u32 member) {
const Id pointer{ctx.TypePointer(spv::StorageClass::PushConstant, ctx.U32[1])};
return ctx.OpLoad(ctx.U32[1], ctx.OpAccessChain(pointer, ctx.rescaling_push_constants, ctx.Const(member)));
}
Id WorkgroupId(EmitContext& ctx);
Id FallbackSlot(EmitContext& ctx);
Id BufferWordAt(EmitContext& ctx, Id base, Id word);
Id BufferWord(EmitContext& ctx, u32 member, Id word);
Id PayloadWord(EmitContext& ctx, Id index) {
if (ctx.mesh_fallback && ctx.stage == Stage::Mesh && ctx.runtime_info.mesh_fallback_task) {
// (the task wrote it out: the mesh workgroups are a later dispatch, and read their task workgroup's payload, a slot
// being task workgroup * threads + its index in the task)
const Id task{ctx.OpUDiv(ctx.U32[1], FallbackSlot(ctx), ctx.Const(ctx.runtime_info.mesh_fallback_task_threads))};
const Id first{ctx.OpIAdd(
ctx.U32[1], ctx.Const(MESH_FALLBACK_CAPACITY),
ctx.OpIMul(ctx.U32[1], task, ctx.Const(ctx.mesh_info.payload_words)))};
return BufferWordAt(ctx, PushU64(ctx, ctx.mesh_fallback_payload_member_index),
ctx.OpIAdd(ctx.U32[1], first,
Clamped(ctx, index, ctx.mesh_info.payload_words)));
}
return ctx.OpAccessChain(ctx.mesh_payload_u32, ctx.mesh_payload,
Clamped(ctx, index, ctx.mesh_info.payload_words));
}
Id SharedWord(EmitContext& ctx, Id array, Id index) {
return ctx.OpAccessChain(ctx.mesh_shared_u32, array, index);
}
// Runs body under `if (condition)`.
template <typename Func>
void If(EmitContext& ctx, Id condition, Func&& body) {
const Id then_label{ctx.OpLabel()};
const Id merge_label{ctx.OpLabel()};
ctx.OpSelectionMerge(merge_label, spv::SelectionControlMask::MaskNone);
ctx.OpBranchConditional(condition, then_label, merge_label);
ctx.AddLabel(then_label);
body();
ctx.OpBranch(merge_label);
ctx.AddLabel(merge_label);
}
// Copies the lines of element `index` (a vertex or a primitive) from the attribute region.
void CopyLines(EmitContext& ctx, Id index, u32 base, u32 num_lines,
const std::vector<EmitContext::MeshLineTarget>& targets) {
// Word of line 0: base + (index / 32) * num_lines * 32 + index % 32.
const Id group{ctx.OpShiftRightLogical(ctx.U32[1], index, ctx.Const(5U))};
const Id slot{ctx.OpBitwiseAnd(ctx.U32[1], index, ctx.Const(31U))};
const Id first{ctx.OpIAdd(ctx.U32[1],
ctx.OpIMul(ctx.U32[1], group, ctx.Const(num_lines * 32)),
ctx.OpIAdd(ctx.U32[1], slot, ctx.Const(base)))};
for (u32 line = 0; line < num_lines; ++line) {
const EmitContext::MeshLineTarget& target{targets[line]};
if (!Sirit::ValidId(target.variable)) {
continue;
}
const Id word{ctx.OpLoad(
ctx.U32[1],
SharedWord(ctx, ctx.mesh_attributes, ctx.OpIAdd(ctx.U32[1], first, ctx.Const(line * 32))))};
const Id value{target.is_float ? ctx.OpBitcast(ctx.F32[1], word) : word};
const Id pointer_type{target.is_float ? ctx.output_f32 : ctx.output_u32};
const Id pointer{target.element == ~0U
? ctx.OpAccessChain(pointer_type, target.variable, index)
: ctx.OpAccessChain(pointer_type, target.variable, index,
ctx.Const(target.element))};
ctx.OpStore(pointer, value);
}
}
// OpenGL-style clip space: z = (z + w) / 2, from the stage buffer's position lines.
void ConvertDepth(EmitContext& ctx, Id vertex) {
const auto& lines{ctx.mesh_info.vertex_lines};
const auto z_line{std::ranges::find(lines, static_cast<u32>(IR::Attribute::PositionZ))};
const auto w_line{std::ranges::find(lines, static_cast<u32>(IR::Attribute::PositionW))};
if (z_line == lines.end() || w_line == lines.end()) {
return;
}
const u32 num_lines{static_cast<u32>(lines.size())};
const Id group{ctx.OpShiftRightLogical(ctx.U32[1], vertex, ctx.Const(5U))};
const Id slot{ctx.OpBitwiseAnd(ctx.U32[1], vertex, ctx.Const(31U))};
const Id first{ctx.OpIAdd(ctx.U32[1], ctx.OpIMul(ctx.U32[1], group, ctx.Const(num_lines * 32)),
slot)};
const auto read = [&](auto line) {
const u32 offset{static_cast<u32>(line - lines.begin()) * 32};
return ctx.OpBitcast(ctx.F32[1],
ctx.OpLoad(ctx.U32[1], SharedWord(ctx, ctx.mesh_attributes,
ctx.OpIAdd(ctx.U32[1], first,
ctx.Const(offset)))));
};
const Id z{ctx.OpFMul(ctx.F32[1], ctx.OpFAdd(ctx.F32[1], read(z_line), read(w_line)),
ctx.Const(0.5f))};
ctx.OpStore(ctx.OpAccessChain(ctx.output_f32, ctx.output_position, vertex, ctx.Const(2U)), z);
}
/// (hookshot) The fallback's records: one per primitive corner, its validity word and the stage buffer's vertex and
/// primitive lines. A mesh workgroup writes its workgroup index and a fixed number of records for it; the generated
/// vertex stage reads one record per vertex.
u32 FallbackRecordWords(const MeshInfo& mesh) {
return 1 + static_cast<u32>(mesh.vertex_lines.size()) +
static_cast<u32>(mesh.primitive_lines.size());
}
Id PushU64(EmitContext& ctx, u32 member) {
const Id pointer_type{ctx.TypePointer(spv::StorageClass::PushConstant, ctx.U64)};
return ctx.OpLoad(
ctx.U64, ctx.OpAccessChain(pointer_type, ctx.rescaling_push_constants, ctx.Const(member)));
}
/// The word at a device address.
Id BufferWordAt(EmitContext& ctx, Id base, Id word) {
const Id address{
ctx.OpIAdd(ctx.U64, base,
ctx.OpUConvert(ctx.U64, ctx.OpIMul(ctx.U32[1], word, ctx.Const(4U))))};
return ctx.MeshFallbackWord(address);
}
/// The word at a pushed device address.
Id BufferWord(EmitContext& ctx, u32 member, Id word) {
return BufferWordAt(ctx, PushU64(ctx, member), word);
}
/// A load and a store of such a word. (PhysicalStorageBuffer accesses must say their alignment: spirv-val rejects them
/// without it.) Workgroup words take the same, which is valid there too.
Id LoadWord(EmitContext& ctx, Id pointer) {
return ctx.OpLoad(ctx.U32[1], pointer, spv::MemoryAccessMask::Aligned, 4U);
}
void StoreWord(EmitContext& ctx, Id pointer, Id value) {
ctx.OpStore(pointer, value, spv::MemoryAccessMask::Aligned, 4U);
}
Id WorkgroupId(EmitContext& ctx) {
return ctx.OpCompositeExtract(ctx.U32[1], ctx.OpLoad(ctx.U32[3], ctx.workgroup_id), 0U);
}
/// A fallback mesh workgroup's slot: the one its task's map gave it (EmitMeshPrologue), else its workgroup id
Id FallbackSlot(EmitContext& ctx) {
return Sirit::ValidId(ctx.mesh_fallback_slot) ? ctx.mesh_fallback_slot : WorkgroupId(ctx);
}
/// Runs body once for each index invocation, invocation + threads, ... below count. (Unrolled: count and threads are
/// known here. A loop's index variable declared at the call, mid-function, was invalid SPIR-V: a function's OpVariables
/// must open its first block)
template <typename Func>
void Strided(EmitContext& ctx, u32 count, u32 threads, Id invocation, Func&& body) {
for (u32 first = 0; first < count; first += std::max(threads, 1U)) {
const Id index{first ? ctx.OpIAdd(ctx.U32[1], invocation, ctx.Const(first)) : invocation};
If(ctx, ctx.OpULessThan(ctx.U1, index, ctx.Const(count)), [&] { body(index); });
}
}
/// The raw stage buffer word of line `line` for element `element` (a vertex or a primitive).
Id LineWord(EmitContext& ctx, Id array, Id element, u32 base, u32 num_lines, u32 line) {
const Id group{ctx.OpShiftRightLogical(ctx.U32[1], element, ctx.Const(5U))};
const Id slot{ctx.OpBitwiseAnd(ctx.U32[1], element, ctx.Const(31U))};
const Id first{ctx.OpIAdd(
ctx.U32[1], ctx.OpIMul(ctx.U32[1], group, ctx.Const(num_lines * 32)),
ctx.OpIAdd(ctx.U32[1], slot, ctx.Const(base)))};
return ctx.OpLoad(
ctx.U32[1],
SharedWord(ctx, array, ctx.OpIAdd(ctx.U32[1], first, ctx.Const(line * 32))));
}
/// A mesh workgroup takes records for its primitives' corners from the records buffer's allocator and writes them; the vertex
/// stage's indirect draw covers the records taken (MESH_FALLBACK_DRAW_WORD). A task's flag first bounds its count.
void MeshFallbackEpilogue(EmitContext& ctx) {
const MeshInfo& mesh{ctx.mesh_info};
const u32 corners{mesh.Corners()};
const u32 words{FallbackRecordWords(mesh)};
const u32 fit{MESH_FALLBACK_RECORD_WORDS / words}; // (this program's records the buffer holds)
const Id none{ctx.Const(0xFFFFFFFFU)};
Barrier(ctx);
// (with a task program every workgroup dispatched was started by it: the dispatch is the map's, TaskEpilogue)
const Id slot{FallbackSlot(ctx)};
const Id in_range{ctx.OpULessThan(ctx.U1, slot, ctx.Const(MESH_FALLBACK_CAPACITY))};
Id count{ctx.OpUMin(ctx.U32[1], ctx.OpLoad(ctx.U32[1], ctx.mesh_primitive_count),
ctx.Const(mesh.max_primitives))};
count = ctx.OpSelect(ctx.U32[1], in_range, count, ctx.u32_zero_value);
const Id taken{ctx.OpIMul(ctx.U32[1], count, ctx.Const(corners))};
const auto atomic_add = [&](u32 word, Id value) {
return ctx.OpAtomicIAdd(ctx.U32[1], BufferWord(ctx, ctx.mesh_fallback_records_member_index, ctx.Const(word)),
ctx.Const(static_cast<u32>(spv::Scope::Device)), ctx.u32_zero_value, value);
};
// (every thread has read the primitive count above: thread 0 hands the others its first record in that word. Records
// that do not fit are not taken; taking only grows, so the ones that fit are the first, and the draw's count is theirs)
Barrier(ctx);
If(ctx, ctx.OpIEqual(ctx.U1, LocalInvocation(ctx), ctx.u32_zero_value), [&] {
ctx.OpStore(ctx.mesh_primitive_count, none);
If(ctx, ctx.OpINotEqual(ctx.U1, taken, ctx.u32_zero_value), [&] {
const Id base{atomic_add(MESH_FALLBACK_ALLOC_WORD, taken)};
const Id fits{ctx.OpLogicalAnd(ctx.U1, ctx.OpULessThanEqual(ctx.U1, base, ctx.Const(fit)),
ctx.OpULessThanEqual(ctx.U1, taken,
ctx.OpISub(ctx.U32[1], ctx.Const(fit), base)))};
If(ctx, fits, [&] {
atomic_add(MESH_FALLBACK_DRAW_WORD, taken);
ctx.OpStore(ctx.mesh_primitive_count, base);
});
});
});
Barrier(ctx);
const Id base{ctx.OpLoad(ctx.U32[1], ctx.mesh_primitive_count)};
If(ctx, ctx.OpINotEqual(ctx.U1, base, none), [&] {
const Id invocation{LocalInvocation(ctx)};
Strided(ctx, mesh.max_primitives, ctx.mesh_threads, invocation, [&](Id primitive) {
If(ctx, ctx.OpULessThan(ctx.U1, primitive, count), [&] {
for (u32 corner = 0; corner < corners; ++corner) {
const Id byte{ctx.OpIAdd(
ctx.U32[1], ctx.OpIMul(ctx.U32[1], primitive, ctx.Const(corners)),
ctx.Const(4U + corner))};
// (an index byte the program left unwritten is held to the vertex count)
const Id vertex{Clamped(
ctx, ctx.OpLoad(ctx.U32[1], SharedWord(ctx, ctx.mesh_map, byte)),
mesh.max_vertices)};
const Id record{ctx.OpIAdd(
ctx.U32[1], base,
ctx.OpIAdd(ctx.U32[1], ctx.OpIMul(ctx.U32[1], primitive, ctx.Const(corners)),
ctx.Const(corner)))};
const Id word{ctx.OpIAdd(ctx.U32[1], ctx.Const(MESH_FALLBACK_HEADER_WORDS),
ctx.OpIMul(ctx.U32[1], record, ctx.Const(words)))};
StoreWord(ctx, BufferWord(ctx, ctx.mesh_fallback_records_member_index, word),
ctx.Const(1U));
for (u32 line = 0; line < mesh.vertex_lines.size(); ++line) {
const Id value{LineWord(ctx, ctx.mesh_attributes, vertex, 0,
static_cast<u32>(mesh.vertex_lines.size()), line)};
StoreWord(ctx, BufferWord(ctx, ctx.mesh_fallback_records_member_index,
ctx.OpIAdd(ctx.U32[1], word, ctx.Const(1U + line))),
value);
}
for (u32 line = 0; line < mesh.primitive_lines.size(); ++line) {
const Id value{LineWord(ctx, ctx.mesh_attributes, primitive,
ctx.mesh_primitive_base,
static_cast<u32>(mesh.primitive_lines.size()), line)};
StoreWord(ctx, BufferWord(ctx, ctx.mesh_fallback_records_member_index,
ctx.OpIAdd(ctx.U32[1], word,
ctx.Const(static_cast<u32>(1U + mesh.vertex_lines.size()) +
line))),
value);
}
}
});
});
});
}
void StorePosition(EmitContext& ctx, u32 element, Id value) {
ctx.OpStore(ctx.OpAccessChain(ctx.output_f32, ctx.output_position, ctx.Const(element)), value);
}
void StoreFallbackOutput(EmitContext& ctx, IR::Attribute attribute, Id value) {
if (IR::IsGeneric(attribute)) {
const u32 index{IR::GenericAttributeIndex(attribute)};
const u32 element{IR::GenericAttributeElement(attribute)};
const GenericElementInfo& info{ctx.output_generics.at(index).at(element)};
if (!Sirit::ValidId(info.id)) {
return;
}
if (info.num_components == 1) {
ctx.OpStore(info.id, value);
} else {
ctx.OpStore(ctx.OpAccessChain(ctx.output_f32, info.id,
ctx.Const(element - info.first_element)),
value);
}
return;
}
if (attribute == IR::Attribute::PointSize && Sirit::ValidId(ctx.output_point_size)) {
ctx.OpStore(ctx.output_point_size, value);
}
}
u32 task_cap(u32 threads) { // (RX_VK_TASK_CAP=<n>: mesh workgroups a task may start; 0: one a thread)
static const char* e = getenv("RX_VK_TASK_CAP");
const u32 n = e ? static_cast<u32>(atoi(e)) : 0U;
return n ? n : threads;
}
void TaskEpilogue(EmitContext& ctx) {
Barrier(ctx);
// (hookshot) at most one mesh workgroup a task thread: each thread keeps or culls one meshlet (the programs compact them by subgroup
// ballot). It was 65535, and a payload word a task left unwritten is anything: when the task culled everything, tens of thousands of
// mesh workgroups for a draw held the GPU past Windows' two seconds, and the device was lost, on about half the first starts
const Id count{ctx.OpUMin(ctx.U32[1], LoadWord(ctx, PayloadWord(ctx, ctx.Const(1U))),
ctx.Const(task_cap(ctx.mesh_threads)))};
if (ctx.mesh_fallback) {
// (no OpEmitMeshTasksEXT: each thread that started a mesh workgroup takes an index of the mesh dispatch
// (MESH_FALLBACK_DISPATCH_WORD) and writes its slot, task workgroup * threads + its index, at that index of the
// payload buffer's map. The mesh dispatch is those indices: no more, whatever the CPU knew of the task count)
const Id slot{ctx.OpIAdd(ctx.U32[1],
ctx.OpIMul(ctx.U32[1], WorkgroupId(ctx), ctx.Const(ctx.mesh_threads)),
LocalInvocation(ctx))};
// (the count is how many meshlets this task workgroup started: the first that many threads, not all of them)
const Id started{ctx.OpLogicalAnd(ctx.U1, ctx.OpULessThan(ctx.U1, LocalInvocation(ctx), count),
ctx.OpULessThan(ctx.U1, slot, ctx.Const(MESH_FALLBACK_CAPACITY)))};
If(ctx, started, [&] {
const Id index{ctx.OpAtomicIAdd(
ctx.U32[1],
BufferWord(ctx, ctx.mesh_fallback_records_member_index, ctx.Const(MESH_FALLBACK_DISPATCH_WORD)),
ctx.Const(static_cast<u32>(spv::Scope::Device)), ctx.u32_zero_value, ctx.Const(1U))};
// (fewer than MESH_FALLBACK_CAPACITY: a started slot is below it)
StoreWord(ctx, BufferWordAt(ctx, PushU64(ctx, ctx.mesh_fallback_payload_member_index), index), slot);
});
// (the payload once a task workgroup, its threads sharing the words, where its mesh workgroups read it (PayloadWord).
// A copy a slot was 32 times the writes and the memory: 52 MB a draw at 16384 slots)
const Id task{WorkgroupId(ctx)};
const Id first_slot{ctx.OpIMul(ctx.U32[1], task, ctx.Const(ctx.mesh_threads))};
If(ctx, ctx.OpULessThan(ctx.U1, first_slot, ctx.Const(MESH_FALLBACK_CAPACITY)), [&] {
const Id first{ctx.OpIAdd(
ctx.U32[1], ctx.Const(MESH_FALLBACK_CAPACITY),
ctx.OpIMul(ctx.U32[1], task, ctx.Const(ctx.mesh_info.payload_words)))};
Strided(ctx, ctx.mesh_info.payload_words, ctx.mesh_threads, LocalInvocation(ctx), [&](Id index) {
StoreWord(ctx, BufferWordAt(ctx, PushU64(ctx, ctx.mesh_fallback_payload_member_index),
ctx.OpIAdd(ctx.U32[1], first, index)),
LoadWord(ctx, PayloadWord(ctx, index)));
});
});
return;
}
const Id one{ctx.Const(1U)};
// Marker for OpEmitMeshTasksEXT count, 1, 1, payload (PatchMeshInstructions).
ctx.OpStore(count, one);
ctx.OpStore(one, ctx.mesh_payload);
ctx.mesh_tasks_marker = {count, one, one, ctx.mesh_payload};
}
void MeshEpilogue(EmitContext& ctx) {
const MeshInfo& mesh{ctx.mesh_info};
if (ctx.mesh_fallback) {
MeshFallbackEpilogue(ctx);
return;
}
Barrier(ctx);
const Id count{ctx.OpUMin(ctx.U32[1], ctx.OpLoad(ctx.U32[1], ctx.mesh_primitive_count),
ctx.Const(mesh.max_primitives))};
const Id max_vertices{ctx.Const(mesh.max_vertices)};
// Marker for OpSetMeshOutputsEXT max_vertices, count (PatchMeshInstructions).
ctx.OpStore(max_vertices, count);
ctx.mesh_outputs_marker = {max_vertices, count};
const Id invocation{LocalInvocation(ctx)};
const u32 threads{ctx.mesh_threads};
const bool convert_depth{ctx.runtime_info.convert_depth_mode && !ctx.profile.support_native_ndc &&
Sirit::ValidId(ctx.output_position)};
const u32 vertex_lines{static_cast<u32>(mesh.vertex_lines.size())};
for (u32 first = 0; first < mesh.max_vertices; first += threads) {
const Id vertex{ctx.OpIAdd(ctx.U32[1], invocation, ctx.Const(first))};
const auto copy = [&] {
for (const auto& d : ctx.mesh_vertex_defaults) {
for (u32 c = 0; c < 4; ++c) {
if (d.missing & (1U << c)) {
ctx.OpStore(ctx.OpAccessChain(ctx.output_f32, d.variable, vertex, ctx.Const(c)),
ctx.Const(c == 3 ? 1.0f : 0.0f));
}
}
}
CopyLines(ctx, vertex, 0, vertex_lines, ctx.mesh_vertex_targets);
if (convert_depth) {
ConvertDepth(ctx, vertex);
}
};
if (first + threads > mesh.max_vertices) {
If(ctx, ctx.OpULessThan(ctx.U1, vertex, max_vertices), copy);
} else {
copy();
}
}
const u32 primitive_lines{static_cast<u32>(mesh.primitive_lines.size())};
for (u32 first = 0; first < mesh.max_primitives; first += threads) {
const Id primitive{ctx.OpIAdd(ctx.U32[1], invocation, ctx.Const(first))};
If(ctx, ctx.OpULessThan(ctx.U1, primitive, count), [&] {
// Vertex indices: map bytes 4 + 3 * primitive (lines: 4 + 2 * primitive, points: 4 + primitive).
const u32 corners{mesh.Corners()};
const Id byte{ctx.OpIAdd(ctx.U32[1], ctx.OpIMul(ctx.U32[1], primitive, ctx.Const(corners)),
ctx.Const(4U))};
std::array<Id, 3> indices;
for (u32 element = 0; element < corners; ++element) {
const Id address{ctx.OpIAdd(ctx.U32[1], byte, ctx.Const(element))};
// (hookshot) within the vertices: an index byte the program left unwritten is anything, and an index
// past the vertex count is undefined (a device fault on NVIDIA)
indices[element] = Clamped(ctx, ctx.OpLoad(ctx.U32[1], SharedWord(ctx, ctx.mesh_map, address)),
mesh.max_vertices);
}
const Id output_indices{ctx.TypePointer(spv::StorageClass::Output, ctx.U32[corners])};
ctx.OpStore(ctx.OpAccessChain(output_indices, ctx.mesh_primitive_indices, primitive),
corners == 1 ? indices[0] // (a point's index is a scalar)
: ctx.OpCompositeConstruct(ctx.U32[corners],
std::span<const Id>(indices.data(), corners)));
CopyLines(ctx, primitive, ctx.mesh_primitive_base, primitive_lines,
ctx.mesh_primitive_targets);
});
}
}
} // Anonymous namespace
void EmitMeshFallbackVertexBody(EmitContext& ctx) {
// (the fragment stage's inputs for the mesh program's per-primitive generics are Flat: the generated vertex
// stage's outputs must match, and DefineOutputs only knows the stores, not which of them are per-primitive)
for (u32 index = 0; index < IR::NUM_GENERICS; ++index) {
if (!ctx.runtime_info.per_primitive_generics.test(index)) {
continue;
}
for (u32 element = 0; element < 4; ++element) {
const Id id{ctx.output_generics[index][element].id};
if (!Sirit::ValidId(id)) {
continue;
}
bool seen{};
for (u32 before = 0; before < element; ++before) {
seen |= ctx.output_generics[index][before].id.value == id.value;
}
if (!seen) {
ctx.Decorate(id, spv::Decoration::Flat);
}
}
}
// (every output component starts at (0,0,0,1), as the native mesh stage's do (6dcf6f1) and a vertex stage's; the record's
// lines below go over the ones the mesh program wrote. Left undefined, AMD's driver gave the fragment stage whatever its
// registers held: specks of colour in Kokiri Forest's bushes on an RX 5500M)
for (u32 index = 0; index < IR::NUM_GENERICS; ++index) {
for (u32 element = 0; element < 4; ++element) {
const GenericElementInfo& info{ctx.output_generics[index][element]};
if (!Sirit::ValidId(info.id)) {
continue;
}
const Id value{element == 3 ? ctx.Const(1.0f) : ctx.f32_zero_value};
if (info.num_components == 1) {
ctx.OpStore(info.id, value);
} else {
ctx.OpStore(ctx.OpAccessChain(ctx.output_f32, info.id, ctx.Const(element - info.first_element)), value);
}
}
}
if (Sirit::ValidId(ctx.output_point_size)) {
ctx.OpStore(ctx.output_point_size, ctx.Const(1.0f));
}
const MeshInfo& mesh{ctx.mesh_info};
const u32 vertex_lines{static_cast<u32>(mesh.vertex_lines.size())};
const u32 primitive_lines{static_cast<u32>(mesh.primitive_lines.size())};
const u32 words{FallbackRecordWords(mesh)};
// (a record a vertex, in the order the mesh workgroups took them (MeshFallbackEpilogue): the indirect draw's count is the
// records taken, so every one it reaches was written)
const Id vertex{ctx.OpLoad(ctx.U32[1], ctx.vertex_index)};
const Id first{ctx.OpIAdd(ctx.U32[1], ctx.Const(MESH_FALLBACK_HEADER_WORDS),
ctx.OpIMul(ctx.U32[1], vertex, ctx.Const(words)))};
const Id valid{LoadWord(ctx, BufferWord(ctx, ctx.mesh_fallback_records_member_index, first))};
const auto read = [&](u32 word) {
return ctx.OpBitcast(
ctx.F32[1],
LoadWord(ctx, BufferWord(ctx, ctx.mesh_fallback_records_member_index,
ctx.OpIAdd(ctx.U32[1], first, ctx.Const(word)))));
};
std::array<Id, 4> position{};
for (u32 line = 0; line < vertex_lines; ++line) {
const IR::Attribute attribute{mesh.vertex_lines[line]};
const Id value{read(1U + line)};
const u32 raw{static_cast<u32>(attribute)};
if (raw >= static_cast<u32>(IR::Attribute::PositionX) &&
raw <= static_cast<u32>(IR::Attribute::PositionW)) {
position[raw - static_cast<u32>(IR::Attribute::PositionX)] = value;
} else {
StoreFallbackOutput(ctx, attribute, value);
}
}
for (u32 line = 0; line < primitive_lines; ++line) {
StoreFallbackOutput(ctx, IR::Attribute{mesh.primitive_lines[line]},
read(1U + vertex_lines + line));
}
// (OpenGL-style clip space, as the mesh epilogue's ConvertDepth)
if (ctx.runtime_info.convert_depth_mode && !ctx.profile.support_native_ndc &&
Sirit::ValidId(position[2]) && Sirit::ValidId(position[3])) {
position[2] = ctx.OpFMul(ctx.F32[1], ctx.OpFAdd(ctx.F32[1], position[2], position[3]),
ctx.Const(0.5f));
}
for (u32 element = 0; element < 4; ++element) {
if (Sirit::ValidId(position[element])) {
StorePosition(ctx, element, position[element]);
}
}
// (a record no mesh workgroup wrote, which the draw's count should never reach: outside the clip volume)
If(ctx, ctx.OpIEqual(ctx.U1, valid, ctx.u32_zero_value),
[&] {
StorePosition(ctx, 0, ctx.f32_zero_value);
StorePosition(ctx, 1, ctx.f32_zero_value);
StorePosition(ctx, 2, ctx.f32_zero_value);
StorePosition(ctx, 3, ctx.Const(-1.0f));
});
}
Id EmitIsbeRead(EmitContext& ctx, Id address) {
const Id native_workgroup{
ctx.OpCompositeExtract(ctx.U32[1], ctx.OpLoad(ctx.U32[3], ctx.workgroup_id), 0U)};
if (ctx.stage == Stage::Task) {
const Id workgroup{native_workgroup};
if (ctx.mesh_fallback) {
const Id input{PushU64(ctx, ctx.mesh_fallback_indirect_input_member_index)};
const Id indirect{ctx.OpINotEqual(ctx.U1, input, ctx.Constant(ctx.U64, u64{0}))};
// On a direct draw the header's instance count is a valid, read-only word for the unused load.
const Id base{ctx.OpSelect(ctx.U64, indirect, input, PushU64(ctx, ctx.mesh_fallback_records_member_index))};
const Id gpu_first{LoadWord(ctx, BufferWordAt(ctx, base, ctx.Const(1U)))};
const Id first{ctx.OpSelect(ctx.U32[1], indirect, gpu_first, PushU32(ctx, ctx.mesh_first_member_index))};
return ctx.OpIAdd(ctx.U32[1], workgroup, first);
}
if (ctx.wants_mesh_first && Sirit::ValidId(ctx.rescaling_push_constants)) {
// (hookshot) + the draw's first workgroup: NVN draws [first, first + count). Without it every draw began at 0: the
// foliage's draws (first up to ~1900) drew the first records again and again, and the forest had no leaves
const Id table{ctx.OpLoad(ctx.U64, ctx.OpAccessChain(ctx.TypePointer(spv::StorageClass::PushConstant, ctx.U64),
ctx.rescaling_push_constants, ctx.Const(ctx.mesh_first_member_index)))};
const Id draw{ctx.OpLoad(ctx.U32[1], ctx.draw_index)};
const Id at{ctx.OpIAdd(ctx.U64, table, ctx.OpUConvert(ctx.U64, ctx.OpIMul(ctx.U32[1], draw, ctx.Const(16U))))};
const Id member{ctx.OpAccessChain(ctx.mesh_first_member_pointer, ctx.OpConvertUToPtr(ctx.mesh_first_block_pointer, at),
ctx.Const(0U))};
const Id first{ctx.OpLoad(ctx.U32[1], member, spv::MemoryAccessMask::Aligned, 4U)};
return ctx.OpIAdd(ctx.U32[1], workgroup, first);
}
return workgroup; // the input region holds the workgroup index (VERTEX_ID)
}
const Id workgroup{native_workgroup};
const Id index{ctx.OpShiftRightLogical(ctx.U32[1], address, ctx.Const(2U))};
const Id pointer{PayloadWord(ctx, index)};
const Id word{ctx.mesh_fallback ? LoadWord(ctx, pointer) : ctx.OpLoad(ctx.U32[1], pointer)}; // (the fallback's: a device address)
// (hookshot) a fallback mesh workgroup is a slot, task workgroup * threads + its index: address 0 is that index in its task,
// from 0 as OpEmitMeshTasksEXT numbers them. It was the slot, which picked a meshlet past the end of every task's list
const Id within{ctx.mesh_fallback && ctx.runtime_info.mesh_fallback_task
? ctx.OpUMod(ctx.U32[1], FallbackSlot(ctx), ctx.Const(ctx.runtime_info.mesh_fallback_task_threads))
: workgroup};
return ctx.OpSelect(ctx.U32[1], ctx.OpIEqual(ctx.U1, address, ctx.u32_zero_value), within,
word);
}
void EmitIsbeWriteAttribute(EmitContext& ctx, Id address, Id value) {
const Id index{ctx.OpShiftRightLogical(ctx.U32[1], address, ctx.Const(2U))};
if (ctx.stage == Stage::Task) {
StoreWord(ctx, PayloadWord(ctx, index), value);
return;
}
ctx.OpStore(SharedWord(ctx, ctx.mesh_attributes, Clamped(ctx, index, ctx.mesh_attribute_words)),
value);
}
void EmitIsbeWriteMapU8(EmitContext& ctx, Id address, Id value) {
if (ctx.stage != Stage::Mesh) {
throw NotImplementedException("Stage buffer map write outside mesh programs");
}
ctx.OpStore(SharedWord(ctx, ctx.mesh_map, Clamped(ctx, address, ctx.mesh_map_bytes)),
ctx.OpBitwiseAnd(ctx.U32[1], value, ctx.Const(0xffU)));
}
void EmitIsbeWriteMapU32(EmitContext& ctx, const IR::Value& address, Id value) {
// The primitive count: the map is written from its end, so its first word is at byte 3.
if (ctx.stage != Stage::Mesh || !address.IsImmediate() || address.U32() != 3) {
throw NotImplementedException("32-bit stage buffer map write");
}
ctx.OpStore(ctx.mesh_primitive_count, value);
}
void EmitMeshPrologue(EmitContext& ctx) {
if (ctx.stage == Stage::Task) {
// (hookshot) the count starts at nothing: task payload memory holds whatever the last task left, and a task that keeps no meshlet
// never writes it. The barrier: every thread has done this before any of them goes on to write the real count
StoreWord(ctx, PayloadWord(ctx, ctx.Const(1U)), ctx.u32_zero_value);
Barrier(ctx);
}
if (ctx.stage != Stage::Mesh) {
return;
}
if (ctx.mesh_fallback && ctx.runtime_info.mesh_fallback_task) {
// (hookshot) the slot its task gave this workgroup, from the map at its index of the dispatch (TaskEpilogue). Read here,
// in the entry block, it serves every use; a load, no branch (see below)
ctx.mesh_fallback_slot =
LoadWord(ctx, BufferWordAt(ctx, PushU64(ctx, ctx.mesh_fallback_payload_member_index), WorkgroupId(ctx)));
}
// (local) The stage buffer's map and attribute regions start at zero, as they come out on NVIDIA: workgroup memory holds
// whatever the last workgroup on that unit left, and a vertex or an index byte the program left unwritten was that. On an
// RX 7700S the foliage drawn into the 2400 x 2400 shadow map around Link came out with hundreds of long thin triangles
// across the whole map (vertices from other meshlets' leftovers), dark lines on the ground, the water and Link that
// changed every frame. Zeroed, such a primitive is (0,0,0): no area. Straight-line stores, each clamped into its array
// (no branch: see below). RX_VK_MESH_NO_ZERO=1: as before
static const bool no_zero{std::getenv("RX_VK_MESH_NO_ZERO") != nullptr};
if (!no_zero && ctx.mesh_threads) {
const Id invocation{LocalInvocation(ctx)};
const auto zero_all = [&](Id array, u32 count) {
if (!Sirit::ValidId(array) || count == 0) {
return;
}
for (u32 first = 0; first < count; first += ctx.mesh_threads) {
const Id at{Clamped(ctx, ctx.OpIAdd(ctx.U32[1], invocation, ctx.Const(first)), count)};
ctx.OpStore(SharedWord(ctx, array, at), ctx.u32_zero_value);
}
};
zero_all(ctx.mesh_map, ctx.mesh_map_bytes);
zero_all(ctx.mesh_attributes, ctx.mesh_attribute_words);
}
if (ctx.mesh_fallback && !ctx.runtime_info.mesh_fallback_task && ctx.mesh_threads) {
// (a mesh program with no task reads its workgroup index from the payload: NVN's hardware puts it at byte 0)
const Id invocation{LocalInvocation(ctx)};
for (u32 first = 0; first < ctx.mesh_info.payload_words; first += ctx.mesh_threads) {
const Id at{Clamped(ctx, ctx.OpIAdd(ctx.U32[1], invocation, ctx.Const(first)),
ctx.mesh_info.payload_words)};
ctx.OpStore(SharedWord(ctx, ctx.mesh_payload, at), ctx.u32_zero_value);
}
if (ctx.mesh_info.payload_words) {
ctx.OpStore(SharedWord(ctx, ctx.mesh_payload, ctx.u32_zero_value), WorkgroupId(ctx));
}
Barrier(ctx);
}
// Every invocation stores the zero: no branch here, the entry block's label is a phi
// predecessor for later blocks.
ctx.OpStore(ctx.mesh_primitive_count, ctx.u32_zero_value);
Barrier(ctx);
}
void EmitMeshEpilogue(EmitContext& ctx) {
if (ctx.stage == Stage::Task) {
TaskEpilogue(ctx);
} else if (ctx.stage == Stage::Mesh) {
MeshEpilogue(ctx);
}
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,41 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
Id EmitSelectU1(EmitContext& ctx, Id cond, Id true_value, Id false_value) {
return ctx.OpSelect(ctx.U1, cond, true_value, false_value);
}
Id EmitSelectU8(EmitContext&, Id, Id, Id) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitSelectU16(EmitContext& ctx, Id cond, Id true_value, Id false_value) {
return ctx.OpSelect(ctx.U16, cond, true_value, false_value);
}
Id EmitSelectU32(EmitContext& ctx, Id cond, Id true_value, Id false_value) {
return ctx.OpSelect(ctx.U32[1], cond, true_value, false_value);
}
Id EmitSelectU64(EmitContext& ctx, Id cond, Id true_value, Id false_value) {
return ctx.OpSelect(ctx.U64, cond, true_value, false_value);
}
Id EmitSelectF16(EmitContext& ctx, Id cond, Id true_value, Id false_value) {
return ctx.OpSelect(ctx.F16[1], cond, true_value, false_value);
}
Id EmitSelectF32(EmitContext& ctx, Id cond, Id true_value, Id false_value) {
return ctx.OpSelect(ctx.F32[1], cond, true_value, false_value);
}
Id EmitSelectF64(EmitContext& ctx, Id cond, Id true_value, Id false_value) {
return ctx.OpSelect(ctx.F64[1], cond, true_value, false_value);
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,176 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
namespace {
Id Pointer(EmitContext& ctx, Id pointer_type, Id array, Id offset, u32 shift) {
const Id shift_id{ctx.Const(shift)};
const Id index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)};
return ctx.OpAccessChain(pointer_type, array, ctx.u32_zero_value, index);
}
Id Word(EmitContext& ctx, Id offset) {
const Id shift_id{ctx.Const(2U)};
const Id index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)};
const Id pointer{ctx.OpAccessChain(ctx.shared_u32, ctx.shared_memory_u32, index)};
return ctx.OpLoad(ctx.U32[1], pointer);
}
std::pair<Id, Id> ExtractArgs(EmitContext& ctx, Id offset, u32 mask, u32 count) {
const Id shift{ctx.OpShiftLeftLogical(ctx.U32[1], offset, ctx.Const(3U))};
const Id bit{ctx.OpBitwiseAnd(ctx.U32[1], shift, ctx.Const(mask))};
const Id count_id{ctx.Const(count)};
return {bit, count_id};
}
} // Anonymous namespace
Id EmitLoadSharedU8(EmitContext& ctx, Id offset) {
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{
ctx.OpAccessChain(ctx.shared_u8, ctx.shared_memory_u8, ctx.u32_zero_value, offset)};
return ctx.OpUConvert(ctx.U32[1], ctx.OpLoad(ctx.U8, pointer));
} else {
const auto [bit, count]{ExtractArgs(ctx, offset, 24, 8)};
return ctx.OpBitFieldUExtract(ctx.U32[1], Word(ctx, offset), bit, count);
}
}
Id EmitLoadSharedS8(EmitContext& ctx, Id offset) {
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{
ctx.OpAccessChain(ctx.shared_u8, ctx.shared_memory_u8, ctx.u32_zero_value, offset)};
return ctx.OpSConvert(ctx.U32[1], ctx.OpLoad(ctx.U8, pointer));
} else {
const auto [bit, count]{ExtractArgs(ctx, offset, 24, 8)};
return ctx.OpBitFieldSExtract(ctx.U32[1], Word(ctx, offset), bit, count);
}
}
Id EmitLoadSharedU16(EmitContext& ctx, Id offset) {
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u16, ctx.shared_memory_u16, offset, 1)};
return ctx.OpUConvert(ctx.U32[1], ctx.OpLoad(ctx.U16, pointer));
} else {
const auto [bit, count]{ExtractArgs(ctx, offset, 16, 16)};
return ctx.OpBitFieldUExtract(ctx.U32[1], Word(ctx, offset), bit, count);
}
}
Id EmitLoadSharedS16(EmitContext& ctx, Id offset) {
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u16, ctx.shared_memory_u16, offset, 1)};
return ctx.OpSConvert(ctx.U32[1], ctx.OpLoad(ctx.U16, pointer));
} else {
const auto [bit, count]{ExtractArgs(ctx, offset, 16, 16)};
return ctx.OpBitFieldSExtract(ctx.U32[1], Word(ctx, offset), bit, count);
}
}
Id EmitLoadSharedU32(EmitContext& ctx, Id offset) {
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32, ctx.shared_memory_u32, offset, 2)};
return ctx.OpLoad(ctx.U32[1], pointer);
} else {
return Word(ctx, offset);
}
}
Id EmitLoadSharedU64(EmitContext& ctx, Id offset) {
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32x2, ctx.shared_memory_u32x2, offset, 3)};
return ctx.OpLoad(ctx.U32[2], pointer);
} else {
const Id shift_id{ctx.Const(2U)};
const Id base_index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)};
const Id next_index{ctx.OpIAdd(ctx.U32[1], base_index, ctx.Const(1U))};
const Id lhs_pointer{ctx.OpAccessChain(ctx.shared_u32, ctx.shared_memory_u32, base_index)};
const Id rhs_pointer{ctx.OpAccessChain(ctx.shared_u32, ctx.shared_memory_u32, next_index)};
return ctx.OpCompositeConstruct(ctx.U32[2], ctx.OpLoad(ctx.U32[1], lhs_pointer),
ctx.OpLoad(ctx.U32[1], rhs_pointer));
}
}
Id EmitLoadSharedU128(EmitContext& ctx, Id offset) {
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32x4, ctx.shared_memory_u32x4, offset, 4)};
return ctx.OpLoad(ctx.U32[4], pointer);
}
const Id shift_id{ctx.Const(2U)};
const Id base_index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift_id)};
std::array<Id, 4> values{};
for (u32 i = 0; i < 4; ++i) {
const Id index{i == 0 ? base_index : ctx.OpIAdd(ctx.U32[1], base_index, ctx.Const(i))};
const Id pointer{ctx.OpAccessChain(ctx.shared_u32, ctx.shared_memory_u32, index)};
values[i] = ctx.OpLoad(ctx.U32[1], pointer);
}
return ctx.OpCompositeConstruct(ctx.U32[4], values);
}
void EmitWriteSharedU8(EmitContext& ctx, Id offset, Id value) {
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{
ctx.OpAccessChain(ctx.shared_u8, ctx.shared_memory_u8, ctx.u32_zero_value, offset)};
ctx.OpStore(pointer, ctx.OpUConvert(ctx.U8, value));
} else {
ctx.OpFunctionCall(ctx.void_id, ctx.shared_store_u8_func, offset, value);
}
}
void EmitWriteSharedU16(EmitContext& ctx, Id offset, Id value) {
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u16, ctx.shared_memory_u16, offset, 1)};
ctx.OpStore(pointer, ctx.OpUConvert(ctx.U16, value));
} else {
ctx.OpFunctionCall(ctx.void_id, ctx.shared_store_u16_func, offset, value);
}
}
void EmitWriteSharedU32(EmitContext& ctx, Id offset, Id value) {
Id pointer{};
if (ctx.uses_explicit_workgroup_layout) {
pointer = Pointer(ctx, ctx.shared_u32, ctx.shared_memory_u32, offset, 2);
} else {
const Id shift{ctx.Const(2U)};
const Id word_offset{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift)};
pointer = ctx.OpAccessChain(ctx.shared_u32, ctx.shared_memory_u32, word_offset);
}
ctx.OpStore(pointer, value);
}
void EmitWriteSharedU64(EmitContext& ctx, Id offset, Id value) {
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32x2, ctx.shared_memory_u32x2, offset, 3)};
ctx.OpStore(pointer, value);
return;
}
const Id shift{ctx.Const(2U)};
const Id word_offset{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift)};
const Id next_offset{ctx.OpIAdd(ctx.U32[1], word_offset, ctx.Const(1U))};
const Id lhs_pointer{ctx.OpAccessChain(ctx.shared_u32, ctx.shared_memory_u32, word_offset)};
const Id rhs_pointer{ctx.OpAccessChain(ctx.shared_u32, ctx.shared_memory_u32, next_offset)};
ctx.OpStore(lhs_pointer, ctx.OpCompositeExtract(ctx.U32[1], value, 0U));
ctx.OpStore(rhs_pointer, ctx.OpCompositeExtract(ctx.U32[1], value, 1U));
}
void EmitWriteSharedU128(EmitContext& ctx, Id offset, Id value) {
if (ctx.uses_explicit_workgroup_layout) {
const Id pointer{Pointer(ctx, ctx.shared_u32x4, ctx.shared_memory_u32x4, offset, 4)};
ctx.OpStore(pointer, value);
return;
}
const Id shift{ctx.Const(2U)};
const Id base_index{ctx.OpShiftRightArithmetic(ctx.U32[1], offset, shift)};
for (u32 i = 0; i < 4; ++i) {
const Id index{i == 0 ? base_index : ctx.OpIAdd(ctx.U32[1], base_index, ctx.Const(i))};
const Id pointer{ctx.OpAccessChain(ctx.shared_u32, ctx.shared_memory_u32, index)};
ctx.OpStore(pointer, ctx.OpCompositeExtract(ctx.U32[1], value, i));
}
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,186 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv.h"
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
namespace {
void ConvertDepthMode(EmitContext& ctx) {
const Id type{ctx.F32[1]};
const Id position{ctx.OpLoad(ctx.F32[4], ctx.output_position)};
const Id z{ctx.OpCompositeExtract(type, position, 2u)};
const Id w{ctx.OpCompositeExtract(type, position, 3u)};
const Id screen_depth{ctx.OpFMul(type, ctx.OpFAdd(type, z, w), ctx.Constant(type, 0.5f))};
const Id vector{ctx.OpCompositeInsert(ctx.F32[4], screen_depth, position, 2u)};
ctx.OpStore(ctx.output_position, vector);
}
void SetFixedPipelinePointSize(EmitContext& ctx) {
if (ctx.runtime_info.fixed_state_point_size) {
const float point_size{*ctx.runtime_info.fixed_state_point_size};
ctx.OpStore(ctx.output_point_size, ctx.Const(point_size));
}
}
Id DefaultVarying(EmitContext& ctx, u32 num_components, u32 element, Id zero, Id one,
Id default_vector) {
switch (num_components) {
case 1:
return element == 3 ? one : zero;
case 2:
return ctx.ConstantComposite(ctx.F32[2], zero, element + 1 == 3 ? one : zero);
case 3:
return ctx.ConstantComposite(ctx.F32[3], zero, zero, element + 2 == 3 ? one : zero);
case 4:
return default_vector;
}
throw InvalidArgument("Bad element");
}
Id ComparisonFunction(EmitContext& ctx, CompareFunction comparison, Id operand_1, Id operand_2) {
switch (comparison) {
case CompareFunction::Never:
return ctx.false_value;
case CompareFunction::Less:
return ctx.OpFOrdLessThan(ctx.U1, operand_1, operand_2);
case CompareFunction::Equal:
return ctx.OpFOrdEqual(ctx.U1, operand_1, operand_2);
case CompareFunction::LessThanEqual:
return ctx.OpFOrdLessThanEqual(ctx.U1, operand_1, operand_2);
case CompareFunction::Greater:
return ctx.OpFOrdGreaterThan(ctx.U1, operand_1, operand_2);
case CompareFunction::NotEqual:
return ctx.OpFOrdNotEqual(ctx.U1, operand_1, operand_2);
case CompareFunction::GreaterThanEqual:
return ctx.OpFOrdGreaterThanEqual(ctx.U1, operand_1, operand_2);
case CompareFunction::Always:
return ctx.true_value;
}
throw InvalidArgument("Comparison function {}", comparison);
}
void AlphaTest(EmitContext& ctx) {
if (!ctx.runtime_info.alpha_test_func) {
return;
}
const auto comparison{*ctx.runtime_info.alpha_test_func};
if (comparison == CompareFunction::Always) {
return;
}
if (!Sirit::ValidId(ctx.frag_color[0])) {
return;
}
const Id type{ctx.F32[1]};
const Id rt0_color{ctx.OpLoad(ctx.F32[4], ctx.frag_color[0])};
const Id alpha{ctx.OpCompositeExtract(type, rt0_color, 3u)};
const Id true_label{ctx.OpLabel()};
const Id discard_label{ctx.OpLabel()};
const Id alpha_reference{ctx.Const(ctx.runtime_info.alpha_test_reference)};
const Id condition{ComparisonFunction(ctx, comparison, alpha, alpha_reference)};
ctx.OpSelectionMerge(true_label, spv::SelectionControlMask::MaskNone);
ctx.OpBranchConditional(condition, true_label, discard_label);
ctx.AddLabel(discard_label);
ctx.OpKill();
ctx.AddLabel(true_label);
}
} // Anonymous namespace
void EmitPrologue(EmitContext& ctx) {
if (ctx.stage == Stage::Fragment && ctx.runtime_info.dual_source_blend) {
// Initialize dual-source blending outputs - prevents MoltenVK crash.
const Id zero{ctx.Const(0.0f)};
const Id one{ctx.Const(1.0f)};
const Id default_color{ctx.ConstantComposite(ctx.F32[4], zero, zero, zero, one)};
for (u32 i = 0; i < 2; ++i) {
if (Sirit::ValidId(ctx.frag_color[i])) {
ctx.OpStore(ctx.frag_color[i], default_color);
}
}
}
if (ctx.stage == Stage::VertexB) {
const Id zero{ctx.Const(0.0f)};
const Id one{ctx.Const(1.0f)};
const Id default_vector{ctx.ConstantComposite(ctx.F32[4], zero, zero, zero, one)};
ctx.OpStore(ctx.output_position, default_vector);
for (const auto& info : ctx.output_generics) {
if (info[0].num_components == 0) {
continue;
}
u32 element{0};
while (element < 4) {
const auto& element_info{info[element]};
const u32 num{element_info.num_components};
const Id value{DefaultVarying(ctx, num, element, zero, one, default_vector)};
ctx.OpStore(element_info.id, value);
element += num;
}
}
if (Sirit::ValidId(ctx.clip_distances)) {
for (u32 i = 0; i < ctx.profile.max_user_clip_distances; ++i) {
if (!clip_distance_written.test(i)) {
const Id idx = ctx.Const(i);
const Id element = ctx.OpAccessChain(ctx.output_f32, ctx.clip_distances, idx);
ctx.OpStore(element, ctx.Const(0.0f));
}
}
}
}
if (ctx.stage == Stage::VertexB || ctx.stage == Stage::Geometry) {
SetFixedPipelinePointSize(ctx);
}
if (ctx.stage == Stage::Task || ctx.stage == Stage::Mesh) {
EmitMeshPrologue(ctx);
}
}
void EmitEpilogue(EmitContext& ctx) {
if (ctx.stage == Stage::Task || ctx.stage == Stage::Mesh) {
EmitMeshEpilogue(ctx);
return;
}
if (ctx.stage == Stage::VertexB && ctx.runtime_info.convert_depth_mode &&
!ctx.profile.support_native_ndc) {
ConvertDepthMode(ctx);
}
if (ctx.stage == Stage::Fragment) {
AlphaTest(ctx);
}
}
void EmitEmitVertex(EmitContext& ctx, const IR::Value& stream) {
if (ctx.runtime_info.convert_depth_mode && !ctx.profile.support_native_ndc) {
ConvertDepthMode(ctx);
}
if (!ctx.profile.support_geometry_streams) {
throw NotImplementedException("Geometry streams");
} else if (stream.IsImmediate()) {
ctx.OpEmitStreamVertex(ctx.Def(stream));
} else {
LOG_WARNING(Shader_SPIRV, "Stream is not immediate");
ctx.OpEmitStreamVertex(ctx.u32_zero_value);
}
// Restore fixed pipeline point size after emitting the vertex
SetFixedPipelinePointSize(ctx);
}
void EmitEndPrimitive(EmitContext& ctx, const IR::Value& stream) {
if (!ctx.profile.support_geometry_streams) {
throw NotImplementedException("Geometry streams");
} else if (stream.IsImmediate()) {
ctx.OpEndStreamPrimitive(ctx.Def(stream));
} else {
LOG_WARNING(Shader_SPIRV, "Stream is not immediate");
ctx.OpEndStreamPrimitive(ctx.u32_zero_value);
}
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,47 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <cstdlib>
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
namespace Shader::Backend::SPIRV {
namespace {
// Registers read before any write are zero rather than OpUndef, as NVIDIA hardware and drivers
// give them; AMD's OpUndef is whatever the register held. Title 1's glow pass exits before
// writing its colour on some pixels, which AMD blended in as green patches on the road.
// EDEN_NX2_UNDEF_ZERO=0 emits OpUndef again.
bool UndefIsZero() {
static const bool zero = [] {
const char* const value = std::getenv("EDEN_NX2_UNDEF_ZERO");
return value == nullptr || value[0] != '0';
}();
return zero;
}
} // Anonymous namespace
Id EmitUndefU1(EmitContext& ctx) {
return UndefIsZero() ? ctx.false_value : ctx.OpUndef(ctx.U1);
}
Id EmitUndefU8(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitUndefU16(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
Id EmitUndefU32(EmitContext& ctx) {
return UndefIsZero() ? ctx.u32_zero_value : ctx.OpUndef(ctx.U32[1]);
}
Id EmitUndefU64(EmitContext&) {
throw NotImplementedException("SPIR-V Instruction");
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,540 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/backend/spirv/emit_spirv_instructions.h"
#include "shader_recompiler/backend/spirv/spirv_emit_context.h"
#include "shader_recompiler/backend/spirv/mma_layout.h"
namespace Shader::Backend::SPIRV {
namespace {
Id SubgroupScope(EmitContext& ctx) {
return ctx.Const(static_cast<u32>(spv::Scope::Subgroup));
}
bool StageSupportsSubgroups(EmitContext& ctx) {
return ctx.profile.SupportsSubgroupStage(ctx.stage);
}
Id GetThreadId(EmitContext& ctx) {
if (!StageSupportsSubgroups(ctx)) {
return ctx.u32_zero_value;
}
return ctx.OpLoad(ctx.U32[1], ctx.subgroup_local_invocation_id);
}
Id WarpExtract(EmitContext& ctx, Id value) {
const Id thread_id{GetThreadId(ctx)};
const Id local_index{ctx.OpShiftRightArithmetic(ctx.U32[1], thread_id, ctx.Const(5U))};
if (ctx.profile.has_broken_spirv_subgroup_mask_vector_extract_dynamic) {
const Id c0_sel{ctx.OpSelect(ctx.U32[1], ctx.OpIEqual(ctx.U1, local_index, ctx.Const(0U)),
ctx.OpCompositeExtract(ctx.U32[1], value, 0U), ctx.Const(0U))};
const Id c1_sel{ctx.OpSelect(ctx.U32[1], ctx.OpIEqual(ctx.U1, local_index, ctx.Const(1U)),
ctx.OpCompositeExtract(ctx.U32[1], value, 1U), ctx.Const(0U))};
const Id c2_sel{ctx.OpSelect(ctx.U32[1], ctx.OpIEqual(ctx.U1, local_index, ctx.Const(2U)),
ctx.OpCompositeExtract(ctx.U32[1], value, 2U), ctx.Const(0U))};
const Id c3_sel{ctx.OpSelect(ctx.U32[1], ctx.OpIEqual(ctx.U1, local_index, ctx.Const(3U)),
ctx.OpCompositeExtract(ctx.U32[1], value, 3U), ctx.Const(0U))};
const Id c0_or_c1{ctx.OpBitwiseOr(ctx.U32[1], c0_sel, c1_sel)};
const Id c2_or_c3{ctx.OpBitwiseOr(ctx.U32[1], c2_sel, c3_sel)};
const Id c0_or_c1_or_c2_or_c3{ctx.OpBitwiseOr(ctx.U32[1], c0_or_c1, c2_or_c3)};
return c0_or_c1_or_c2_or_c3;
} else {
return ctx.OpVectorExtractDynamic(ctx.U32[1], value, local_index);
}
}
Id LoadMask(EmitContext& ctx, Id mask) {
const Id value{ctx.OpLoad(ctx.U32[4], mask)};
if (!ctx.profile.warp_size_potentially_larger_than_guest) {
return ctx.OpCompositeExtract(ctx.U32[1], value, 0U);
}
return WarpExtract(ctx, value);
}
void SetInBoundsFlag(IR::Inst* inst, Id result) {
IR::Inst* const in_bounds{inst->GetAssociatedPseudoOperation(IR::Opcode::GetInBoundsFromOp)};
if (!in_bounds) {
return;
}
in_bounds->SetDefinition(result);
in_bounds->Invalidate();
}
Id ComputeMinThreadId(EmitContext& ctx, Id thread_id, Id segmentation_mask) {
return ctx.OpBitwiseAnd(ctx.U32[1], thread_id, segmentation_mask);
}
Id ComputeMaxThreadId(EmitContext& ctx, Id min_thread_id, Id clamp, Id not_seg_mask) {
return ctx.OpBitwiseOr(ctx.U32[1], min_thread_id,
ctx.OpBitwiseAnd(ctx.U32[1], clamp, not_seg_mask));
}
Id GetMaxThreadId(EmitContext& ctx, Id thread_id, Id clamp, Id segmentation_mask) {
const Id not_seg_mask{ctx.OpNot(ctx.U32[1], segmentation_mask)};
const Id min_thread_id{ComputeMinThreadId(ctx, thread_id, segmentation_mask)};
return ComputeMaxThreadId(ctx, min_thread_id, clamp, not_seg_mask);
}
Id HostThreadId(EmitContext& ctx, Id thread_id) {
if (!ctx.profile.warp_size_potentially_larger_than_guest) {
return thread_id;
}
const Id partition_idx{ctx.OpShiftRightLogical(ctx.U32[1], GetThreadId(ctx), ctx.Const(5u))};
const Id partition_base{ctx.OpShiftLeftLogical(ctx.U32[1], partition_idx, ctx.Const(5u))};
return ctx.OpIAdd(ctx.U32[1], thread_id, partition_base);
}
Id GuestLane(EmitContext& ctx, Id index) {
return ctx.OpBitwiseAnd(ctx.U32[1], index, ctx.Const(31U));
}
bool UseShuffleFallback(EmitContext& ctx) {
return ctx.profile.has_broken_spirv_subgroup_shuffle ||
(ctx.profile.has_broken_spirv_fragment_shuffle && ctx.stage == Stage::Fragment);
}
Id ShuffleAbsolute(EmitContext& ctx, Id value, Id src_thread_id) {
if (!UseShuffleFallback(ctx)) {
return ctx.OpGroupNonUniformShuffle(ctx.U32[1], SubgroupScope(ctx), value, src_thread_id);
}
Id result{ctx.u32_zero_value};
for (u32 lane = 0; lane < ctx.profile.max_subgroup_size; ++lane) {
const Id read{
ctx.OpGroupNonUniformBroadcast(ctx.U32[1], SubgroupScope(ctx), value, ctx.Const(lane))};
const Id matches{ctx.OpIEqual(ctx.U1, src_thread_id, ctx.Const(lane))};
result = ctx.OpSelect(ctx.U32[1], matches, read, result);
}
return result;
}
Id ShuffleRelative(EmitContext& ctx, Id value, Id delta, Id src_thread_id, spv::Op op) {
if (!ctx.profile.support_shuffle_relative || UseShuffleFallback(ctx)) {
return ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id));
}
const Id result{ctx.OpGroupNonUniformShuffleXor(ctx.U32[1], SubgroupScope(ctx), value, delta)};
ctx.opcode_rewrites.emplace_back(result.value, op);
return result;
}
Id BroadcastLane(EmitContext& ctx, Id value, u32 lane) {
Id result{
ctx.OpGroupNonUniformBroadcast(ctx.U32[1], SubgroupScope(ctx), value, ctx.Const(lane))};
if (!ctx.profile.warp_size_potentially_larger_than_guest) {
return result;
}
const Id partition_idx{ctx.OpShiftRightLogical(ctx.U32[1], GetThreadId(ctx), ctx.Const(5u))};
for (u32 base = 32; base < ctx.profile.max_subgroup_size; base += 32) {
const Id read{ctx.OpGroupNonUniformBroadcast(ctx.U32[1], SubgroupScope(ctx), value,
ctx.Const(base + lane))};
const Id matches{ctx.OpIEqual(ctx.U1, partition_idx, ctx.Const(base >> 5))};
result = ctx.OpSelect(ctx.U32[1], matches, read, result);
}
return result;
}
} // Anonymous namespace
Id EmitLaneId(EmitContext& ctx) {
const Id id{GetThreadId(ctx)};
if (!ctx.profile.warp_size_potentially_larger_than_guest) {
return id;
}
return ctx.OpBitwiseAnd(ctx.U32[1], id, ctx.Const(31U));
}
Id EmitVoteAll(EmitContext& ctx, Id pred) {
if (!StageSupportsSubgroups(ctx)) {
return pred;
}
if (!ctx.profile.warp_size_potentially_larger_than_guest) {
return ctx.OpGroupNonUniformAll(ctx.U1, SubgroupScope(ctx), pred);
}
const Id mask_ballot{
ctx.OpGroupNonUniformBallot(ctx.U32[4], SubgroupScope(ctx), ctx.true_value)};
const Id active_mask{WarpExtract(ctx, mask_ballot)};
const Id ballot{
WarpExtract(ctx, ctx.OpGroupNonUniformBallot(ctx.U32[4], SubgroupScope(ctx), pred))};
const Id lhs{ctx.OpBitwiseAnd(ctx.U32[1], ballot, active_mask)};
return ctx.OpIEqual(ctx.U1, lhs, active_mask);
}
Id EmitVoteAny(EmitContext& ctx, Id pred) {
if (!StageSupportsSubgroups(ctx)) {
return pred;
}
if (!ctx.profile.warp_size_potentially_larger_than_guest) {
return ctx.OpGroupNonUniformAny(ctx.U1, SubgroupScope(ctx), pred);
}
const Id mask_ballot{
ctx.OpGroupNonUniformBallot(ctx.U32[4], SubgroupScope(ctx), ctx.true_value)};
const Id active_mask{WarpExtract(ctx, mask_ballot)};
const Id ballot{
WarpExtract(ctx, ctx.OpGroupNonUniformBallot(ctx.U32[4], SubgroupScope(ctx), pred))};
const Id lhs{ctx.OpBitwiseAnd(ctx.U32[1], ballot, active_mask)};
return ctx.OpINotEqual(ctx.U1, lhs, ctx.u32_zero_value);
}
Id EmitVoteEqual(EmitContext& ctx, Id pred) {
if (!StageSupportsSubgroups(ctx)) {
return ctx.true_value;
}
if (!ctx.profile.warp_size_potentially_larger_than_guest) {
return ctx.OpGroupNonUniformAllEqual(ctx.U1, SubgroupScope(ctx), pred);
}
const Id mask_ballot{
ctx.OpGroupNonUniformBallot(ctx.U32[4], SubgroupScope(ctx), ctx.true_value)};
const Id active_mask{WarpExtract(ctx, mask_ballot)};
const Id ballot{
WarpExtract(ctx, ctx.OpGroupNonUniformBallot(ctx.U32[4], SubgroupScope(ctx), pred))};
const Id lhs{ctx.OpBitwiseXor(ctx.U32[1], ballot, active_mask)};
return ctx.OpLogicalOr(ctx.U1, ctx.OpIEqual(ctx.U1, lhs, ctx.u32_zero_value),
ctx.OpIEqual(ctx.U1, lhs, active_mask));
}
Id EmitSubgroupBallot(EmitContext& ctx, Id pred) {
if (!StageSupportsSubgroups(ctx)) {
return ctx.OpSelect(ctx.U32[1], pred, ctx.Const(1u), ctx.u32_zero_value);
}
const Id ballot{ctx.OpGroupNonUniformBallot(ctx.U32[4], SubgroupScope(ctx), pred)};
if (!ctx.profile.warp_size_potentially_larger_than_guest) {
return ctx.OpCompositeExtract(ctx.U32[1], ballot, 0U);
}
return WarpExtract(ctx, ballot);
}
Id EmitSubgroupEqMask(EmitContext& ctx) {
if (!StageSupportsSubgroups(ctx)) {
return ctx.Const(1u);
}
return LoadMask(ctx, ctx.subgroup_mask_eq);
}
Id EmitSubgroupLtMask(EmitContext& ctx) {
if (!StageSupportsSubgroups(ctx)) {
return ctx.u32_zero_value;
}
return LoadMask(ctx, ctx.subgroup_mask_lt);
}
Id EmitSubgroupLeMask(EmitContext& ctx) {
if (!StageSupportsSubgroups(ctx)) {
return ctx.Const(1u);
}
return LoadMask(ctx, ctx.subgroup_mask_le);
}
Id EmitSubgroupGtMask(EmitContext& ctx) {
if (!StageSupportsSubgroups(ctx)) {
return ctx.u32_zero_value;
}
return LoadMask(ctx, ctx.subgroup_mask_gt);
}
Id EmitSubgroupGeMask(EmitContext& ctx) {
if (!StageSupportsSubgroups(ctx)) {
return ctx.Const(1u);
}
return LoadMask(ctx, ctx.subgroup_mask_ge);
}
Id EmitShuffleIndex(EmitContext& ctx, IR::Inst* inst, Id value, Id index, Id clamp,
Id segmentation_mask) {
const Id not_seg_mask{ctx.OpNot(ctx.U32[1], segmentation_mask)};
const Id thread_id{EmitLaneId(ctx)};
const Id min_thread_id{ComputeMinThreadId(ctx, thread_id, segmentation_mask)};
const Id max_thread_id{ComputeMaxThreadId(ctx, min_thread_id, clamp, not_seg_mask)};
const Id lhs{ctx.OpBitwiseAnd(ctx.U32[1], GuestLane(ctx, index), not_seg_mask)};
const Id src_thread_id{ctx.OpBitwiseOr(ctx.U32[1], lhs, min_thread_id)};
const Id in_range{ctx.OpSLessThanEqual(ctx.U1, src_thread_id, max_thread_id)};
SetInBoundsFlag(inst, in_range);
if (!StageSupportsSubgroups(ctx)) {
return value;
}
const IR::Value lane{inst->Arg(1).Resolve()};
const IR::Value segment{inst->Arg(3).Resolve()};
if (lane.IsImmediate() && segment.IsImmediate() && segment.U32() == 0) {
return ctx.OpSelect(ctx.U32[1], in_range, BroadcastLane(ctx, value, lane.U32() & 31),
value);
}
const Id shuffled{ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id))};
return ctx.OpSelect(ctx.U32[1], in_range, shuffled, value);
}
Id EmitShuffleUp(EmitContext& ctx, IR::Inst* inst, Id value, Id index, Id clamp,
Id segmentation_mask) {
if (!StageSupportsSubgroups(ctx)) {
SetInBoundsFlag(inst, ctx.false_value);
return value;
}
const Id delta{GuestLane(ctx, index)};
const Id thread_id{EmitLaneId(ctx)};
const Id max_thread_id{GetMaxThreadId(ctx, thread_id, clamp, segmentation_mask)};
const Id src_thread_id{ctx.OpISub(ctx.U32[1], thread_id, delta)};
const Id in_range{ctx.OpSGreaterThanEqual(ctx.U1, src_thread_id, max_thread_id)};
SetInBoundsFlag(inst, in_range);
const Id shuffled{ShuffleRelative(ctx, value, delta, src_thread_id,
spv::Op::OpGroupNonUniformShuffleUp)};
return ctx.OpSelect(ctx.U32[1], in_range, shuffled, value);
}
Id EmitShuffleDown(EmitContext& ctx, IR::Inst* inst, Id value, Id index, Id clamp,
Id segmentation_mask) {
if (!StageSupportsSubgroups(ctx)) {
SetInBoundsFlag(inst, ctx.false_value);
return value;
}
const Id delta{GuestLane(ctx, index)};
const Id thread_id{EmitLaneId(ctx)};
const Id max_thread_id{GetMaxThreadId(ctx, thread_id, clamp, segmentation_mask)};
const Id src_thread_id{ctx.OpIAdd(ctx.U32[1], thread_id, delta)};
const Id in_range{ctx.OpSLessThanEqual(ctx.U1, src_thread_id, max_thread_id)};
SetInBoundsFlag(inst, in_range);
const Id shuffled{ShuffleRelative(ctx, value, delta, src_thread_id,
spv::Op::OpGroupNonUniformShuffleDown)};
return ctx.OpSelect(ctx.U32[1], in_range, shuffled, value);
}
Id EmitShuffleButterfly(EmitContext& ctx, IR::Inst* inst, Id value, Id index, Id clamp,
Id segmentation_mask) {
if (!StageSupportsSubgroups(ctx)) {
SetInBoundsFlag(inst, ctx.false_value);
return value;
}
const Id mask{GuestLane(ctx, index)};
const Id thread_id{EmitLaneId(ctx)};
const Id max_thread_id{GetMaxThreadId(ctx, thread_id, clamp, segmentation_mask)};
const Id src_thread_id{ctx.OpBitwiseXor(ctx.U32[1], thread_id, mask)};
const Id in_range{ctx.OpSLessThanEqual(ctx.U1, src_thread_id, max_thread_id)};
SetInBoundsFlag(inst, in_range);
// (hookshot) through the broadcasts where shuffles are broken (UseShuffleFallback: AMD's fragment stages, Lichii #273),
// as the absolute and relative shuffles are; #273 left SHFL.BFLY native, and some of Kokiri Forest's plants use it
const Id shuffled{UseShuffleFallback(ctx)
? ShuffleAbsolute(ctx, value, HostThreadId(ctx, src_thread_id))
: ctx.OpGroupNonUniformShuffleXor(ctx.U32[1], SubgroupScope(ctx), value, mask)};
return ctx.OpSelect(ctx.U32[1], in_range, shuffled, value);
}
Id EmitQuadBroadcast(EmitContext& ctx, Id value, Id lane) {
if (ctx.profile.support_quad_shuffles) {
return ctx.OpGroupNonUniformQuadBroadcast(ctx.U32[1], SubgroupScope(ctx), value, lane);
}
const Id base{ctx.OpBitwiseAnd(ctx.U32[1], GetThreadId(ctx), ctx.Const(~3u))};
const Id local_lane{ctx.OpBitwiseAnd(ctx.U32[1], lane, ctx.Const(3u))};
const Id src_thread_id{ctx.OpBitwiseOr(ctx.U32[1], base, local_lane)};
return ShuffleAbsolute(ctx, value, src_thread_id);
}
Id EmitQuadSwap(EmitContext& ctx, Id value, Id direction) {
if (ctx.profile.support_quad_shuffles) {
const Id result{
ctx.OpGroupNonUniformQuadBroadcast(ctx.U32[1], SubgroupScope(ctx), value, direction)};
ctx.opcode_rewrites.emplace_back(result.value, spv::Op::OpGroupNonUniformQuadSwap);
return result;
}
const Id xor_mask{ctx.OpIAdd(ctx.U32[1], direction, ctx.Const(1u))};
// (hookshot) not through UseShuffleFallback's broadcasts: a program with quad shuffles alone (uses_quad_shuffles) has
// neither the lane id input nor the ballot capability they need, and the SPIR-V made with them crashed the start
return ctx.OpGroupNonUniformShuffleXor(ctx.U32[1], SubgroupScope(ctx), value, xor_mask);
}
Id EmitFSwizzleAdd(EmitContext& ctx, Id op_a, Id op_b, Id swizzle) {
const Id three{ctx.Const(3U)};
Id mask{GetThreadId(ctx)};
mask = ctx.OpBitwiseAnd(ctx.U32[1], mask, three);
mask = ctx.OpShiftLeftLogical(ctx.U32[1], mask, ctx.Const(1U));
mask = ctx.OpShiftRightLogical(ctx.U32[1], swizzle, mask);
mask = ctx.OpBitwiseAnd(ctx.U32[1], mask, three);
const Id modifier_a{ctx.OpVectorExtractDynamic(ctx.F32[1], ctx.fswzadd_lut_a, mask)};
const Id modifier_b{ctx.OpVectorExtractDynamic(ctx.F32[1], ctx.fswzadd_lut_b, mask)};
const Id result_a{ctx.OpFMul(ctx.F32[1], op_a, modifier_a)};
const Id result_b{ctx.OpFMul(ctx.F32[1], op_b, modifier_b)};
return ctx.OpFAdd(ctx.F32[1], result_a, result_b);
}
Id EmitDPdxFine(EmitContext& ctx, Id op_a) {
return ctx.OpDPdxFine(ctx.F32[1], op_a);
}
Id EmitDPdyFine(EmitContext& ctx, Id op_a) {
return ctx.OpDPdyFine(ctx.F32[1], op_a);
}
Id EmitDPdxCoarse(EmitContext& ctx, Id op_a) {
return ctx.OpDPdxCoarse(ctx.F32[1], op_a);
}
Id EmitDPdyCoarse(EmitContext& ctx, Id op_a) {
return ctx.OpDPdyCoarse(ctx.F32[1], op_a);
}
// (hookshot) HMMA in software, from Lichii (zortos293/Lichii, emit_spirv_tensor.cpp, GPL-3.0): the 32 lanes' fragments
// through subgroup shuffles in one helper function, accumulated with FMA in f32 and rounded to f16 after every 8
// columns of K, as HMMA.F16 does. It needs no cooperative matrices (RX_VK_MMA_SOFTWARE=1, vk_coopmat.cpp).
void DefineSoftwareMatrixFunction(EmitContext& ctx) {
{
constexpr bool variant = true;
ctx.AddCapability(spv::Capability::GroupNonUniform);
ctx.AddCapability(spv::Capability::GroupNonUniformShuffle);
if (!ctx.profile.SupportsSubgroupStage(ctx.stage) ||
ctx.profile.has_broken_spirv_subgroup_shuffle)
throw NotImplementedException("Software HMMA requires subgroup shuffle support");
const Id u32ty = ctx.U32[1], f32ty = ctx.F32[1];
const auto imm = [&](u32 v) { return ctx.Const(v); };
const auto add = [&](Id a, Id b) { return ctx.OpIAdd(u32ty, a, b); };
const auto and_ = [&](Id a, u32 b) { return ctx.OpBitwiseAnd(u32ty, a, imm(b)); };
const auto unpack = [&](Id v) { return ctx.OpUnpackHalf2x16(ctx.F32[2], v); };
const auto round_half = [&](Id value) {
// Do not express the intermediate rounding as unpack(pack(value)): drivers may
// eliminate that round trip. Round normal significands explicitly, ties to even;
// half subnormals lie on the exactly representable 2^-24 grid.
const Id bits = ctx.OpBitcast(u32ty, value);
const Id magnitude = and_(bits, 0x7fffffffU);
const Id tie = and_(ctx.OpShiftRightLogical(u32ty, magnitude, imm(13)), 1);
const Id rounded = and_(add(add(magnitude, imm(0xfff)), tie), ~0x1fffU);
const Id normal = ctx.OpSelect(u32ty,
ctx.OpUGreaterThanEqual(ctx.U1, rounded, imm(0x47800000)), imm(0x7f800000), rounded);
const Id scaled = ctx.OpFMul(f32ty, ctx.OpBitcast(f32ty, magnitude), ctx.Const(16777216.0f));
const Id subnormal = ctx.OpBitcast(u32ty,
ctx.OpFMul(f32ty, ctx.OpRoundEven(f32ty, scaled), ctx.Const(0x1p-24f)));
const Id finite = ctx.OpSelect(u32ty,
ctx.OpUGreaterThanEqual(ctx.U1, magnitude, imm(0x38800000)), normal, subnormal);
const Id result = ctx.OpSelect(u32ty,
ctx.OpUGreaterThanEqual(ctx.U1, magnitude, imm(0x7f800000)), magnitude, finite);
return ctx.OpBitcast(f32ty, ctx.OpBitwiseOr(u32ty, result, and_(bits, 0x80000000)));
};
const Id type = ctx.TypeFunction(ctx.U32[2], ctx.U32[4], ctx.U32[2], ctx.U32[2]);
const Id fn = ctx.OpFunction(ctx.U32[2], spv::FunctionControlMask::DontInline, type);
ctx.software_matrix_function = fn;
const Id matrix_a = ctx.OpFunctionParameter(ctx.U32[4]);
const Id matrix_b = ctx.OpFunctionParameter(ctx.U32[2]);
const Id matrix_c = ctx.OpFunctionParameter(ctx.U32[2]);
ctx.AddLabel();
const Id counter = ctx.AddLocalVariable(ctx.TypePointer(spv::StorageClass::Function, u32ty),
spv::StorageClass::Function);
std::array<Id, 4> accumulators;
for (auto& acc : accumulators)
acc = ctx.AddLocalVariable(ctx.TypePointer(spv::StorageClass::Function, f32ty),
spv::StorageClass::Function);
std::array<Id, 8> inputs;
for (u32 i = 0; i < 4; ++i)
inputs[i] = ctx.OpCompositeExtract(u32ty, matrix_a, i);
for (u32 i = 0; i < 2; ++i) {
inputs[4 + i] = ctx.OpCompositeExtract(u32ty, matrix_b, i);
inputs[6 + i] = ctx.OpCompositeExtract(u32ty, matrix_c, i);
}
for (u32 row = 0; row < 2; ++row) {
const Id c = unpack(inputs[6 + row]);
for (u32 col = 0; col < 2; ++col)
ctx.OpStore(accumulators[row * 2 + col], ctx.OpCompositeExtract(f32ty, c, col));
}
ctx.OpStore(counter, imm(0));
const Id lane = ctx.OpLoad(u32ty, ctx.subgroup_local_invocation_id);
const Id warp_base = and_(lane, ~31U);
const Id row_base = and_(lane, 28);
const Id col_base = ctx.OpShiftLeftLogical(u32ty, and_(lane, 3), imm(3));
const Id header = ctx.OpLabel(), body = ctx.OpLabel(), next = ctx.OpLabel(), end = ctx.OpLabel();
ctx.OpBranch(header);
ctx.AddLabel(header);
const Id pair = ctx.OpLoad(u32ty, counter);
const Id more = ctx.OpULessThan(ctx.U1, pair, imm(variant ? 8 : 4));
ctx.OpLoopMerge(end, next, spv::LoopControlMask::DontUnroll);
ctx.OpBranchConditional(more, body, end);
ctx.AddLabel(body);
const Id upper = ctx.OpUGreaterThanEqual(ctx.U1, pair, imm(4));
const Id component = and_(pair, 3);
const Id a_lane = add(warp_base, add(row_base, component));
const Id b_lane = add(warp_base, add(col_base, component));
const Id b = ctx.OpSelect(u32ty, upper, inputs[5], inputs[4]);
std::array<Id, 2> bs;
for (u32 col = 0; col < 2; ++col)
bs[col] = unpack(ctx.OpGroupNonUniformShuffle(u32ty, imm(3), b,
add(b_lane, imm(col * 4))));
const Id round_now = ctx.OpIEqual(ctx.U1, component, imm(3));
for (u32 row = 0; row < 2; ++row) {
const Id a = ctx.OpSelect(u32ty, upper, inputs[row + 2], inputs[row]);
const Id as = unpack(ctx.OpGroupNonUniformShuffle(u32ty, imm(3), a, a_lane));
for (u32 col = 0; col < 2; ++col) {
Id value = ctx.OpLoad(f32ty, accumulators[row * 2 + col]);
for (u32 half = 0; half < 2; ++half) {
value = ctx.OpFma(f32ty, ctx.OpCompositeExtract(f32ty, as, half),
ctx.OpCompositeExtract(f32ty, bs[col], half), value);
ctx.Decorate(value, spv::Decoration::NoContraction);
}
ctx.OpStore(accumulators[row * 2 + col], value);
}
}
// HMMA.F16 rounds after each K=8 group, including the midpoint of K=16.
// OpSelect evaluates both operands: keep the expensive half conversion in
// a branch so the other three pairs do no rounding work. The counter is
// identical in every participating lane and all shuffles precede this branch.
const Id round_block = ctx.OpLabel(), round_merge = ctx.OpLabel();
ctx.OpSelectionMerge(round_merge, spv::SelectionControlMask::DontFlatten);
ctx.OpBranchConditional(round_now, round_block, round_merge);
ctx.AddLabel(round_block);
for (const Id accumulator : accumulators) {
ctx.OpStore(accumulator, round_half(ctx.OpLoad(f32ty, accumulator)));
}
ctx.OpBranch(round_merge);
ctx.AddLabel(round_merge);
ctx.OpBranch(next);
ctx.AddLabel(next);
ctx.OpStore(counter, add(pair, imm(1)));
ctx.OpBranch(header);
ctx.AddLabel(end);
std::array<Id, 2> packed;
for (u32 row = 0; row < 2; ++row)
packed[row] = ctx.OpPackHalf2x16(u32ty, ctx.OpCompositeConstruct(ctx.F32[2],
ctx.OpLoad(f32ty, accumulators[row * 2]), ctx.OpLoad(f32ty, accumulators[row * 2 + 1])));
ctx.OpReturnValue(ctx.OpCompositeConstruct(ctx.U32[2], packed[0], packed[1]));
ctx.OpFunctionEnd();
}
}
// (hookshot) HMMA_nx2: D(16x8) = A(16x16) * B(16x8) + C on the subgroup's cooperative matrices. Each
// lane's fragment registers hold its elements in the console's order (two f16 a word); they go to and come
// from the device's elements in its own order (mma_layout.h, measured by vk_coopmat.cpp).
Id EmitMatrixMulAddF16(EmitContext& ctx, Id a, Id b, Id c) {
if (mma_software) return ctx.OpFunctionCall(ctx.U32[2], ctx.software_matrix_function, a, b, c);
const Id f16{ctx.F16[1]};
const Id scope{ctx.Const(static_cast<u32>(spv::Scope::Subgroup))};
const auto type = [&](u32 rows, u32 columns, spv::CooperativeMatrixUse use) {
return ctx.TypeCooperativeMatrixKHR(f16, scope, ctx.Const(rows), ctx.Const(columns),
ctx.Const(static_cast<u32>(use)));
};
const Id a_type{type(16, 16, spv::CooperativeMatrixUse::MatrixAKHR)};
const Id b_type{type(16, 8, spv::CooperativeMatrixUse::MatrixBKHR)};
const Id c_type{type(16, 8, spv::CooperativeMatrixUse::MatrixAccumulatorKHR)};
const Id zero{ctx.Constant(f16, 0U)};
const auto load = [&](Id matrix_type, Id words, u32 count, const auto& order) {
Id m{ctx.OpCompositeConstruct(matrix_type, zero)};
for (u32 w = 0; w < count; ++w) {
const Id word{count == 1 ? words : ctx.OpCompositeExtract(ctx.U32[1], words, w)};
const Id pair{ctx.OpBitcast(ctx.F16[2], word)};
for (u32 h = 0; h < 2; ++h) {
m = ctx.OpCompositeInsert(matrix_type, ctx.OpCompositeExtract(f16, pair, h), m, order[w * 2 + h]);
}
}
return m;
};
const Id d{ctx.OpCooperativeMatrixMulAddKHR(c_type, load(a_type, a, 4, mma_order_a), load(b_type, b, 2, mma_order_b),
load(c_type, c, 2, mma_order_c))};
Id out[2];
for (u32 w = 0; w < 2; ++w) {
const Id pair{ctx.OpCompositeConstruct(ctx.F16[2], ctx.OpCompositeExtract(f16, d, mma_order_c[w * 2]),
ctx.OpCompositeExtract(f16, d, mma_order_c[w * 2 + 1]))};
out[w] = ctx.OpBitcast(ctx.U32[1], pair);
}
return ctx.OpCompositeConstruct(ctx.U32[2], out[0], out[1]);
}
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,29 @@
// (hookshot) The device's element order of the 16x8x16 f16 cooperative matrices HMMA_nx2 runs on.
//
// A lane's HMMA fragment registers hold its matrix elements in the console's order (PTX mma.m16n8k16's: two f16
// a register, low half first). The host's cooperative matrices hold each lane's share in an order of the
// implementation's own; vk_coopmat.cpp measures it at start, and only when every lane holds the same positions in
// both does the back end turn HMMA_nx2 on. Element e of the console's fragment is then the device's element
// mma_order_x[e].
#pragma once
#include <array>
#include <cstdint>
namespace Shader::Backend::SPIRV {
inline std::array<uint32_t, 8> mma_order_a{0, 1, 2, 3, 4, 5, 6, 7};
inline std::array<uint32_t, 4> mma_order_b{0, 1, 2, 3};
inline std::array<uint32_t, 4> mma_order_c{0, 1, 2, 3};
inline bool mma_software = false; // (HMMA_nx2 in software: emit_spirv_warp.cpp, RX_VK_MMA_SOFTWARE=1)
// (amd-fix) float colour outputs carry no NaN or infinity to their target (EmitSetFragColor): one NaN texel spreads
// through bloom/blur/downsample averages into glowing blocks (OoT's flowers on AMD, Hookshot issue 606). On for every
// card but NVIDIA (vk_coopmat.cpp coopmat_init); RX_VK_NO_NAN_GUARD=1 turns it off. Idea from NX2navi patch 0027.
inline bool frag_nan_guard = false;
// (amd-fix) gl_Position decorated Invariant (vertex and mesh outputs): the game draws a depth pre-pass, then the colour
// pass again with depth EQUAL; Maxwell computes the same position in both, AMD's compiler may not without Invariant, and
// the colour pass is then rejected in spots: holes showing the bright sky, blown up by bloom (the glowing flowers).
// On for every card but NVIDIA (vk_coopmat.cpp); RX_VK_NO_INVARIANT=1 off. From NX2navi patch 0007.
inline bool position_invariant = false;
} // namespace Shader::Backend::SPIRV

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,477 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <array>
#include <sirit/sirit.h>
#include "common/container/unordered_set.h"
#include "shader_recompiler/backend/bindings.h"
#include "shader_recompiler/frontend/ir/program.h"
#include "shader_recompiler/profile.h"
#include "shader_recompiler/runtime_info.h"
#include "shader_recompiler/shader_info.h"
namespace Shader::Backend::SPIRV {
static std::bitset<8> clip_distance_written;
using Sirit::Id;
class VectorTypes {
public:
void Define(Sirit::Module& sirit_ctx, Id base_type, std::string_view name);
[[nodiscard]] Id operator[](size_t size) const noexcept {
return defs[size - 1];
}
private:
std::array<Id, 4> defs{};
};
struct TextureDefinition {
Id id;
Id sampled_type;
Id pointer_type;
Id image_type;
u32 count;
bool is_multisample;
bool is_integer;
};
struct TextureBufferDefinition {
Id id;
u32 count;
};
struct ImageBufferDefinition {
Id id;
Id image_type;
Id pointer_type;
u32 count;
bool is_integer;
};
struct ImageDefinition {
Id id;
Id image_type;
Id pointer_type;
u32 count;
bool is_integer;
};
struct UniformDefinitions {
Id U8{};
Id S8{};
Id U16{};
Id S16{};
Id U32{};
Id F32{};
Id U32x2{};
Id U32x4{};
constexpr static size_t NumElements(Id UniformDefinitions::*member_ptr) {
if (member_ptr == &UniformDefinitions::U8) {
return 1;
}
if (member_ptr == &UniformDefinitions::S8) {
return 1;
}
if (member_ptr == &UniformDefinitions::U16) {
return 1;
}
if (member_ptr == &UniformDefinitions::S16) {
return 1;
}
if (member_ptr == &UniformDefinitions::U32) {
return 1;
}
if (member_ptr == &UniformDefinitions::F32) {
return 1;
}
if (member_ptr == &UniformDefinitions::U32x2) {
return 2;
}
if (member_ptr == &UniformDefinitions::U32x4) {
return 4;
}
ASSERT(false);
return 1;
}
constexpr static bool IsFloat(Id UniformDefinitions::*member_ptr) {
if (member_ptr == &UniformDefinitions::F32) {
return true;
}
return false;
}
};
struct StorageTypeDefinition {
Id array{};
Id element{};
};
struct StorageTypeDefinitions {
StorageTypeDefinition U8{};
StorageTypeDefinition S8{};
StorageTypeDefinition U16{};
StorageTypeDefinition S16{};
StorageTypeDefinition U32{};
StorageTypeDefinition U64{};
StorageTypeDefinition F32{};
StorageTypeDefinition U32x2{};
StorageTypeDefinition U32x4{};
};
struct StorageDefinitions {
Id U8{};
Id S8{};
Id U16{};
Id S16{};
Id U32{};
Id F32{};
Id U64{};
Id U32x2{};
Id U32x4{};
};
enum class InputGenericLoadOp {
None,
Bitcast,
SToF,
UToF,
};
struct InputGenericInfo {
Id id;
Id pointer_type;
Id component_type;
InputGenericLoadOp load_op;
};
struct GenericElementInfo {
Id id{};
u32 first_element{};
u32 num_components{};
};
class EmitContext final : public Sirit::Module {
public:
explicit EmitContext(const Profile& profile, const RuntimeInfo& runtime_info,
IR::Program& program, Bindings& binding);
~EmitContext();
[[nodiscard]] Id Def(const IR::Value& value);
[[nodiscard]] Id BitOffset8(const IR::Value& offset);
[[nodiscard]] Id BitOffset16(const IR::Value& offset);
Id Const(u32 value) {
return Constant(U32[1], value);
}
Id Const(u32 element_1, u32 element_2) {
return ConstantComposite(U32[2], Const(element_1), Const(element_2));
}
Id Const(u32 element_1, u32 element_2, u32 element_3) {
return ConstantComposite(U32[3], Const(element_1), Const(element_2), Const(element_3));
}
Id Const(u32 element_1, u32 element_2, u32 element_3, u32 element_4) {
return ConstantComposite(U32[4], Const(element_1), Const(element_2), Const(element_3),
Const(element_4));
}
Id SConst(s32 value) {
return Constant(S32[1], value);
}
Id SConst(s32 element_1, s32 element_2) {
return ConstantComposite(S32[2], SConst(element_1), SConst(element_2));
}
Id SConst(s32 element_1, s32 element_2, s32 element_3) {
return ConstantComposite(S32[3], SConst(element_1), SConst(element_2), SConst(element_3));
}
Id SConst(s32 element_1, s32 element_2, s32 element_3, s32 element_4) {
return ConstantComposite(S32[4], SConst(element_1), SConst(element_2), SConst(element_3),
SConst(element_4));
}
Id Const(f32 value) {
return Constant(F32[1], value);
}
const Profile& profile;
const RuntimeInfo& runtime_info;
Stage stage{};
Id void_id{};
Id U1{};
Id U8{};
Id S8{};
Id U16{};
Id S16{};
Id U64{};
VectorTypes F32;
VectorTypes U32;
VectorTypes S32;
VectorTypes F16;
VectorTypes F64;
Id true_value{};
Id false_value{};
Id u32_zero_value{};
Id f32_zero_value{};
UniformDefinitions uniform_types;
StorageTypeDefinitions storage_types;
Id private_u32{};
Id shared_u8{};
Id shared_u16{};
Id shared_u32{};
Id shared_u64{};
Id shared_u32x2{};
Id shared_u32x4{};
Id input_f32{};
Id input_u32{};
Id input_s32{};
Id output_f32{};
Id output_u32{};
Id image_buffer_type{};
Id image_u32{};
std::array<UniformDefinitions, Info::MAX_CBUFS> cbufs{};
std::array<StorageDefinitions, Info::MAX_SSBOS> ssbos{};
std::vector<TextureBufferDefinition> texture_buffers;
std::vector<ImageBufferDefinition> image_buffers;
std::vector<TextureDefinition> textures;
std::vector<ImageDefinition> images;
Id workgroup_id{};
Id local_invocation_id{};
Id invocation_id{};
Id patch_vertices_in{};
Id sample_id{};
Id is_helper_invocation{};
Id subgroup_local_invocation_id{};
Id software_matrix_function{}; // (hookshot: emit_spirv_warp.cpp)
Id subgroup_mask_eq{};
Id subgroup_mask_lt{};
Id subgroup_mask_le{};
Id subgroup_mask_gt{};
Id subgroup_mask_ge{};
Id instance_id{};
Id instance_index{};
Id base_instance{};
Id vertex_id{};
Id vertex_index{};
Id draw_index{};
Id base_vertex{};
Id front_face{};
Id point_coord{};
Id tess_coord{};
Id clip_distances{};
Id layer{};
Id viewport_index{};
Id viewport_mask{};
Id primitive_id{};
Id fswzadd_lut_a{};
Id fswzadd_lut_b{};
Id indexed_load_func{};
Id indexed_store_func{};
Id rescaling_uniform_constant{};
Id rescaling_push_constants{};
Id rescaling_textures_type{};
Id rescaling_images_type{};
u32 rescaling_textures_member_index{};
u32 rescaling_images_member_index{};
u32 rescaling_downfactor_member_index{};
u32 guest_pages_member_index{}; // (hookshot: GUEST_PAGES_PUSH_OFFSET's member, compute only)
bool wants_guest_pages{};
Id probe_log{}; // (EvilOOT, RX_VK_BLOCK_PROBE: the miss log's U32 storage buffer, per-block entry counters at word 128+)
u32 mesh_first_member_index{}; // (hookshot: MESH_FIRST_PUSH_OFFSET's member, task programs)
bool wants_mesh_first{};
Id mesh_first_block_pointer{}; // (a PhysicalStorageBuffer pointer to a block of one u32, and to its member)
Id mesh_first_member_pointer{};
u32 texture_rescaling_index{};
u32 image_rescaling_index{};
Id render_area_push_constant{};
u32 render_are_member_index{};
Id local_memory{};
bool uses_explicit_workgroup_layout{};
Id shared_memory_u8{};
Id shared_memory_u16{};
Id shared_memory_u32{};
Id shared_memory_u64{};
Id shared_memory_u32x2{};
Id shared_memory_u32x4{};
Id shared_memory_u32_type{};
Id shared_store_u8_func{};
Id shared_store_u16_func{};
Id increment_cas_shared{};
Id increment_cas_ssbo{};
Id decrement_cas_shared{};
Id decrement_cas_ssbo{};
Id f32_add_cas{};
Id f16x2_add_cas{};
Id f16x2_min_cas{};
Id f16x2_max_cas{};
Id f32x2_add_cas{};
Id f32x2_min_cas{};
Id f32x2_max_cas{};
Id write_storage_cas_loop_func{};
// (hookshot) storage buffer tables (Info::global_tables): one array variable a type, which only the global
// memory fallbacks use; and those fallbacks for the 32-bit atomics (IAdd SMin UMin SMax UMax And Or Xor Exchange)
struct GlobalTableVars {
u32 cbuf_index{};
u32 cbuf_offset{};
u32 count{};
StorageDefinitions vars{};
};
std::vector<GlobalTableVars> ssbo_tables;
std::array<Id, 9> atomic_global_func_u32{};
Id atomic_global_cas_func_u32{}; // (hookshot) (address, comparator, value) -> the old value
Id load_global_func_u32{};
Id load_global_func_u32x2{};
Id load_global_func_u32x4{};
Id write_global_func_u32{};
Id write_global_func_u32x2{};
Id write_global_func_u32x4{};
Id write_global_cas_loop_func{};
Id atomic_global_add_f32_func{}; // (hookshot) (address, value) -> the old value
bool need_input_position_indirect{};
Id input_position{};
std::array<InputGenericInfo, 32> input_generics{};
Id output_point_size{};
Id output_position{};
std::array<std::array<GenericElementInfo, 4>, 32> output_generics{};
Id output_tess_level_outer{};
Id output_tess_level_inner{};
std::array<Id, 30> patches{};
std::array<Id, 8> frag_color{};
Id sample_mask{};
Id frag_depth{};
std::vector<Id> interfaces;
std::vector<std::pair<u32, spv::Op>> opcode_rewrites;
// Task and mesh programs. Their stage buffer: the task's output region is the task payload
// (the mesh stage's input region); the mesh output regions live in workgroup memory until
// the epilogue copies them to the outputs. The two mesh instructions Sirit lacks are emitted
// as OpStore markers with these operands and replaced after assembly (PatchMeshInstructions).
struct MeshLineTarget {
Id variable; ///< arrayed output
u32 element; ///< vector element, or ~0 for a scalar output
bool is_float; ///< bitcast the stage buffer word to F32
};
Id mesh_payload{};
Id mesh_payload_u32{}; ///< pointer type to one payload word
Id mesh_attributes{};
Id mesh_map{};
Id mesh_primitive_count{};
/// (hookshot) a fallback mesh workgroup's slot (task workgroup * threads + its index), read from the task's map in the
/// entry block (EmitMeshPrologue); invalid without a task program, where the slot is the workgroup id
Id mesh_fallback_slot{};
Id mesh_shared_u32{}; ///< pointer type to one workgroup word
Id mesh_primitive_indices{};
MeshInfo mesh_info{};
u32 mesh_threads{};
u32 mesh_attribute_words{};
u32 mesh_map_bytes{};
u32 mesh_primitive_base{}; ///< first word of the per-primitive groups
std::vector<MeshLineTarget> mesh_vertex_targets;
// (hookshot) a vertex generic output and the components no vertex line writes: they get (0,0,0,1) as in the vertex stage
struct MeshDefault {
Id variable;
u32 missing;
};
std::vector<MeshDefault> mesh_vertex_defaults;
std::vector<MeshLineTarget> mesh_primitive_targets;
std::array<Id, 2> mesh_outputs_marker{};
bool mesh_zero_outputs_marker{}; ///< (hookshot) OpStore(zero, zero): OpSetMeshOutputsEXT 0 0 where a branch has no target
std::array<Id, 4> mesh_tasks_marker{};
// (hookshot) a GPU without VK_EXT_mesh_shader: Task and Mesh are compute shaders. Their stage buffer outputs go to
// the payload, map and record buffers, pushed as device addresses; a generated vertex stage reads the records.
bool mesh_fallback{};
bool wants_mesh_fallback{};
Id mesh_fallback_word_ptr{}; ///< PhysicalStorageBuffer pointer to a block of one u32
u32 mesh_fallback_payload_member_index{};
u32 mesh_fallback_records_member_index{};
u32 mesh_fallback_indirect_input_member_index{};
/// A pointer to the u32 at a PhysicalStorageBuffer device address
Id MeshFallbackWord(Id address);
Id load_const_func_u8{};
Id load_const_func_u16{};
Id load_const_func_u32{};
Id load_const_func_f32{};
Id load_const_func_u32x2{};
Id load_const_func_u32x4{};
// Sirit::Id doesn't play nice with *::set<>
::Common::unordered_set<u32> non_uniform_ids;
bool uses_nonuniform_sampled_image{};
bool uses_nonuniform_storage_image{};
bool uses_nonuniform_uniform_texel_buffer{};
bool uses_nonuniform_storage_texel_buffer{};
private:
void DefineCommonTypes(const Info& info);
void DefineCommonConstants();
void DefineInterfaces(const IR::Program& program);
void DefineLocalMemory(const IR::Program& program);
void DefineSharedMemory(const IR::Program& program);
void DefineSharedMemoryFunctions(const IR::Program& program);
void DefineConstantBuffers(const Info& info, u32& binding);
void DefineConstantBufferIndirectFunctions(const Info& info);
void DefineStorageBuffers(const Info& info, u32& binding);
void DefineTextureBuffers(const Info& info, u32& binding);
void DefineImageBuffers(const Info& info, u32& binding);
void DefineTextures(const Info& info, u32& binding, u32& scaling_index);
void DefineImages(const Info& info, u32& binding, u32& scaling_index);
void DefineAttributeMemAccess(const Info& info);
void DefineWriteStorageCasLoopFunction(const Info& info);
void DefineGlobalMemoryFunctions(const Info& info);
void DefineRescalingInput(const Info& info);
void DefineRescalingInputPushConstant();
void DefineRescalingInputUniformConstant();
void DefineRenderArea(const Info& info);
void DefineInputs(const IR::Program& program);
void DefineOutputs(const IR::Program& program);
void DefineMesh(const IR::Program& program);
};
} // namespace Shader::Backend::SPIRV

View File

@@ -0,0 +1,160 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <array>
#include "common/container/unordered_map.h"
#include "common/common_types.h"
#include "shader_recompiler/program_header.h"
#include "shader_recompiler/shader_info.h"
#include "shader_recompiler/stage.h"
#include "shader_recompiler/frontend/ir/value.h"
namespace Shader::IR {
class Inst;
}
namespace Shader {
struct CbufWordKey {
u32 index;
u32 offset;
constexpr bool operator==(const CbufWordKey& o) const noexcept {
return index == o.index && offset == o.offset;
}
};
struct CbufWordKeyHash {
constexpr size_t operator()(const CbufWordKey& k) const noexcept {
return (size_t(k.index) << 32) ^ k.offset;
}
};
struct HandleKey {
u32 index, offset, shift_left;
u32 sec_index, sec_offset, sec_shift_left;
u32 count;
bool has_secondary;
constexpr bool operator==(const HandleKey& o) const noexcept {
return std::tie(index, offset, shift_left, sec_index, sec_offset, sec_shift_left, count, has_secondary)
== std::tie(o.index, o.offset, o.shift_left, o.sec_index, o.sec_offset, o.sec_shift_left, o.count, o.has_secondary);
}
};
struct HandleKeyHash {
constexpr size_t operator()(const HandleKey& k) const noexcept {
size_t h = (size_t(k.index) << 32) ^ k.offset;
h ^= (size_t(k.shift_left) << 1);
h ^= (size_t(k.sec_index) << 33) ^ (size_t(k.sec_offset) << 2);
h ^= (size_t(k.sec_shift_left) << 3);
h ^= (size_t(k.count) << 7);
return h ^ (k.has_secondary ? 0x9e3779b97f4a7c15ULL : 0ULL);
}
};
struct ConstBufferAddr {
u32 index;
u32 offset;
u32 shift_left;
u32 secondary_index;
u32 secondary_offset;
u32 secondary_shift_left;
IR::U32 dynamic_offset;
u32 count;
bool has_secondary;
u32 handle_program{~0U}; ///< index in Info::handle_programs for a handle computed per draw
};
class Environment {
public:
virtual ~Environment() = default;
[[nodiscard]] virtual u64 ReadInstruction(u32 address) = 0;
[[nodiscard]] virtual u32 TranslateProgramAddress(u32 address) const { return address; }
[[nodiscard]] virtual u32 ReadCbufValue(u32 cbuf_index, u32 cbuf_offset) = 0;
[[nodiscard]] virtual TextureType ReadTextureType(u32 raw_handle) = 0;
[[nodiscard]] virtual TexturePixelFormat ReadTexturePixelFormat(u32 raw_handle) = 0;
[[nodiscard]] virtual bool IsTexturePixelFormatInteger(u32 raw_handle) = 0;
[[nodiscard]] virtual u32 ReadViewportTransformState() = 0;
[[nodiscard]] virtual u32 TextureBoundBuffer() const = 0;
[[nodiscard]] virtual u32 LocalMemorySize() const = 0;
[[nodiscard]] virtual u32 SharedMemorySize() const = 0;
[[nodiscard]] virtual std::array<u32, 3> WorkgroupSize() const = 0;
[[nodiscard]] virtual bool HasHLEMacroState() const = 0;
[[nodiscard]] virtual bool HasNativeInstructions() const { return false; } // (from Lichii)
/// (from Fritchii) A Switch 2 native program lowered to Maxwell instructions. Its storage buffer pointers are
/// raw 8-byte arguments in constant buffers, without the NVN descriptor layout.
[[nodiscard]] virtual bool IsNativeProgram() const { return false; }
/// (from Fritchii) Native programs: whether the 8-byte constant at (index, offset) is a mapped GPU address.
/// Native argument tables mix pointers with scalars (sizes, strides).
[[nodiscard]] virtual bool IsCbufStorageBufferPointer(u32, u32) { return true; }
[[nodiscard]] virtual std::optional<ReplaceConstant> GetReplaceConstBuffer(u32 bank,
u32 offset) = 0;
virtual void Dump(u64 pipeline_hash, u64 shader_hash) = 0;
[[nodiscard]] const ProgramHeader& SPH() const noexcept {
return sph;
}
[[nodiscard]] const std::array<u32, 8>& GpPassthroughMask() const noexcept {
return gp_passthrough_mask;
}
[[nodiscard]] Stage ShaderStage() const noexcept {
return stage;
}
[[nodiscard]] u32 StartAddress() const noexcept {
return start_address;
}
[[nodiscard]] bool IsProprietaryDriver() const noexcept {
return is_proprietary_driver;
}
/// Task and mesh programs: SET_MESH_SHADER_A, SET_MESH_SHADER_B and SET_MESH_INIT_SHADER.
[[nodiscard]] const std::array<u32, 3>& MeshRegisters() const noexcept {
return mesh_registers;
}
/// Mesh programs: the header of the pass-through geometry program bound with them, whose
/// output map lists the per-primitive outputs.
[[nodiscard]] const ProgramHeader& MeshPrimitiveHeader() const noexcept {
return mesh_primitive_header;
}
protected:
ProgramHeader sph{};
std::array<u32, 8> gp_passthrough_mask{};
std::array<u32, 3> mesh_registers{};
ProgramHeader mesh_primitive_header{};
Stage stage{};
u32 start_address{};
bool is_proprietary_driver{};
public:
::Common::unordered_map<CbufWordKey, u32, CbufWordKeyHash> cbuf_word_cache;
::Common::unordered_map<HandleKey, u32, HandleKeyHash> handle_cache;
::Common::unordered_map<const IR::Inst*, ConstBufferAddr> track_cache;
};
} // namespace Shader

View File

@@ -0,0 +1,67 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <exception>
#include <string>
#include <utility>
#include "common/logging.h"
namespace Shader {
class Exception : public std::exception {
public:
explicit Exception(std::string message) noexcept : err_message{std::move(message)} {}
[[nodiscard]] const char* what() const noexcept override {
return err_message.c_str();
}
void Prepend(std::string_view prepend) {
err_message.insert(0, prepend);
}
void Append(std::string_view append) {
err_message += append;
}
private:
std::string err_message;
};
class LogicError : public Exception {
public:
template <typename... Args>
explicit LogicError(const char* message, Args&&... args)
: Exception{fmt::format(fmt::runtime(message), std::forward<Args>(args)...)} {}
};
class RuntimeError : public Exception {
public:
template <typename... Args>
explicit RuntimeError(const char* message, Args&&... args)
: Exception{fmt::format(fmt::runtime(message), std::forward<Args>(args)...)} {}
};
class NotImplementedException : public Exception {
public:
template <typename... Args>
explicit NotImplementedException(const char* message, Args&&... args)
: Exception{fmt::format(fmt::runtime(message), std::forward<Args>(args)...)} {
Append(" is not implemented");
}
};
class InvalidArgument : public Exception {
public:
template <typename... Args>
explicit InvalidArgument(const char* message, Args&&... args)
: Exception{fmt::format(fmt::runtime(message), std::forward<Args>(args)...)} {}
};
} // namespace Shader

View File

@@ -0,0 +1,199 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#pragma once
#include <string_view>
#include <vector>
#include "common/common_types.h"
namespace Shader::Ampere {
struct GeometryExitEnvelope {
struct Exit {
u32 offset;
u32 predicate;
};
std::vector<Exit> exits;
u32 handler{};
bool has_returns{};
u32 error_offset{};
std::string_view error;
explicit operator bool() const { return error.empty(); }
};
inline constexpr u32 GeometryExitSetupInstructions = 5;
template <typename Code>
GeometryExitEnvelope AnalyzeGeometryExit(const Code& code) {
constexpr u32 ModeMask = 0xc00001ffU;
constexpr size_t Setup = GeometryExitSetupInstructions;
GeometryExitEnvelope result;
const auto fail = [&](size_t index, std::string_view reason) {
result.exits.clear();
result.handler = 0;
result.has_returns = false;
result.error_offset = static_cast<u32>(index * 16);
result.error = reason;
return result;
};
const auto bits = [&](size_t index, u32 offset, u32 count) {
const auto& words = code[index].words;
u64 value = words[offset / 32];
if (offset / 32 + 1 < 4) value |= u64{words[offset / 32 + 1]} << 32;
return static_cast<u32>((value >> (offset % 32)) & ((u64{1} << count) - 1));
};
const auto is = [&](size_t index, u32 word0, u32 word1, u32 word2) {
const auto& words = code[index].words;
return words[0] == word0 && words[1] == word1 && words[2] == word2 &&
(words[3] & ModeMask) == 0;
};
const auto is_exit = [&](size_t index) {
const auto& words = code[index].words;
return (words[0] & ~0xf000U) == 0x94d && words[1] == 0 && words[2] == 0x03800000 &&
(words[3] & ModeMask) == 0 && bits(index, 12, 4) != 15;
};
const auto is_return = [&](size_t index) {
const auto& words = code[index].words;
const u64 raw = words[1] | (u64{words[2] & 0x3ffff} << 32);
const s64 relative = static_cast<s64>(raw << 14) >> 14;
return (words[0] & 0x00ffffffU) == 0x00007950 && bits(index, 24, 8) < 249 &&
(words[2] & ~0x3ffffU) == 0x03c00000 && (words[3] & ModeMask) == 0 &&
static_cast<s64>(index + 1) * 16 + relative == 0;
};
const size_t count = code.size();
if (count < Setup + 4) return fail(count, "native exit envelope truncated");
const u32 target = code[1].words[1];
if (!is(0, 0x734e, 0, 0) || !is(1, 0x37802, target, 0xf00) || !is(2, 0x7210, 3, 0x07f1e0ff) ||
!(is(3, 0x01017210, 0xff, 0x007fe4ff) || is(3, 0xff017224, 0xff, 0x000e0401)) ||
!is(4, 0x7357, 0, 0))
return fail(0, "native exit handler installation");
const size_t handler = target / 16;
if (target % 16 || handler <= Setup + 1 || handler + 3 != count)
return fail(1, "native exit handler location");
if (!is(handler, 0x00ff7324, 0xff, 0) || !is(handler + 1, 0x7957, 0, 0) ||
!is(handler + 2, 0x794d, 0, 0x03800000))
return fail(handler, "native exit handler body");
if (!(is_exit(handler - 1) && bits(handler - 1, 12, 4) == 7) && !is_return(handler - 1))
return fail(handler - 1, "native exit handler reached by fallthrough");
for (size_t index = Setup; index < handler; ++index) {
const u32 opcode = bits(index, 0, 12), family = opcode & 0x1ff;
if (family == 0x14e || (family >= 0x154 && family <= 0x157))
return fail(index, "native exit handler state access in body");
if (opcode == 0x94d) {
if (!is_exit(index)) return fail(index, "native body EXIT mode");
result.exits.push_back({static_cast<u32>(index * 16), bits(index, 12, 4)});
continue;
}
if (is_return(index)) {
result.has_returns = true;
continue;
}
if (family < 0x140 || family >= 0x160 || opcode == 0x941 || opcode == 0x942 ||
opcode == 0x948)
continue;
const auto& words = code[index].words;
const bool noinc_call = opcode == 0x944 && (words[2] & 0x400000) != 0;
if (opcode != 0x945 && opcode != 0x947 && !noinc_call)
return fail(index, "native control flow in exit envelope body");
const u64 raw = words[1] | (u64{words[2] & 0x3ffff} << 32);
const s64 relative = opcode == 0x945 ? s64{static_cast<s32>(words[1])}
: static_cast<s64>(raw << 14) >> 14;
const s64 destination = static_cast<s64>(index) * 16 + 16 + relative;
if (destination % 16 || destination < static_cast<s64>(Setup * 16) ||
destination >= static_cast<s64>(handler * 16))
return fail(index, "native branch leaves exit envelope body");
}
u32 live_registers = 0b1011, live_predicates = 0b1;
for (size_t index = Setup; live_registers || live_predicates; ++index) {
const u32 opcode = bits(index, 0, 12), family = opcode & 0x1ff;
const u32 form = (opcode >> 9) & 7, guard = bits(index, 12, 4);
u32 reads = 0, read_predicates = 0, writes = 0, write_predicates = 0;
const auto read_register = [&](u32 reg) { if (reg < 32) reads |= 1U << reg; };
const auto write_register = [&](u32 reg) { if (reg < 32) writes |= 1U << reg; };
const auto read_predicate = [&](u32 pred) { if (pred < 7) read_predicates |= 1U << pred; };
const auto write_predicate = [&](u32 pred) { if (pred < 7) write_predicates |= 1U << pred; };
const auto read_sources = [&](bool has_c) {
read_register(bits(index, 24, 8));
if (form == 1) read_register(bits(index, 32, 8));
if (form == 2 || form == 3 || form == 7) read_register(bits(index, 64, 8));
if (has_c && (form == 1 || form == 4 || form == 5 || form == 6)) read_register(bits(index, 64, 8));
};
read_predicate(guard & 7);
if (is_exit(index)) {
if (guard != 7)
return fail(index, "native instruction before exit setup state is dead");
// The handler finalizes the output through R0; the rest of the setup state may stay
// live when the body ends without reading it.
if ((live_registers & 1) || (read_predicates & live_predicates))
return fail(index, "native exit handler reads exit setup state");
break;
}
if (opcode == 0x919 || opcode == 0x923) {
if (opcode == 0x923) read_register(bits(index, 24, 8));
write_register(bits(index, 16, 8));
} else if (opcode == 0x805 || opcode == 0xab9 || opcode == 0x945) {
} else if (opcode == 0x381 || opcode == 0x981) {
const u32 address = bits(index, 24, 8), size = bits(index, 73, 3);
if (address != 255) {
read_register(address);
read_register(address + 1);
}
const u32 results = size < 5 ? 1 : size == 5 ? 2 : size == 6 ? 4 : 0;
for (u32 i = 0; i < results; ++i) write_register(bits(index, 16, 8) + i);
} else if (family == 0x002 && (form == 1 || form == 4 || form == 5 || form == 6)) {
if (form == 1) read_register(bits(index, 32, 8));
write_register(bits(index, 16, 8));
} else if ((family == 0x024 || family == 0x016) && form != 0) {
read_sources(true);
if (family == 0x024 && bits(index, 74, 1)) read_predicate(bits(index, 87, 3));
write_register(bits(index, 16, 8));
} else if (family == 0x105 && (form == 1 || form == 5 || form == 6)) {
// F2I: register, constant buffer or uniform source in operand b; a is unused.
if (form == 1) read_register(bits(index, 32, 8));
write_register(bits(index, 16, 8));
} else if (family == 0x011 && form != 0) {
// LEA (.X reads the carry predicate), optional carry out.
read_sources(true);
if (bits(index, 74, 1)) read_predicate(bits(index, 87, 3));
write_register(bits(index, 16, 8));
write_predicate(bits(index, 81, 3));
} else if (opcode == 0xb82) {
// LDC with a register index; a 64-bit load writes a pair.
if (bits(index, 24, 8) != 255) read_register(bits(index, 24, 8));
write_register(bits(index, 16, 8));
if (bits(index, 73, 3) == 5) write_register(bits(index, 16, 8) + 1);
} else if (family == 0x012 && form != 0) {
// LOP3.LUT with an optional predicate result.
read_sources(true);
write_register(bits(index, 16, 8));
write_predicate(bits(index, 81, 3));
} else if (opcode == 0x322) {
// AST: stores 1-4 components from operand b for the vertex in c; writes nothing.
const u32 value = bits(index, 32, 8), components = bits(index, 74, 2) + 1;
if (value != 255)
for (u32 i = 0; i < components; ++i) read_register(value + i);
read_register(bits(index, 64, 8));
} else if (family == 0x00c && form != 0 && !bits(index, 72, 1)) {
read_sources(false);
read_predicate(bits(index, 87, 3));
write_predicate(bits(index, 81, 3));
write_predicate(bits(index, 84, 3));
} else {
return fail(index, "native instruction before exit setup state is dead");
}
if ((reads & live_registers) || (read_predicates & live_predicates))
return fail(index, "native body reads exit setup state");
if (guard == 7) {
live_registers &= ~writes;
live_predicates &= ~write_predicates;
}
}
result.handler = target;
return result;
}
} // namespace Shader::Ampere

View File

@@ -0,0 +1,20 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#pragma once
#include <array>
#include "common/common_types.h"
namespace Shader::Ampere {
struct Instruction {
std::array<u32, 4> words;
u32 Bits(u32 offset, u32 count) const {
u64 bits = words[offset / 32];
if (offset / 32 + 1 < words.size())
bits |= u64{words[offset / 32 + 1]} << 32;
return static_cast<u32>((bits >> (offset % 32)) & ((u64{1} << count) - 1));
}
};
static_assert(sizeof(Instruction) == 16);
} // namespace Shader::Ampere

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,29 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#pragma once
#include <optional>
#include <span>
#include <vector>
#include "common/common_types.h"
namespace Shader::Ampere {
constexpr u64 NativeAddressMapMagic = 0x50414d4e4544454eULL;
inline bool HasNativeAddressMap(std::span<const u64> code) {
return code.size() >= 2 && code.back() == NativeAddressMapMagic;
}
inline void AppendNativeAddressMap(std::vector<u64>& code, std::span<const u32> offsets) {
if (offsets.empty()) return;
for (u32 index : offsets) code.push_back(0x50U + index * 8 + ((index + 1) / 3) * 8);
code.push_back(offsets.size());
code.push_back(NativeAddressMapMagic);
}
inline std::optional<u32> NativeProgramAddress(std::span<const u64> code, u32 address) {
if (!HasNativeAddressMap(code) || address % 16) return {};
const u64 count = code[code.size() - 2];
if (!count || count > 65536 || count + 2 > code.size() || address / 16 >= count) return {};
const size_t start = code.size() - static_cast<size_t>(count) - 2;
const u64 target = code[start + address / 16];
if (target < 0x50 || target >= start * 8 || target % 8 || target % 32 == 0) return {};
return static_cast<u32>(target);
}
} // namespace Shader::Ampere

View File

@@ -0,0 +1,81 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#pragma once
#include "shader_recompiler/frontend/ampere/lower_to_maxwell.h"
namespace Shader::Ampere {
class ProgramCaptureState {
public:
enum class Result { Continue, Complete, InvalidTarget };
static constexpr u32 MaximumInstructions = 65536;
Result Observe(const Instruction& instruction, u32 index) {
const u32 opcode = instruction.Bits(0, 12);
if (index < exit_setup.size()) {
exit_setup[index] = instruction;
if (index + 1 == exit_setup.size() &&
std::all_of(exit_setup.begin(), exit_setup.end(), [](const Instruction& setup) {
return (setup.words[3] & 0xc00001ffU) == 0;
}) &&
exit_setup[0].words[0] == 0x0000734e &&
exit_setup[0].words[1] == 0 && exit_setup[0].words[2] == 0 &&
exit_setup[1].words[0] == 0x00037802 && exit_setup[1].words[2] == 0xf00 &&
exit_setup[2].words[0] == 0x00007210 && exit_setup[2].words[1] == 3 &&
exit_setup[2].words[2] == 0x07f1e0ff &&
exit_setup[3].words[1] == 0xff &&
((exit_setup[3].words[0] == 0x01017210 &&
exit_setup[3].words[2] == 0x007fe4ff) ||
(exit_setup[3].words[0] == 0xff017224 &&
exit_setup[3].words[2] == 0x000e0401)) &&
instruction.words[0] == 0x00007357 && instruction.words[1] == 0 &&
instruction.words[2] == 0) {
const u32 target = exit_setup[1].words[1];
if (target < exit_setup.size() * 16 || target % 16 ||
target >= MaximumInstructions * 16)
return Result::InvalidTarget;
last_forward_target = std::max(last_forward_target, target / 16);
}
}
if (opcode == 0x944 || opcode == 0x947 || opcode == 0x945) {
// BRA keeps its .DIV/.CONV mode in the offset's two low bits.
const u32 low = instruction.words[1] & (opcode == 0x947 ? ~3U : ~0U);
const u64 raw = low | (u64{instruction.words[2] & 0x3ffff} << 32);
const s64 relative = opcode == 0x945 ? static_cast<s32>(instruction.words[1])
: static_cast<s64>(raw << 14) >> 14;
const s64 target = s64{index} * 16 + 16 + relative;
if (target < 0 || target % 16 || target >= MaximumInstructions * 16)
return Result::InvalidTarget;
// CALL.REL.NOINC (bit 86) pushes no return address, but its callee still returns
// with RET.REL.NODEC through the link register set ahead of the call.
if (opcode == 0x944) {
last_call_target = std::max(last_call_target, static_cast<u32>(target / 16));
} else {
last_forward_target = std::max(last_forward_target, static_cast<u32>(target / 16));
}
// The compiler ends every program with an unconditional branch to itself; with a jump
// table (whose targets live in a constant buffer) that is the only reliable end.
if (opcode == 0x947 && computed_branch && instruction.Bits(12, 4) == 7 &&
target == s64{index} * 16)
return Result::Complete;
}
if (opcode == 0x949 || opcode == 0x958) computed_branch = true;
if (computed_branch) return Result::Continue;
if (opcode == 0x94d && instruction.Bits(12, 4) == 7 && index >= last_forward_target) {
if (index >= last_call_target) return Result::Complete;
main_exited = true;
}
if (opcode == 0x950 && instruction.Bits(12, 4) == 7 && main_exited &&
index >= last_call_target && index >= last_forward_target)
return Result::Complete;
return Result::Continue;
}
private:
std::array<Instruction, 5> exit_setup{};
u32 last_call_target{};
u32 last_forward_target{};
bool main_exited{};
bool computed_branch{};
};
} // namespace Shader::Ampere

View File

@@ -0,0 +1,85 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#pragma once
#include <cstring>
#include "shader_recompiler/frontend/ampere/lower_to_maxwell.h"
namespace Shader::Ampere {
inline u64 NativeProgramKey(std::span<const Instruction> source) {
u64 hash = 14695981039346656037ULL;
for (const auto& instruction : source) {
for (u32 word : instruction.words) {
for (u32 byte = 0; byte < 4; ++byte) hash = (hash ^ ((word >> (byte * 8)) & 255)) * 1099511628211ULL;
}
}
return hash;
}
// Private iteration format: version, lengths, texture bank, exact original
// instructions, then the newly generated canonical instructions. Source bytes
// are compared directly; the filename hash is only an index.
inline std::vector<u8> EncodeProgramOverride(std::span<const Instruction> source,
const LoweredProgram& program) {
if (!program || program.source_size != source.size_bytes() || program.source_instruction_offsets.size() != source.size()) return {};
const std::array<u32, 6> header{0x4e454445, 0x3230564f,
static_cast<u32>(source.size_bytes()), static_cast<u32>(program.instructions.size()),
program.required_texture_buffer, static_cast<u32>(program.source_instruction_offsets.size())};
std::vector<u8> bytes(sizeof(header) + source.size_bytes() + program.instructions.size() * 8 + program.source_instruction_offsets.size() * 4);
std::memcpy(bytes.data(), header.data(), sizeof(header));
std::memcpy(bytes.data() + sizeof(header), source.data(), source.size_bytes());
std::memcpy(bytes.data() + sizeof(header) + source.size_bytes(), program.instructions.data(), program.instructions.size() * 8);
std::memcpy(bytes.data() + sizeof(header) + source.size_bytes() + program.instructions.size() * 8,
program.source_instruction_offsets.data(), program.source_instruction_offsets.size() * 4);
return bytes;
}
inline u32 ProgramOverrideSourceSize(std::span<const u8> bytes) {
std::array<u32, 6> header{};
if (bytes.size() < sizeof(header)) return 0;
std::memcpy(header.data(), bytes.data(), sizeof(header));
const bool mapped = header[1] == 0x3230564f;
if (header[0] != 0x4e454445 || (!mapped && header[1] != 0x3130564f) ||
(mapped ? header[5] != header[2] / 16 : header[5] != 0) ||
header[2] == 0 || header[2] % sizeof(Instruction) || header[2] > 1048576 ||
header[3] == 0 || header[3] > 1048576 || (header[4] != ~0U && header[4] >= 18)) return 0;
if (bytes.size() != sizeof(header) + size_t{header[2]} + size_t{header[3]} * 8 + size_t{header[5]} * 4) return 0;
return header[2];
}
inline LoweredProgram DecodeProgramOverride(std::span<const u8> bytes,
std::span<const Instruction> source) {
LoweredProgram result;
const auto fail = [&result](std::string_view message) {
result.instructions.clear();
result.source_instruction_offsets.clear();
result.error = message;
return result;
};
std::array<u32, 6> header{};
if (!ProgramOverrideSourceSize(bytes)) return fail("invalid native override header or length");
std::memcpy(header.data(), bytes.data(), sizeof(header));
if (header[2] != source.size_bytes()) return fail("native override source length mismatch");
if (std::memcmp(bytes.data() + sizeof(header), source.data(), source.size_bytes()))
return fail("native override source mismatch");
result.instructions.resize(header[3]);
std::memcpy(result.instructions.data(), bytes.data() + sizeof(header) + source.size_bytes(), header[3] * 8);
if (header[5]) {
result.source_instruction_offsets.resize(header[5]);
std::memcpy(result.source_instruction_offsets.data(),
bytes.data() + sizeof(header) + source.size_bytes() + size_t{header[3]} * 8,
size_t{header[5]} * 4);
u32 previous = 0;
for (u32 offset : result.source_instruction_offsets) {
if (offset >= result.instructions.size() || offset < previous)
return fail("native override address map mismatch");
previous = offset;
}
}
result.source_size = header[2];
result.required_texture_buffer = header[4];
return result;
}
} // namespace Shader::Ampere

View File

@@ -0,0 +1,316 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
#pragma once
#include <array>
#include <vector>
#include "shader_recompiler/frontend/ampere/instruction.h"
namespace Shader::Ampere {
struct RegisterPressure {
Instruction instruction;
std::vector<u64> prefix, suffix;
bool valid = true;
};
inline u64 NativeRegisterCopy(u32 destination_bank, u32 destination, u32 source_bank, u32 source) {
return 0x5c98000000070000ULL | (u64{10} << 44) | destination | (u64{destination_bank} << 8) |
(u64{source_bank} << 10) | (u64{source} << 20);
}
// Preserve high guest registers in SSA while borrowing aligned low quartets
// for one native instruction. The expanded instruction owns its temporary bank.
inline RegisterPressure PrepareRegisterPressure(const Instruction& input, u32 reserved) {
RegisterPressure result{input};
if (reserved < 4 || reserved > 246 || (reserved & 3)) {
result.valid = false;
return result;
}
struct Operand {
u32 bit, count;
};
std::vector<Operand> operands;
const auto operand = [&](u32 bit, u32 count = 1) { operands.push_back({bit, count}); };
const u32 opcode = input.Bits(0, 12);
switch (opcode) {
case 0xa02:
case 0xc3e:
case 0x802:
case 0xb08:
case 0xd08:
case 0x908:
case 0x919:
operand(16);
break;
// (triforce 11-10) Forms this table lacked: 0x3c2, 0x803, 0x809, 0xb82, 0xc0c, 0xc12, 0xc19, 0xc20, 0xc23, 0xc24, 0xc45.
// A mesh program that reached one of them with a high register stopped as "native register shadowing mode"; the old
// lowering refused all but two of those too, so their draws were never made (tests/seed_lowering.cpp over the game's
// 13,773 shader stages: 26 of its 491 mesh stages lowered by neither, 9 of them the models'; 5 after this). Each form
// lists the fields its lowering (lower_to_maxwell.h) reads as general registers: the uniform register of a 0xc.. form
// (bits 32-39) and R2UR's destination are not among them.
case 0x3c2: // R2UR: its source (the destination is a uniform register)
case 0xc0c: // ISETP against a uniform register: its first source (it writes predicates only)
case 0xa0c:
case 0x804:
case 0x80b:
case 0x80c:
operand(24);
break;
case 0xb82: { // LDC, indexed: the destination (a pair at width 5) and the index
const u32 width = input.Bits(73, 3);
operand(16, width == 6 ? 4 : width == 5 ? 2 : 1);
operand(24);
break;
}
case 0x20c:
case 0x20b:
operand(24);
operand(32);
break;
case 0x803: // (triforce) P2R: destination and merge source
case 0x809: // (triforce) FMNMX with an immediate
case 0xc20: // (triforce) FMUL by a uniform register
case 0x807:
case 0x808:
case 0x923:
case 0x820:
case 0xa20:
case 0x421:
operand(16);
operand(24);
break;
case 0x207:
case 0x208:
case 0x220:
case 0x221:
case 0x209:
operand(16);
operand(24);
operand(32);
break;
case 0x805:
operand(16, 2);
break;
case 0xc17:
case 0x817:
operand(16);
operand(24);
break;
case 0xc23: // (triforce) FFMA with a uniform multiplier: destination, a and the addend
case 0xa24:
case 0xa23:
case 0x623:
case 0x423:
operand(16);
operand(24);
operand(64);
break;
case 0x621:
operand(16);
operand(24);
break;
case 0x321:
operand(16, input.Bits(74, 2) + 1);
operand(24);
operand(32);
break;
case 0x322:
operand(24);
operand(32, input.Bits(74, 2) + 1);
break;
case 0x927:
operand(24);
operand(32);
break;
case 0x224:
case 0x223:
operand(16);
operand(24);
operand(32);
operand(64);
break;
case 0x81a:
operand(16);
operand(24);
break;
case 0x308:
case 0x202:
case 0x243:
case 0x245:
case 0x305:
case 0x306:
case 0x307:
operand(16);
operand(32);
break;
case 0xc45: // (triforce) I2FP of a uniform register
case 0xb05:
case 0xd05:
operand(16);
break;
case 0x59f:
operand(24, 2);
operand(32);
break;
case 0x98e:
operand(24, input.Bits(72, 1) ? 2 : 1);
operand(32, input.Bits(73, 3) == 2 || input.Bits(73, 3) == 5 ? 2 : 1);
break;
case 0x433:
case 0x633:
operand(16);
operand(24);
break;
case 0x233:
operand(16);
operand(24);
operand(32);
break;
case 0xc12: // (triforce) LOP3 with a uniform second source
case 0xc24: // (triforce) IMAD with a uniform multiplier
case 0x810:
case 0xa10:
case 0xc10:
case 0x812:
case 0x824:
case 0x823:
case 0x831:
case 0x424:
case 0x624:
operand(16);
operand(24);
operand(64);
break;
case 0x230:
case 0x231:
operand(16);
operand(24);
operand(32);
operand(64);
break;
case 0xc19: // (triforce) SHF by a uniform amount: destination, low and high words
case 0x419:
case 0x819:
operand(16);
operand(24);
operand(64);
break;
case 0x210:
case 0x211:
case 0x212:
case 0x219:
operand(16);
operand(24);
operand(32);
operand(64);
break;
case 0xa11:
operand(16);
operand(24);
operand(64);
break;
case 0x225:
case 0x227:
case 0x625:
case 0x825:
operand(16, opcode == 0x227 ? 1 : 2);
operand(24);
if (opcode == 0x225 || opcode == 0x227) {
operand(32);
operand(64, 2);
}
if (opcode == 0x625)
operand(64);
if (opcode == 0x825)
operand(64, 2);
break;
case 0x83b:
operand(16, 1U << input.Bits(72, 2));
operand(24);
break;
case 0x23c:
operand(16, 2);
operand(24, input.Bits(75, 1) ? 4 : 2);
operand(32, input.Bits(75, 1) ? 2 : 1);
operand(64, 2);
break;
case 0x388:
case 0x984:
case 0x988: {
const u32 width = input.Bits(73, 3), count = width == 6 ? 4 : width == 5 ? 2 : 1;
operand(opcode == 0x984 ? 16 : 32, count);
operand(24);
break;
}
case 0x387:
case 0x983:
case 0x987: {
const u32 width = input.Bits(73, 3), count = width == 6 ? 4 : width == 5 ? 2 : 1;
operand(opcode == 0x983 ? 16 : 32, count);
operand(24);
break;
}
case 0x381:
case 0x386:
case 0x981:
case 0x986: {
const u32 width = input.Bits(73, 3), count = width == 6 ? 4 : width == 5 ? 2 : 1;
operand(opcode == 0x386 || opcode == 0x986 ? 32 : 16, count);
operand(24, opcode == 0x381 || opcode == 0x386 || input.Bits(90, 1) ? 2 : 1);
break;
}
default:
result.valid = false;
return result;
}
std::array<bool, 256> used{};
std::array<bool, 64> groups{};
for (const auto [bit, count] : operands) {
const u32 reg = input.Bits(bit, 8);
if (reg == 255)
continue;
if (reg + count > 255 ||
(reg < reserved && reg + count > reserved) ||
(reg >= reserved && reg / 4 != (reg + count - 1) / 4)) {
result.valid = false;
return result;
}
for (u32 component = 0; component < count; ++component) {
used[reg + component] = true;
if (reg + component >= reserved)
groups[(reg + component) / 4] = true;
}
}
std::array<u32, 64> mapped{};
for (u32 group = 0; group < groups.size(); ++group) {
if (!groups[group])
continue;
u32 borrow = reserved;
while (borrow >= 4) {
borrow -= 4;
if (!used[borrow] && !used[borrow + 1] && !used[borrow + 2] && !used[borrow + 3])
break;
}
if (used[borrow] || used[borrow + 1] || used[borrow + 2] || used[borrow + 3]) {
result.valid = false;
return result;
}
mapped[group] = borrow;
for (u32 component = 0; component < 4; ++component) {
const u32 low = borrow + component, high = group * 4 + component;
used[low] = true;
result.prefix.push_back(NativeRegisterCopy(2, low, 0, low));
result.prefix.push_back(NativeRegisterCopy(0, low, 1, high));
result.suffix.push_back(NativeRegisterCopy(1, high, 0, low));
result.suffix.push_back(NativeRegisterCopy(0, low, 2, low));
}
}
for (const auto [bit, count] : operands) {
const u32 reg = input.Bits(bit, 8);
if (reg == 255 || !groups[reg / 4])
continue;
const u32 replacement = mapped[reg / 4] + reg % 4;
const u32 word = bit / 32, shift = bit % 32;
result.instruction.words[word] =
(result.instruction.words[word] & ~(255U << shift)) | (replacement << shift);
}
return result;
}
} // namespace Shader::Ampere

View File

@@ -0,0 +1,57 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <vector>
#include "shader_recompiler/frontend/ir/value.h"
namespace Shader::IR {
class Block;
struct AbstractSyntaxNode {
enum class Type {
Block,
If,
EndIf,
Loop,
Repeat,
Break,
Return,
Unreachable,
};
union Data {
Block* block;
struct {
U1 cond;
Block* body;
Block* merge;
} if_node;
struct {
Block* merge;
} end_if;
struct {
Block* body;
Block* continue_block;
Block* merge;
} loop;
struct {
U1 cond;
Block* loop_header;
Block* merge;
} repeat;
struct {
U1 cond;
Block* merge;
Block* skip;
} break_node;
};
Data data{};
Type type{};
};
using AbstractSyntaxList = std::vector<AbstractSyntaxNode>;
} // namespace Shader::IR

View File

@@ -0,0 +1,288 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <fmt/ranges.h>
#include "common/common_types.h"
#include "shader_recompiler/exception.h"
namespace Shader::IR {
enum class Attribute : u64 {
#define SRIR_ATTRIBUTE_LIST \
SRIR_ATTRIBUTE_ELEM(PrimitiveId, 24) \
SRIR_ATTRIBUTE_ELEM(Layer, 25) \
SRIR_ATTRIBUTE_ELEM(ViewportIndex, 26) \
SRIR_ATTRIBUTE_ELEM(PointSize, 27) \
SRIR_ATTRIBUTE_ELEM(PositionX, 28) \
SRIR_ATTRIBUTE_ELEM(PositionY, 29) \
SRIR_ATTRIBUTE_ELEM(PositionZ, 30) \
SRIR_ATTRIBUTE_ELEM(PositionW, 31) \
SRIR_ATTRIBUTE_ELEM(Generic0X, 32) \
SRIR_ATTRIBUTE_ELEM(Generic0Y, 33) \
SRIR_ATTRIBUTE_ELEM(Generic0Z, 34) \
SRIR_ATTRIBUTE_ELEM(Generic0W, 35) \
SRIR_ATTRIBUTE_ELEM(Generic1X, 36) \
SRIR_ATTRIBUTE_ELEM(Generic1Y, 37) \
SRIR_ATTRIBUTE_ELEM(Generic1Z, 38) \
SRIR_ATTRIBUTE_ELEM(Generic1W, 39) \
SRIR_ATTRIBUTE_ELEM(Generic2X, 40) \
SRIR_ATTRIBUTE_ELEM(Generic2Y, 41) \
SRIR_ATTRIBUTE_ELEM(Generic2Z, 42) \
SRIR_ATTRIBUTE_ELEM(Generic2W, 43) \
SRIR_ATTRIBUTE_ELEM(Generic3X, 44) \
SRIR_ATTRIBUTE_ELEM(Generic3Y, 45) \
SRIR_ATTRIBUTE_ELEM(Generic3Z, 46) \
SRIR_ATTRIBUTE_ELEM(Generic3W, 47) \
SRIR_ATTRIBUTE_ELEM(Generic4X, 48) \
SRIR_ATTRIBUTE_ELEM(Generic4Y, 49) \
SRIR_ATTRIBUTE_ELEM(Generic4Z, 50) \
SRIR_ATTRIBUTE_ELEM(Generic4W, 51) \
SRIR_ATTRIBUTE_ELEM(Generic5X, 52) \
SRIR_ATTRIBUTE_ELEM(Generic5Y, 53) \
SRIR_ATTRIBUTE_ELEM(Generic5Z, 54) \
SRIR_ATTRIBUTE_ELEM(Generic5W, 55) \
SRIR_ATTRIBUTE_ELEM(Generic6X, 56) \
SRIR_ATTRIBUTE_ELEM(Generic6Y, 57) \
SRIR_ATTRIBUTE_ELEM(Generic6Z, 58) \
SRIR_ATTRIBUTE_ELEM(Generic6W, 59) \
SRIR_ATTRIBUTE_ELEM(Generic7X, 60) \
SRIR_ATTRIBUTE_ELEM(Generic7Y, 61) \
SRIR_ATTRIBUTE_ELEM(Generic7Z, 62) \
SRIR_ATTRIBUTE_ELEM(Generic7W, 63) \
SRIR_ATTRIBUTE_ELEM(Generic8X, 64) \
SRIR_ATTRIBUTE_ELEM(Generic8Y, 65) \
SRIR_ATTRIBUTE_ELEM(Generic8Z, 66) \
SRIR_ATTRIBUTE_ELEM(Generic8W, 67) \
SRIR_ATTRIBUTE_ELEM(Generic9X, 68) \
SRIR_ATTRIBUTE_ELEM(Generic9Y, 69) \
SRIR_ATTRIBUTE_ELEM(Generic9Z, 70) \
SRIR_ATTRIBUTE_ELEM(Generic9W, 71) \
SRIR_ATTRIBUTE_ELEM(Generic10X, 72) \
SRIR_ATTRIBUTE_ELEM(Generic10Y, 73) \
SRIR_ATTRIBUTE_ELEM(Generic10Z, 74) \
SRIR_ATTRIBUTE_ELEM(Generic10W, 75) \
SRIR_ATTRIBUTE_ELEM(Generic11X, 76) \
SRIR_ATTRIBUTE_ELEM(Generic11Y, 77) \
SRIR_ATTRIBUTE_ELEM(Generic11Z, 78) \
SRIR_ATTRIBUTE_ELEM(Generic11W, 79) \
SRIR_ATTRIBUTE_ELEM(Generic12X, 80) \
SRIR_ATTRIBUTE_ELEM(Generic12Y, 81) \
SRIR_ATTRIBUTE_ELEM(Generic12Z, 82) \
SRIR_ATTRIBUTE_ELEM(Generic12W, 83) \
SRIR_ATTRIBUTE_ELEM(Generic13X, 84) \
SRIR_ATTRIBUTE_ELEM(Generic13Y, 85) \
SRIR_ATTRIBUTE_ELEM(Generic13Z, 86) \
SRIR_ATTRIBUTE_ELEM(Generic13W, 87) \
SRIR_ATTRIBUTE_ELEM(Generic14X, 88) \
SRIR_ATTRIBUTE_ELEM(Generic14Y, 89) \
SRIR_ATTRIBUTE_ELEM(Generic14Z, 90) \
SRIR_ATTRIBUTE_ELEM(Generic14W, 91) \
SRIR_ATTRIBUTE_ELEM(Generic15X, 92) \
SRIR_ATTRIBUTE_ELEM(Generic15Y, 93) \
SRIR_ATTRIBUTE_ELEM(Generic15Z, 94) \
SRIR_ATTRIBUTE_ELEM(Generic15W, 95) \
SRIR_ATTRIBUTE_ELEM(Generic16X, 96) \
SRIR_ATTRIBUTE_ELEM(Generic16Y, 97) \
SRIR_ATTRIBUTE_ELEM(Generic16Z, 98) \
SRIR_ATTRIBUTE_ELEM(Generic16W, 99) \
SRIR_ATTRIBUTE_ELEM(Generic17X, 100) \
SRIR_ATTRIBUTE_ELEM(Generic17Y, 101) \
SRIR_ATTRIBUTE_ELEM(Generic17Z, 102) \
SRIR_ATTRIBUTE_ELEM(Generic17W, 103) \
SRIR_ATTRIBUTE_ELEM(Generic18X, 104) \
SRIR_ATTRIBUTE_ELEM(Generic18Y, 105) \
SRIR_ATTRIBUTE_ELEM(Generic18Z, 106) \
SRIR_ATTRIBUTE_ELEM(Generic18W, 107) \
SRIR_ATTRIBUTE_ELEM(Generic19X, 108) \
SRIR_ATTRIBUTE_ELEM(Generic19Y, 109) \
SRIR_ATTRIBUTE_ELEM(Generic19Z, 110) \
SRIR_ATTRIBUTE_ELEM(Generic19W, 111) \
SRIR_ATTRIBUTE_ELEM(Generic20X, 112) \
SRIR_ATTRIBUTE_ELEM(Generic20Y, 113) \
SRIR_ATTRIBUTE_ELEM(Generic20Z, 114) \
SRIR_ATTRIBUTE_ELEM(Generic20W, 115) \
SRIR_ATTRIBUTE_ELEM(Generic21X, 116) \
SRIR_ATTRIBUTE_ELEM(Generic21Y, 117) \
SRIR_ATTRIBUTE_ELEM(Generic21Z, 118) \
SRIR_ATTRIBUTE_ELEM(Generic21W, 119) \
SRIR_ATTRIBUTE_ELEM(Generic22X, 120) \
SRIR_ATTRIBUTE_ELEM(Generic22Y, 121) \
SRIR_ATTRIBUTE_ELEM(Generic22Z, 122) \
SRIR_ATTRIBUTE_ELEM(Generic22W, 123) \
SRIR_ATTRIBUTE_ELEM(Generic23X, 124) \
SRIR_ATTRIBUTE_ELEM(Generic23Y, 125) \
SRIR_ATTRIBUTE_ELEM(Generic23Z, 126) \
SRIR_ATTRIBUTE_ELEM(Generic23W, 127) \
SRIR_ATTRIBUTE_ELEM(Generic24X, 128) \
SRIR_ATTRIBUTE_ELEM(Generic24Y, 129) \
SRIR_ATTRIBUTE_ELEM(Generic24Z, 130) \
SRIR_ATTRIBUTE_ELEM(Generic24W, 131) \
SRIR_ATTRIBUTE_ELEM(Generic25X, 132) \
SRIR_ATTRIBUTE_ELEM(Generic25Y, 133) \
SRIR_ATTRIBUTE_ELEM(Generic25Z, 134) \
SRIR_ATTRIBUTE_ELEM(Generic25W, 135) \
SRIR_ATTRIBUTE_ELEM(Generic26X, 136) \
SRIR_ATTRIBUTE_ELEM(Generic26Y, 137) \
SRIR_ATTRIBUTE_ELEM(Generic26Z, 138) \
SRIR_ATTRIBUTE_ELEM(Generic26W, 139) \
SRIR_ATTRIBUTE_ELEM(Generic27X, 140) \
SRIR_ATTRIBUTE_ELEM(Generic27Y, 141) \
SRIR_ATTRIBUTE_ELEM(Generic27Z, 142) \
SRIR_ATTRIBUTE_ELEM(Generic27W, 143) \
SRIR_ATTRIBUTE_ELEM(Generic28X, 144) \
SRIR_ATTRIBUTE_ELEM(Generic28Y, 145) \
SRIR_ATTRIBUTE_ELEM(Generic28Z, 146) \
SRIR_ATTRIBUTE_ELEM(Generic28W, 147) \
SRIR_ATTRIBUTE_ELEM(Generic29X, 148) \
SRIR_ATTRIBUTE_ELEM(Generic29Y, 149) \
SRIR_ATTRIBUTE_ELEM(Generic29Z, 150) \
SRIR_ATTRIBUTE_ELEM(Generic29W, 151) \
SRIR_ATTRIBUTE_ELEM(Generic30X, 152) \
SRIR_ATTRIBUTE_ELEM(Generic30Y, 153) \
SRIR_ATTRIBUTE_ELEM(Generic30Z, 154) \
SRIR_ATTRIBUTE_ELEM(Generic30W, 155) \
SRIR_ATTRIBUTE_ELEM(Generic31X, 156) \
SRIR_ATTRIBUTE_ELEM(Generic31Y, 157) \
SRIR_ATTRIBUTE_ELEM(Generic31Z, 158) \
SRIR_ATTRIBUTE_ELEM(Generic31W, 159) \
SRIR_ATTRIBUTE_ELEM(ColorFrontDiffuseR, 160) \
SRIR_ATTRIBUTE_ELEM(ColorFrontDiffuseG, 161) \
SRIR_ATTRIBUTE_ELEM(ColorFrontDiffuseB, 162) \
SRIR_ATTRIBUTE_ELEM(ColorFrontDiffuseA, 163) \
SRIR_ATTRIBUTE_ELEM(ColorFrontSpecularR, 164) \
SRIR_ATTRIBUTE_ELEM(ColorFrontSpecularG, 165) \
SRIR_ATTRIBUTE_ELEM(ColorFrontSpecularB, 166) \
SRIR_ATTRIBUTE_ELEM(ColorFrontSpecularA, 167) \
SRIR_ATTRIBUTE_ELEM(ColorBackDiffuseR, 168) \
SRIR_ATTRIBUTE_ELEM(ColorBackDiffuseG, 169) \
SRIR_ATTRIBUTE_ELEM(ColorBackDiffuseB, 170) \
SRIR_ATTRIBUTE_ELEM(ColorBackDiffuseA, 171) \
SRIR_ATTRIBUTE_ELEM(ColorBackSpecularR, 172) \
SRIR_ATTRIBUTE_ELEM(ColorBackSpecularG, 173) \
SRIR_ATTRIBUTE_ELEM(ColorBackSpecularB, 174) \
SRIR_ATTRIBUTE_ELEM(ColorBackSpecularA, 175) \
SRIR_ATTRIBUTE_ELEM(ClipDistance0, 176) \
SRIR_ATTRIBUTE_ELEM(ClipDistance1, 177) \
SRIR_ATTRIBUTE_ELEM(ClipDistance2, 178) \
SRIR_ATTRIBUTE_ELEM(ClipDistance3, 179) \
SRIR_ATTRIBUTE_ELEM(ClipDistance4, 180) \
SRIR_ATTRIBUTE_ELEM(ClipDistance5, 181) \
SRIR_ATTRIBUTE_ELEM(ClipDistance6, 182) \
SRIR_ATTRIBUTE_ELEM(ClipDistance7, 183) \
SRIR_ATTRIBUTE_ELEM(PointSpriteS, 184) \
SRIR_ATTRIBUTE_ELEM(PointSpriteT, 185) \
SRIR_ATTRIBUTE_ELEM(FogCoordinate, 186) \
SRIR_ATTRIBUTE_ELEM(TessellationEvaluationPointU, 188) \
SRIR_ATTRIBUTE_ELEM(TessellationEvaluationPointV, 189) \
SRIR_ATTRIBUTE_ELEM(InstanceId, 190) \
SRIR_ATTRIBUTE_ELEM(VertexId, 191) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture0S, 192) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture0T, 193) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture0R, 194) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture0Q, 195) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture1S, 196) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture1T, 197) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture1R, 198) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture1Q, 199) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture2S, 200) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture2T, 201) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture2R, 202) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture2Q, 203) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture3S, 204) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture3T, 205) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture3R, 206) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture3Q, 207) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture4S, 208) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture4T, 209) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture4R, 210) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture4Q, 211) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture5S, 212) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture5T, 213) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture5R, 214) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture5Q, 215) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture6S, 216) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture6T, 217) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture6R, 218) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture6Q, 219) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture7S, 220) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture7T, 221) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture7R, 222) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture7Q, 223) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture8S, 224) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture8T, 225) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture8R, 226) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture8Q, 227) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture9S, 228) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture9T, 229) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture9R, 230) \
SRIR_ATTRIBUTE_ELEM(FixedFncTexture9Q, 231) \
SRIR_ATTRIBUTE_ELEM(ViewportMask, 232) \
SRIR_ATTRIBUTE_ELEM(FrontFace, 255) \
/* Implementation attributes */ \
SRIR_ATTRIBUTE_ELEM(BaseInstance, 256) \
SRIR_ATTRIBUTE_ELEM(BaseVertex, 257) \
SRIR_ATTRIBUTE_ELEM(DrawID, 258)
#define SRIR_ATTRIBUTE_ELEM(n, v) n = v,
SRIR_ATTRIBUTE_LIST
#undef SRIR_ATTRIBUTE_ELEM
};
constexpr size_t NUM_GENERICS = 32;
constexpr size_t NUM_FIXEDFNCTEXTURE = 10;
[[nodiscard]] inline bool IsGeneric(Attribute attribute) noexcept {
return attribute >= Attribute::Generic0X && attribute <= Attribute::Generic31X;
}
[[nodiscard]] inline bool IsLegacyAttribute(Attribute attribute) noexcept {
return (attribute >= Attribute::ColorFrontDiffuseR &&
attribute <= Attribute::ColorBackSpecularA) ||
attribute == Attribute::FogCoordinate ||
(attribute >= Attribute::FixedFncTexture0S &&
attribute <= Attribute::FixedFncTexture9Q);
}
[[nodiscard]] inline u32 GenericAttributeIndex(Attribute attribute) {
if (!IsGeneric(attribute))
throw InvalidArgument("Attribute is not generic {}", attribute);
return (u32(attribute) - u32(Attribute::Generic0X)) / 4u;
}
[[nodiscard]] inline u32 GenericAttributeElement(Attribute attribute) {
if (!IsGeneric(attribute))
throw InvalidArgument("Attribute is not generic {}", attribute);
return u32(attribute) % 4;
}
[[nodiscard]] inline std::string NameOf(Attribute attribute) {
switch (attribute) {
#define SRIR_ATTRIBUTE_ELEM(n, v) case Attribute::n: return #n;
SRIR_ATTRIBUTE_LIST
#undef SRIR_ATTRIBUTE_ELEM
default:
return fmt::format("<reserved attribute {}>", int(attribute));
}
}
[[nodiscard]] constexpr IR::Attribute operator+(IR::Attribute attribute, size_t value) noexcept {
return IR::Attribute(size_t(attribute) + value);
}
} // namespace Shader::IR
template <>
struct fmt::formatter<Shader::IR::Attribute> {
constexpr auto parse(format_parse_context& ctx) {
return ctx.begin();
}
template <typename FormatContext>
auto format(const Shader::IR::Attribute& attribute, FormatContext& ctx) const {
return fmt::format_to(ctx.out(), "{}", Shader::IR::NameOf(attribute));
}
};

View File

@@ -0,0 +1,154 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <algorithm>
#include <initializer_list>
#include <map>
#include "common/common_types.h"
#include "shader_recompiler/frontend/ir/basic_block.h"
#include "shader_recompiler/frontend/ir/value.h"
namespace Shader::IR {
Block::Block(ObjectPool<Inst>& inst_pool_) : inst_pool{&inst_pool_} {}
Block::~Block() = default;
void Block::AppendNewInst(Opcode op, std::initializer_list<Value> args) {
PrependNewInst(instructions.end(), op, args);
}
Block::iterator Block::PrependNewInst(iterator insertion_point, const Inst& base_inst) {
Inst* const inst{inst_pool->Create(base_inst)};
return instructions.insert(insertion_point, *inst);
}
Block::iterator Block::PrependNewInst(iterator insertion_point, Opcode op,
std::initializer_list<Value> args, u32 flags) {
Inst* const inst{inst_pool->Create(op, flags)};
const auto result_it{instructions.insert(insertion_point, *inst)};
if (inst->NumArgs() != args.size()) {
throw InvalidArgument("Invalid number of arguments {} in {}", args.size(), op);
}
std::ranges::for_each(args, [inst, index = size_t{0}](const Value& arg) mutable {
inst->SetArg(index, arg);
++index;
});
return result_it;
}
void Block::AddBranch(Block* block) {
if (std::ranges::find(imm_successors, block) != imm_successors.end()) {
throw LogicError("Successor already inserted");
}
if (std::ranges::find(block->imm_predecessors, this) != block->imm_predecessors.end()) {
throw LogicError("Predecessor already inserted");
}
imm_successors.push_back(block);
block->imm_predecessors.push_back(this);
}
static std::string BlockToIndex(const std::map<const Block*, size_t>& block_to_index,
Block* block) {
if (const auto it{block_to_index.find(block)}; it != block_to_index.end()) {
return fmt::format("{{Block ${}}}", it->second);
}
return fmt::format("$<unknown block {:016x}>", reinterpret_cast<u64>(block));
}
static size_t InstIndex(std::map<const Inst*, size_t>& inst_to_index, size_t& inst_index,
const Inst* inst) {
const auto [it, is_inserted]{inst_to_index.emplace(inst, inst_index + 1)};
if (is_inserted) {
++inst_index;
}
return it->second;
}
static std::string ArgToIndex(std::map<const Inst*, size_t>& inst_to_index, size_t& inst_index,
const Value& arg) {
if (arg.IsEmpty()) {
return "<null>";
}
if (!arg.IsImmediate() || arg.IsIdentity()) {
return fmt::format("%{}", InstIndex(inst_to_index, inst_index, arg.Inst()));
}
switch (arg.Type()) {
case Type::U1:
return fmt::format("#{}", arg.U1() ? "true" : "false");
case Type::U8:
return fmt::format("#{}", arg.U8());
case Type::U16:
return fmt::format("#{}", arg.U16());
case Type::U32:
return fmt::format("#{}", arg.U32());
case Type::U64:
return fmt::format("#{}", arg.U64());
case Type::F32:
return fmt::format("#{}", arg.F32());
case Type::Reg:
return fmt::format("{}", arg.Reg());
case Type::Pred:
return fmt::format("{}", arg.Pred());
case Type::Attribute:
return fmt::format("{}", arg.Attribute());
default:
return "<unknown immediate type>";
}
}
std::string DumpBlock(const Block& block) {
size_t inst_index{0};
std::map<const Inst*, size_t> inst_to_index;
return DumpBlock(block, {}, inst_to_index, inst_index);
}
std::string DumpBlock(const Block& block, const std::map<const Block*, size_t>& block_to_index,
std::map<const Inst*, size_t>& inst_to_index, size_t& inst_index) {
std::string ret{"Block"};
if (const auto it{block_to_index.find(&block)}; it != block_to_index.end()) {
ret += fmt::format(" ${}", it->second);
}
ret += '\n';
for (const Inst& inst : block) {
const Opcode op{inst.GetOpcode()};
ret += fmt::format("[{:016x}] ", reinterpret_cast<u64>(&inst));
if (TypeOf(op) != Type::Void) {
ret += fmt::format("%{:<5} = {}", InstIndex(inst_to_index, inst_index, &inst), op);
} else {
ret += fmt::format(" {}", op); // '%00000 = ' -> 1 + 5 + 3 = 9 spaces
}
const size_t arg_count{inst.NumArgs()};
for (size_t arg_index = 0; arg_index < arg_count; ++arg_index) {
const Value arg{inst.Arg(arg_index)};
const std::string arg_str{ArgToIndex(inst_to_index, inst_index, arg)};
ret += arg_index != 0 ? ", " : " ";
if (op == Opcode::Phi) {
ret += fmt::format("[ {}, {} ]", arg_str,
BlockToIndex(block_to_index, inst.PhiBlock(arg_index)));
} else {
ret += arg_str;
}
if (op != Opcode::Phi) {
const Type actual_type{arg.Type()};
const Type expected_type{ArgTypeOf(op, arg_index)};
if (!AreTypesCompatible(actual_type, expected_type)) {
ret += fmt::format("<type error: {} != {}>", actual_type, expected_type);
}
}
}
if (TypeOf(op) != Type::Void) {
ret += fmt::format(" (uses: {})\n", inst.UseCount());
} else {
ret += '\n';
}
}
return ret;
}
} // namespace Shader::IR

View File

@@ -0,0 +1,204 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <initializer_list>
#include <map>
#include <span>
#include <vector>
#include <bit>
#include <numeric>
#include <boost/intrusive/list.hpp>
#include "common/common_types.h"
#include "shader_recompiler/frontend/ir/condition.h"
#include "shader_recompiler/frontend/ir/value.h"
#include "shader_recompiler/object_pool.h"
namespace Shader::IR {
class Block {
public:
using InstructionList = boost::intrusive::list<Inst>;
using size_type = InstructionList::size_type;
using iterator = InstructionList::iterator;
using const_iterator = InstructionList::const_iterator;
using reverse_iterator = InstructionList::reverse_iterator;
using const_reverse_iterator = InstructionList::const_reverse_iterator;
explicit Block(ObjectPool<Inst>& inst_pool_);
~Block();
Block(const Block&) = delete;
Block& operator=(const Block&) = delete;
Block(Block&&) = default;
Block& operator=(Block&&) = default;
/// Appends a new instruction to the end of this basic block.
void AppendNewInst(Opcode op, std::initializer_list<Value> args);
/// Prepends a copy of an instruction to this basic block before the insertion point.
iterator PrependNewInst(iterator insertion_point, const Inst& base_inst);
/// Prepends a new instruction to this basic block before the insertion point.
iterator PrependNewInst(iterator insertion_point, Opcode op,
std::initializer_list<Value> args = {}, u32 flags = 0);
/// Adds a new branch to this basic block.
void AddBranch(Block* block);
/// Gets a mutable reference to the instruction list for this basic block.
[[nodiscard]] InstructionList& Instructions() noexcept {
return instructions;
}
/// Gets an immutable reference to the instruction list for this basic block.
[[nodiscard]] const InstructionList& Instructions() const noexcept {
return instructions;
}
/// Gets an immutable span to the immediate predecessors.
[[nodiscard]] std::span<Block* const> ImmPredecessors() const noexcept {
return imm_predecessors;
}
/// Gets an immutable span to the immediate successors.
[[nodiscard]] std::span<Block* const> ImmSuccessors() const noexcept {
return imm_successors;
}
/// Intrusively store the host definition of this instruction.
template <typename DefinitionType>
void SetDefinition(DefinitionType def) {
definition = std::bit_cast<u32>(def);
}
/// Return the intrusively stored host definition of this instruction.
template <typename DefinitionType>
[[nodiscard]] DefinitionType Definition() const noexcept {
return std::bit_cast<DefinitionType>(definition);
}
void SetSsaRegValue(IR::Reg reg, const Value& value) noexcept {
ssa_reg_values[RegIndex(reg)] = value;
}
const Value& SsaRegValue(IR::Reg reg) const noexcept {
return ssa_reg_values[RegIndex(reg)];
}
void SsaSeal() noexcept {
is_ssa_sealed = true;
}
[[nodiscard]] bool IsSsaSealed() const noexcept {
return is_ssa_sealed;
}
[[nodiscard]] bool empty() const {
return instructions.empty();
}
[[nodiscard]] size_type size() const {
return instructions.size();
}
[[nodiscard]] Inst& front() {
return instructions.front();
}
[[nodiscard]] const Inst& front() const {
return instructions.front();
}
[[nodiscard]] Inst& back() {
return instructions.back();
}
[[nodiscard]] const Inst& back() const {
return instructions.back();
}
[[nodiscard]] iterator begin() {
return instructions.begin();
}
[[nodiscard]] const_iterator begin() const {
return instructions.begin();
}
[[nodiscard]] iterator end() {
return instructions.end();
}
[[nodiscard]] const_iterator end() const {
return instructions.end();
}
[[nodiscard]] reverse_iterator rbegin() {
return instructions.rbegin();
}
[[nodiscard]] const_reverse_iterator rbegin() const {
return instructions.rbegin();
}
[[nodiscard]] reverse_iterator rend() {
return instructions.rend();
}
[[nodiscard]] const_reverse_iterator rend() const {
return instructions.rend();
}
[[nodiscard]] const_iterator cbegin() const {
return instructions.cbegin();
}
[[nodiscard]] const_iterator cend() const {
return instructions.cend();
}
[[nodiscard]] const_reverse_iterator crbegin() const {
return instructions.crbegin();
}
[[nodiscard]] const_reverse_iterator crend() const {
return instructions.crend();
}
// Set the order of the block, it can be set pre order, the user decides
void SetOrder(u32 new_order) {
order = new_order;
}
// Get the order of the block.
// The higher, the closer is the block to the end.
[[nodiscard]] u32 GetOrder() const {
return order;
}
private:
/// Memory pool for instruction list
ObjectPool<Inst>* inst_pool;
/// List of instructions in this block
InstructionList instructions;
/// Block immediate predecessors
std::vector<Block*> imm_predecessors;
/// Block immediate successors
std::vector<Block*> imm_successors;
/// Intrusively store the value of a register in the block.
std::array<Value, NUM_REGS> ssa_reg_values;
/// Intrusively store if the block is sealed in the SSA pass.
bool is_ssa_sealed{false};
/// Intrusively stored host definition of this block.
u32 definition{};
/// Order of the block.
u32 order{};
};
using BlockList = std::vector<Block*>;
[[nodiscard]] std::string DumpBlock(const Block& block);
[[nodiscard]] std::string DumpBlock(const Block& block,
const std::map<const Block*, size_t>& block_to_index,
std::map<const Inst*, size_t>& inst_to_index,
size_t& inst_index);
} // namespace Shader::IR

View File

@@ -0,0 +1,78 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <algorithm>
#include <functional>
#include <optional>
#include <type_traits>
#include <queue>
#include <unordered_set>
#include <boost/container/small_vector.hpp>
#include "shader_recompiler/frontend/ir/value.h"
namespace Shader::IR {
template <typename Pred, typename Resolve = std::identity>
auto BreadthFirstSearch(Value value, Pred&& pred, Resolve&& resolve = {})
-> std::invoke_result_t<Pred, const Inst*> {
value = resolve(value);
if (value.IsImmediate()) {
// Nothing to do with immediates
return std::nullopt;
}
// Breadth-first search visiting the right most arguments first
// Small vector has been determined from shaders in Super Smash Bros. Ultimate
boost::container::small_vector<const Inst*, 2> visited;
// Once a search grows, a linear scan of `visited` per argument made it quadratic: a 28,000
// instruction Switch 2 compute shader took 3.4 s to translate, most of it here. Same visit
// order, so the same result.
std::unordered_set<const Inst*> visited_set;
const auto first_visit = [&](const Inst* inst) {
if (!visited_set.empty()) {
return visited_set.insert(inst).second;
}
if (std::ranges::find(visited, inst) != visited.end()) {
return false;
}
visited.push_back(inst);
if (visited.size() > 64) {
visited_set.insert(visited.begin(), visited.end());
}
return true;
};
std::queue<const Inst*> queue;
queue.push(value.InstRecursive());
while (!queue.empty()) {
// Pop one instruction from the queue
const Inst* const inst{queue.front()};
queue.pop();
if (const std::optional result = pred(inst)) {
// This is the instruction we were looking for
return result;
}
// Visit the right most arguments first
for (size_t arg = inst->NumArgs(); arg--;) {
const Value arg_value{resolve(inst->Arg(arg))};
if (arg_value.IsImmediate()) {
continue;
}
// Queue instruction if it hasn't been visited
const Inst* const arg_inst{arg_value.InstRecursive()};
if (first_visit(arg_inst)) {
queue.push(arg_inst);
}
}
}
// SSA tree has been traversed and the result hasn't been found
return std::nullopt;
}
} // namespace Shader::IR

View File

@@ -0,0 +1,28 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <string>
#include <fmt/ranges.h>
#include "shader_recompiler/frontend/ir/condition.h"
namespace Shader::IR {
std::string NameOf(Condition condition) {
std::string ret;
if (condition.GetFlowTest() != FlowTest::T) {
ret = fmt::to_string(condition.GetFlowTest());
}
const auto [pred, negated]{condition.GetPred()};
if (!ret.empty()) {
ret += '&';
}
if (negated) {
ret += '!';
}
ret += fmt::to_string(pred);
return ret;
}
} // namespace Shader::IR

View File

@@ -0,0 +1,58 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <string>
#include <fmt/ranges.h>
#include "common/common_types.h"
#include "shader_recompiler/frontend/ir/flow_test.h"
#include "shader_recompiler/frontend/ir/pred.h"
namespace Shader::IR {
class Condition {
public:
Condition() noexcept = default;
explicit Condition(FlowTest flow_test_, Pred pred_, bool pred_negated_ = false) noexcept
: flow_test{static_cast<u16>(flow_test_)}, pred{static_cast<u8>(pred_)},
pred_negated{pred_negated_ ? u8{1} : u8{0}} {}
explicit Condition(Pred pred_, bool pred_negated_ = false) noexcept
: Condition(FlowTest::T, pred_, pred_negated_) {}
explicit Condition(bool value) : Condition(Pred::PT, !value) {}
auto operator<=>(const Condition&) const noexcept = default;
[[nodiscard]] IR::FlowTest GetFlowTest() const noexcept {
return static_cast<IR::FlowTest>(flow_test);
}
[[nodiscard]] std::pair<IR::Pred, bool> GetPred() const noexcept {
return {static_cast<IR::Pred>(pred), pred_negated != 0};
}
private:
u16 flow_test;
u8 pred;
u8 pred_negated;
};
std::string NameOf(Condition condition);
} // namespace Shader::IR
template <>
struct fmt::formatter<Shader::IR::Condition> {
constexpr auto parse(format_parse_context& ctx) {
return ctx.begin();
}
template <typename FormatContext>
auto format(const Shader::IR::Condition& cond, FormatContext& ctx) const {
return fmt::format_to(ctx.out(), "{}", Shader::IR::NameOf(cond));
}
};

View File

@@ -0,0 +1,77 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <string>
#include <fmt/ranges.h>
#include "common/common_types.h"
#include "shader_recompiler/exception.h"
namespace Shader::IR {
enum class FlowTest : u64 {
#define SRIR_FLOW_TEST_LIST \
SRIR_FLOW_TEST_ELEM(F) \
SRIR_FLOW_TEST_ELEM(LT) \
SRIR_FLOW_TEST_ELEM(EQ) \
SRIR_FLOW_TEST_ELEM(LE) \
SRIR_FLOW_TEST_ELEM(GT) \
SRIR_FLOW_TEST_ELEM(NE) \
SRIR_FLOW_TEST_ELEM(GE) \
SRIR_FLOW_TEST_ELEM(NUM) \
SRIR_FLOW_TEST_ELEM(NaN) \
SRIR_FLOW_TEST_ELEM(LTU) \
SRIR_FLOW_TEST_ELEM(EQU) \
SRIR_FLOW_TEST_ELEM(LEU) \
SRIR_FLOW_TEST_ELEM(GTU) \
SRIR_FLOW_TEST_ELEM(NEU) \
SRIR_FLOW_TEST_ELEM(GEU) \
SRIR_FLOW_TEST_ELEM(T) \
SRIR_FLOW_TEST_ELEM(OFF) \
SRIR_FLOW_TEST_ELEM(LO) \
SRIR_FLOW_TEST_ELEM(SFF) \
SRIR_FLOW_TEST_ELEM(LS) \
SRIR_FLOW_TEST_ELEM(HI) \
SRIR_FLOW_TEST_ELEM(SFT) \
SRIR_FLOW_TEST_ELEM(HS) \
SRIR_FLOW_TEST_ELEM(OFT) \
SRIR_FLOW_TEST_ELEM(CSM_TA) \
SRIR_FLOW_TEST_ELEM(CSM_TR) \
SRIR_FLOW_TEST_ELEM(CSM_MX) \
SRIR_FLOW_TEST_ELEM(FCSM_TA) \
SRIR_FLOW_TEST_ELEM(FCSM_TR) \
SRIR_FLOW_TEST_ELEM(FCSM_MX) \
SRIR_FLOW_TEST_ELEM(RLE) \
SRIR_FLOW_TEST_ELEM(RGT)
#define SRIR_FLOW_TEST_ELEM(n) n,
SRIR_FLOW_TEST_LIST
#undef SRIR_FLOW_TEST_ELEM
};
[[nodiscard]] inline std::string NameOf(FlowTest flow_test) {
switch (flow_test) {
#define SRIR_FLOW_TEST_ELEM(n) case FlowTest::n: return #n;
SRIR_FLOW_TEST_LIST
#undef SRIR_FLOW_TEST_ELEM
default:
return fmt::format("<invalid flow test {}>", int(flow_test));
}
}
} // namespace Shader::IR
template <>
struct fmt::formatter<Shader::IR::FlowTest> {
constexpr auto parse(format_parse_context& ctx) {
return ctx.begin();
}
template <typename FormatContext>
auto format(const Shader::IR::FlowTest& flow_test, FormatContext& ctx) const {
return fmt::format_to(ctx.out(), "{}", Shader::IR::NameOf(flow_test));
}
};

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,446 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <cstring>
#include <type_traits>
#include "shader_recompiler/frontend/ir/attribute.h"
#include "shader_recompiler/frontend/ir/basic_block.h"
#include "shader_recompiler/frontend/ir/modifiers.h"
#include "shader_recompiler/frontend/ir/value.h"
namespace Shader::IR {
class IREmitter {
public:
explicit IREmitter(Block& block_) : block{&block_}, insertion_point{block->end()} {}
explicit IREmitter(Block& block_, Block::iterator insertion_point_)
: block{&block_}, insertion_point{insertion_point_} {}
Block* block;
[[nodiscard]] U1 Imm1(bool value) const;
[[nodiscard]] U8 Imm8(u8 value) const;
[[nodiscard]] U16 Imm16(u16 value) const;
[[nodiscard]] U32 Imm32(u32 value) const;
[[nodiscard]] U32 Imm32(s32 value) const;
[[nodiscard]] F32 Imm32(f32 value) const;
[[nodiscard]] U64 Imm64(u64 value) const;
[[nodiscard]] U64 Imm64(s64 value) const;
[[nodiscard]] F64 Imm64(f64 value) const;
U1 ConditionRef(const U1& value);
void Reference(const Value& value);
void PhiMove(IR::Inst& phi, const Value& value);
void Prologue();
void Epilogue();
void DemoteToHelperInvocation();
void EmitVertex(const U32& stream);
void EndPrimitive(const U32& stream);
[[nodiscard]] U32 GetReg(IR::Reg reg);
void SetReg(IR::Reg reg, const U32& value);
[[nodiscard]] U1 GetPred(IR::Pred pred, bool is_negated = false);
void SetPred(IR::Pred pred, const U1& value);
[[nodiscard]] U1 GetGotoVariable(u32 id);
void SetGotoVariable(u32 id, const U1& value);
[[nodiscard]] U32 GetIndirectBranchVariable();
void SetIndirectBranchVariable(const U32& value);
[[nodiscard]] U32 GetCbuf(const U32& binding, const U32& byte_offset);
[[nodiscard]] Value GetCbuf(const U32& binding, const U32& byte_offset, size_t bitsize,
bool is_signed);
[[nodiscard]] F32 GetFloatCbuf(const U32& binding, const U32& byte_offset);
[[nodiscard]] U1 GetZFlag();
[[nodiscard]] U1 GetSFlag();
[[nodiscard]] U1 GetCFlag();
[[nodiscard]] U1 GetOFlag();
void SetZFlag(const U1& value);
void SetSFlag(const U1& value);
void SetCFlag(const U1& value);
void SetOFlag(const U1& value);
[[nodiscard]] U1 Condition(IR::Condition cond);
[[nodiscard]] U1 GetFlowTestResult(FlowTest test);
[[nodiscard]] F32 GetAttribute(IR::Attribute attribute);
[[nodiscard]] F32 GetAttribute(IR::Attribute attribute, const U32& vertex);
[[nodiscard]] U32 GetAttributeU32(IR::Attribute attribute);
[[nodiscard]] U32 GetAttributeU32(IR::Attribute attribute, const U32& vertex);
void SetAttribute(IR::Attribute attribute, const F32& value, const U32& vertex);
[[nodiscard]] F32 GetAttributeIndexed(const U32& phys_address);
[[nodiscard]] F32 GetAttributeIndexed(const U32& phys_address, const U32& vertex);
void SetAttributeIndexed(const U32& phys_address, const F32& value, const U32& vertex);
[[nodiscard]] F32 GetPatch(Patch patch);
void SetPatch(Patch patch, const F32& value);
[[nodiscard]] U32 IsbeRead(const U32& address);
void IsbeWriteAttribute(const U32& address, const U32& value);
void IsbeWriteMapU8(const U32& address, const U32& value);
void IsbeWriteMapU32(const U32& address, const U32& value);
void SetFragColor(u32 index, u32 component, const F32& value);
void SetSampleMask(const U32& value);
void SetFragDepth(const F32& value);
[[nodiscard]] U32 WorkgroupIdX();
[[nodiscard]] U32 WorkgroupIdY();
[[nodiscard]] U32 WorkgroupIdZ();
[[nodiscard]] Value LocalInvocationId();
[[nodiscard]] U32 LocalInvocationIdX();
[[nodiscard]] U32 LocalInvocationIdY();
[[nodiscard]] U32 LocalInvocationIdZ();
[[nodiscard]] U32 InvocationId();
[[nodiscard]] U32 InvocationInfo();
[[nodiscard]] U32 SampleId();
[[nodiscard]] U1 IsHelperInvocation();
[[nodiscard]] U32 SR_WScaleFactorXY();
[[nodiscard]] U32 SR_WScaleFactorZ();
[[nodiscard]] F32 YDirection();
[[nodiscard]] F32 ResolutionDownFactor();
[[nodiscard]] F32 RenderAreaWidth();
[[nodiscard]] F32 RenderAreaHeight();
[[nodiscard]] U32 LaneId();
[[nodiscard]] U32 LoadGlobalU8(const U64& address);
[[nodiscard]] U32 LoadGlobalS8(const U64& address);
[[nodiscard]] U32 LoadGlobalU16(const U64& address);
[[nodiscard]] U32 LoadGlobalS16(const U64& address);
[[nodiscard]] U32 LoadGlobal32(const U64& address);
[[nodiscard]] Value LoadGlobal64(const U64& address);
[[nodiscard]] Value LoadGlobal128(const U64& address);
void WriteGlobalU8(const U64& address, const U32& value);
void WriteGlobalS8(const U64& address, const U32& value);
void WriteGlobalU16(const U64& address, const U32& value);
void WriteGlobalS16(const U64& address, const U32& value);
void WriteGlobal32(const U64& address, const U32& value);
void WriteGlobal64(const U64& address, const IR::Value& vector);
void WriteGlobal128(const U64& address, const IR::Value& vector);
[[nodiscard]] U32 LoadLocal(const U32& word_offset);
void WriteLocal(const U32& word_offset, const U32& value);
[[nodiscard]] Value LoadShared(int bit_size, bool is_signed, const U32& offset);
void WriteShared(int bit_size, const U32& offset, const Value& value);
[[nodiscard]] U1 GetZeroFromOp(const Value& op);
[[nodiscard]] U1 GetSignFromOp(const Value& op);
[[nodiscard]] U1 GetCarryFromOp(const Value& op);
[[nodiscard]] U1 GetOverflowFromOp(const Value& op);
[[nodiscard]] U1 GetSparseFromOp(const Value& op);
[[nodiscard]] U1 GetInBoundsFromOp(const Value& op);
[[nodiscard]] Value CompositeConstruct(const Value& e1, const Value& e2);
[[nodiscard]] Value CompositeConstruct(const Value& e1, const Value& e2, const Value& e3);
[[nodiscard]] Value CompositeConstruct(const Value& e1, const Value& e2, const Value& e3,
const Value& e4);
[[nodiscard]] Value CompositeExtract(const Value& vector, size_t element);
[[nodiscard]] Value CompositeInsert(const Value& vector, const Value& object, size_t element);
[[nodiscard]] Value Select(const U1& condition, const Value& true_value,
const Value& false_value);
void Barrier();
void SubgroupBarrier(); // (hookshot: the warp's lanes meet, their shared memory accesses ordered: Ampere LDSM)
void WorkgroupMemoryBarrier();
void DeviceMemoryBarrier();
template <typename Dest, typename Source>
[[nodiscard]] Dest BitCast(const Source& value);
[[nodiscard]] U64 PackUint2x32(const Value& vector);
[[nodiscard]] Value UnpackUint2x32(const U64& value);
[[nodiscard]] U32 PackFloat2x16(const Value& vector);
[[nodiscard]] Value UnpackFloat2x16(const U32& value);
[[nodiscard]] U32 PackHalf2x16(const Value& vector);
[[nodiscard]] Value UnpackHalf2x16(const U32& value);
[[nodiscard]] F64 PackDouble2x32(const Value& vector);
[[nodiscard]] Value UnpackDouble2x32(const F64& value);
[[nodiscard]] F16F32F64 FPAdd(const F16F32F64& a, const F16F32F64& b, FpControl control = {});
[[nodiscard]] F16F32F64 FPMul(const F16F32F64& a, const F16F32F64& b, FpControl control = {});
[[nodiscard]] F16F32F64 FPFma(const F16F32F64& a, const F16F32F64& b, const F16F32F64& c,
FpControl control = {});
[[nodiscard]] F16F32F64 FPAbs(const F16F32F64& value);
[[nodiscard]] F16F32F64 FPNeg(const F16F32F64& value);
[[nodiscard]] F16F32F64 FPAbsNeg(const F16F32F64& value, bool abs, bool neg);
[[nodiscard]] F32 FPCos(const F32& value);
[[nodiscard]] F32 FPSin(const F32& value);
[[nodiscard]] F32 FPExp2(const F32& value);
[[nodiscard]] F32 FPLog2(const F32& value);
[[nodiscard]] F32F64 FPRecip(const F32F64& value);
[[nodiscard]] F32F64 FPRecipSqrt(const F32F64& value);
[[nodiscard]] F32 FPSqrt(const F32& value);
[[nodiscard]] F16F32F64 FPSaturate(const F16F32F64& value);
[[nodiscard]] F16F32F64 FPClamp(const F16F32F64& value, const F16F32F64& min_value,
const F16F32F64& max_value);
[[nodiscard]] F16F32F64 FPRoundEven(const F16F32F64& value, FpControl control = {});
[[nodiscard]] F16F32F64 FPFloor(const F16F32F64& value, FpControl control = {});
[[nodiscard]] F16F32F64 FPCeil(const F16F32F64& value, FpControl control = {});
[[nodiscard]] F16F32F64 FPTrunc(const F16F32F64& value, FpControl control = {});
[[nodiscard]] U1 FPEqual(const F16F32F64& lhs, const F16F32F64& rhs, FpControl control = {},
bool ordered = true);
[[nodiscard]] U1 FPNotEqual(const F16F32F64& lhs, const F16F32F64& rhs, FpControl control = {},
bool ordered = true);
[[nodiscard]] U1 FPLessThan(const F16F32F64& lhs, const F16F32F64& rhs, FpControl control = {},
bool ordered = true);
[[nodiscard]] U1 FPGreaterThan(const F16F32F64& lhs, const F16F32F64& rhs,
FpControl control = {}, bool ordered = true);
[[nodiscard]] U1 FPLessThanEqual(const F16F32F64& lhs, const F16F32F64& rhs,
FpControl control = {}, bool ordered = true);
[[nodiscard]] U1 FPGreaterThanEqual(const F16F32F64& lhs, const F16F32F64& rhs,
FpControl control = {}, bool ordered = true);
[[nodiscard]] U1 FPIsNan(const F16F32F64& value);
[[nodiscard]] U1 FPOrdered(const F16F32F64& lhs, const F16F32F64& rhs);
[[nodiscard]] U1 FPUnordered(const F16F32F64& lhs, const F16F32F64& rhs);
[[nodiscard]] F32F64 FPMax(const F32F64& lhs, const F32F64& rhs, FpControl control = {});
[[nodiscard]] F32F64 FPMin(const F32F64& lhs, const F32F64& rhs, FpControl control = {});
[[nodiscard]] U32U64 IAdd(const U32U64& a, const U32U64& b);
[[nodiscard]] U32U64 ISub(const U32U64& a, const U32U64& b);
[[nodiscard]] U32 IMul(const U32& a, const U32& b);
[[nodiscard]] U32 IDiv(const U32& a, const U32& b, bool is_signed = false);
[[nodiscard]] U32U64 INeg(const U32U64& value);
[[nodiscard]] U32U64 IAbs(const U32U64& value);
[[nodiscard]] U32U64 ShiftLeftLogical(const U32U64& base, const U32& shift);
[[nodiscard]] U32U64 ShiftRightLogical(const U32U64& base, const U32& shift);
[[nodiscard]] U32U64 ShiftRightArithmetic(const U32U64& base, const U32& shift);
[[nodiscard]] U32 BitwiseAnd(const U32& a, const U32& b);
[[nodiscard]] U32 BitwiseOr(const U32& a, const U32& b);
[[nodiscard]] U32 BitwiseXor(const U32& a, const U32& b);
[[nodiscard]] U32 BitFieldInsert(const U32& base, const U32& insert, const U32& offset,
const U32& count);
[[nodiscard]] U32 BitFieldExtract(const U32& base, const U32& offset, const U32& count,
bool is_signed = false);
[[nodiscard]] U32 BitReverse(const U32& value);
[[nodiscard]] U32 BitCount(const U32& value);
[[nodiscard]] U32 BitwiseNot(const U32& value);
[[nodiscard]] U32 FindSMsb(const U32& value);
[[nodiscard]] U32 FindUMsb(const U32& value);
[[nodiscard]] U32 SMin(const U32& a, const U32& b);
[[nodiscard]] U32 UMin(const U32& a, const U32& b);
[[nodiscard]] U32 IMin(const U32& a, const U32& b, bool is_signed);
[[nodiscard]] U32 SMax(const U32& a, const U32& b);
[[nodiscard]] U32 UMax(const U32& a, const U32& b);
[[nodiscard]] U32 IMax(const U32& a, const U32& b, bool is_signed);
[[nodiscard]] U32 SClamp(const U32& value, const U32& min, const U32& max);
[[nodiscard]] U32 UClamp(const U32& value, const U32& min, const U32& max);
[[nodiscard]] U1 ILessThan(const U32& lhs, const U32& rhs, bool is_signed);
[[nodiscard]] U1 IEqual(const U32U64& lhs, const U32U64& rhs);
[[nodiscard]] U1 ILessThanEqual(const U32& lhs, const U32& rhs, bool is_signed);
[[nodiscard]] U1 IGreaterThan(const U32& lhs, const U32& rhs, bool is_signed);
[[nodiscard]] U1 INotEqual(const U32& lhs, const U32& rhs);
[[nodiscard]] U1 IGreaterThanEqual(const U32& lhs, const U32& rhs, bool is_signed);
[[nodiscard]] U32 SharedAtomicIAdd(const U32& pointer_offset, const U32& value);
[[nodiscard]] U32 SharedAtomicSMin(const U32& pointer_offset, const U32& value);
[[nodiscard]] U32 SharedAtomicUMin(const U32& pointer_offset, const U32& value);
[[nodiscard]] U32 SharedAtomicIMin(const U32& pointer_offset, const U32& value, bool is_signed);
[[nodiscard]] U32 SharedAtomicSMax(const U32& pointer_offset, const U32& value);
[[nodiscard]] U32 SharedAtomicUMax(const U32& pointer_offset, const U32& value);
[[nodiscard]] U32 SharedAtomicIMax(const U32& pointer_offset, const U32& value, bool is_signed);
[[nodiscard]] U32 SharedAtomicInc(const U32& pointer_offset, const U32& value);
[[nodiscard]] U32 SharedAtomicDec(const U32& pointer_offset, const U32& value);
[[nodiscard]] U32 SharedAtomicAnd(const U32& pointer_offset, const U32& value);
[[nodiscard]] U32 SharedAtomicOr(const U32& pointer_offset, const U32& value);
[[nodiscard]] U32 SharedAtomicXor(const U32& pointer_offset, const U32& value);
[[nodiscard]] U32U64 SharedAtomicExchange(const U32& pointer_offset, const U32U64& value);
[[nodiscard]] U32U64 GlobalAtomicIAdd(const U64& pointer_offset, const U32U64& value);
[[nodiscard]] U32U64 GlobalAtomicSMin(const U64& pointer_offset, const U32U64& value);
[[nodiscard]] U32U64 GlobalAtomicUMin(const U64& pointer_offset, const U32U64& value);
[[nodiscard]] U32U64 GlobalAtomicIMin(const U64& pointer_offset, const U32U64& value,
bool is_signed);
[[nodiscard]] U32U64 GlobalAtomicSMax(const U64& pointer_offset, const U32U64& value);
[[nodiscard]] U32U64 GlobalAtomicUMax(const U64& pointer_offset, const U32U64& value);
[[nodiscard]] U32U64 GlobalAtomicIMax(const U64& pointer_offset, const U32U64& value,
bool is_signed);
[[nodiscard]] U32 GlobalAtomicInc(const U64& pointer_offset, const U32& value);
[[nodiscard]] U32 GlobalAtomicDec(const U64& pointer_offset, const U32& value);
[[nodiscard]] U32U64 GlobalAtomicAnd(const U64& pointer_offset, const U32U64& value);
[[nodiscard]] U32U64 GlobalAtomicOr(const U64& pointer_offset, const U32U64& value);
[[nodiscard]] U32U64 GlobalAtomicXor(const U64& pointer_offset, const U32U64& value);
[[nodiscard]] U32U64 GlobalAtomicExchange(const U64& pointer_offset, const U32U64& value);
[[nodiscard]] U32 GlobalAtomicCompareExchange(const U64& pointer_offset, const U32& comparator, const U32& value);
[[nodiscard]] F32 GlobalAtomicF32Add(const U64& pointer_offset, const Value& value,
const FpControl control = {});
[[nodiscard]] Value GlobalAtomicF16x2Add(const U64& pointer_offset, const Value& value,
const FpControl control = {});
[[nodiscard]] Value GlobalAtomicF16x2Min(const U64& pointer_offset, const Value& value,
const FpControl control = {});
[[nodiscard]] Value GlobalAtomicF16x2Max(const U64& pointer_offset, const Value& value,
const FpControl control = {});
[[nodiscard]] U1 LogicalOr(const U1& a, const U1& b);
[[nodiscard]] U1 LogicalAnd(const U1& a, const U1& b);
[[nodiscard]] U1 LogicalXor(const U1& a, const U1& b);
[[nodiscard]] U1 LogicalNot(const U1& value);
[[nodiscard]] U32U64 ConvertFToS(size_t bitsize, const F16F32F64& value);
[[nodiscard]] U32U64 ConvertFToU(size_t bitsize, const F16F32F64& value);
[[nodiscard]] U32U64 ConvertFToI(size_t bitsize, bool is_signed, const F16F32F64& value);
[[nodiscard]] F16F32F64 ConvertSToF(size_t dest_bitsize, size_t src_bitsize, const Value& value,
FpControl control = {});
[[nodiscard]] F16F32F64 ConvertUToF(size_t dest_bitsize, size_t src_bitsize, const Value& value,
FpControl control = {});
[[nodiscard]] F16F32F64 ConvertIToF(size_t dest_bitsize, size_t src_bitsize, bool is_signed,
const Value& value, FpControl control = {});
[[nodiscard]] U8U16U32U64 UConvert(size_t result_bitsize, const U8U16U32U64& value);
[[nodiscard]] U8U16U32U64 SConvert(size_t result_bitsize, const U8U16U32U64& value);
[[nodiscard]] F16F32F64 FPConvert(size_t result_bitsize, const F16F32F64& value,
FpControl control = {});
[[nodiscard]] Value ImageSampleImplicitLod(const Value& handle, const Value& coords,
const F32& bias, const Value& offset,
const F32& lod_clamp, TextureInstInfo info);
[[nodiscard]] Value ImageSampleExplicitLod(const Value& handle, const Value& coords,
const F32& lod, const Value& offset,
TextureInstInfo info);
[[nodiscard]] F32 ImageSampleDrefImplicitLod(const Value& handle, const Value& coords,
const F32& dref, const F32& bias,
const Value& offset, const F32& lod_clamp,
TextureInstInfo info);
[[nodiscard]] F32 ImageSampleDrefExplicitLod(const Value& handle, const Value& coords,
const F32& dref, const F32& lod,
const Value& offset, TextureInstInfo info);
[[nodiscard]] Value ImageQueryDimension(const Value& handle, const IR::U32& lod,
const IR::U1& skip_mips);
[[nodiscard]] Value ImageQueryDimension(const Value& handle, const IR::U32& lod,
const IR::U1& skip_mips, TextureInstInfo info);
[[nodiscard]] Value ImageQueryLod(const Value& handle, const Value& coords,
TextureInstInfo info);
[[nodiscard]] Value ImageGather(const Value& handle, const Value& coords, const Value& offset,
const Value& offset2, TextureInstInfo info);
[[nodiscard]] Value ImageGatherDref(const Value& handle, const Value& coords,
const Value& offset, const Value& offset2, const F32& dref,
TextureInstInfo info);
[[nodiscard]] Value ImageFetch(const Value& handle, const Value& coords, const Value& offset,
const U32& lod, const U32& multisampling, TextureInstInfo info);
[[nodiscard]] Value ImageGradient(const Value& handle, const Value& coords,
const Value& derivatives, const Value& offset,
const F32& lod_clamp, TextureInstInfo info);
[[nodiscard]] Value ImageRead(const Value& handle, const Value& coords, TextureInstInfo info);
void ImageWrite(const Value& handle, const Value& coords, const Value& color,
TextureInstInfo info);
[[nodiscard]] Value ImageAtomicIAdd(const Value& handle, const Value& coords,
const Value& value, TextureInstInfo info);
[[nodiscard]] Value ImageAtomicSMin(const Value& handle, const Value& coords,
const Value& value, TextureInstInfo info);
[[nodiscard]] Value ImageAtomicUMin(const Value& handle, const Value& coords,
const Value& value, TextureInstInfo info);
[[nodiscard]] Value ImageAtomicIMin(const Value& handle, const Value& coords,
const Value& value, bool is_signed, TextureInstInfo info);
[[nodiscard]] Value ImageAtomicSMax(const Value& handle, const Value& coords,
const Value& value, TextureInstInfo info);
[[nodiscard]] Value ImageAtomicUMax(const Value& handle, const Value& coords,
const Value& value, TextureInstInfo info);
[[nodiscard]] Value ImageAtomicIMax(const Value& handle, const Value& coords,
const Value& value, bool is_signed, TextureInstInfo info);
[[nodiscard]] Value ImageAtomicInc(const Value& handle, const Value& coords, const Value& value,
TextureInstInfo info);
[[nodiscard]] Value ImageAtomicDec(const Value& handle, const Value& coords, const Value& value,
TextureInstInfo info);
[[nodiscard]] Value ImageAtomicAnd(const Value& handle, const Value& coords, const Value& value,
TextureInstInfo info);
[[nodiscard]] Value ImageAtomicOr(const Value& handle, const Value& coords, const Value& value,
TextureInstInfo info);
[[nodiscard]] Value ImageAtomicXor(const Value& handle, const Value& coords, const Value& value,
TextureInstInfo info);
[[nodiscard]] Value ImageAtomicExchange(const Value& handle, const Value& coords,
const Value& value, TextureInstInfo info);
[[nodiscard]] U1 IsTextureScaled(const U32& index);
[[nodiscard]] U1 IsImageScaled(const U32& index);
[[nodiscard]] U1 VoteAll(const U1& value);
[[nodiscard]] U1 VoteAny(const U1& value);
[[nodiscard]] U1 VoteEqual(const U1& value);
[[nodiscard]] U32 SubgroupBallot(const U1& value);
[[nodiscard]] U32 SubgroupEqMask();
[[nodiscard]] U32 SubgroupLtMask();
[[nodiscard]] U32 SubgroupLeMask();
[[nodiscard]] U32 SubgroupGtMask();
[[nodiscard]] U32 SubgroupGeMask();
[[nodiscard]] U32 ShuffleIndex(const IR::U32& value, const IR::U32& index, const IR::U32& clamp,
const IR::U32& seg_mask);
[[nodiscard]] U32 ShuffleUp(const IR::U32& value, const IR::U32& index, const IR::U32& clamp,
const IR::U32& seg_mask);
[[nodiscard]] U32 ShuffleDown(const IR::U32& value, const IR::U32& index, const IR::U32& clamp,
const IR::U32& seg_mask);
[[nodiscard]] U32 ShuffleButterfly(const IR::U32& value, const IR::U32& index,
const IR::U32& clamp, const IR::U32& seg_mask);
[[nodiscard]] U32 QuadBroadcast(const IR::U32& value, const IR::U32& lane);
// (hookshot) the subgroup's 16x8x16 f16 matrix multiply: a U32x4, b and c U32x2 fragments
[[nodiscard]] Value MatrixMulAddF16(const Value& a, const Value& b, const Value& c);
[[nodiscard]] U32 QuadSwap(const IR::U32& value, const IR::U32& direction);
[[nodiscard]] F32 FSwizzleAdd(const F32& a, const F32& b, const U32& swizzle,
FpControl control = {});
[[nodiscard]] F32 DPdxFine(const F32& a);
[[nodiscard]] F32 DPdyFine(const F32& a);
[[nodiscard]] F32 DPdxCoarse(const F32& a);
[[nodiscard]] F32 DPdyCoarse(const F32& a);
private:
IR::Block::iterator insertion_point;
template <typename T = Value, typename... Args>
T Inst(Opcode op, Args... args) {
auto it{block->PrependNewInst(insertion_point, op, {Value{args}...})};
return T{Value{&*it}};
}
template <typename T>
requires(sizeof(T) <= sizeof(u32) && std::is_trivially_copyable_v<T>)
struct Flags {
Flags() = default;
Flags(T proxy_) : proxy{proxy_} {}
T proxy;
};
template <typename T = Value, typename FlagType, typename... Args>
T Inst(Opcode op, Flags<FlagType> flags, Args... args) {
u32 raw_flags{};
std::memcpy(&raw_flags, &flags.proxy, sizeof(flags.proxy));
auto it{block->PrependNewInst(insertion_point, op, {Value{args}...}, raw_flags)};
return T{Value{&*it}};
}
};
} // namespace Shader::IR

View File

@@ -0,0 +1,461 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <algorithm>
#include <memory>
#include "shader_recompiler/exception.h"
#include "shader_recompiler/frontend/ir/basic_block.h"
#include "shader_recompiler/frontend/ir/type.h"
#include "shader_recompiler/frontend/ir/value.h"
namespace Shader::IR {
namespace {
void CheckPseudoInstruction(IR::Inst* inst, IR::Opcode opcode) {
if (inst && inst->GetOpcode() != opcode) {
throw LogicError("Invalid pseudo-instruction");
}
}
void SetPseudoInstruction(IR::Inst*& dest_inst, IR::Inst* pseudo_inst) {
if (dest_inst) {
throw LogicError("Only one of each type of pseudo-op allowed");
}
dest_inst = pseudo_inst;
}
void RemovePseudoInstruction(IR::Inst*& inst, IR::Opcode expected_opcode) {
if (inst->GetOpcode() != expected_opcode) {
throw LogicError("Undoing use of invalid pseudo-op");
}
inst = nullptr;
}
void AllocAssociatedInsts(std::unique_ptr<AssociatedInsts>& associated_insts) {
if (!associated_insts) {
associated_insts = std::make_unique<AssociatedInsts>();
}
}
} // Anonymous namespace
Inst::Inst(IR::Opcode op_, u32 flags_) noexcept : op{op_}, flags{flags_} {
if (op == Opcode::Phi) {
std::construct_at(&phi_args);
} else {
std::construct_at(&args);
}
}
Inst::Inst(const Inst& base) : op{base.op}, flags{base.flags} {
if (base.op == Opcode::Phi) {
throw NotImplementedException("Copying phi node");
}
std::construct_at(&args);
const size_t num_args{base.NumArgs()};
for (size_t index = 0; index < num_args; ++index) {
SetArg(index, base.Arg(index));
}
}
Inst::~Inst() {
if (op == Opcode::Phi) {
std::destroy_at(&phi_args);
} else {
std::destroy_at(&args);
}
}
bool Inst::MayHaveSideEffects() const noexcept {
switch (op) {
case Opcode::ConditionRef:
case Opcode::Reference:
case Opcode::PhiMove:
case Opcode::Prologue:
case Opcode::Epilogue:
case Opcode::Join:
case Opcode::DemoteToHelperInvocation:
case Opcode::Barrier:
case Opcode::SubgroupBarrier:
case Opcode::WorkgroupMemoryBarrier:
case Opcode::DeviceMemoryBarrier:
case Opcode::EmitVertex:
case Opcode::EndPrimitive:
case Opcode::SetAttribute:
case Opcode::SetAttributeIndexed:
case Opcode::SetPatch:
case Opcode::IsbeWriteAttribute:
case Opcode::IsbeWriteMapU8:
case Opcode::IsbeWriteMapU32:
case Opcode::SetFragColor:
case Opcode::SetSampleMask:
case Opcode::SetFragDepth:
case Opcode::WriteGlobalU8:
case Opcode::WriteGlobalS8:
case Opcode::WriteGlobalU16:
case Opcode::WriteGlobalS16:
case Opcode::WriteGlobal32:
case Opcode::WriteGlobal64:
case Opcode::WriteGlobal128:
case Opcode::WriteStorageU8:
case Opcode::WriteStorageS8:
case Opcode::WriteStorageU16:
case Opcode::WriteStorageS16:
case Opcode::WriteStorage32:
case Opcode::WriteStorage64:
case Opcode::WriteStorage128:
case Opcode::WriteLocal:
case Opcode::WriteSharedU8:
case Opcode::WriteSharedU16:
case Opcode::WriteSharedU32:
case Opcode::WriteSharedU64:
case Opcode::WriteSharedU128:
case Opcode::SharedAtomicIAdd32:
case Opcode::SharedAtomicSMin32:
case Opcode::SharedAtomicUMin32:
case Opcode::SharedAtomicSMax32:
case Opcode::SharedAtomicUMax32:
case Opcode::SharedAtomicInc32:
case Opcode::SharedAtomicDec32:
case Opcode::SharedAtomicAnd32:
case Opcode::SharedAtomicOr32:
case Opcode::SharedAtomicXor32:
case Opcode::SharedAtomicExchange32:
case Opcode::SharedAtomicExchange64:
case Opcode::SharedAtomicExchange32x2:
case Opcode::GlobalAtomicIAdd32:
case Opcode::GlobalAtomicSMin32:
case Opcode::GlobalAtomicUMin32:
case Opcode::GlobalAtomicSMax32:
case Opcode::GlobalAtomicUMax32:
case Opcode::GlobalAtomicInc32:
case Opcode::GlobalAtomicDec32:
case Opcode::GlobalAtomicAnd32:
case Opcode::GlobalAtomicOr32:
case Opcode::GlobalAtomicXor32:
case Opcode::GlobalAtomicExchange32:
case Opcode::GlobalAtomicCompareExchange32:
case Opcode::GlobalAtomicIAdd64:
case Opcode::GlobalAtomicSMin64:
case Opcode::GlobalAtomicUMin64:
case Opcode::GlobalAtomicSMax64:
case Opcode::GlobalAtomicUMax64:
case Opcode::GlobalAtomicAnd64:
case Opcode::GlobalAtomicOr64:
case Opcode::GlobalAtomicXor64:
case Opcode::GlobalAtomicExchange64:
case Opcode::GlobalAtomicIAdd32x2:
case Opcode::GlobalAtomicSMin32x2:
case Opcode::GlobalAtomicUMin32x2:
case Opcode::GlobalAtomicSMax32x2:
case Opcode::GlobalAtomicUMax32x2:
case Opcode::GlobalAtomicAnd32x2:
case Opcode::GlobalAtomicOr32x2:
case Opcode::GlobalAtomicXor32x2:
case Opcode::GlobalAtomicExchange32x2:
case Opcode::GlobalAtomicAddF32:
case Opcode::GlobalAtomicAddF16x2:
case Opcode::GlobalAtomicAddF32x2:
case Opcode::GlobalAtomicMinF16x2:
case Opcode::GlobalAtomicMinF32x2:
case Opcode::GlobalAtomicMaxF16x2:
case Opcode::GlobalAtomicMaxF32x2:
case Opcode::StorageAtomicIAdd32:
case Opcode::StorageAtomicSMin32:
case Opcode::StorageAtomicUMin32:
case Opcode::StorageAtomicSMax32:
case Opcode::StorageAtomicUMax32:
case Opcode::StorageAtomicInc32:
case Opcode::StorageAtomicDec32:
case Opcode::StorageAtomicAnd32:
case Opcode::StorageAtomicOr32:
case Opcode::StorageAtomicXor32:
case Opcode::StorageAtomicExchange32:
case Opcode::StorageAtomicCompareExchange32:
case Opcode::StorageAtomicIAdd64:
case Opcode::StorageAtomicSMin64:
case Opcode::StorageAtomicUMin64:
case Opcode::StorageAtomicSMax64:
case Opcode::StorageAtomicUMax64:
case Opcode::StorageAtomicAnd64:
case Opcode::StorageAtomicOr64:
case Opcode::StorageAtomicXor64:
case Opcode::StorageAtomicExchange64:
case Opcode::StorageAtomicIAdd32x2:
case Opcode::StorageAtomicSMin32x2:
case Opcode::StorageAtomicUMin32x2:
case Opcode::StorageAtomicSMax32x2:
case Opcode::StorageAtomicUMax32x2:
case Opcode::StorageAtomicAnd32x2:
case Opcode::StorageAtomicOr32x2:
case Opcode::StorageAtomicXor32x2:
case Opcode::StorageAtomicExchange32x2:
case Opcode::StorageAtomicAddF32:
case Opcode::StorageAtomicAddF16x2:
case Opcode::StorageAtomicAddF32x2:
case Opcode::StorageAtomicMinF16x2:
case Opcode::StorageAtomicMinF32x2:
case Opcode::StorageAtomicMaxF16x2:
case Opcode::StorageAtomicMaxF32x2:
case Opcode::BindlessImageWrite:
case Opcode::BoundImageWrite:
case Opcode::ImageWrite:
case IR::Opcode::BindlessImageAtomicIAdd32:
case IR::Opcode::BindlessImageAtomicSMin32:
case IR::Opcode::BindlessImageAtomicUMin32:
case IR::Opcode::BindlessImageAtomicSMax32:
case IR::Opcode::BindlessImageAtomicUMax32:
case IR::Opcode::BindlessImageAtomicInc32:
case IR::Opcode::BindlessImageAtomicDec32:
case IR::Opcode::BindlessImageAtomicAnd32:
case IR::Opcode::BindlessImageAtomicOr32:
case IR::Opcode::BindlessImageAtomicXor32:
case IR::Opcode::BindlessImageAtomicExchange32:
case IR::Opcode::BoundImageAtomicIAdd32:
case IR::Opcode::BoundImageAtomicSMin32:
case IR::Opcode::BoundImageAtomicUMin32:
case IR::Opcode::BoundImageAtomicSMax32:
case IR::Opcode::BoundImageAtomicUMax32:
case IR::Opcode::BoundImageAtomicInc32:
case IR::Opcode::BoundImageAtomicDec32:
case IR::Opcode::BoundImageAtomicAnd32:
case IR::Opcode::BoundImageAtomicOr32:
case IR::Opcode::BoundImageAtomicXor32:
case IR::Opcode::BoundImageAtomicExchange32:
case IR::Opcode::ImageAtomicIAdd32:
case IR::Opcode::ImageAtomicSMin32:
case IR::Opcode::ImageAtomicUMin32:
case IR::Opcode::ImageAtomicSMax32:
case IR::Opcode::ImageAtomicUMax32:
case IR::Opcode::ImageAtomicInc32:
case IR::Opcode::ImageAtomicDec32:
case IR::Opcode::ImageAtomicAnd32:
case IR::Opcode::ImageAtomicOr32:
case IR::Opcode::ImageAtomicXor32:
case IR::Opcode::ImageAtomicExchange32:
return true;
default:
return false;
}
}
bool Inst::IsPseudoInstruction() const noexcept {
switch (op) {
case Opcode::GetZeroFromOp:
case Opcode::GetSignFromOp:
case Opcode::GetCarryFromOp:
case Opcode::GetOverflowFromOp:
case Opcode::GetSparseFromOp:
case Opcode::GetInBoundsFromOp:
return true;
default:
return false;
}
}
bool Inst::AreAllArgsImmediates() const {
if (op == Opcode::Phi) {
throw LogicError("Testing for all arguments are immediates on phi instruction");
}
return std::all_of(args.begin(), args.begin() + NumArgs(),
[](const IR::Value& value) { return value.IsImmediate(); });
}
Inst* Inst::GetAssociatedPseudoOperation(IR::Opcode opcode) {
if (!associated_insts) {
return nullptr;
}
switch (opcode) {
case Opcode::GetZeroFromOp:
CheckPseudoInstruction(associated_insts->zero_inst, Opcode::GetZeroFromOp);
return associated_insts->zero_inst;
case Opcode::GetSignFromOp:
CheckPseudoInstruction(associated_insts->sign_inst, Opcode::GetSignFromOp);
return associated_insts->sign_inst;
case Opcode::GetCarryFromOp:
CheckPseudoInstruction(associated_insts->carry_inst, Opcode::GetCarryFromOp);
return associated_insts->carry_inst;
case Opcode::GetOverflowFromOp:
CheckPseudoInstruction(associated_insts->overflow_inst, Opcode::GetOverflowFromOp);
return associated_insts->overflow_inst;
case Opcode::GetSparseFromOp:
CheckPseudoInstruction(associated_insts->sparse_inst, Opcode::GetSparseFromOp);
return associated_insts->sparse_inst;
case Opcode::GetInBoundsFromOp:
CheckPseudoInstruction(associated_insts->in_bounds_inst, Opcode::GetInBoundsFromOp);
return associated_insts->in_bounds_inst;
default:
throw InvalidArgument("{} is not a pseudo-instruction", opcode);
}
}
IR::Type Inst::Type() const {
if (op == IR::Opcode::Phi) {
// The type of a phi node is stored in its flags
return Flags<IR::Type>();
}
return TypeOf(op);
}
void Inst::SetArg(size_t index, Value value) {
if (index >= NumArgs()) {
throw InvalidArgument("Out of bounds argument index {} in opcode {}", index, op);
}
const IR::Value arg{Arg(index)};
if (!arg.IsImmediate()) {
UndoUse(arg);
}
if (!value.IsImmediate()) {
Use(value);
}
if (op == Opcode::Phi) {
phi_args[index].second = value;
} else {
args[index] = value;
}
}
Block* Inst::PhiBlock(size_t index) const {
if (op != Opcode::Phi) {
throw LogicError("{} is not a Phi instruction", op);
}
if (index >= phi_args.size()) {
throw InvalidArgument("Out of bounds argument index {} in phi instruction");
}
return phi_args[index].first;
}
void Inst::AddPhiOperand(Block* predecessor, const Value& value) {
if (!value.IsImmediate()) {
Use(value);
}
phi_args.emplace_back(predecessor, value);
}
void Inst::OrderPhiArgs() {
if (op != Opcode::Phi) {
throw LogicError("{} is not a Phi instruction", op);
}
std::sort(phi_args.begin(), phi_args.end(),
[](const std::pair<Block*, Value>& a, const std::pair<Block*, Value>& b) {
return a.first->GetOrder() < b.first->GetOrder();
});
}
void Inst::Invalidate() {
ClearArgs();
ReplaceOpcode(Opcode::Void);
}
void Inst::ClearArgs() {
if (op == Opcode::Phi) {
for (auto& pair : phi_args) {
IR::Value& value{pair.second};
if (!value.IsImmediate()) {
UndoUse(value);
}
}
phi_args.clear();
} else {
for (auto& value : args) {
if (!value.IsImmediate()) {
UndoUse(value);
}
}
// Reset arguments to null
// std::memset was measured to be faster on MSVC than std::ranges:fill
std::memset(reinterpret_cast<char*>(&args), 0, sizeof(args));
}
}
void Inst::ReplaceUsesWith(Value replacement) {
Invalidate();
ReplaceOpcode(Opcode::Identity);
if (!replacement.IsImmediate()) {
Use(replacement);
}
args[0] = replacement;
}
void Inst::ReplaceOpcode(IR::Opcode opcode) {
if (opcode == IR::Opcode::Phi) {
throw LogicError("Cannot transition into Phi");
}
if (op == Opcode::Phi) {
// Transition out of phi arguments into non-phi
std::destroy_at(&phi_args);
std::construct_at(&args);
}
op = opcode;
}
void Inst::Use(const Value& value) {
Inst* const inst{value.Inst()};
++inst->use_count;
std::unique_ptr<AssociatedInsts>& assoc_inst{inst->associated_insts};
switch (op) {
case Opcode::GetZeroFromOp:
AllocAssociatedInsts(assoc_inst);
SetPseudoInstruction(assoc_inst->zero_inst, this);
break;
case Opcode::GetSignFromOp:
AllocAssociatedInsts(assoc_inst);
SetPseudoInstruction(assoc_inst->sign_inst, this);
break;
case Opcode::GetCarryFromOp:
AllocAssociatedInsts(assoc_inst);
SetPseudoInstruction(assoc_inst->carry_inst, this);
break;
case Opcode::GetOverflowFromOp:
AllocAssociatedInsts(assoc_inst);
SetPseudoInstruction(assoc_inst->overflow_inst, this);
break;
case Opcode::GetSparseFromOp:
AllocAssociatedInsts(assoc_inst);
SetPseudoInstruction(assoc_inst->sparse_inst, this);
break;
case Opcode::GetInBoundsFromOp:
AllocAssociatedInsts(assoc_inst);
SetPseudoInstruction(assoc_inst->in_bounds_inst, this);
break;
default:
break;
}
}
void Inst::UndoUse(const Value& value) {
Inst* const inst{value.Inst()};
--inst->use_count;
std::unique_ptr<AssociatedInsts>& assoc_inst{inst->associated_insts};
switch (op) {
case Opcode::GetZeroFromOp:
AllocAssociatedInsts(assoc_inst);
RemovePseudoInstruction(assoc_inst->zero_inst, Opcode::GetZeroFromOp);
break;
case Opcode::GetSignFromOp:
AllocAssociatedInsts(assoc_inst);
RemovePseudoInstruction(assoc_inst->sign_inst, Opcode::GetSignFromOp);
break;
case Opcode::GetCarryFromOp:
AllocAssociatedInsts(assoc_inst);
RemovePseudoInstruction(assoc_inst->carry_inst, Opcode::GetCarryFromOp);
break;
case Opcode::GetOverflowFromOp:
AllocAssociatedInsts(assoc_inst);
RemovePseudoInstruction(assoc_inst->overflow_inst, Opcode::GetOverflowFromOp);
break;
case Opcode::GetSparseFromOp:
AllocAssociatedInsts(assoc_inst);
RemovePseudoInstruction(assoc_inst->sparse_inst, Opcode::GetSparseFromOp);
break;
case Opcode::GetInBoundsFromOp:
AllocAssociatedInsts(assoc_inst);
RemovePseudoInstruction(assoc_inst->in_bounds_inst, Opcode::GetInBoundsFromOp);
break;
default:
break;
}
}
} // namespace Shader::IR

View File

@@ -0,0 +1,56 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "common/bit_field.h"
#include "common/common_types.h"
#include "shader_recompiler/shader_info.h"
namespace Shader::IR {
enum class FmzMode : u8 {
DontCare, // Not specified for this instruction
FTZ, // Flush denorms to zero, NAN is propagated (D3D11, NVN, GL, VK)
FMZ, // Flush denorms to zero, x * 0 == 0 (D3D9)
None, // Denorms are not flushed, NAN is propagated (nouveau)
};
enum class FpRounding : u8 {
DontCare, // Not specified for this instruction
RN, // Round to nearest even,
RM, // Round towards negative infinity
RP, // Round towards positive infinity
RZ, // Round towards zero
};
struct FpControl {
bool no_contraction{false};
FpRounding rounding{FpRounding::DontCare};
FmzMode fmz_mode{FmzMode::DontCare};
};
static_assert(sizeof(FpControl) <= sizeof(u32));
union TextureInstInfo {
u32 raw;
BitField<0, 16, u32> descriptor_index;
BitField<16, 3, TextureType> type;
BitField<19, 1, u32> is_depth;
BitField<20, 1, u32> has_bias;
BitField<21, 1, u32> has_lod_clamp;
BitField<22, 1, u32> relaxed_precision;
BitField<23, 2, u32> gather_component;
BitField<25, 2, u32> num_derivatives;
BitField<27, 3, ImageFormat> image_format;
BitField<30, 1, u32> ndv_is_active;
/// Only meaningful for the sampled-texture family (ImageSampleImplicitLod, ImageFetch,
/// ImageGather, etc.); unused/zero for storage-image opcodes, which carry their own
/// is_integer via ImageDescriptor/ImageBufferDescriptor instead.
BitField<31, 1, u32> is_integer;
};
static_assert(sizeof(TextureInstInfo) <= sizeof(u32));
} // namespace Shader::IR

View File

@@ -0,0 +1,36 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/frontend/ir/opcodes.h"
namespace Shader::IR {
namespace Detail {
OpcodeMeta META_TABLE[542] = {
#define OPCODE(name_token, type_token, ...) \
{ \
.name{#name_token}, \
.type = type_token, \
.arg_types{__VA_ARGS__}, \
},
#include "opcodes.inc"
#undef OPCODE
};
u8 NUM_ARGS[542] = {
#define OPCODE(name_token, type_token, ...) u8(CalculateNumArgsOf(Opcode::name_token)),
#include "opcodes.inc"
#undef OPCODE
};
}
std::string_view NameOf(Opcode op) {
return Detail::META_TABLE[static_cast<size_t>(op)].name;
}
} // namespace Shader::IR

View File

@@ -0,0 +1,97 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <algorithm>
#include <array>
#include <fmt/ranges.h>
#include <ranges>
#include "shader_recompiler/frontend/ir/type.h"
namespace Shader::IR {
enum class Opcode {
#define OPCODE(name, ...) name,
#include "opcodes.inc"
#undef OPCODE
};
namespace Detail {
struct OpcodeMeta {
std::string_view name;
Type type;
std::array<Type, 5> arg_types;
};
// using enum Type;
static constexpr Type Void{Type::Void};
static constexpr Type Opaque{Type::Opaque};
static constexpr Type Reg{Type::Reg};
static constexpr Type Pred{Type::Pred};
static constexpr Type Attribute{Type::Attribute};
static constexpr Type Patch{Type::Patch};
static constexpr Type U1{Type::U1};
static constexpr Type U8{Type::U8};
static constexpr Type U16{Type::U16};
static constexpr Type U32{Type::U32};
static constexpr Type U64{Type::U64};
static constexpr Type F16{Type::F16};
static constexpr Type F32{Type::F32};
static constexpr Type F64{Type::F64};
static constexpr Type U32x2{Type::U32x2};
static constexpr Type U32x3{Type::U32x3};
static constexpr Type U32x4{Type::U32x4};
static constexpr Type F16x2{Type::F16x2};
static constexpr Type F16x3{Type::F16x3};
static constexpr Type F16x4{Type::F16x4};
static constexpr Type F32x2{Type::F32x2};
static constexpr Type F32x3{Type::F32x3};
static constexpr Type F32x4{Type::F32x4};
static constexpr Type F64x2{Type::F64x2};
static constexpr Type F64x3{Type::F64x3};
static constexpr Type F64x4{Type::F64x4};
extern OpcodeMeta META_TABLE[542];
constexpr size_t CalculateNumArgsOf(Opcode op) noexcept {
const auto& arg_types = META_TABLE[size_t(op)].arg_types;
return size_t(std::distance(arg_types.begin(), std::ranges::find(arg_types, Type::Void)));
}
extern u8 NUM_ARGS[542];
} // namespace Detail
/// Get return type of an opcode
[[nodiscard]] inline Type TypeOf(Opcode op) noexcept {
return Detail::META_TABLE[size_t(op)].type;
}
/// Get the number of arguments an opcode accepts
[[nodiscard]] inline size_t NumArgsOf(Opcode op) noexcept {
return size_t(Detail::NUM_ARGS[size_t(op)]);
}
/// Get the required type of an argument of an opcode
[[nodiscard]] inline Type ArgTypeOf(Opcode op, size_t arg_index) noexcept {
return Detail::META_TABLE[size_t(op)].arg_types[arg_index];
}
/// Get the name of an opcode
[[nodiscard]] std::string_view NameOf(Opcode op);
} // namespace Shader::IR
template <>
struct fmt::formatter<Shader::IR::Opcode> {
constexpr auto parse(format_parse_context& ctx) {
return ctx.begin();
}
template <typename FormatContext>
auto format(const Shader::IR::Opcode& op, FormatContext& ctx) const {
return fmt::format_to(ctx.out(), "{}", Shader::IR::NameOf(op));
}
};

View File

@@ -0,0 +1,599 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
// opcode name, return type, arg1 type, arg2 type, arg3 type, arg4 type, arg4 type, ...
OPCODE(Phi, Opaque, )
OPCODE(Identity, Opaque, Opaque, )
OPCODE(Void, Void, )
OPCODE(ConditionRef, U1, U1, )
OPCODE(Reference, Void, Opaque, )
OPCODE(PhiMove, Void, Opaque, Opaque, )
// Special operations
OPCODE(Prologue, Void, )
OPCODE(Epilogue, Void, )
OPCODE(Join, Void, )
OPCODE(DemoteToHelperInvocation, Void, )
OPCODE(EmitVertex, Void, U32, )
OPCODE(EndPrimitive, Void, U32, )
// Barriers
OPCODE(Barrier, Void, )
OPCODE(WorkgroupMemoryBarrier, Void, )
OPCODE(SubgroupBarrier, Void, )
OPCODE(DeviceMemoryBarrier, Void, )
// Context getters/setters
OPCODE(GetRegister, U32, Reg, )
OPCODE(SetRegister, Void, Reg, U32, )
OPCODE(GetPred, U1, Pred, )
OPCODE(SetPred, Void, Pred, U1, )
OPCODE(GetGotoVariable, U1, U32, )
OPCODE(SetGotoVariable, Void, U32, U1, )
OPCODE(GetIndirectBranchVariable, U32, )
OPCODE(SetIndirectBranchVariable, Void, U32, )
OPCODE(GetCbufU8, U32, U32, U32, )
OPCODE(GetCbufS8, U32, U32, U32, )
OPCODE(GetCbufU16, U32, U32, U32, )
OPCODE(GetCbufS16, U32, U32, U32, )
OPCODE(GetCbufU32, U32, U32, U32, )
OPCODE(GetCbufF32, F32, U32, U32, )
OPCODE(GetCbufU32x2, U32x2, U32, U32, )
OPCODE(GetAttribute, F32, Attribute, U32, )
OPCODE(GetAttributeU32, U32, Attribute, U32, )
OPCODE(SetAttribute, Void, Attribute, F32, U32, )
OPCODE(GetAttributeIndexed, F32, U32, U32, )
OPCODE(SetAttributeIndexed, Void, U32, F32, U32, )
OPCODE(GetPatch, F32, Patch, )
OPCODE(SetPatch, Void, Patch, F32, )
OPCODE(IsbeRead, U32, U32, )
OPCODE(IsbeWriteAttribute, Void, U32, U32, )
OPCODE(IsbeWriteMapU8, Void, U32, U32, )
OPCODE(IsbeWriteMapU32, Void, U32, U32, )
OPCODE(SetFragColor, Void, U32, U32, F32, )
OPCODE(SetSampleMask, Void, U32, )
OPCODE(SetFragDepth, Void, F32, )
OPCODE(GetZFlag, U1, Void, )
OPCODE(GetSFlag, U1, Void, )
OPCODE(GetCFlag, U1, Void, )
OPCODE(GetOFlag, U1, Void, )
OPCODE(SetZFlag, Void, U1, )
OPCODE(SetSFlag, Void, U1, )
OPCODE(SetCFlag, Void, U1, )
OPCODE(SetOFlag, Void, U1, )
OPCODE(WorkgroupId, U32x3, )
OPCODE(LocalInvocationId, U32x3, )
OPCODE(InvocationId, U32, )
OPCODE(InvocationInfo, U32, )
OPCODE(SampleId, U32, )
OPCODE(IsHelperInvocation, U1, )
OPCODE(SR_WScaleFactorXY, U32, )
OPCODE(SR_WScaleFactorZ, U32, )
OPCODE(YDirection, F32, )
OPCODE(ResolutionDownFactor, F32, )
OPCODE(RenderArea, F32x4, )
// Undefined
OPCODE(UndefU1, U1, )
OPCODE(UndefU8, U8, )
OPCODE(UndefU16, U16, )
OPCODE(UndefU32, U32, )
OPCODE(UndefU64, U64, )
// Memory operations
OPCODE(LoadGlobalU8, U32, Opaque, )
OPCODE(LoadGlobalS8, U32, Opaque, )
OPCODE(LoadGlobalU16, U32, Opaque, )
OPCODE(LoadGlobalS16, U32, Opaque, )
OPCODE(LoadGlobal32, U32, Opaque, )
OPCODE(LoadGlobal64, U32x2, Opaque, )
OPCODE(LoadGlobal128, U32x4, Opaque, )
OPCODE(WriteGlobalU8, Void, Opaque, U32, )
OPCODE(WriteGlobalS8, Void, Opaque, U32, )
OPCODE(WriteGlobalU16, Void, Opaque, U32, )
OPCODE(WriteGlobalS16, Void, Opaque, U32, )
OPCODE(WriteGlobal32, Void, Opaque, U32, )
OPCODE(WriteGlobal64, Void, Opaque, U32x2, )
OPCODE(WriteGlobal128, Void, Opaque, U32x4, )
// Storage buffer operations
OPCODE(LoadStorageU8, U32, U32, U32, )
OPCODE(LoadStorageS8, U32, U32, U32, )
OPCODE(LoadStorageU16, U32, U32, U32, )
OPCODE(LoadStorageS16, U32, U32, U32, )
OPCODE(LoadStorage32, U32, U32, U32, )
OPCODE(LoadStorage64, U32x2, U32, U32, )
OPCODE(LoadStorage128, U32x4, U32, U32, )
OPCODE(WriteStorageU8, Void, U32, U32, U32, )
OPCODE(WriteStorageS8, Void, U32, U32, U32, )
OPCODE(WriteStorageU16, Void, U32, U32, U32, )
OPCODE(WriteStorageS16, Void, U32, U32, U32, )
OPCODE(WriteStorage32, Void, U32, U32, U32, )
OPCODE(WriteStorage64, Void, U32, U32, U32x2, )
OPCODE(WriteStorage128, Void, U32, U32, U32x4, )
// Local memory operations
OPCODE(LoadLocal, U32, U32, )
OPCODE(WriteLocal, Void, U32, U32, )
// Shared memory operations
OPCODE(LoadSharedU8, U32, U32, )
OPCODE(LoadSharedS8, U32, U32, )
OPCODE(LoadSharedU16, U32, U32, )
OPCODE(LoadSharedS16, U32, U32, )
OPCODE(LoadSharedU32, U32, U32, )
OPCODE(LoadSharedU64, U32x2, U32, )
OPCODE(LoadSharedU128, U32x4, U32, )
OPCODE(WriteSharedU8, Void, U32, U32, )
OPCODE(WriteSharedU16, Void, U32, U32, )
OPCODE(WriteSharedU32, Void, U32, U32, )
OPCODE(WriteSharedU64, Void, U32, U32x2, )
OPCODE(WriteSharedU128, Void, U32, U32x4, )
// Vector utility
OPCODE(CompositeConstructU32x2, U32x2, U32, U32, )
OPCODE(CompositeConstructU32x3, U32x3, U32, U32, U32, )
OPCODE(CompositeConstructU32x4, U32x4, U32, U32, U32, U32, )
OPCODE(CompositeExtractU32x2, U32, U32x2, U32, )
OPCODE(CompositeExtractU32x3, U32, U32x3, U32, )
OPCODE(CompositeExtractU32x4, U32, U32x4, U32, )
OPCODE(CompositeInsertU32x2, U32x2, U32x2, U32, U32, )
OPCODE(CompositeInsertU32x3, U32x3, U32x3, U32, U32, )
OPCODE(CompositeInsertU32x4, U32x4, U32x4, U32, U32, )
OPCODE(CompositeConstructF16x2, F16x2, F16, F16, )
OPCODE(CompositeConstructF16x3, F16x3, F16, F16, F16, )
OPCODE(CompositeConstructF16x4, F16x4, F16, F16, F16, F16, )
OPCODE(CompositeExtractF16x2, F16, F16x2, U32, )
OPCODE(CompositeExtractF16x3, F16, F16x3, U32, )
OPCODE(CompositeExtractF16x4, F16, F16x4, U32, )
OPCODE(CompositeInsertF16x2, F16x2, F16x2, F16, U32, )
OPCODE(CompositeInsertF16x3, F16x3, F16x3, F16, U32, )
OPCODE(CompositeInsertF16x4, F16x4, F16x4, F16, U32, )
OPCODE(CompositeConstructF32x2, F32x2, F32, F32, )
OPCODE(CompositeConstructF32x3, F32x3, F32, F32, F32, )
OPCODE(CompositeConstructF32x4, F32x4, F32, F32, F32, F32, )
OPCODE(CompositeExtractF32x2, F32, F32x2, U32, )
OPCODE(CompositeExtractF32x3, F32, F32x3, U32, )
OPCODE(CompositeExtractF32x4, F32, F32x4, U32, )
OPCODE(CompositeInsertF32x2, F32x2, F32x2, F32, U32, )
OPCODE(CompositeInsertF32x3, F32x3, F32x3, F32, U32, )
OPCODE(CompositeInsertF32x4, F32x4, F32x4, F32, U32, )
OPCODE(CompositeConstructF64x2, F64x2, F64, F64, )
OPCODE(CompositeConstructF64x3, F64x3, F64, F64, F64, )
OPCODE(CompositeConstructF64x4, F64x4, F64, F64, F64, F64, )
OPCODE(CompositeExtractF64x2, F64, F64x2, U32, )
OPCODE(CompositeExtractF64x3, F64, F64x3, U32, )
OPCODE(CompositeExtractF64x4, F64, F64x4, U32, )
OPCODE(CompositeInsertF64x2, F64x2, F64x2, F64, U32, )
OPCODE(CompositeInsertF64x3, F64x3, F64x3, F64, U32, )
OPCODE(CompositeInsertF64x4, F64x4, F64x4, F64, U32, )
// Select operations
OPCODE(SelectU1, U1, U1, U1, U1, )
OPCODE(SelectU8, U8, U1, U8, U8, )
OPCODE(SelectU16, U16, U1, U16, U16, )
OPCODE(SelectU32, U32, U1, U32, U32, )
OPCODE(SelectU64, U64, U1, U64, U64, )
OPCODE(SelectF16, F16, U1, F16, F16, )
OPCODE(SelectF32, F32, U1, F32, F32, )
OPCODE(SelectF64, F64, U1, F64, F64, )
// Bitwise conversions
OPCODE(BitCastU16F16, U16, F16, )
OPCODE(BitCastU32F32, U32, F32, )
OPCODE(BitCastU64F64, U64, F64, )
OPCODE(BitCastF16U16, F16, U16, )
OPCODE(BitCastF32U32, F32, U32, )
OPCODE(BitCastF64U64, F64, U64, )
OPCODE(PackUint2x32, U64, U32x2, )
OPCODE(UnpackUint2x32, U32x2, U64, )
OPCODE(PackFloat2x16, U32, F16x2, )
OPCODE(UnpackFloat2x16, F16x2, U32, )
OPCODE(PackHalf2x16, U32, F32x2, )
OPCODE(UnpackHalf2x16, F32x2, U32, )
OPCODE(PackDouble2x32, F64, U32x2, )
OPCODE(UnpackDouble2x32, U32x2, F64, )
// Pseudo-operation, handled specially at final emit
OPCODE(GetZeroFromOp, U1, Opaque, )
OPCODE(GetSignFromOp, U1, Opaque, )
OPCODE(GetCarryFromOp, U1, Opaque, )
OPCODE(GetOverflowFromOp, U1, Opaque, )
OPCODE(GetSparseFromOp, U1, Opaque, )
OPCODE(GetInBoundsFromOp, U1, Opaque, )
// Floating-point operations
OPCODE(FPAbs16, F16, F16, )
OPCODE(FPAbs32, F32, F32, )
OPCODE(FPAbs64, F64, F64, )
OPCODE(FPAdd16, F16, F16, F16, )
OPCODE(FPAdd32, F32, F32, F32, )
OPCODE(FPAdd64, F64, F64, F64, )
OPCODE(FPFma16, F16, F16, F16, F16, )
OPCODE(FPFma32, F32, F32, F32, F32, )
OPCODE(FPFma64, F64, F64, F64, F64, )
OPCODE(FPMax32, F32, F32, F32, )
OPCODE(FPMax64, F64, F64, F64, )
OPCODE(FPMin32, F32, F32, F32, )
OPCODE(FPMin64, F64, F64, F64, )
OPCODE(FPMul16, F16, F16, F16, )
OPCODE(FPMul32, F32, F32, F32, )
OPCODE(FPMul64, F64, F64, F64, )
OPCODE(FPNeg16, F16, F16, )
OPCODE(FPNeg32, F32, F32, )
OPCODE(FPNeg64, F64, F64, )
OPCODE(FPRecip32, F32, F32, )
OPCODE(FPRecip64, F64, F64, )
OPCODE(FPRecipSqrt32, F32, F32, )
OPCODE(FPRecipSqrt64, F64, F64, )
OPCODE(FPSqrt, F32, F32, )
OPCODE(FPSin, F32, F32, )
OPCODE(FPExp2, F32, F32, )
OPCODE(FPCos, F32, F32, )
OPCODE(FPLog2, F32, F32, )
OPCODE(FPSaturate16, F16, F16, )
OPCODE(FPSaturate32, F32, F32, )
OPCODE(FPSaturate64, F64, F64, )
OPCODE(FPClamp16, F16, F16, F16, F16, )
OPCODE(FPClamp32, F32, F32, F32, F32, )
OPCODE(FPClamp64, F64, F64, F64, F64, )
OPCODE(FPRoundEven16, F16, F16, )
OPCODE(FPRoundEven32, F32, F32, )
OPCODE(FPRoundEven64, F64, F64, )
OPCODE(FPFloor16, F16, F16, )
OPCODE(FPFloor32, F32, F32, )
OPCODE(FPFloor64, F64, F64, )
OPCODE(FPCeil16, F16, F16, )
OPCODE(FPCeil32, F32, F32, )
OPCODE(FPCeil64, F64, F64, )
OPCODE(FPTrunc16, F16, F16, )
OPCODE(FPTrunc32, F32, F32, )
OPCODE(FPTrunc64, F64, F64, )
OPCODE(FPOrdEqual16, U1, F16, F16, )
OPCODE(FPOrdEqual32, U1, F32, F32, )
OPCODE(FPOrdEqual64, U1, F64, F64, )
OPCODE(FPUnordEqual16, U1, F16, F16, )
OPCODE(FPUnordEqual32, U1, F32, F32, )
OPCODE(FPUnordEqual64, U1, F64, F64, )
OPCODE(FPOrdNotEqual16, U1, F16, F16, )
OPCODE(FPOrdNotEqual32, U1, F32, F32, )
OPCODE(FPOrdNotEqual64, U1, F64, F64, )
OPCODE(FPUnordNotEqual16, U1, F16, F16, )
OPCODE(FPUnordNotEqual32, U1, F32, F32, )
OPCODE(FPUnordNotEqual64, U1, F64, F64, )
OPCODE(FPOrdLessThan16, U1, F16, F16, )
OPCODE(FPOrdLessThan32, U1, F32, F32, )
OPCODE(FPOrdLessThan64, U1, F64, F64, )
OPCODE(FPUnordLessThan16, U1, F16, F16, )
OPCODE(FPUnordLessThan32, U1, F32, F32, )
OPCODE(FPUnordLessThan64, U1, F64, F64, )
OPCODE(FPOrdGreaterThan16, U1, F16, F16, )
OPCODE(FPOrdGreaterThan32, U1, F32, F32, )
OPCODE(FPOrdGreaterThan64, U1, F64, F64, )
OPCODE(FPUnordGreaterThan16, U1, F16, F16, )
OPCODE(FPUnordGreaterThan32, U1, F32, F32, )
OPCODE(FPUnordGreaterThan64, U1, F64, F64, )
OPCODE(FPOrdLessThanEqual16, U1, F16, F16, )
OPCODE(FPOrdLessThanEqual32, U1, F32, F32, )
OPCODE(FPOrdLessThanEqual64, U1, F64, F64, )
OPCODE(FPUnordLessThanEqual16, U1, F16, F16, )
OPCODE(FPUnordLessThanEqual32, U1, F32, F32, )
OPCODE(FPUnordLessThanEqual64, U1, F64, F64, )
OPCODE(FPOrdGreaterThanEqual16, U1, F16, F16, )
OPCODE(FPOrdGreaterThanEqual32, U1, F32, F32, )
OPCODE(FPOrdGreaterThanEqual64, U1, F64, F64, )
OPCODE(FPUnordGreaterThanEqual16, U1, F16, F16, )
OPCODE(FPUnordGreaterThanEqual32, U1, F32, F32, )
OPCODE(FPUnordGreaterThanEqual64, U1, F64, F64, )
OPCODE(FPIsNan16, U1, F16, )
OPCODE(FPIsNan32, U1, F32, )
OPCODE(FPIsNan64, U1, F64, )
// Integer operations
OPCODE(IAdd32, U32, U32, U32, )
OPCODE(IAdd64, U64, U64, U64, )
OPCODE(ISub32, U32, U32, U32, )
OPCODE(ISub64, U64, U64, U64, )
OPCODE(IMul32, U32, U32, U32, )
OPCODE(SDiv32, U32, U32, U32, )
OPCODE(UDiv32, U32, U32, U32, )
OPCODE(INeg32, U32, U32, )
OPCODE(INeg64, U64, U64, )
OPCODE(IAbs32, U32, U32, )
OPCODE(IAbs64, U64, U64, )
OPCODE(ShiftLeftLogical32, U32, U32, U32, )
OPCODE(ShiftLeftLogical64, U64, U64, U32, )
OPCODE(ShiftRightLogical32, U32, U32, U32, )
OPCODE(ShiftRightLogical64, U64, U64, U32, )
OPCODE(ShiftRightArithmetic32, U32, U32, U32, )
OPCODE(ShiftRightArithmetic64, U64, U64, U32, )
OPCODE(BitwiseAnd32, U32, U32, U32, )
OPCODE(BitwiseOr32, U32, U32, U32, )
OPCODE(BitwiseXor32, U32, U32, U32, )
OPCODE(BitFieldInsert, U32, U32, U32, U32, U32, )
OPCODE(BitFieldSExtract, U32, U32, U32, U32, )
OPCODE(BitFieldUExtract, U32, U32, U32, U32, )
OPCODE(BitReverse32, U32, U32, )
OPCODE(BitCount32, U32, U32, )
OPCODE(BitwiseNot32, U32, U32, )
OPCODE(FindSMsb32, U32, U32, )
OPCODE(FindUMsb32, U32, U32, )
OPCODE(SMin32, U32, U32, U32, )
OPCODE(UMin32, U32, U32, U32, )
OPCODE(SMax32, U32, U32, U32, )
OPCODE(UMax32, U32, U32, U32, )
OPCODE(SClamp32, U32, U32, U32, U32, )
OPCODE(UClamp32, U32, U32, U32, U32, )
OPCODE(SLessThan, U1, U32, U32, )
OPCODE(ULessThan, U1, U32, U32, )
OPCODE(IEqual, U1, U32, U32, )
OPCODE(SLessThanEqual, U1, U32, U32, )
OPCODE(ULessThanEqual, U1, U32, U32, )
OPCODE(SGreaterThan, U1, U32, U32, )
OPCODE(UGreaterThan, U1, U32, U32, )
OPCODE(INotEqual, U1, U32, U32, )
OPCODE(SGreaterThanEqual, U1, U32, U32, )
OPCODE(UGreaterThanEqual, U1, U32, U32, )
// Atomic operations
OPCODE(SharedAtomicIAdd32, U32, U32, U32, )
OPCODE(SharedAtomicSMin32, U32, U32, U32, )
OPCODE(SharedAtomicUMin32, U32, U32, U32, )
OPCODE(SharedAtomicSMax32, U32, U32, U32, )
OPCODE(SharedAtomicUMax32, U32, U32, U32, )
OPCODE(SharedAtomicInc32, U32, U32, U32, )
OPCODE(SharedAtomicDec32, U32, U32, U32, )
OPCODE(SharedAtomicAnd32, U32, U32, U32, )
OPCODE(SharedAtomicOr32, U32, U32, U32, )
OPCODE(SharedAtomicXor32, U32, U32, U32, )
OPCODE(SharedAtomicExchange32, U32, U32, U32, )
OPCODE(SharedAtomicExchange64, U64, U32, U64, )
OPCODE(SharedAtomicExchange32x2, U32x2, U32, U32x2, )
OPCODE(GlobalAtomicIAdd32, U32, U64, U32, )
OPCODE(GlobalAtomicSMin32, U32, U64, U32, )
OPCODE(GlobalAtomicUMin32, U32, U64, U32, )
OPCODE(GlobalAtomicSMax32, U32, U64, U32, )
OPCODE(GlobalAtomicUMax32, U32, U64, U32, )
OPCODE(GlobalAtomicInc32, U32, U64, U32, )
OPCODE(GlobalAtomicDec32, U32, U64, U32, )
OPCODE(GlobalAtomicAnd32, U32, U64, U32, )
OPCODE(GlobalAtomicOr32, U32, U64, U32, )
OPCODE(GlobalAtomicXor32, U32, U64, U32, )
OPCODE(GlobalAtomicExchange32, U32, U64, U32, )
OPCODE(GlobalAtomicCompareExchange32, U32, U64, U32, U32, )
OPCODE(GlobalAtomicIAdd64, U64, U64, U64, )
OPCODE(GlobalAtomicSMin64, U64, U64, U64, )
OPCODE(GlobalAtomicUMin64, U64, U64, U64, )
OPCODE(GlobalAtomicSMax64, U64, U64, U64, )
OPCODE(GlobalAtomicUMax64, U64, U64, U64, )
OPCODE(GlobalAtomicAnd64, U64, U64, U64, )
OPCODE(GlobalAtomicOr64, U64, U64, U64, )
OPCODE(GlobalAtomicXor64, U64, U64, U64, )
OPCODE(GlobalAtomicExchange64, U64, U64, U64, )
OPCODE(GlobalAtomicIAdd32x2, U32x2, U32x2, U32x2, )
OPCODE(GlobalAtomicSMin32x2, U32x2, U32x2, U32x2, )
OPCODE(GlobalAtomicUMin32x2, U32x2, U32x2, U32x2, )
OPCODE(GlobalAtomicSMax32x2, U32x2, U32x2, U32x2, )
OPCODE(GlobalAtomicUMax32x2, U32x2, U32x2, U32x2, )
OPCODE(GlobalAtomicAnd32x2, U32x2, U32x2, U32x2, )
OPCODE(GlobalAtomicOr32x2, U32x2, U32x2, U32x2, )
OPCODE(GlobalAtomicXor32x2, U32x2, U32x2, U32x2, )
OPCODE(GlobalAtomicExchange32x2, U32x2, U32x2, U32x2, )
OPCODE(GlobalAtomicAddF32, F32, U64, F32, )
OPCODE(GlobalAtomicAddF16x2, U32, U64, F16x2, )
OPCODE(GlobalAtomicAddF32x2, U32, U64, F32x2, )
OPCODE(GlobalAtomicMinF16x2, U32, U64, F16x2, )
OPCODE(GlobalAtomicMinF32x2, U32, U64, F32x2, )
OPCODE(GlobalAtomicMaxF16x2, U32, U64, F16x2, )
OPCODE(GlobalAtomicMaxF32x2, U32, U64, F32x2, )
OPCODE(StorageAtomicIAdd32, U32, U32, U32, U32, )
OPCODE(StorageAtomicSMin32, U32, U32, U32, U32, )
OPCODE(StorageAtomicUMin32, U32, U32, U32, U32, )
OPCODE(StorageAtomicSMax32, U32, U32, U32, U32, )
OPCODE(StorageAtomicUMax32, U32, U32, U32, U32, )
OPCODE(StorageAtomicInc32, U32, U32, U32, U32, )
OPCODE(StorageAtomicDec32, U32, U32, U32, U32, )
OPCODE(StorageAtomicAnd32, U32, U32, U32, U32, )
OPCODE(StorageAtomicOr32, U32, U32, U32, U32, )
OPCODE(StorageAtomicXor32, U32, U32, U32, U32, )
OPCODE(StorageAtomicExchange32, U32, U32, U32, U32, )
OPCODE(StorageAtomicCompareExchange32, U32, U32, U32, U32, U32, )
OPCODE(StorageAtomicIAdd64, U64, U32, U32, U64, )
OPCODE(StorageAtomicSMin64, U64, U32, U32, U64, )
OPCODE(StorageAtomicUMin64, U64, U32, U32, U64, )
OPCODE(StorageAtomicSMax64, U64, U32, U32, U64, )
OPCODE(StorageAtomicUMax64, U64, U32, U32, U64, )
OPCODE(StorageAtomicAnd64, U64, U32, U32, U64, )
OPCODE(StorageAtomicOr64, U64, U32, U32, U64, )
OPCODE(StorageAtomicXor64, U64, U32, U32, U64, )
OPCODE(StorageAtomicExchange64, U64, U32, U32, U64, )
OPCODE(StorageAtomicIAdd32x2, U32x2, U32, U32, U32x2, )
OPCODE(StorageAtomicSMin32x2, U32x2, U32, U32, U32x2, )
OPCODE(StorageAtomicUMin32x2, U32x2, U32, U32, U32x2, )
OPCODE(StorageAtomicSMax32x2, U32x2, U32, U32, U32x2, )
OPCODE(StorageAtomicUMax32x2, U32x2, U32, U32, U32x2, )
OPCODE(StorageAtomicAnd32x2, U32x2, U32, U32, U32x2, )
OPCODE(StorageAtomicOr32x2, U32x2, U32, U32, U32x2, )
OPCODE(StorageAtomicXor32x2, U32x2, U32, U32, U32x2, )
OPCODE(StorageAtomicExchange32x2, U32x2, U32, U32, U32x2, )
OPCODE(StorageAtomicAddF32, F32, U32, U32, F32, )
OPCODE(StorageAtomicAddF16x2, U32, U32, U32, F16x2, )
OPCODE(StorageAtomicAddF32x2, U32, U32, U32, F32x2, )
OPCODE(StorageAtomicMinF16x2, U32, U32, U32, F16x2, )
OPCODE(StorageAtomicMinF32x2, U32, U32, U32, F32x2, )
OPCODE(StorageAtomicMaxF16x2, U32, U32, U32, F16x2, )
OPCODE(StorageAtomicMaxF32x2, U32, U32, U32, F32x2, )
// Logical operations
OPCODE(LogicalOr, U1, U1, U1, )
OPCODE(LogicalAnd, U1, U1, U1, )
OPCODE(LogicalXor, U1, U1, U1, )
OPCODE(LogicalNot, U1, U1, )
// Conversion operations
OPCODE(ConvertS16F16, U32, F16, )
OPCODE(ConvertS16F32, U32, F32, )
OPCODE(ConvertS16F64, U32, F64, )
OPCODE(ConvertS32F16, U32, F16, )
OPCODE(ConvertS32F32, U32, F32, )
OPCODE(ConvertS32F64, U32, F64, )
OPCODE(ConvertS64F16, U64, F16, )
OPCODE(ConvertS64F32, U64, F32, )
OPCODE(ConvertS64F64, U64, F64, )
OPCODE(ConvertU16F16, U32, F16, )
OPCODE(ConvertU16F32, U32, F32, )
OPCODE(ConvertU16F64, U32, F64, )
OPCODE(ConvertU32F16, U32, F16, )
OPCODE(ConvertU32F32, U32, F32, )
OPCODE(ConvertU32F64, U32, F64, )
OPCODE(ConvertU64F16, U64, F16, )
OPCODE(ConvertU64F32, U64, F32, )
OPCODE(ConvertU64F64, U64, F64, )
OPCODE(ConvertU64U32, U64, U32, )
OPCODE(ConvertU32U64, U32, U64, )
OPCODE(ConvertF16F32, F16, F32, )
OPCODE(ConvertF32F16, F32, F16, )
OPCODE(ConvertF32F64, F32, F64, )
OPCODE(ConvertF64F32, F64, F32, )
OPCODE(ConvertF16S8, F16, U32, )
OPCODE(ConvertF16S16, F16, U32, )
OPCODE(ConvertF16S32, F16, U32, )
OPCODE(ConvertF16S64, F16, U64, )
OPCODE(ConvertF16U8, F16, U32, )
OPCODE(ConvertF16U16, F16, U32, )
OPCODE(ConvertF16U32, F16, U32, )
OPCODE(ConvertF16U64, F16, U64, )
OPCODE(ConvertF32S8, F32, U32, )
OPCODE(ConvertF32S16, F32, U32, )
OPCODE(ConvertF32S32, F32, U32, )
OPCODE(ConvertF32S64, F32, U64, )
OPCODE(ConvertF32U8, F32, U32, )
OPCODE(ConvertF32U16, F32, U32, )
OPCODE(ConvertF32U32, F32, U32, )
OPCODE(ConvertF32U64, F32, U64, )
OPCODE(ConvertF64S8, F64, U32, )
OPCODE(ConvertF64S16, F64, U32, )
OPCODE(ConvertF64S32, F64, U32, )
OPCODE(ConvertF64S64, F64, U64, )
OPCODE(ConvertF64U8, F64, U32, )
OPCODE(ConvertF64U16, F64, U32, )
OPCODE(ConvertF64U32, F64, U32, )
OPCODE(ConvertF64U64, F64, U64, )
OPCODE(ConvertU16U32, U16, U32, )
OPCODE(ConvertU32U16, U32, U16, )
OPCODE(ConvertU8U32, U8, U32, )
OPCODE(ConvertU32U8, U32, U8, )
OPCODE(ConvertS32S8, U32, U8, )
OPCODE(ConvertS32S16, U32, U16, )
// Image operations
OPCODE(BindlessImageSampleImplicitLod, F32x4, U32, Opaque, Opaque, Opaque, )
OPCODE(BindlessImageSampleExplicitLod, F32x4, U32, Opaque, Opaque, Opaque, )
OPCODE(BindlessImageSampleDrefImplicitLod, F32, U32, Opaque, F32, Opaque, Opaque, )
OPCODE(BindlessImageSampleDrefExplicitLod, F32, U32, Opaque, F32, Opaque, Opaque, )
OPCODE(BindlessImageGather, F32x4, U32, Opaque, Opaque, Opaque, )
OPCODE(BindlessImageGatherDref, F32x4, U32, Opaque, Opaque, Opaque, F32, )
OPCODE(BindlessImageFetch, F32x4, U32, Opaque, Opaque, U32, Opaque, )
OPCODE(BindlessImageQueryDimensions, U32x4, U32, U32, U1, )
OPCODE(BindlessImageQueryLod, F32x4, U32, Opaque, )
OPCODE(BindlessImageGradient, F32x4, U32, Opaque, Opaque, Opaque, Opaque, )
OPCODE(BindlessImageRead, U32x4, U32, Opaque, )
OPCODE(BindlessImageWrite, Void, U32, Opaque, U32x4, )
OPCODE(BoundImageSampleImplicitLod, F32x4, U32, Opaque, Opaque, Opaque, )
OPCODE(BoundImageSampleExplicitLod, F32x4, U32, Opaque, Opaque, Opaque, )
OPCODE(BoundImageSampleDrefImplicitLod, F32, U32, Opaque, F32, Opaque, Opaque, )
OPCODE(BoundImageSampleDrefExplicitLod, F32, U32, Opaque, F32, Opaque, Opaque, )
OPCODE(BoundImageGather, F32x4, U32, Opaque, Opaque, Opaque, )
OPCODE(BoundImageGatherDref, F32x4, U32, Opaque, Opaque, Opaque, F32, )
OPCODE(BoundImageFetch, F32x4, U32, Opaque, Opaque, U32, Opaque, )
OPCODE(BoundImageQueryDimensions, U32x4, U32, U32, U1, )
OPCODE(BoundImageQueryLod, F32x4, U32, Opaque, )
OPCODE(BoundImageGradient, F32x4, U32, Opaque, Opaque, Opaque, Opaque, )
OPCODE(BoundImageRead, U32x4, U32, Opaque, )
OPCODE(BoundImageWrite, Void, U32, Opaque, U32x4, )
OPCODE(ImageSampleImplicitLod, F32x4, Opaque, Opaque, Opaque, Opaque, )
OPCODE(ImageSampleExplicitLod, F32x4, Opaque, Opaque, Opaque, Opaque, )
OPCODE(ImageSampleDrefImplicitLod, F32, Opaque, Opaque, F32, Opaque, Opaque, )
OPCODE(ImageSampleDrefExplicitLod, F32, Opaque, Opaque, F32, Opaque, Opaque, )
OPCODE(ImageGather, F32x4, Opaque, Opaque, Opaque, Opaque, )
OPCODE(ImageGatherDref, F32x4, Opaque, Opaque, Opaque, Opaque, F32, )
OPCODE(ImageFetch, F32x4, Opaque, Opaque, Opaque, U32, Opaque, )
OPCODE(ImageQueryDimensions, U32x4, Opaque, U32, U1, )
OPCODE(ImageQueryLod, F32x4, Opaque, Opaque, )
OPCODE(ImageGradient, F32x4, Opaque, Opaque, Opaque, Opaque, Opaque, )
OPCODE(ImageRead, U32x4, Opaque, Opaque, )
OPCODE(ImageWrite, Void, Opaque, Opaque, U32x4, )
OPCODE(IsTextureScaled, U1, U32, )
OPCODE(IsImageScaled, U1, U32, )
// Atomic Image operations
OPCODE(BindlessImageAtomicIAdd32, U32, U32, Opaque, U32, )
OPCODE(BindlessImageAtomicSMin32, U32, U32, Opaque, U32, )
OPCODE(BindlessImageAtomicUMin32, U32, U32, Opaque, U32, )
OPCODE(BindlessImageAtomicSMax32, U32, U32, Opaque, U32, )
OPCODE(BindlessImageAtomicUMax32, U32, U32, Opaque, U32, )
OPCODE(BindlessImageAtomicInc32, U32, U32, Opaque, U32, )
OPCODE(BindlessImageAtomicDec32, U32, U32, Opaque, U32, )
OPCODE(BindlessImageAtomicAnd32, U32, U32, Opaque, U32, )
OPCODE(BindlessImageAtomicOr32, U32, U32, Opaque, U32, )
OPCODE(BindlessImageAtomicXor32, U32, U32, Opaque, U32, )
OPCODE(BindlessImageAtomicExchange32, U32, U32, Opaque, U32, )
OPCODE(BoundImageAtomicIAdd32, U32, U32, Opaque, U32, )
OPCODE(BoundImageAtomicSMin32, U32, U32, Opaque, U32, )
OPCODE(BoundImageAtomicUMin32, U32, U32, Opaque, U32, )
OPCODE(BoundImageAtomicSMax32, U32, U32, Opaque, U32, )
OPCODE(BoundImageAtomicUMax32, U32, U32, Opaque, U32, )
OPCODE(BoundImageAtomicInc32, U32, U32, Opaque, U32, )
OPCODE(BoundImageAtomicDec32, U32, U32, Opaque, U32, )
OPCODE(BoundImageAtomicAnd32, U32, U32, Opaque, U32, )
OPCODE(BoundImageAtomicOr32, U32, U32, Opaque, U32, )
OPCODE(BoundImageAtomicXor32, U32, U32, Opaque, U32, )
OPCODE(BoundImageAtomicExchange32, U32, U32, Opaque, U32, )
OPCODE(ImageAtomicIAdd32, U32, Opaque, Opaque, U32, )
OPCODE(ImageAtomicSMin32, U32, Opaque, Opaque, U32, )
OPCODE(ImageAtomicUMin32, U32, Opaque, Opaque, U32, )
OPCODE(ImageAtomicSMax32, U32, Opaque, Opaque, U32, )
OPCODE(ImageAtomicUMax32, U32, Opaque, Opaque, U32, )
OPCODE(ImageAtomicInc32, U32, Opaque, Opaque, U32, )
OPCODE(ImageAtomicDec32, U32, Opaque, Opaque, U32, )
OPCODE(ImageAtomicAnd32, U32, Opaque, Opaque, U32, )
OPCODE(ImageAtomicOr32, U32, Opaque, Opaque, U32, )
OPCODE(ImageAtomicXor32, U32, Opaque, Opaque, U32, )
OPCODE(ImageAtomicExchange32, U32, Opaque, Opaque, U32, )
// Warp operations
OPCODE(LaneId, U32, )
OPCODE(VoteAll, U1, U1, )
OPCODE(VoteAny, U1, U1, )
OPCODE(VoteEqual, U1, U1, )
OPCODE(SubgroupBallot, U32, U1, )
OPCODE(SubgroupEqMask, U32, )
OPCODE(SubgroupLtMask, U32, )
OPCODE(SubgroupLeMask, U32, )
OPCODE(SubgroupGtMask, U32, )
OPCODE(SubgroupGeMask, U32, )
OPCODE(ShuffleIndex, U32, U32, U32, U32, U32, )
OPCODE(ShuffleUp, U32, U32, U32, U32, U32, )
OPCODE(ShuffleDown, U32, U32, U32, U32, U32, )
OPCODE(ShuffleButterfly, U32, U32, U32, U32, U32, )
OPCODE(MatrixMulAddF16, U32x2, U32x4, U32x2, U32x2, )
OPCODE(QuadBroadcast, U32, U32, U32, )
OPCODE(QuadSwap, U32, U32, U32, )
OPCODE(FSwizzleAdd, F32, F32, F32, U32, )
OPCODE(DPdxFine, F32, F32, )
OPCODE(DPdyFine, F32, F32, )
OPCODE(DPdxCoarse, F32, F32, )
OPCODE(DPdyCoarse, F32, F32, )

View File

@@ -0,0 +1,27 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/exception.h"
#include "shader_recompiler/frontend/ir/patch.h"
namespace Shader::IR {
bool IsGeneric(Patch patch) noexcept {
return patch >= Patch::Component0 && patch <= Patch::Component119;
}
u32 GenericPatchIndex(Patch patch) {
if (!IsGeneric(patch)) {
throw InvalidArgument("Patch {} is not generic", patch);
}
return (static_cast<u32>(patch) - static_cast<u32>(Patch::Component0)) / 4;
}
u32 GenericPatchElement(Patch patch) {
if (!IsGeneric(patch)) {
throw InvalidArgument("Patch {} is not generic", patch);
}
return (static_cast<u32>(patch) - static_cast<u32>(Patch::Component0)) % 4;
}
} // namespace Shader::IR

View File

@@ -0,0 +1,146 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "common/common_types.h"
namespace Shader::IR {
enum class Patch : u64 {
TessellationLodLeft,
TessellationLodTop,
TessellationLodRight,
TessellationLodBottom,
TessellationLodInteriorU,
TessellationLodInteriorV,
Component0,
Component1,
Component2,
Component3,
Component4,
Component5,
Component6,
Component7,
Component8,
Component9,
Component10,
Component11,
Component12,
Component13,
Component14,
Component15,
Component16,
Component17,
Component18,
Component19,
Component20,
Component21,
Component22,
Component23,
Component24,
Component25,
Component26,
Component27,
Component28,
Component29,
Component30,
Component31,
Component32,
Component33,
Component34,
Component35,
Component36,
Component37,
Component38,
Component39,
Component40,
Component41,
Component42,
Component43,
Component44,
Component45,
Component46,
Component47,
Component48,
Component49,
Component50,
Component51,
Component52,
Component53,
Component54,
Component55,
Component56,
Component57,
Component58,
Component59,
Component60,
Component61,
Component62,
Component63,
Component64,
Component65,
Component66,
Component67,
Component68,
Component69,
Component70,
Component71,
Component72,
Component73,
Component74,
Component75,
Component76,
Component77,
Component78,
Component79,
Component80,
Component81,
Component82,
Component83,
Component84,
Component85,
Component86,
Component87,
Component88,
Component89,
Component90,
Component91,
Component92,
Component93,
Component94,
Component95,
Component96,
Component97,
Component98,
Component99,
Component100,
Component101,
Component102,
Component103,
Component104,
Component105,
Component106,
Component107,
Component108,
Component109,
Component110,
Component111,
Component112,
Component113,
Component114,
Component115,
Component116,
Component117,
Component118,
Component119,
};
static_assert(static_cast<u64>(Patch::Component119) == 125);
[[nodiscard]] bool IsGeneric(Patch patch) noexcept;
[[nodiscard]] u32 GenericPatchIndex(Patch patch);
[[nodiscard]] u32 GenericPatchElement(Patch patch);
} // namespace Shader::IR

View File

@@ -0,0 +1,45 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <algorithm>
#include <boost/container/flat_set.hpp>
#include <boost/container/small_vector.hpp>
#include "shader_recompiler/frontend/ir/basic_block.h"
#include "shader_recompiler/frontend/ir/post_order.h"
namespace Shader::IR {
BlockList PostOrder(const AbstractSyntaxNode& root) {
boost::container::small_vector<Block*, 16> block_stack;
boost::container::flat_set<Block*> visited;
BlockList post_order_blocks;
if (root.type != AbstractSyntaxNode::Type::Block) {
throw LogicError("First node in abstract syntax list root is not a block");
}
Block* const first_block{root.data.block};
visited.insert(first_block);
block_stack.push_back(first_block);
while (!block_stack.empty()) {
Block* const block{block_stack.back()};
const auto visit{[&](Block* branch) {
if (!visited.insert(branch).second) {
return false;
}
// Calling push_back twice is faster than insert on MSVC
block_stack.push_back(block);
block_stack.push_back(branch);
return true;
}};
block_stack.pop_back();
if (std::ranges::none_of(block->ImmSuccessors(), visit)) {
post_order_blocks.push_back(block);
}
}
return post_order_blocks;
}
} // namespace Shader::IR

View File

@@ -0,0 +1,13 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "shader_recompiler/frontend/ir/abstract_syntax_list.h"
#include "shader_recompiler/frontend/ir/basic_block.h"
namespace Shader::IR {
BlockList PostOrder(const AbstractSyntaxNode& root);
} // namespace Shader::IR

View File

@@ -0,0 +1,43 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <fmt/ranges.h>
namespace Shader::IR {
enum class Pred : u64 {
P0,
P1,
P2,
P3,
P4,
P5,
P6,
PT,
};
constexpr size_t NUM_USER_PREDS = 7;
constexpr size_t NUM_PREDS = 8;
[[nodiscard]] constexpr size_t PredIndex(Pred pred) noexcept {
return static_cast<size_t>(pred);
}
} // namespace Shader::IR
template <>
struct fmt::formatter<Shader::IR::Pred> {
constexpr auto parse(format_parse_context& ctx) {
return ctx.begin();
}
template <typename FormatContext>
auto format(const Shader::IR::Pred& pred, FormatContext& ctx) const {
if (pred == Shader::IR::Pred::PT) {
return fmt::format_to(ctx.out(), "PT");
} else {
return fmt::format_to(ctx.out(), "P{}", static_cast<int>(pred));
}
}
};

View File

@@ -0,0 +1,31 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <map>
#include <string>
#include <fmt/ranges.h>
#include "shader_recompiler/frontend/ir/basic_block.h"
#include "shader_recompiler/frontend/ir/program.h"
#include "shader_recompiler/frontend/ir/value.h"
namespace Shader::IR {
std::string DumpProgram(const Program& program) {
size_t index{0};
std::map<const IR::Inst*, size_t> inst_to_index;
std::map<const IR::Block*, size_t> block_to_index;
for (const IR::Block* const block : program.blocks) {
block_to_index.emplace(block, index);
++index;
}
std::string ret;
for (const auto& block : program.blocks) {
ret += IR::DumpBlock(*block, block_to_index, inst_to_index, index) + '\n';
}
return ret;
}
} // namespace Shader::IR

View File

@@ -0,0 +1,35 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <array>
#include <string>
#include "shader_recompiler/frontend/ir/abstract_syntax_list.h"
#include "shader_recompiler/frontend/ir/basic_block.h"
#include "shader_recompiler/program_header.h"
#include "shader_recompiler/shader_info.h"
#include "shader_recompiler/stage.h"
namespace Shader::IR {
struct Program {
AbstractSyntaxList syntax_list;
BlockList blocks;
BlockList post_order_blocks;
Info info;
Stage stage{};
std::array<u32, 3> workgroup_size{};
OutputTopology output_topology{};
u32 output_vertices{};
u32 invocations{};
u32 local_memory_size{};
u32 shared_memory_size{};
bool is_geometry_passthrough{};
MeshInfo mesh{};
};
[[nodiscard]] std::string DumpProgram(const Program& program);
} // namespace Shader::IR

View File

@@ -0,0 +1,333 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <fmt/ranges.h>
#include "common/common_types.h"
#include "shader_recompiler/exception.h"
namespace Shader::IR {
enum class Reg : u64 {
R0,
R1,
R2,
R3,
R4,
R5,
R6,
R7,
R8,
R9,
R10,
R11,
R12,
R13,
R14,
R15,
R16,
R17,
R18,
R19,
R20,
R21,
R22,
R23,
R24,
R25,
R26,
R27,
R28,
R29,
R30,
R31,
R32,
R33,
R34,
R35,
R36,
R37,
R38,
R39,
R40,
R41,
R42,
R43,
R44,
R45,
R46,
R47,
R48,
R49,
R50,
R51,
R52,
R53,
R54,
R55,
R56,
R57,
R58,
R59,
R60,
R61,
R62,
R63,
R64,
R65,
R66,
R67,
R68,
R69,
R70,
R71,
R72,
R73,
R74,
R75,
R76,
R77,
R78,
R79,
R80,
R81,
R82,
R83,
R84,
R85,
R86,
R87,
R88,
R89,
R90,
R91,
R92,
R93,
R94,
R95,
R96,
R97,
R98,
R99,
R100,
R101,
R102,
R103,
R104,
R105,
R106,
R107,
R108,
R109,
R110,
R111,
R112,
R113,
R114,
R115,
R116,
R117,
R118,
R119,
R120,
R121,
R122,
R123,
R124,
R125,
R126,
R127,
R128,
R129,
R130,
R131,
R132,
R133,
R134,
R135,
R136,
R137,
R138,
R139,
R140,
R141,
R142,
R143,
R144,
R145,
R146,
R147,
R148,
R149,
R150,
R151,
R152,
R153,
R154,
R155,
R156,
R157,
R158,
R159,
R160,
R161,
R162,
R163,
R164,
R165,
R166,
R167,
R168,
R169,
R170,
R171,
R172,
R173,
R174,
R175,
R176,
R177,
R178,
R179,
R180,
R181,
R182,
R183,
R184,
R185,
R186,
R187,
R188,
R189,
R190,
R191,
R192,
R193,
R194,
R195,
R196,
R197,
R198,
R199,
R200,
R201,
R202,
R203,
R204,
R205,
R206,
R207,
R208,
R209,
R210,
R211,
R212,
R213,
R214,
R215,
R216,
R217,
R218,
R219,
R220,
R221,
R222,
R223,
R224,
R225,
R226,
R227,
R228,
R229,
R230,
R231,
R232,
R233,
R234,
R235,
R236,
R237,
R238,
R239,
R240,
R241,
R242,
R243,
R244,
R245,
R246,
R247,
R248,
R249,
R250,
R251,
R252,
R253,
R254,
RZ,
};
static_assert(static_cast<int>(Reg::RZ) == 255);
constexpr size_t NUM_USER_REGS = 255;
// (hookshot: 256-510 are REGMAP_nx2's extended registers) (triforce, from EvilOOT: the native lowering keeps shadowed guest
// registers and borrowed scratch registers in two more banks of 256, R256-R767, reached only by native register copies)
constexpr size_t NUM_REGS = 768;
[[nodiscard]] constexpr Reg operator+(Reg reg, int num) {
if (reg == Reg::RZ) {
// Adding or subtracting registers from RZ yields RZ
return Reg::RZ;
}
const int result{static_cast<int>(reg) + num};
if (result >= static_cast<int>(Reg::RZ)) {
throw LogicError("Overflow on register arithmetic");
}
if (result < 0) {
throw LogicError("Underflow on register arithmetic");
}
return static_cast<Reg>(result);
}
[[nodiscard]] constexpr Reg operator-(Reg reg, int num) {
return reg + (-num);
}
constexpr Reg operator++(Reg& reg) {
reg = reg + 1;
return reg;
}
constexpr Reg operator++(Reg& reg, int) {
const Reg copy{reg};
reg = reg + 1;
return copy;
}
[[nodiscard]] constexpr size_t RegIndex(Reg reg) noexcept {
return static_cast<size_t>(reg);
}
[[nodiscard]] constexpr bool IsAligned(Reg reg, size_t align) {
return RegIndex(reg) % align == 0 || reg == Reg::RZ;
}
} // namespace Shader::IR
template <>
struct fmt::formatter<Shader::IR::Reg> {
constexpr auto parse(format_parse_context& ctx) {
return ctx.begin();
}
template <typename FormatContext>
auto format(const Shader::IR::Reg& reg, FormatContext& ctx) const {
if (reg == Shader::IR::Reg::RZ) {
return fmt::format_to(ctx.out(), "RZ");
} else if (static_cast<int>(reg) >= 0 && static_cast<int>(reg) < static_cast<int>(Shader::IR::NUM_REGS)) {
return fmt::format_to(ctx.out(), "R{}", static_cast<int>(reg));
} else {
throw Shader::LogicError("Invalid register with raw value {}", static_cast<int>(reg));
}
}
};

View File

@@ -0,0 +1,37 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <array>
#include <string>
#include "shader_recompiler/frontend/ir/type.h"
namespace Shader::IR {
std::string NameOf(Type type) {
static constexpr std::array names{
"Opaque", "Label", "Reg", "Pred", "Attribute", "U1", "U8", "U16", "U32",
"U64", "F16", "F32", "F64", "U32x2", "U32x3", "U32x4", "F16x2", "F16x3",
"F16x4", "F32x2", "F32x3", "F32x4", "F64x2", "F64x3", "F64x4",
};
const size_t bits{static_cast<size_t>(type)};
if (bits == 0) {
return "Void";
}
std::string result;
for (size_t i = 0; i < names.size(); i++) {
if ((bits & (size_t{1} << i)) != 0) {
if (!result.empty()) {
result += '|';
}
result += names[i];
}
}
return result;
}
bool AreTypesCompatible(Type lhs, Type rhs) noexcept {
return lhs == rhs || lhs == Type::Opaque || rhs == Type::Opaque;
}
} // namespace Shader::IR

View File

@@ -0,0 +1,60 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <string>
#include <fmt/ranges.h>
#include "common/common_funcs.h"
#include "shader_recompiler/exception.h"
namespace Shader::IR {
enum class Type {
Void = 0,
Opaque = 1 << 0,
Reg = 1 << 1,
Pred = 1 << 2,
Attribute = 1 << 3,
Patch = 1 << 4,
U1 = 1 << 5,
U8 = 1 << 6,
U16 = 1 << 7,
U32 = 1 << 8,
U64 = 1 << 9,
F16 = 1 << 10,
F32 = 1 << 11,
F64 = 1 << 12,
U32x2 = 1 << 13,
U32x3 = 1 << 14,
U32x4 = 1 << 15,
F16x2 = 1 << 16,
F16x3 = 1 << 17,
F16x4 = 1 << 18,
F32x2 = 1 << 19,
F32x3 = 1 << 20,
F32x4 = 1 << 21,
F64x2 = 1 << 22,
F64x3 = 1 << 23,
F64x4 = 1 << 24,
};
DECLARE_ENUM_FLAG_OPERATORS(Type)
[[nodiscard]] std::string NameOf(Type type);
[[nodiscard]] bool AreTypesCompatible(Type lhs, Type rhs) noexcept;
} // namespace Shader::IR
template <>
struct fmt::formatter<Shader::IR::Type> {
constexpr auto parse(format_parse_context& ctx) {
return ctx.begin();
}
template <typename FormatContext>
auto format(const Shader::IR::Type& type, FormatContext& ctx) const {
return fmt::format_to(ctx.out(), "{}", NameOf(type));
}
};

View File

@@ -0,0 +1,97 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "shader_recompiler/frontend/ir/value.h"
namespace Shader::IR {
Value::Value(IR::Inst* value) noexcept : type{Type::Opaque}, inst{value} {}
Value::Value(IR::Reg value) noexcept : type{Type::Reg}, reg{value} {}
Value::Value(IR::Pred value) noexcept : type{Type::Pred}, pred{value} {}
Value::Value(IR::Attribute value) noexcept : type{Type::Attribute}, attribute{value} {}
Value::Value(IR::Patch value) noexcept : type{Type::Patch}, patch{value} {}
Value::Value(bool value) noexcept : type{Type::U1}, imm_u1{value} {}
Value::Value(u8 value) noexcept : type{Type::U8}, imm_u8{value} {}
Value::Value(u16 value) noexcept : type{Type::U16}, imm_u16{value} {}
Value::Value(u32 value) noexcept : type{Type::U32}, imm_u32{value} {}
Value::Value(f32 value) noexcept : type{Type::F32}, imm_f32{value} {}
Value::Value(u64 value) noexcept : type{Type::U64}, imm_u64{value} {}
Value::Value(f64 value) noexcept : type{Type::F64}, imm_f64{value} {}
IR::Type Value::Type() const noexcept {
if (IsPhi()) {
// The type of a phi node is stored in its flags
return inst->Flags<IR::Type>();
}
if (IsIdentity()) {
return inst->Arg(0).Type();
}
if (type == Type::Opaque) {
return inst->Type();
}
return type;
}
bool Value::operator==(const Value& other) const {
if (type != other.type) {
return false;
}
switch (type) {
case Type::Void:
return true;
case Type::Opaque:
return inst == other.inst;
case Type::Reg:
return reg == other.reg;
case Type::Pred:
return pred == other.pred;
case Type::Attribute:
return attribute == other.attribute;
case Type::Patch:
return patch == other.patch;
case Type::U1:
return imm_u1 == other.imm_u1;
case Type::U8:
return imm_u8 == other.imm_u8;
case Type::U16:
case Type::F16:
return imm_u16 == other.imm_u16;
case Type::U32:
case Type::F32:
return imm_u32 == other.imm_u32;
case Type::U64:
case Type::F64:
return imm_u64 == other.imm_u64;
case Type::U32x2:
case Type::U32x3:
case Type::U32x4:
case Type::F16x2:
case Type::F16x3:
case Type::F16x4:
case Type::F32x2:
case Type::F32x3:
case Type::F32x4:
case Type::F64x2:
case Type::F64x3:
case Type::F64x4:
break;
}
throw LogicError("Invalid type {}", type);
}
bool Value::operator!=(const Value& other) const {
return !operator==(other);
}
} // namespace Shader::IR

View File

@@ -0,0 +1,410 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <array>
#include <cstring>
#include <memory>
#include <type_traits>
#include <utility>
#include <vector>
#include <boost/container/small_vector.hpp>
#include <boost/intrusive/list.hpp>
#include "common/assert.h"
#include <numeric>
#include "common/common_types.h"
#include "shader_recompiler/exception.h"
#include "shader_recompiler/frontend/ir/attribute.h"
#include "shader_recompiler/frontend/ir/opcodes.h"
#include "shader_recompiler/frontend/ir/patch.h"
#include "shader_recompiler/frontend/ir/pred.h"
#include "shader_recompiler/frontend/ir/reg.h"
#include "shader_recompiler/frontend/ir/type.h"
namespace Shader::IR {
class Block;
class Inst;
struct AssociatedInsts;
class Value {
public:
Value() noexcept = default;
explicit Value(IR::Inst* value) noexcept;
explicit Value(IR::Reg value) noexcept;
explicit Value(IR::Pred value) noexcept;
explicit Value(IR::Attribute value) noexcept;
explicit Value(IR::Patch value) noexcept;
explicit Value(bool value) noexcept;
explicit Value(u8 value) noexcept;
explicit Value(u16 value) noexcept;
explicit Value(u32 value) noexcept;
explicit Value(f32 value) noexcept;
explicit Value(u64 value) noexcept;
explicit Value(f64 value) noexcept;
[[nodiscard]] bool IsIdentity() const noexcept;
[[nodiscard]] bool IsPhi() const noexcept;
[[nodiscard]] bool IsEmpty() const noexcept;
[[nodiscard]] bool IsImmediate() const noexcept;
[[nodiscard]] IR::Type Type() const noexcept;
[[nodiscard]] IR::Inst* Inst() const;
[[nodiscard]] IR::Inst* InstRecursive() const;
[[nodiscard]] IR::Inst* TryInstRecursive() const;
[[nodiscard]] IR::Value Resolve() const;
[[nodiscard]] IR::Reg Reg() const;
[[nodiscard]] IR::Pred Pred() const;
[[nodiscard]] IR::Attribute Attribute() const;
[[nodiscard]] IR::Patch Patch() const;
[[nodiscard]] bool U1() const;
[[nodiscard]] u8 U8() const;
[[nodiscard]] u16 U16() const;
[[nodiscard]] u32 U32() const;
[[nodiscard]] f32 F32() const;
[[nodiscard]] u64 U64() const;
[[nodiscard]] f64 F64() const;
[[nodiscard]] bool operator==(const Value& other) const;
[[nodiscard]] bool operator!=(const Value& other) const;
private:
IR::Type type{};
union {
IR::Inst* inst{};
IR::Reg reg;
IR::Pred pred;
IR::Attribute attribute;
IR::Patch patch;
bool imm_u1;
u8 imm_u8;
u16 imm_u16;
u32 imm_u32;
f32 imm_f32;
u64 imm_u64;
f64 imm_f64;
};
};
static_assert(static_cast<u32>(IR::Type::Void) == 0, "memset relies on IR::Type being zero");
static_assert(std::is_trivially_copyable_v<Value>);
template <IR::Type type_>
class TypedValue : public Value {
public:
TypedValue() = default;
template <IR::Type other_type>
requires((other_type & type_) != IR::Type::Void)
explicit(false) TypedValue(const TypedValue<other_type>& value) : Value(value) {}
explicit TypedValue(const Value& value) : Value(value) {
if ((value.Type() & type_) == IR::Type::Void) {
throw InvalidArgument("Incompatible types {} and {}", type_, value.Type());
}
}
explicit TypedValue(IR::Inst* inst_) : TypedValue(Value(inst_)) {}
};
class Inst : public boost::intrusive::list_base_hook<> {
public:
explicit Inst(IR::Opcode op_, u32 flags_) noexcept;
explicit Inst(const Inst& base);
~Inst();
Inst& operator=(const Inst&) = delete;
Inst& operator=(Inst&&) = delete;
Inst(Inst&&) = delete;
/// Get the number of uses this instruction has.
[[nodiscard]] int UseCount() const noexcept {
return use_count;
}
/// Determines whether this instruction has uses or not.
[[nodiscard]] bool HasUses() const noexcept {
return use_count > 0;
}
/// Get the opcode this microinstruction represents.
[[nodiscard]] IR::Opcode GetOpcode() const noexcept {
return op;
}
/// Determines if there is a pseudo-operation associated with this instruction.
[[nodiscard]] bool HasAssociatedPseudoOperation() const noexcept {
return associated_insts != nullptr;
}
/// Determines whether or not this instruction may have side effects.
[[nodiscard]] bool MayHaveSideEffects() const noexcept;
/// Determines whether or not this instruction is a pseudo-instruction.
/// Pseudo-instructions depend on their parent instructions for their semantics.
[[nodiscard]] bool IsPseudoInstruction() const noexcept;
/// Determines if all arguments of this instruction are immediates.
[[nodiscard]] bool AreAllArgsImmediates() const;
/// Gets a pseudo-operation associated with this instruction
[[nodiscard]] Inst* GetAssociatedPseudoOperation(IR::Opcode opcode);
/// Get the type this instruction returns.
[[nodiscard]] IR::Type Type() const;
/// Get the number of arguments this instruction has.
[[nodiscard]] size_t NumArgs() const {
return op == IR::Opcode::Phi ? phi_args.size() : NumArgsOf(op);
}
/// Get the value of a given argument index.
[[nodiscard]] Value Arg(size_t index) const noexcept {
if (op == IR::Opcode::Phi) {
return phi_args[index].second;
} else {
return args[index];
}
}
/// Set the value of a given argument index.
void SetArg(size_t index, Value value);
/// Get a pointer to the block of a phi argument.
[[nodiscard]] Block* PhiBlock(size_t index) const;
/// Add phi operand to a phi instruction.
void AddPhiOperand(Block* predecessor, const Value& value);
/// Orders the Phi arguments from farthest away to nearest.
void OrderPhiArgs();
void Invalidate();
void ClearArgs();
void ReplaceUsesWith(Value replacement);
void ReplaceOpcode(IR::Opcode opcode);
template <typename FlagsType>
requires(sizeof(FlagsType) <= sizeof(u32) && std::is_trivially_copyable_v<FlagsType>)
[[nodiscard]] FlagsType Flags() const noexcept {
FlagsType ret;
std::memcpy(reinterpret_cast<char*>(&ret), &flags, sizeof(ret));
return ret;
}
template <typename FlagsType>
requires(sizeof(FlagsType) <= sizeof(u32) && std::is_trivially_copyable_v<FlagsType>)
void SetFlags(FlagsType value) noexcept {
std::memcpy(&flags, &value, sizeof(value));
}
/// Intrusively store the host definition of this instruction.
template <typename DefinitionType>
void SetDefinition(DefinitionType def) {
definition = std::bit_cast<u32>(def);
}
/// Return the intrusively stored host definition of this instruction.
template <typename DefinitionType>
[[nodiscard]] DefinitionType Definition() const noexcept {
return std::bit_cast<DefinitionType>(definition);
}
/// Destructively remove one reference count from the instruction
/// Useful for register allocation
void DestructiveRemoveUsage() {
--use_count;
}
/// Destructively add usages to the instruction
/// Useful for register allocation
void DestructiveAddUsage(int count) {
use_count += count;
}
private:
struct NonTriviallyDummy {
NonTriviallyDummy() noexcept {}
};
void Use(const Value& value);
void UndoUse(const Value& value);
IR::Opcode op{};
int use_count{};
u32 flags{};
u32 definition{};
union {
NonTriviallyDummy dummy{};
boost::container::small_vector<std::pair<Block*, Value>, 2> phi_args;
std::array<Value, 5> args;
};
std::unique_ptr<AssociatedInsts> associated_insts;
};
static_assert(sizeof(Inst) <= 128, "Inst size unintentionally increased");
struct AssociatedInsts {
union {
Inst* in_bounds_inst;
Inst* sparse_inst;
Inst* zero_inst{};
};
Inst* sign_inst{};
Inst* carry_inst{};
Inst* overflow_inst{};
};
using U1 = TypedValue<Type::U1>;
using U8 = TypedValue<Type::U8>;
using U16 = TypedValue<Type::U16>;
using U32 = TypedValue<Type::U32>;
using U64 = TypedValue<Type::U64>;
using F16 = TypedValue<Type::F16>;
using F32 = TypedValue<Type::F32>;
using F64 = TypedValue<Type::F64>;
using U32U64 = TypedValue<Type::U32 | Type::U64>;
using F32F64 = TypedValue<Type::F32 | Type::F64>;
using U8U16U32U64 = TypedValue<Type::U8 | Type::U16 | Type::U32 | Type::U64>;
using U16U32U64 = TypedValue<Type::U16 | Type::U32 | Type::U64>;
using F16F32F64 = TypedValue<Type::F16 | Type::F32 | Type::F64>;
using UAny = TypedValue<Type::U8 | Type::U16 | Type::U32 | Type::U64>;
inline bool Value::IsIdentity() const noexcept {
return type == Type::Opaque && inst->GetOpcode() == Opcode::Identity;
}
inline bool Value::IsPhi() const noexcept {
return type == Type::Opaque && inst->GetOpcode() == Opcode::Phi;
}
inline bool Value::IsEmpty() const noexcept {
return type == Type::Void;
}
inline bool Value::IsImmediate() const noexcept {
IR::Type current_type{type};
const IR::Inst* current_inst{inst};
while (current_type == Type::Opaque && current_inst->GetOpcode() == Opcode::Identity) {
const Value& arg{current_inst->Arg(0)};
current_type = arg.type;
current_inst = arg.inst;
}
return current_type != Type::Opaque;
}
inline IR::Inst* Value::Inst() const {
DEBUG_ASSERT(type == Type::Opaque);
return inst;
}
inline IR::Inst* Value::InstRecursive() const {
DEBUG_ASSERT(type == Type::Opaque);
if (IsIdentity()) {
return inst->Arg(0).InstRecursive();
}
return inst;
}
inline IR::Inst* Value::TryInstRecursive() const {
if (IsIdentity()) {
return inst->Arg(0).TryInstRecursive();
}
return type == Type::Opaque ? inst : nullptr;
}
inline IR::Value Value::Resolve() const {
if (IsIdentity()) {
return inst->Arg(0).Resolve();
}
return *this;
}
inline IR::Reg Value::Reg() const {
DEBUG_ASSERT(type == Type::Reg);
return reg;
}
inline IR::Pred Value::Pred() const {
DEBUG_ASSERT(type == Type::Pred);
return pred;
}
inline IR::Attribute Value::Attribute() const {
DEBUG_ASSERT(type == Type::Attribute);
return attribute;
}
inline IR::Patch Value::Patch() const {
DEBUG_ASSERT(type == Type::Patch);
return patch;
}
inline bool Value::U1() const {
if (IsIdentity()) {
return inst->Arg(0).U1();
}
DEBUG_ASSERT(type == Type::U1);
return imm_u1;
}
inline u8 Value::U8() const {
if (IsIdentity()) {
return inst->Arg(0).U8();
}
DEBUG_ASSERT(type == Type::U8);
return imm_u8;
}
inline u16 Value::U16() const {
if (IsIdentity()) {
return inst->Arg(0).U16();
}
DEBUG_ASSERT(type == Type::U16);
return imm_u16;
}
inline u32 Value::U32() const {
if (IsIdentity()) {
return inst->Arg(0).U32();
}
DEBUG_ASSERT(type == Type::U32);
return imm_u32;
}
inline f32 Value::F32() const {
if (IsIdentity()) {
return inst->Arg(0).F32();
}
DEBUG_ASSERT(type == Type::F32);
return imm_f32;
}
inline u64 Value::U64() const {
if (IsIdentity()) {
return inst->Arg(0).U64();
}
DEBUG_ASSERT(type == Type::U64);
return imm_u64;
}
inline f64 Value::F64() const {
if (IsIdentity()) {
return inst->Arg(0).F64();
}
DEBUG_ASSERT(type == Type::F64);
return imm_f64;
}
[[nodiscard]] inline bool IsPhi(const Inst& inst) {
return inst.GetOpcode() == Opcode::Phi;
}
} // namespace Shader::IR

View File

@@ -0,0 +1,652 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <algorithm>
#include <array>
#include <optional>
#include <string>
#include <utility>
#include <fmt/ranges.h>
#include <ranges>
#include "shader_recompiler/exception.h"
#include "shader_recompiler/frontend/maxwell/control_flow.h"
#include "shader_recompiler/frontend/maxwell/decode.h"
#include "shader_recompiler/frontend/maxwell/indirect_branch_table_track.h"
#include "shader_recompiler/frontend/maxwell/location.h"
namespace Shader::Maxwell::Flow {
namespace {
struct Compare {
bool operator()(const Block& lhs, Location rhs) const noexcept {
return lhs.begin < rhs;
}
bool operator()(Location lhs, const Block& rhs) const noexcept {
return lhs < rhs.begin;
}
bool operator()(const Block& lhs, const Block& rhs) const noexcept {
return lhs.begin < rhs.begin;
}
};
u32 BranchOffset(Location pc, Instruction inst) {
return pc.Offset() + static_cast<u32>(inst.branch.Offset()) + 8u;
}
void Split(Block* old_block, Block* new_block, Location pc) {
if (pc <= old_block->begin || pc >= old_block->end) {
throw InvalidArgument("Invalid address to split={}", pc);
}
*new_block = Block{};
new_block->begin = pc;
new_block->end = old_block->end;
new_block->end_class = old_block->end_class;
new_block->cond = old_block->cond;
new_block->stack = old_block->stack;
new_block->branch_true = old_block->branch_true;
new_block->branch_false = old_block->branch_false;
new_block->function_call = old_block->function_call;
new_block->return_block = old_block->return_block;
new_block->branch_reg = old_block->branch_reg;
new_block->branch_offset = old_block->branch_offset;
new_block->indirect_branches = std::move(old_block->indirect_branches);
const Location old_begin{old_block->begin};
Stack old_stack{std::move(old_block->stack)};
*old_block = Block{};
old_block->begin = old_begin;
old_block->end = pc;
old_block->end_class = EndClass::Branch;
old_block->cond = IR::Condition(true);
old_block->stack = old_stack;
old_block->branch_true = new_block;
old_block->branch_false = nullptr;
}
Token OpcodeToken(Opcode opcode) {
switch (opcode) {
case Opcode::PBK:
case Opcode::BRK:
return Token::PBK;
case Opcode::PCNT:
case Opcode::CONT:
return Token::PCNT;
case Opcode::PEXIT:
case Opcode::EXIT:
return Token::PEXIT;
case Opcode::PLONGJMP:
case Opcode::LONGJMP:
return Token::PLONGJMP;
case Opcode::PRET:
case Opcode::RET:
case Opcode::CAL:
return Token::PRET;
case Opcode::SSY:
case Opcode::SYNC:
return Token::SSY;
default:
throw InvalidArgument("{}", opcode);
}
}
bool IsAbsoluteJump(Opcode opcode) {
switch (opcode) {
case Opcode::JCAL:
case Opcode::JMP:
case Opcode::JMX:
return true;
default:
return false;
}
}
bool HasFlowTest(Opcode opcode) {
switch (opcode) {
case Opcode::BRA:
case Opcode::BRX:
case Opcode::EXIT:
case Opcode::JMP:
case Opcode::JMX:
case Opcode::KIL:
case Opcode::BRK:
case Opcode::CONT:
case Opcode::LONGJMP:
case Opcode::RET:
case Opcode::SYNC:
return true;
case Opcode::CAL:
case Opcode::JCAL:
return false;
default:
throw InvalidArgument("Invalid branch {}", opcode);
}
}
std::string NameOf(const Block& block) {
if (block.begin.IsVirtual()) {
return fmt::format("\"Virtual {}\"", block.begin);
} else {
return fmt::format("\"{}\"", block.begin);
}
}
} // Anonymous namespace
void Stack::Push(Token token, Location target) {
entries.push_back({
.token = token,
.target{target},
});
}
std::pair<Location, Stack> Stack::Pop(Token token) const {
const std::optional<Location> pc{Peek(token)};
if (!pc) {
throw LogicError("Token could not be found");
}
return {*pc, Remove(token)};
}
std::optional<Location> Stack::Peek(Token token) const {
const auto it{std::find_if(entries.rbegin(), entries.rend(),
[token](const auto& entry) { return entry.token == token; })};
if (it == entries.rend()) {
return std::nullopt;
}
return it->target;
}
Stack Stack::Remove(Token token) const {
const auto it{std::find_if(entries.rbegin(), entries.rend(),
[token](const auto& entry) { return entry.token == token; })};
const auto pos{std::distance(entries.rbegin(), it)};
Stack result;
result.entries.insert(result.entries.end(), entries.begin(), entries.end() - pos - 1);
return result;
}
bool Block::Contains(Location pc) const noexcept {
return pc >= begin && pc < end;
}
Function::Function(ObjectPool<Block>& block_pool, Location start_address)
: entrypoint{start_address} {
Label& label{labels.emplace_back()};
label.address = start_address;
label.block = block_pool.Create(Block{});
label.block->begin = start_address;
label.block->end = start_address;
label.block->end_class = EndClass::Branch;
label.block->cond = IR::Condition(true);
label.block->branch_true = nullptr;
label.block->branch_false = nullptr;
}
CFG::CFG(Environment& env_, ObjectPool<Block>& block_pool_, Location start_address,
bool exits_to_dispatcher_)
: env{env_}, block_pool{block_pool_}, program_start{start_address}, exits_to_dispatcher{
exits_to_dispatcher_} {
if (exits_to_dispatcher) {
dispatch_block = block_pool.Create(Block{});
dispatch_block->begin = {};
dispatch_block->end = {};
dispatch_block->end_class = EndClass::Exit;
dispatch_block->cond = IR::Condition(true);
dispatch_block->stack = {};
dispatch_block->branch_true = nullptr;
dispatch_block->branch_false = nullptr;
}
functions.emplace_back(block_pool, start_address);
for (FunctionId function_id = 0; function_id < functions.size(); ++function_id) {
while (!functions[function_id].labels.empty()) {
Function& function{functions[function_id]};
Label label{function.labels.back()};
function.labels.pop_back();
AnalyzeLabel(function_id, label);
}
}
if (exits_to_dispatcher) {
const auto last_block{functions[0].blocks.rbegin()};
dispatch_block->begin = last_block->end + 1;
dispatch_block->end = last_block->end + 1;
functions[0].blocks.insert(*dispatch_block);
}
}
void CFG::AnalyzeLabel(FunctionId function_id, Label& label) {
if (InspectVisitedBlocks(function_id, label)) {
// Label address has been visited
return;
}
// Try to find the next block
Function* const function{&functions[function_id]};
Location pc{label.address};
const auto next_it{function->blocks.upper_bound(pc, Compare{})};
const bool is_last{next_it == function->blocks.end()};
Block* const next{is_last ? nullptr : &*next_it};
// Insert before the next block
Block* const block{label.block};
// Analyze instructions until it reaches an already visited block or there's a branch
bool is_branch{false};
while (!next || pc < next->begin) {
is_branch = AnalyzeInst(block, function_id, pc) == AnalysisState::Branch;
if (is_branch) {
break;
}
++pc;
}
if (!is_branch) {
// If the block finished without a branch,
// it means that the next instruction is already visited, jump to it
block->end = pc;
block->cond = IR::Condition{true};
block->branch_true = next;
block->branch_false = nullptr;
}
// Function's pointer might be invalid, resolve it again
// Insert the new block
functions[function_id].blocks.insert(*block);
}
bool CFG::InspectVisitedBlocks(FunctionId function_id, const Label& label) {
const Location pc{label.address};
Function& function{functions[function_id]};
const auto it{
std::ranges::find_if(function.blocks, [pc](auto& block) { return block.Contains(pc); })};
if (it == function.blocks.end()) {
// Address has not been visited
return false;
}
Block* const visited_block{&*it};
if (visited_block->begin == pc) {
throw LogicError("Dangling block");
}
Block* const new_block{label.block};
Split(visited_block, new_block, pc);
function.blocks.insert(it, *new_block);
return true;
}
CFG::AnalysisState CFG::AnalyzeInst(Block* block, FunctionId function_id, Location pc) {
const Instruction inst{env.ReadInstruction(pc.Offset())};
const Opcode opcode{Decode(inst.raw)};
switch (opcode) {
case Opcode::BRA:
case Opcode::JMP:
case Opcode::RET:
if (!AnalyzeBranch(block, function_id, pc, inst, opcode)) {
return AnalysisState::Continue;
}
switch (opcode) {
case Opcode::BRA:
case Opcode::JMP:
AnalyzeBRA(block, function_id, pc, inst, IsAbsoluteJump(opcode));
break;
case Opcode::RET:
block->end_class = EndClass::Return;
break;
default:
break;
}
block->end = pc;
return AnalysisState::Branch;
case Opcode::BRK:
case Opcode::CONT:
case Opcode::LONGJMP:
case Opcode::SYNC: {
if (!AnalyzeBranch(block, function_id, pc, inst, opcode)) {
return AnalysisState::Continue;
}
const auto [stack_pc, new_stack]{block->stack.Pop(OpcodeToken(opcode))};
block->branch_true = AddLabel(block, new_stack, stack_pc, function_id);
block->end = pc;
return AnalysisState::Branch;
}
case Opcode::KIL: {
const Predicate pred{inst.Pred()};
const auto ir_pred{static_cast<IR::Pred>(pred.index)};
const IR::Condition cond{inst.branch.flow_test, ir_pred, pred.negated};
AnalyzeCondInst(block, function_id, pc, EndClass::Kill, cond);
return AnalysisState::Branch;
}
case Opcode::PBK:
case Opcode::PCNT:
case Opcode::PEXIT:
case Opcode::PLONGJMP:
case Opcode::SSY:
block->stack.Push(OpcodeToken(opcode), BranchOffset(pc, inst));
return AnalysisState::Continue;
case Opcode::BRX:
case Opcode::JMX:
return AnalyzeBRX(block, pc, inst, IsAbsoluteJump(opcode), function_id);
case Opcode::EXIT:
return AnalyzeEXIT(block, function_id, pc, inst);
case Opcode::PRET:
throw NotImplementedException("PRET flow analysis");
case Opcode::CAL:
case Opcode::JCAL: {
const bool is_absolute{IsAbsoluteJump(opcode)};
const Location cal_pc{is_absolute ? inst.branch.Absolute() : BranchOffset(pc, inst)};
// Technically CAL pushes into PRET, but that's implicit in the function call for us
// Insert the function into the list if it doesn't exist
const auto it{std::ranges::find(functions, cal_pc, &Function::entrypoint)};
const bool exists{it != functions.end()};
const FunctionId call_id{exists ? static_cast<size_t>(std::distance(functions.begin(), it))
: functions.size()};
if (!exists) {
functions.emplace_back(block_pool, cal_pc);
}
block->end_class = EndClass::Call;
block->function_call = call_id;
block->return_block = AddLabel(block, block->stack, pc + 1, function_id);
block->end = pc;
return AnalysisState::Branch;
}
default:
break;
}
const Predicate pred{inst.Pred()};
if (pred == Predicate{true} || pred == Predicate{false}) {
return AnalysisState::Continue;
}
const IR::Condition cond{static_cast<IR::Pred>(pred.index), pred.negated};
AnalyzeCondInst(block, function_id, pc, EndClass::Branch, cond);
return AnalysisState::Branch;
}
void CFG::AnalyzeCondInst(Block* block, FunctionId function_id, Location pc,
EndClass insn_end_class, IR::Condition cond) {
if (block->begin != pc) {
// If the block doesn't start in the conditional instruction
// mark it as a label to visit it later
block->end = pc;
block->cond = IR::Condition{true};
block->branch_true = AddLabel(block, block->stack, pc, function_id);
block->branch_false = nullptr;
return;
}
// Create a virtual block and a conditional block
Block* const conditional_block{block_pool.Create()};
Block virtual_block{};
virtual_block.begin = block->begin.Virtual();
virtual_block.end = block->begin.Virtual();
virtual_block.end_class = EndClass::Branch;
virtual_block.stack = block->stack;
virtual_block.cond = cond;
virtual_block.branch_true = conditional_block;
virtual_block.branch_false = nullptr;
// Save the contents of the visited block in the conditional block
*conditional_block = std::move(*block);
// Impersonate the visited block with a virtual block
*block = std::move(virtual_block);
// Set the end properties of the conditional instruction
conditional_block->end = pc + 1;
conditional_block->end_class = insn_end_class;
// Add a label to the instruction after the conditional instruction
Block* const endif_block{AddLabel(conditional_block, block->stack, pc + 1, function_id)};
// Branch to the next instruction from the virtual block
block->branch_false = endif_block;
// And branch to it from the conditional instruction if it is a branch or a kill instruction
// Kill instructions are considered a branch because they demote to a helper invocation and
// execution may continue.
if (insn_end_class == EndClass::Branch || insn_end_class == EndClass::Kill) {
conditional_block->cond = IR::Condition{true};
conditional_block->branch_true = endif_block;
conditional_block->branch_false = nullptr;
}
// Finally insert the condition block into the list of blocks
functions[function_id].blocks.insert(*conditional_block);
}
bool CFG::AnalyzeBranch(Block* block, FunctionId function_id, Location pc, Instruction inst,
Opcode opcode) {
if (inst.branch.is_cbuf) {
throw NotImplementedException("Branch with constant buffer offset");
}
const Predicate pred{inst.Pred()};
if (pred == Predicate{false}) {
return false;
}
const bool has_flow_test{HasFlowTest(opcode)};
const IR::FlowTest flow_test{has_flow_test ? inst.branch.flow_test.Value() : IR::FlowTest::T};
if (pred != Predicate{true} || flow_test != IR::FlowTest::T) {
block->cond = IR::Condition(flow_test, static_cast<IR::Pred>(pred.index), pred.negated);
block->branch_false = AddLabel(block, block->stack, pc + 1, function_id);
} else {
block->cond = IR::Condition{true};
}
return true;
}
void CFG::AnalyzeBRA(Block* block, FunctionId function_id, Location pc, Instruction inst,
bool is_absolute) {
const Location bra_pc{is_absolute ? inst.branch.Absolute() : BranchOffset(pc, inst)};
block->branch_true = AddLabel(block, block->stack, bra_pc, function_id);
}
CFG::AnalysisState CFG::AnalyzeBRX(Block* block, Location pc, Instruction inst, bool is_absolute,
FunctionId function_id) {
const std::optional brx_table{TrackIndirectBranchTable(env, pc, program_start)};
if (!brx_table) {
// (amd-fix, diagnosis) the instructions before the jump, newest first: the pattern the tracker did not know
// (OoT's foliage mesh programs fail here)
std::string seen;
Location at{pc};
for (int k = 0; k < 14 && at >= program_start; ++k, --at) {
const u64 raw{env.ReadInstruction(at.Offset())};
seen += fmt::format(" {}:{:016x}", NameOf(Decode(raw)), raw);
}
throw NotImplementedException("Failed to track indirect branch at {:#x}:{}", pc.Offset(), seen);
}
const IR::FlowTest flow_test{inst.branch.flow_test};
const Predicate pred{inst.Pred()};
if (flow_test != IR::FlowTest::T || pred != Predicate{true}) {
throw NotImplementedException("Conditional indirect branch");
}
std::vector<u32> targets;
targets.reserve(brx_table->num_entries);
for (u32 i = 0; i < brx_table->num_entries; ++i) {
u32 target{env.ReadCbufValue(brx_table->cbuf_index, brx_table->cbuf_offset + i * 4)};
if (!is_absolute) {
target += pc.Offset();
}
target += static_cast<u32>(brx_table->branch_offset);
target += 8;
targets.push_back(target);
}
std::ranges::sort(targets);
targets.erase(std::unique(targets.begin(), targets.end()), targets.end());
block->indirect_branches.reserve(targets.size());
for (const u32 target : targets) {
Block* const branch{AddLabel(block, block->stack, env.TranslateProgramAddress(target), function_id)};
block->indirect_branches.push_back({
.block = branch,
.address = target,
});
}
block->cond = IR::Condition{true};
block->end = pc + 1;
block->end_class = EndClass::IndirectBranch;
block->branch_reg = brx_table->branch_reg;
block->branch_offset = brx_table->branch_offset + 8;
if (!is_absolute) {
block->branch_offset += pc.Offset();
}
return AnalysisState::Branch;
}
CFG::AnalysisState CFG::AnalyzeEXIT(Block* block, FunctionId function_id, Location pc,
Instruction inst) {
const IR::FlowTest flow_test{inst.branch.flow_test};
const Predicate pred{inst.Pred()};
if (pred == Predicate{false} || flow_test == IR::FlowTest::F) {
// EXIT will never be taken
return AnalysisState::Continue;
}
if (exits_to_dispatcher && function_id != 0) {
throw NotImplementedException("Dispatch EXIT on external function");
}
if (pred != Predicate{true} || flow_test != IR::FlowTest::T) {
if (block->stack.Peek(Token::PEXIT).has_value()) {
throw NotImplementedException("Conditional EXIT with PEXIT token");
}
const IR::Condition cond{flow_test, static_cast<IR::Pred>(pred.index), pred.negated};
if (exits_to_dispatcher) {
block->end = pc;
block->end_class = EndClass::Branch;
block->cond = cond;
block->branch_true = dispatch_block;
block->branch_false = AddLabel(block, block->stack, pc + 1, function_id);
return AnalysisState::Branch;
}
AnalyzeCondInst(block, function_id, pc, EndClass::Exit, cond);
return AnalysisState::Branch;
}
if (const std::optional<Location> exit_pc{block->stack.Peek(Token::PEXIT)}) {
const Stack popped_stack{block->stack.Remove(Token::PEXIT)};
block->cond = IR::Condition{true};
block->branch_true = AddLabel(block, popped_stack, *exit_pc, function_id);
block->branch_false = nullptr;
return AnalysisState::Branch;
}
if (exits_to_dispatcher) {
block->cond = IR::Condition{true};
block->end = pc;
block->end_class = EndClass::Branch;
block->branch_true = dispatch_block;
block->branch_false = nullptr;
return AnalysisState::Branch;
}
block->end = pc + 1;
block->end_class = EndClass::Exit;
return AnalysisState::Branch;
}
Block* CFG::AddLabel(Block* block, Stack stack, Location pc, FunctionId function_id) {
Function& function{functions[function_id]};
if (block->begin == pc) {
// Jumps to itself
return block;
}
if (const auto it{function.blocks.find(pc, Compare{})}; it != function.blocks.end()) {
// Block already exists and it has been visited
if (function.blocks.begin() != it) {
// Check if the previous node is the virtual variant of the label
// This won't exist if a virtual node is not needed or it hasn't been visited
// If it hasn't been visited and a virtual node is needed, this will still behave as
// expected because the node impersonated with its virtual node.
const auto prev{std::prev(it)};
if (it->begin.Virtual() == prev->begin) {
return &*prev;
}
}
return &*it;
}
// Make sure we don't insert the same layer twice
const auto label_it{std::ranges::find(function.labels, pc, &Label::address)};
if (label_it != function.labels.end()) {
return label_it->block;
}
Block* const new_block{block_pool.Create()};
new_block->begin = pc;
new_block->end = pc;
new_block->end_class = EndClass::Branch;
new_block->cond = IR::Condition(true);
new_block->stack = stack;
new_block->branch_true = nullptr;
new_block->branch_false = nullptr;
function.labels.push_back(Label{
.address{pc},
.block = new_block,
.stack{std::move(stack)},
});
return new_block;
}
std::string CFG::Dot() const {
int node_uid{0};
std::string dot{"digraph shader {\n"};
for (const Function& function : functions) {
dot += fmt::format("\tsubgraph cluster_{} {{\n", function.entrypoint);
dot += fmt::format("\t\tnode [style=filled];\n");
for (const Block& block : function.blocks) {
const std::string name{NameOf(block)};
const auto add_branch = [&](Block* branch, bool add_label) {
dot += fmt::format("\t\t{}->{}", name, NameOf(*branch));
if (add_label && block.cond != IR::Condition{true} &&
block.cond != IR::Condition{false}) {
dot += fmt::format(" [label=\"{}\"]", block.cond);
}
dot += '\n';
};
dot += fmt::format("\t\t{};\n", name);
switch (block.end_class) {
case EndClass::Branch:
if (block.cond != IR::Condition{false}) {
add_branch(block.branch_true, true);
}
if (block.cond != IR::Condition{true}) {
add_branch(block.branch_false, false);
}
break;
case EndClass::IndirectBranch:
for (const IndirectBranch& branch : block.indirect_branches) {
add_branch(branch.block, false);
}
break;
case EndClass::Call:
dot += fmt::format("\t\t{}->N{};\n", name, node_uid);
dot += fmt::format("\t\tN{}->{};\n", node_uid, NameOf(*block.return_block));
dot += fmt::format("\t\tN{} [label=\"Call {}\"][shape=square][style=stripped];\n",
node_uid, block.function_call);
dot += '\n';
++node_uid;
break;
case EndClass::Exit:
dot += fmt::format("\t\t{}->N{};\n", name, node_uid);
dot += fmt::format("\t\tN{} [label=\"Exit\"][shape=square][style=stripped];\n",
node_uid);
++node_uid;
break;
case EndClass::Return:
dot += fmt::format("\t\t{}->N{};\n", name, node_uid);
dot += fmt::format("\t\tN{} [label=\"Return\"][shape=square][style=stripped];\n",
node_uid);
++node_uid;
break;
case EndClass::Kill:
dot += fmt::format("\t\t{}->N{};\n", name, node_uid);
dot += fmt::format("\t\tN{} [label=\"Kill\"][shape=square][style=stripped];\n",
node_uid);
++node_uid;
break;
}
}
if (function.entrypoint == 8) {
dot += fmt::format("\t\tlabel = \"main\";\n");
} else {
dot += fmt::format("\t\tlabel = \"Function {}\";\n", function.entrypoint);
}
dot += "\t}\n";
}
if (!functions.empty()) {
auto& function{functions.front()};
if (function.blocks.empty()) {
dot += "Start;\n";
} else {
dot += fmt::format("\tStart -> {};\n", NameOf(*function.blocks.begin()));
}
dot += fmt::format("\tStart [shape=diamond];\n");
}
dot += "}\n";
return dot;
}
} // namespace Shader::Maxwell::Flow

View File

@@ -0,0 +1,168 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <optional>
#include <span>
#include <string>
#include <vector>
#include <boost/container/small_vector.hpp>
#include <boost/intrusive/set.hpp>
#include "shader_recompiler/environment.h"
#include "shader_recompiler/frontend/ir/condition.h"
#include "shader_recompiler/frontend/ir/reg.h"
#include "shader_recompiler/frontend/maxwell/instruction.h"
#include "shader_recompiler/frontend/maxwell/location.h"
#include "shader_recompiler/frontend/maxwell/opcodes.h"
#include "shader_recompiler/object_pool.h"
namespace Shader::Maxwell::Flow {
struct Block;
using FunctionId = size_t;
enum class EndClass {
Branch,
IndirectBranch,
Call,
Exit,
Return,
Kill,
};
enum class Token {
SSY,
PBK,
PEXIT,
PRET,
PCNT,
PLONGJMP,
};
struct StackEntry {
auto operator<=>(const StackEntry&) const noexcept = default;
Token token;
Location target;
};
class Stack {
public:
void Push(Token token, Location target);
[[nodiscard]] std::pair<Location, Stack> Pop(Token token) const;
[[nodiscard]] std::optional<Location> Peek(Token token) const;
[[nodiscard]] Stack Remove(Token token) const;
private:
std::vector<StackEntry> entries;
};
struct IndirectBranch {
Block* block;
u32 address;
};
struct Block : boost::intrusive::set_base_hook<
// Normal link is ~2.5% faster compared to safe link
boost::intrusive::link_mode<boost::intrusive::normal_link>> {
[[nodiscard]] bool Contains(Location pc) const noexcept;
bool operator<(const Block& rhs) const noexcept {
return begin < rhs.begin;
}
Location begin;
Location end;
EndClass end_class{};
IR::Condition cond{};
Stack stack;
Block* branch_true{};
Block* branch_false{};
FunctionId function_call{};
Block* return_block{};
IR::Reg branch_reg{};
s32 branch_offset{};
std::vector<IndirectBranch> indirect_branches;
};
struct Label {
Location address;
Block* block;
Stack stack;
};
struct Function {
explicit Function(ObjectPool<Block>& block_pool, Location start_address);
Location entrypoint;
boost::container::small_vector<Label, 16> labels;
boost::intrusive::set<Block> blocks;
};
class CFG {
enum class AnalysisState {
Branch,
Continue,
};
public:
explicit CFG(Environment& env, ObjectPool<Block>& block_pool, Location start_address,
bool exits_to_dispatcher = false);
CFG& operator=(const CFG&) = delete;
CFG(const CFG&) = delete;
CFG& operator=(CFG&&) = delete;
CFG(CFG&&) = delete;
[[nodiscard]] std::string Dot() const;
[[nodiscard]] std::span<const Function> Functions() const noexcept {
return std::span(functions.data(), functions.size());
}
[[nodiscard]] std::span<Function> Functions() noexcept {
return std::span(functions.data(), functions.size());
}
[[nodiscard]] bool ExitsToDispatcher() const {
return exits_to_dispatcher;
}
private:
void AnalyzeLabel(FunctionId function_id, Label& label);
/// Inspect already visited blocks.
/// Return true when the block has already been visited
bool InspectVisitedBlocks(FunctionId function_id, const Label& label);
AnalysisState AnalyzeInst(Block* block, FunctionId function_id, Location pc);
void AnalyzeCondInst(Block* block, FunctionId function_id, Location pc, EndClass insn_end_class,
IR::Condition cond);
/// Return true when the branch instruction is confirmed to be a branch
bool AnalyzeBranch(Block* block, FunctionId function_id, Location pc, Instruction inst,
Opcode opcode);
void AnalyzeBRA(Block* block, FunctionId function_id, Location pc, Instruction inst,
bool is_absolute);
AnalysisState AnalyzeBRX(Block* block, Location pc, Instruction inst, bool is_absolute,
FunctionId function_id);
AnalysisState AnalyzeEXIT(Block* block, FunctionId function_id, Location pc, Instruction inst);
/// Return the branch target block id
Block* AddLabel(Block* block, Stack stack, Location pc, FunctionId function_id);
Environment& env;
ObjectPool<Block>& block_pool;
boost::container::small_vector<Function, 1> functions;
Location program_start;
bool exits_to_dispatcher{};
Block* dispatch_block{};
};
} // namespace Shader::Maxwell::Flow

View File

@@ -0,0 +1,53 @@
// SPDX-FileCopyrightText: Copyright 2026 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <algorithm>
#include <array>
#include <bit>
#include <memory>
#include <ranges>
#include <string_view>
#include "common/assert.h"
#include "common/common_types.h"
#include "shader_recompiler/exception.h"
#include "shader_recompiler/frontend/maxwell/decode.h"
#include "shader_recompiler/frontend/maxwell/opcodes.h"
namespace Shader::Maxwell {
consteval std::pair<u64, u64> MaskValueFromEncoding(const char data[20]) noexcept {
u64 mask = 0, value = 0, bit = u64(1) << 63;
for (int i = 0; i < 20; ++i)
switch (data[i]) {
case '0':
mask |= bit;
bit >>= 1;
break;
case '1':
mask |= bit;
value |= bit;
bit >>= 1;
break;
case '-':
bit >>= 1;
break;
default:
break;
}
return { mask, value };
}
Opcode Decode(u64 insn) {
#define INST(name, cute, encode) \
if (auto const p = MaskValueFromEncoding(encode); (insn & p.first) == p.second) \
return Opcode::name;
#include "maxwell.inc"
#undef INST
ASSERT_MSG(false, "Invalid insn {:#016x}", insn);
return Opcode::NOP;
}
} // namespace Shader::Maxwell

View File

@@ -0,0 +1,13 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "common/common_types.h"
#include "shader_recompiler/frontend/maxwell/opcodes.h"
namespace Shader::Maxwell {
[[nodiscard]] Opcode Decode(u64 insn);
} // namespace Shader::Maxwell

View File

@@ -0,0 +1,191 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <cstdio>
#include <cstdlib>
#include <optional>
#include "common/common_types.h"
#include "shader_recompiler/exception.h"
#include "shader_recompiler/frontend/maxwell/decode.h"
#include "shader_recompiler/frontend/maxwell/indirect_branch_table_track.h"
#include "shader_recompiler/frontend/maxwell/opcodes.h"
#include "shader_recompiler/frontend/maxwell/translate/impl/load_constant.h"
namespace Shader::Maxwell {
namespace {
union Encoding {
u64 raw;
BitField<0, 8, IR::Reg> dest_reg;
BitField<8, 8, IR::Reg> src_reg;
BitField<20, 19, u64> immediate;
BitField<56, 1, u64> is_negative;
BitField<20, 24, s64> brx_offset;
};
template <typename Callable>
std::optional<u64> Track(Environment& env, Location block_begin, Location& pos, Callable&& func) {
while (pos >= block_begin) {
const u64 insn{env.ReadInstruction(pos.Offset())};
--pos;
if (func(insn, Decode(insn))) {
return insn;
}
}
return std::nullopt;
}
std::optional<u64> TrackLDC(Environment& env, Location block_begin, Location& pos,
IR::Reg brx_reg) {
return Track(env, block_begin, pos, [brx_reg](u64 insn, Opcode opcode) {
const LDC::Encoding ldc{insn};
return opcode == Opcode::LDC && ldc.dest_reg == brx_reg && ldc.size == LDC::Size::B32 &&
ldc.mode == LDC::Mode::Default;
});
}
std::optional<u64> TrackSHL(Environment& env, Location block_begin, Location& pos,
IR::Reg ldc_reg) {
return Track(env, block_begin, pos, [ldc_reg](u64 insn, Opcode opcode) {
const Encoding shl{insn};
return opcode == Opcode::SHL_imm && shl.dest_reg == ldc_reg;
});
}
std::optional<u64> TrackIMNMX(Environment& env, Location block_begin, Location& pos,
IR::Reg shl_reg) {
return Track(env, block_begin, pos, [shl_reg](u64 insn, Opcode opcode) {
const Encoding imnmx{insn};
return opcode == Opcode::IMNMX_imm && imnmx.dest_reg == shl_reg;
});
}
} // Anonymous namespace
std::optional<IndirectBranchTableInfo> TrackIndirectBranchTable(Environment& env, Location brx_pos,
Location block_begin) {
const u64 brx_insn{env.ReadInstruction(brx_pos.Offset())};
const Opcode brx_opcode{Decode(brx_insn)};
if (brx_opcode != Opcode::BRX && brx_opcode != Opcode::JMX) {
throw LogicError("Tracked instruction is not BRX or JMX");
}
const IR::Reg brx_reg{Encoding{brx_insn}.src_reg};
const s32 brx_offset{static_cast<s32>(Encoding{brx_insn}.brx_offset)};
Location pos{brx_pos};
const std::optional<u64> ldc_insn{TrackLDC(env, block_begin, pos, brx_reg)};
if (!ldc_insn) {
return std::nullopt;
}
const LDC::Encoding ldc{*ldc_insn};
const u32 cbuf_index{static_cast<u32>(ldc.index)};
const u32 cbuf_offset{static_cast<u32>(static_cast<s32>(ldc.offset.Value()))};
const IR::Reg ldc_reg{ldc.src_reg};
const std::optional<u64> shl_insn{TrackSHL(env, block_begin, pos, ldc_reg)};
if (!shl_insn) {
return std::nullopt;
}
const Encoding shl{*shl_insn};
const IR::Reg shl_reg{shl.src_reg};
const Location after_shl{pos};
const std::optional<u64> imnmx_insn{TrackIMNMX(env, block_begin, pos, shl_reg)};
u32 max_index{};
if (imnmx_insn) {
const Encoding imnmx{*imnmx_insn};
if (imnmx.is_negative != 0) {
return std::nullopt;
}
max_index = static_cast<u32>(imnmx.immediate.Value());
} else {
// (amd-fix) NVN2 programs lowered to Maxwell clamp the index with ISETP + SEL against a register a MOV32I set
// (SEL R, idx, Rbound, P), not with an IMNMX: OoT's 30 foliage mesh programs were rejected here
// A bound read from a constant buffer (MOV from cbuf) is data, not code: OoT's 2 remaining foliage mesh
// programs clamp so. The table is then measured by reading it until an entry is not a plausible target, and
// capped by the bound when its constant is known at translation time.
const auto table_count{[&](std::optional<u64> movc, u32 extra) -> u32 {
const u32 t_index{static_cast<u32>(ldc.index)};
const s64 base{Decode(brx_insn) == Opcode::JMX ? 0 : static_cast<s64>(brx_pos.Offset())};
u32 count{0};
for (u32 i = 0; i < 256; ++i) {
const s32 rel{static_cast<s32>(env.ReadCbufValue(t_index, cbuf_offset + i * 4))};
const s64 target{base + rel + brx_offset + 8};
const bool ok{(rel & 7) == 0 && rel > -0x40000 && rel < 0x40000 &&
target >= static_cast<s64>(block_begin.Offset()) && target < 0x400000};
if (!ok) break;
++count;
}
u32 bound{0};
if (movc) {
bound = env.ReadCbufValue(static_cast<u32>((*movc >> 34) & 0x1f),
static_cast<u32>(((*movc >> 20) & 0x3fff) * 4));
if (bound > 0 && bound + extra < count) count = bound + extra;
}
if (std::getenv("RX_VK_LOG_BRX")) {
std::fprintf(stderr, "brx: cbuf-bounded table at %#x: c%u[%#x], %u entries (bound %u)\n",
static_cast<unsigned>(brx_pos.Offset()), t_index, cbuf_offset, count, bound);
}
return count;
}};
const auto find_movc{[&](Location from, IR::Reg reg) {
return Track(env, block_begin, from, [reg](u64 insn, Opcode opcode) {
return opcode == Opcode::MOV_cbuf && Encoding{insn}.dest_reg == reg;
});
}};
Location at{after_shl};
const std::optional<u64> sel_insn{Track(env, block_begin, at, [shl_reg](u64 insn, Opcode opcode) {
return opcode == Opcode::SEL_reg && Encoding{insn}.dest_reg == shl_reg;
})};
u32 count{0};
bool measured{false};
if (!sel_insn) {
measured = true;
// ISETP.LT against the bound and a branch around the jump: indices 0..bound-1
Location at2{after_shl};
const std::optional<u64> isetp{Track(env, block_begin, at2, [shl_reg](u64 insn, Opcode opcode) {
return opcode == Opcode::ISETP_reg && Encoding{insn}.src_reg == shl_reg;
})};
count = table_count(isetp ? find_movc(at2, static_cast<IR::Reg>((*isetp >> 20) & 0xff))
: std::optional<u64>{}, 0);
} else {
const IR::Reg bound_reg{static_cast<IR::Reg>((*sel_insn >> 20) & 0xff)};
Location at_mov{at};
const std::optional<u64> mov_insn{Track(env, block_begin, at_mov, [bound_reg](u64 insn, Opcode opcode) {
return (opcode == Opcode::MOV32I || opcode == Opcode::MOV_cbuf) && Encoding{insn}.dest_reg == bound_reg;
})};
if (!mov_insn) {
return std::nullopt;
}
if (Decode(*mov_insn) == Opcode::MOV32I) {
max_index = static_cast<u32>((*mov_insn >> 20) & 0xffffffffULL);
if (max_index > 255) { // (a table that large is not a clamp of this kind)
return std::nullopt;
}
} else {
measured = true;
count = table_count(mov_insn, 1); // SEL clamps to the bound itself: indices 0..bound
}
}
if (count) {
return IndirectBranchTableInfo{
.cbuf_index = cbuf_index,
.cbuf_offset = cbuf_offset,
.num_entries = count,
.branch_offset = brx_offset,
.branch_reg = brx_reg,
};
}
if (measured) { // (a measured table with no plausible entry)
return std::nullopt;
}
}
return IndirectBranchTableInfo{
.cbuf_index = cbuf_index,
.cbuf_offset = cbuf_offset,
.num_entries = max_index + 1,
.branch_offset = brx_offset,
.branch_reg = brx_reg,
};
}
} // namespace Shader::Maxwell

View File

@@ -0,0 +1,26 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <optional>
#include "common/common_types.h"
#include "shader_recompiler/environment.h"
#include "shader_recompiler/frontend/ir/reg.h"
#include "shader_recompiler/frontend/maxwell/location.h"
namespace Shader::Maxwell {
struct IndirectBranchTableInfo {
u32 cbuf_index{};
u32 cbuf_offset{};
u32 num_entries{};
s32 branch_offset{};
IR::Reg branch_reg{};
};
std::optional<IndirectBranchTableInfo> TrackIndirectBranchTable(Environment& env, Location brx_pos,
Location block_begin);
} // namespace Shader::Maxwell

View File

@@ -0,0 +1,61 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "common/bit_field.h"
#include "common/common_types.h"
#include "shader_recompiler/frontend/ir/flow_test.h"
namespace Shader::Maxwell {
struct Predicate {
Predicate() = default;
Predicate(unsigned index_, bool negated_ = false) : index{index_}, negated{negated_} {}
Predicate(bool value) : index{7}, negated{!value} {}
Predicate(u64 raw) : index{static_cast<unsigned>(raw & 7)}, negated{(raw & 8) != 0} {}
unsigned index;
bool negated;
};
inline bool operator==(const Predicate& lhs, const Predicate& rhs) noexcept {
return lhs.index == rhs.index && lhs.negated == rhs.negated;
}
inline bool operator!=(const Predicate& lhs, const Predicate& rhs) noexcept {
return !(lhs == rhs);
}
union Instruction {
Instruction(u64 raw_) : raw{raw_} {}
u64 raw;
union {
BitField<5, 1, u64> is_cbuf;
BitField<0, 5, IR::FlowTest> flow_test;
[[nodiscard]] u32 Absolute() const noexcept {
return static_cast<u32>(absolute);
}
[[nodiscard]] s32 Offset() const noexcept {
return static_cast<s32>(offset);
}
private:
BitField<20, 24, s64> offset;
BitField<20, 32, u64> absolute;
} branch;
[[nodiscard]] Predicate Pred() const noexcept {
return Predicate{pred};
}
private:
BitField<16, 4, u64> pred;
};
static_assert(std::is_trivially_copyable_v<Instruction>);
} // namespace Shader::Maxwell

View File

@@ -0,0 +1,108 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <fmt/ranges.h>
#include "common/common_types.h"
#include "shader_recompiler/exception.h"
namespace Shader::Maxwell {
class Location {
static constexpr u32 VIRTUAL_BIAS{4};
public:
constexpr Location() = default;
constexpr Location(u32 initial_offset) : offset{initial_offset} {
if (initial_offset % 8 != 0) {
throw InvalidArgument("initial_offset={} is not a multiple of 8", initial_offset);
}
Align();
}
constexpr Location Virtual() const noexcept {
Location virtual_location;
virtual_location.offset = offset - VIRTUAL_BIAS;
return virtual_location;
}
[[nodiscard]] constexpr u32 Offset() const noexcept {
return offset;
}
[[nodiscard]] constexpr bool IsVirtual() const {
return offset % 8 == VIRTUAL_BIAS;
}
constexpr auto operator<=>(const Location&) const noexcept = default;
constexpr Location operator++() noexcept {
const Location copy{*this};
Step();
return copy;
}
constexpr Location operator++(int) noexcept {
Step();
return *this;
}
constexpr Location operator--() noexcept {
const Location copy{*this};
Back();
return copy;
}
constexpr Location operator--(int) noexcept {
Back();
return *this;
}
constexpr Location operator+(int number) const {
Location new_pc{*this};
while (number > 0) {
--number;
++new_pc;
}
while (number < 0) {
++number;
--new_pc;
}
return new_pc;
}
constexpr Location operator-(int number) const {
return operator+(-number);
}
private:
constexpr void Align() {
offset += offset % 32 == 0 ? 8 : 0;
}
constexpr void Step() {
offset += 8 + (offset % 32 == 24 ? 8 : 0);
}
constexpr void Back() {
offset -= 8 + (offset % 32 == 8 ? 8 : 0);
}
u32 offset{0xcccccccc};
};
} // namespace Shader::Maxwell
template <>
struct fmt::formatter<Shader::Maxwell::Location> {
constexpr auto parse(format_parse_context& ctx) {
return ctx.begin();
}
template <typename FormatContext>
auto format(const Shader::Maxwell::Location& location, FormatContext& ctx) const {
return fmt::format_to(ctx.out(), "{:04x}", location.Offset());
}
};

View File

@@ -0,0 +1,286 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
INST(CCTLT, "CCTLT", "1110 1011 1111 0--0")
INST(AL2P, "AL2P", "1110 1111 1010 0---")
INST(ALD, "ALD", "1110 1111 1101 1---")
INST(AST, "AST", "1110 1111 1111 0---")
INST(B2R, "B2R", "1111 0000 1011 1---")
INST(BAR, "BAR", "1111 0000 1010 1---")
INST(BFE_cbuf, "BFE (cbuf)", "0100 1100 0000 0---")
INST(BFE_reg, "BFE (reg)", "0101 1100 0000 0---")
INST(BFI_cr, "BFI (cr)", "0100 1011 1111 0---")
INST(BFI_rc, "BFI (rc)", "0101 0011 1111 0---")
INST(BFI_reg, "BFI (reg)", "0101 1011 1111 0---")
INST(CS2R, "CS2R", "0101 0000 1100 1---")
INST(CSET, "CSET", "0101 0000 1001 1---")
INST(CSETP, "CSETP", "0101 0000 1010 0---")
INST(DADD_cbuf, "DADD (cbuf)", "0100 1100 0111 0---")
INST(DADD_reg, "DADD (reg)", "0101 1100 0111 0---")
INST(DEPBAR, "DEPBAR", "1111 0000 1111 0---")
INST(DMNMX_cbuf, "DMNMX (cbuf)", "0100 1100 0101 0---")
INST(DMNMX_reg, "DMNMX (reg)", "0101 1100 0101 0---")
INST(DMUL_cbuf, "DMUL (cbuf)", "0100 1100 1000 0---")
INST(DMUL_reg, "DMUL (reg)", "0101 1100 1000 0---")
INST(F2F_cbuf, "F2F (cbuf)", "0100 1100 1010 1---")
INST(F2F_reg, "F2F (reg)", "0101 1100 1010 1---")
INST(F2I_cbuf, "F2I (cbuf)", "0100 1100 1011 0---")
INST(F2I_reg, "F2I (reg)", "0101 1100 1011 0---")
INST(FADD_cbuf, "FADD (cbuf)", "0100 1100 0101 1---")
INST(FADD_reg, "FADD (reg)", "0101 1100 0101 1---")
INST(FCHK_cbuf, "FCHK (cbuf)", "0100 1100 1000 1---")
INST(FCHK_reg, "FCHK (reg)", "0101 1100 1000 1---")
INST(FLO_cbuf, "FLO (cbuf)", "0100 1100 0011 0---")
INST(FLO_reg, "FLO (reg)", "0101 1100 0011 0---")
INST(FMNMX_cbuf, "FMNMX (cbuf)", "0100 1100 0110 0---")
INST(FMNMX_reg, "FMNMX (reg)", "0101 1100 0110 0---")
INST(FMUL_cbuf, "FMUL (cbuf)", "0100 1100 0110 1---")
INST(FMUL_reg, "FMUL (reg)", "0101 1100 0110 1---")
INST(FSWZADD, "FSWZADD", "0101 0000 1111 1---")
INST(HADD2_reg, "HADD2 (reg)", "0101 1101 0001 0---")
INST(HFMA2_reg, "HFMA2 (reg)", "0101 1101 0000 0---")
INST(HMUL2_reg, "HMUL2 (reg)", "0101 1101 0000 1---")
INST(HSET2_reg, "HSET2 (reg)", "0101 1101 0001 1---")
INST(HSETP2_reg, "HSETP2 (reg)", "0101 1101 0010 0---")
INST(I2F_cbuf, "I2F (cbuf)", "0100 1100 1011 1---")
INST(I2F_reg, "I2F (reg)", "0101 1100 1011 1---")
INST(I2I_cbuf, "I2I (cbuf)", "0100 1100 1110 0---")
INST(I2I_reg, "I2I (reg)", "0101 1100 1110 0---")
INST(IADD_cbuf, "IADD (cbuf)", "0100 1100 0001 0---")
INST(IADD_reg, "IADD (reg)", "0101 1100 0001 0---")
INST(IDP_imm, "IDP (imm)", "0101 0011 1101 1---")
INST(IDP_reg, "IDP (reg)", "0101 0011 1111 1---")
INST(IMNMX_cbuf, "IMNMX (cbuf)", "0100 1100 0010 0---")
INST(IMNMX_reg, "IMNMX (reg)", "0101 1100 0010 0---")
INST(IMUL_cbuf, "IMUL (cbuf)", "0100 1100 0011 1---")
INST(IMUL_reg, "IMUL (reg)", "0101 1100 0011 1---")
INST(ISBERD, "ISBERD", "1110 1111 1101 0---")
INST(ISBE_nx2, "ISBE (nx2)", "1110 1111 1100 0---")
INST(HMMA_nx2, "HMMA (nx2)", "1110 1111 1100 1---")
INST(REGMAP_nx2, "REGMAP (nx2)", "1110 1111 1110 0---")
INST(ISCADD_cbuf, "ISCADD (cbuf)", "0100 1100 0001 1---")
INST(ISCADD_reg, "ISCADD (reg)", "0101 1100 0001 1---")
INST(LDC, "LDC", "1110 1111 1001 0---")
INST(LDG, "LDG", "1110 1110 1101 0---")
INST(LDL, "LDL", "1110 1111 0100 0---")
INST(LDS, "LDS", "1110 1111 0100 1---")
INST(LEA_hi_reg, "LEA (hi reg)", "0101 1011 1101 1---")
INST(LEA_lo_reg, "LEA (lo reg)", "0101 1011 1101 0---")
INST(LEPC, "LEPC", "0101 0000 1101 0---")
INST(LOP3_reg, "LOP3 (reg)", "0101 1011 1110 0---")
INST(LOP_cbuf, "LOP (cbuf)", "0100 1100 0100 0---")
INST(LOP_reg, "LOP (reg)", "0101 1100 0100 0---")
INST(MEMBAR, "MEMBAR", "1110 1111 1001 1---")
INST(MOV_cbuf, "MOV (cbuf)", "0100 1100 1001 1---")
INST(MOV_reg, "MOV (reg)", "0101 1100 1001 1---")
INST(MUFU, "MUFU", "0101 0000 1000 0---")
INST(NOP, "NOP", "0101 0000 1011 0---")
INST(OUT_cbuf, "OUT (cbuf)", "1110 1011 1110 0---")
INST(OUT_reg, "OUT (reg)", "1111 1011 1110 0---")
INST(P2R_cbuf, "P2R (cbuf)", "0100 1100 1110 1---")
INST(P2R_imm, "P2R (imm)", "0011 1000 1110 1---")
INST(P2R_reg, "P2R (reg)", "0101 1100 1110 1---")
INST(PIXLD, "PIXLD", "1110 1111 1110 1---")
INST(POPC_cbuf, "POPC (cbuf)", "0100 1100 0000 1---")
INST(POPC_reg, "POPC (reg)", "0101 1100 0000 1---")
INST(PSET, "PSET", "0101 0000 1000 1---")
INST(PSETP, "PSETP", "0101 0000 1001 0---")
INST(R2B, "R2B", "1111 0000 1100 0---")
INST(R2P_cbuf, "R2P (cbuf)", "0100 1100 1111 0---")
INST(R2P_reg, "R2P (reg)", "0101 1100 1111 0---")
INST(RED, "RED", "1110 1011 1111 1---")
INST(RRO_cbuf, "RRO (cbuf)", "0100 1100 1001 0---")
INST(RRO_reg, "RRO (reg)", "0101 1100 1001 0---")
INST(S2R, "S2R", "1111 0000 1100 1---")
INST(SEL_cbuf, "SEL (cbuf)", "0100 1100 1010 0---")
INST(SEL_reg, "SEL (reg)", "0101 1100 1010 0---")
INST(SHFL, "SHFL", "1110 1111 0001 0---")
INST(SHF_l_reg, "SHF (l reg)", "0101 1011 1111 1---")
INST(SHF_r_reg, "SHF (r reg)", "0101 1100 1111 1---")
INST(SHL_cbuf, "SHL (cbuf)", "0100 1100 0100 1---")
INST(SHL_reg, "SHL (reg)", "0101 1100 0100 1---")
INST(SHR_cbuf, "SHR (cbuf)", "0100 1100 0010 1---")
INST(SHR_reg, "SHR (reg)", "0101 1100 0010 1---")
INST(STG, "STG", "1110 1110 1101 1---")
INST(STL, "STL", "1110 1111 0101 0---")
INST(STP, "STP", "1110 1110 1010 0---")
INST(STS, "STS", "1110 1111 0101 1---")
INST(SYNC, "SYNC", "1111 0000 1111 1---")
INST(TMML, "TMML", "1101 1111 0101 1---")
INST(TMML_b, "TMML (b)", "1101 1111 0110 0---")
INST(TXA, "TXA", "1101 1111 0100 0---")
INST(TXQ, "TXQ", "1101 1111 0100 1---")
INST(TXQ_b, "TXQ (b)", "1101 1111 0101 0---")
INST(VOTE, "VOTE", "0101 0000 1101 1---")
INST(VOTE_vtg, "VOTE (vtg)", "0101 0000 1110 0---")
INST(VSETP, "VSETP", "0101 0000 1111 0---")
INST(ATOM_cas, "ATOM (cas)", "1110 1110 1111 ----")
INST(BFE_imm, "BFE (imm)", "0011 100- 0000 0---")
INST(BFI_imm, "BFI (imm)", "0011 011- 1111 0---")
INST(BPT, "BPT", "1110 0011 1010 ----")
INST(BRA, "BRA", "1110 0010 0100 ----")
INST(BRK, "BRK", "1110 0011 0100 ----")
INST(BRX, "BRX", "1110 0010 0101 ----")
INST(CAL, "CAL", "1110 0010 0110 ----")
INST(CONT, "CONT", "1110 0011 0101 ----")
INST(DADD_imm, "DADD (imm)", "0011 100- 0111 0---")
INST(DFMA_cr, "DFMA (cr)", "0100 1011 0111 ----")
INST(DFMA_rc, "DFMA (rc)", "0101 0011 0111 ----")
INST(DFMA_reg, "DFMA (reg)", "0101 1011 0111 ----")
INST(DMNMX_imm, "DMNMX (imm)", "0011 100- 0101 0---")
INST(DMUL_imm, "DMUL (imm)", "0011 100- 1000 0---")
INST(DSETP_cbuf, "DSETP (cbuf)", "0100 1011 1000 ----")
INST(DSETP_reg, "DSETP (reg)", "0101 1011 1000 ----")
INST(EXIT, "EXIT", "1110 0011 0000 ----")
INST(F2F_imm, "F2F (imm)", "0011 100- 1010 1---")
INST(F2I_imm, "F2I (imm)", "0011 100- 1011 0---")
INST(FADD_imm, "FADD (imm)", "0011 100- 0101 1---")
INST(FCHK_imm, "FCHK (imm)", "0011 100- 1000 1---")
INST(FCMP_cr, "FCMP (cr)", "0100 1011 1010 ----")
INST(FCMP_rc, "FCMP (rc)", "0101 0011 1010 ----")
INST(FCMP_reg, "FCMP (reg)", "0101 1011 1010 ----")
INST(FLO_imm, "FLO (imm)", "0011 100- 0011 0---")
INST(FMNMX_imm, "FMNMX (imm)", "0011 100- 0110 0---")
INST(FMUL_imm, "FMUL (imm)", "0011 100- 0110 1---")
INST(FSETP_cbuf, "FSETP (cbuf)", "0100 1011 1011 ----")
INST(FSETP_reg, "FSETP (reg)", "0101 1011 1011 ----")
INST(GETCRSPTR, "GETCRSPTR", "1110 0010 1100 ----")
INST(GETLMEMBASE, "GETLMEMBASE", "1110 0010 1101 ----")
INST(I2F_imm, "I2F (imm)", "0011 100- 1011 1---")
INST(I2I_imm, "I2I (imm)", "0011 100- 1110 0---")
INST(IADD3_cbuf, "IADD3 (cbuf)", "0100 1100 1100 ----")
INST(IADD3_reg, "IADD3 (reg)", "0101 1100 1100 ----")
INST(IADD_imm, "IADD (imm)", "0011 100- 0001 0---")
INST(ICMP_cr, "ICMP (cr)", "0100 1011 0100 ----")
INST(ICMP_rc, "ICMP (rc)", "0101 0011 0100 ----")
INST(ICMP_reg, "ICMP (reg)", "0101 1011 0100 ----")
INST(IDE, "IDE", "1110 0011 1001 ----")
INST(IMNMX_imm, "IMNMX (imm)", "0011 100- 0010 0---")
INST(IMUL_imm, "IMUL (imm)", "0011 100- 0011 1---")
INST(ISCADD_imm, "ISCADD (imm)", "0011 100- 0001 1---")
INST(ISETP_cbuf, "ISETP (cbuf)", "0100 1011 0110 ----")
INST(ISETP_reg, "ISETP (reg)", "0101 1011 0110 ----")
INST(ISET_cbuf, "ISET (cbuf)", "0100 1011 0101 ----")
INST(ISET_reg, "ISET (reg)", "0101 1011 0101 ----")
INST(JCAL, "JCAL", "1110 0010 0010 ----")
INST(JMP, "JMP", "1110 0010 0001 ----")
INST(JMX, "JMX", "1110 0010 0000 ----")
INST(KIL, "KIL", "1110 0011 0011 ----")
INST(LEA_lo_cbuf, "LEA (lo cbuf)", "0100 1011 1101 ----")
INST(LEA_lo_imm, "LEA (lo imm)", "0011 011- 1101 0---")
INST(LONGJMP, "LONGJMP", "1110 0011 0001 ----")
INST(LOP_imm, "LOP (imm)", "0011 100- 0100 0---")
INST(MOV32I, "MOV32I", "0000 0001 0000 ----")
INST(MOV_imm, "MOV (imm)", "0011 100- 1001 1---")
INST(OUT_imm, "OUT (imm)", "1111 011- 1110 0---")
INST(PBK, "PBK", "1110 0010 1010 ----")
INST(PCNT, "PCNT", "1110 0010 1011 ----")
INST(PEXIT, "PEXIT", "1110 0010 0011 ----")
INST(PLONGJMP, "PLONGJMP", "1110 0010 1000 ----")
INST(POPC_imm, "POPC (imm)", "0011 100- 0000 1---")
INST(PRET, "PRET", "1110 0010 0111 ----")
INST(PRMT_cr, "PRMT (cr)", "0100 1011 1100 ----")
INST(PRMT_rc, "PRMT (rc)", "0101 0011 1100 ----")
INST(PRMT_reg, "PRMT (reg)", "0101 1011 1100 ----")
INST(R2P_imm, "R2P (imm)", "0011 100- 1111 0---")
INST(RAM, "RAM", "1110 0011 1000 ----")
INST(RET, "RET", "1110 0011 0010 ----")
INST(RRO_imm, "RRO (imm)", "0011 100- 1001 0---")
INST(RTT, "RTT", "1110 0011 0110 ----")
INST(SAM, "SAM", "1110 0011 0111 ----")
INST(SEL_imm, "SEL (imm)", "0011 100- 1010 0---")
INST(SETCRSPTR, "SETCRSPTR", "1110 0010 1110 ----")
INST(SETLMEMBASE, "SETLMEMBASE", "1110 0010 1111 ----")
INST(SHF_l_imm, "SHF (l imm)", "0011 011- 1111 1---")
INST(SHF_r_imm, "SHF (r imm)", "0011 100- 1111 1---")
INST(SHL_imm, "SHL (imm)", "0011 100- 0100 1---")
INST(SHR_imm, "SHR (imm)", "0011 100- 0010 1---")
INST(SSY, "SSY", "1110 0010 1001 ----")
INST(CCTL, "CCTL", "1110 1111 011- ----")
INST(CCTLL, "CCTLL", "1110 1111 100- ----")
INST(DFMA_imm, "DFMA (imm)", "0011 011- 0111 ----")
INST(DSETP_imm, "DSETP (imm)", "0011 011- 1000 ----")
INST(FCMP_imm, "FCMP (imm)", "0011 011- 1010 ----")
INST(FSETP_imm, "FSETP (imm)", "0011 011- 1011 ----")
INST(IADD3_imm, "IADD3 (imm)", "0011 100- 1100 ----")
INST(ICMP_imm, "ICMP (imm)", "0011 011- 0100 ----")
INST(ISETP_imm, "ISETP (imm)", "0011 011- 0110 ----")
INST(ISET_imm, "ISET (imm)", "0011 011- 0101 ----")
INST(PRMT_imm, "PRMT (imm)", "0011 011- 1100 ----")
INST(SULD, "SULD", "1110 1011 000- ----")
INST(SURED, "SURED", "1110 1011 010- ----")
INST(SUST, "SUST", "1110 1011 001- ----")
INST(TEX_b, "TEX (b)", "1101 1110 10-- ----")
INST(TLD4_b, "TLD4 (b)", "1101 1110 11-- ----")
INST(TXD, "TXD", "1101 1110 00-- ----")
INST(TXD_b, "TXD (b)", "1101 1110 01-- ----")
INST(XMAD_reg, "XMAD (reg)", "0101 1011 00-- ----")
INST(DSET_cbuf, "DSET (cbuf)", "0100 1001 0--- ----")
INST(DSET_reg, "DSET (reg)", "0101 1001 0--- ----")
INST(FFMA_cr, "FFMA (cr)", "0100 1001 1--- ----")
INST(FFMA_rc, "FFMA (rc)", "0101 0001 1--- ----")
INST(FFMA_reg, "FFMA (reg)", "0101 1001 1--- ----")
INST(IMADSP_cr, "IMADSP (cr)", "0100 1010 1--- ----")
INST(IMADSP_rc, "IMADSP (rc)", "0101 0010 1--- ----")
INST(IMADSP_reg, "IMADSP (reg)", "0101 1010 1--- ----")
INST(IMAD_cr, "IMAD (cr)", "0100 1010 0--- ----")
INST(IMAD_rc, "IMAD (rc)", "0101 0010 0--- ----")
INST(IMAD_reg, "IMAD (reg)", "0101 1010 0--- ----")
INST(SUATOM, "SUATOM", "1110 1010 0--- ----")
INST(SUATOM_cas, "SUATOM_cas", "1110 1010 1--- ----")
INST(TLD4S, "TLD4S", "1101 1111 -0-- ----")
INST(VABSDIFF4, "VABSDIFF4", "0101 0000 0--- ----")
INST(XMAD_imm, "XMAD (imm)", "0011 011- 00-- ----")
INST(XMAD_rc, "XMAD (rc)", "0101 0001 0--- ----")
INST(ATOM, "ATOM", "1110 1101 ---- ----")
INST(ATOMS, "ATOMS", "1110 1100 ---- ----")
INST(ATOMS_cas, "ATOMS (cas)", "1110 1110 ---- ----")
INST(DSET_imm, "DSET (imm)", "0011 001- 0--- ----")
INST(FFMA_imm, "FFMA (imm)", "0011 001- 1--- ----")
INST(FMUL32I, "FMUL32I", "0001 1110 ---- ----")
INST(FSET_cbuf, "FSET (cbuf)", "0100 1000 ---- ----")
INST(FSET_reg, "FSET (reg)", "0101 1000 ---- ----")
INST(HADD2_cbuf, "HADD2 (cbuf)", "0111 101- 1--- ----")
INST(HADD2_imm, "HADD2 (imm)", "0111 101- 0--- ----")
INST(HMUL2_cbuf, "HMUL2 (cbuf)", "0111 100- 1--- ----")
INST(HMUL2_imm, "HMUL2 (imm)", "0111 100- 0--- ----")
INST(HSET2_cbuf, "HSET2 (cbuf)", "0111 110- 1--- ----")
INST(HSET2_imm, "HSET2 (imm)", "0111 110- 0--- ----")
INST(HSETP2_cbuf, "HSETP2 (cbuf)", "0111 111- 1--- ----")
INST(HSETP2_imm, "HSETP2 (imm)", "0111 111- 0--- ----")
INST(IMADSP_imm, "IMADSP (imm)", "0011 010- 1--- ----")
INST(IMAD_imm, "IMAD (imm)", "0011 010- 0--- ----")
INST(IMUL32I, "IMUL32I", "0001 1111 ---- ----")
INST(IPA, "IPA", "1110 0000 ---- ----")
INST(TLD, "TLD", "1101 1100 ---- ----")
INST(TLD_b, "TLD (b)", "1101 1101 ---- ----")
INST(VABSDIFF, "VABSDIFF", "0101 0100 ---- ----")
INST(VMAD, "VMAD", "0101 1111 ---- ----")
INST(VSHL, "VSHL", "0101 0111 ---- ----")
INST(VSHR, "VSHR", "0101 0110 ---- ----")
INST(FSET_imm, "FSET (imm)", "0011 000- ---- ----")
INST(HADD2_32I, "HADD2_32I", "0010 110- ---- ----")
INST(HFMA2_32I, "HFMA2_32I", "0010 100- ---- ----")
INST(HMUL2_32I, "HMUL2_32I", "0010 101- ---- ----")
INST(IADD32I, "IADD32I", "0001 110- ---- ----")
INST(LOP3_cbuf, "LOP3 (cbuf)", "0000 001- ---- ----")
INST(VMNMX, "VMNMX", "0011 101- ---- ----")
INST(VSET, "VSET", "0100 000- ---- ----")
INST(XMAD_cr, "XMAD (cr)", "0100 111- ---- ----")
INST(FADD32I, "FADD32I", "0000 10-- ---- ----")
INST(FFMA32I, "FFMA32I", "0000 11-- ---- ----")
INST(HFMA2_cr, "HFMA2 (cr)", "0111 0--- 1--- ----")
INST(HFMA2_imm, "HFMA2 (imm)", "0111 0--- 0--- ----")
INST(HFMA2_rc, "HFMA2 (rc)", "0110 0--- 1--- ----")
INST(IMAD32I, "IMAD32I", "1000 00-- ---- ----")
INST(ISCADD32I, "ISCADD32I", "0001 01-- ---- ----")
INST(LEA_hi_cbuf, "LEA (hi cbuf)", "0001 10-- ---- ----")
INST(LOP32I, "LOP32I", "0000 01-- ---- ----")
INST(LOP3_imm, "LOP3 (imm)", "0011 11-- ---- ----")
INST(TEXS, "TEXS", "1101 -00- ---- ----")
INST(TLD4, "TLD4", "1100 10-- ---- ----")
INST(TLDS, "TLDS", "1101 -01- ---- ----")
INST(VADD, "VADD", "0010 00-- ---- ----")
INST(TEX, "TEX", "1100 0--- ---- ----")
INST(LD, "LD", "100- ---- ---- ----")
INST(ST, "ST", "101- ---- ---- ----")

View File

@@ -0,0 +1,25 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include <array>
#include <string_view>
#include "shader_recompiler/exception.h"
#include "shader_recompiler/frontend/maxwell/opcodes.h"
namespace Shader::Maxwell {
const char* NameOf(Opcode opcode) {
constexpr const char* NAME_TABLE[] = {
#define INST(name, cute, encode) cute,
#include "maxwell.inc"
#undef INST
};
if (size_t(opcode) >= sizeof(NAME_TABLE) / sizeof(NAME_TABLE[0]))
throw InvalidArgument("Invalid opcode with raw value {}", int(opcode));
return NAME_TABLE[size_t(opcode)];
}
} // namespace Shader::Maxwell

View File

@@ -0,0 +1,29 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include <fmt/ranges.h>
namespace Shader::Maxwell {
enum class Opcode {
#define INST(name, cute, encode) name,
#include "maxwell.inc"
#undef INST
};
const char* NameOf(Opcode opcode);
} // namespace Shader::Maxwell
template <>
struct fmt::formatter<Shader::Maxwell::Opcode> {
constexpr auto parse(format_parse_context& ctx) {
return ctx.begin();
}
template <typename FormatContext>
auto format(const Shader::Maxwell::Opcode& opcode, FormatContext& ctx) const {
return fmt::format_to(ctx.out(), "{}", NameOf(opcode));
}
};

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,22 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once
#include "shader_recompiler/environment.h"
#include "shader_recompiler/frontend/ir/abstract_syntax_list.h"
#include "shader_recompiler/frontend/ir/basic_block.h"
#include "shader_recompiler/frontend/ir/value.h"
#include "shader_recompiler/frontend/maxwell/control_flow.h"
#include "shader_recompiler/object_pool.h"
namespace Shader {
struct HostTranslateInfo;
namespace Maxwell {
[[nodiscard]] IR::AbstractSyntaxList BuildASL(ObjectPool<IR::Inst>& inst_pool,
ObjectPool<IR::Block>& block_pool, Environment& env,
Flow::CFG& cfg, const HostTranslateInfo& host_info);
} // namespace Maxwell
} // namespace Shader

View File

@@ -0,0 +1,232 @@
// SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project
// SPDX-License-Identifier: GPL-3.0-or-later
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "common/bit_field.h"
#include "common/common_types.h"
#include "shader_recompiler/frontend/maxwell/translate/impl/impl.h"
namespace Shader::Maxwell {
namespace {
enum class AtomOp : u64 {
ADD,
MIN,
MAX,
INC,
DEC,
AND,
OR,
XOR,
EXCH,
SAFEADD,
};
enum class AtomSize : u64 {
U32,
S32,
U64,
F32,
F16x2,
S64,
};
IR::U32U64 ApplyIntegerAtomOp(IR::IREmitter& ir, const IR::U32U64& offset, const IR::U32U64& op_b, AtomOp op, AtomSize size) {
bool const is_signed = size == AtomSize::S64 || size == AtomSize::S32;
switch (op) {
case AtomOp::ADD:
return ir.GlobalAtomicIAdd(offset, op_b);
case AtomOp::MIN:
return ir.GlobalAtomicIMin(offset, op_b, is_signed);
case AtomOp::MAX:
return ir.GlobalAtomicIMax(offset, op_b, is_signed);
case AtomOp::INC:
return ir.GlobalAtomicInc(offset, op_b);
case AtomOp::DEC:
return ir.GlobalAtomicDec(offset, op_b);
case AtomOp::AND:
return ir.GlobalAtomicAnd(offset, op_b);
case AtomOp::OR:
return ir.GlobalAtomicOr(offset, op_b);
case AtomOp::XOR:
return ir.GlobalAtomicXor(offset, op_b);
case AtomOp::EXCH:
return ir.GlobalAtomicExchange(offset, op_b);
default:
throw NotImplementedException("Integer Atom Operation {}", op);
}
}
IR::Value ApplyFpAtomOp(IR::IREmitter& ir, const IR::U64& offset, const IR::Value& op_b, AtomOp op,
AtomSize size) {
static constexpr IR::FpControl f16_control{
.no_contraction = false,
.rounding = IR::FpRounding::RN,
.fmz_mode = IR::FmzMode::DontCare,
};
static constexpr IR::FpControl f32_control{
.no_contraction = false,
.rounding = IR::FpRounding::RN,
.fmz_mode = IR::FmzMode::FTZ,
};
switch (op) {
case AtomOp::ADD:
return size == AtomSize::F32 ? ir.GlobalAtomicF32Add(offset, op_b, f32_control)
: ir.GlobalAtomicF16x2Add(offset, op_b, f16_control);
case AtomOp::MIN:
return ir.GlobalAtomicF16x2Min(offset, op_b, f16_control);
case AtomOp::MAX:
return ir.GlobalAtomicF16x2Max(offset, op_b, f16_control);
default:
throw NotImplementedException("FP Atom Operation {}", op);
}
}
IR::U64 AtomOffset(TranslatorVisitor& v, u64 insn) {
union {
u64 raw;
BitField<8, 8, IR::Reg> addr_reg;
BitField<28, 20, s64> addr_offset;
BitField<28, 20, u64> rz_addr_offset;
BitField<48, 1, u64> e;
} const mem{insn};
const IR::U64 address{[&]() -> IR::U64 {
if (mem.e == 0)
return v.ir.UConvert(64, v.X(mem.addr_reg));
return v.L(mem.addr_reg);
}()};
const u64 addr_offset{[&]() -> u64 {
if (mem.addr_reg == IR::Reg::RZ) {
// When RZ is used, the address is an absolute address
return u64(mem.rz_addr_offset.Value());
} else {
return u64(mem.addr_offset.Value());
}
}()};
return v.ir.IAdd(address, v.ir.Imm64(addr_offset));
}
// INC, DEC for U32/S32/U64 does nothing
// ADD, INC, DEC for S64 does nothing
// Only ADD does something for F32
// Only ADD, MIN and MAX does something for F16x2
bool AtomOpNotApplicable(AtomSize size, AtomOp op) {
// TODO: SAFEADD
switch (size) {
case AtomSize::U32:
case AtomSize::S32:
case AtomSize::U64:
return (op == AtomOp::INC || op == AtomOp::DEC);
case AtomSize::S64:
return (op == AtomOp::ADD || op == AtomOp::INC || op == AtomOp::DEC);
case AtomSize::F32:
return op != AtomOp::ADD;
case AtomSize::F16x2:
return !(op == AtomOp::ADD || op == AtomOp::MIN || op == AtomOp::MAX);
default:
return false;
}
}
IR::U32U64 LoadGlobal(IR::IREmitter& ir, const IR::U64& offset, AtomSize size) {
switch (size) {
case AtomSize::U32:
case AtomSize::S32:
case AtomSize::F32:
case AtomSize::F16x2:
return ir.LoadGlobal32(offset);
case AtomSize::U64:
case AtomSize::S64:
return ir.PackUint2x32(ir.LoadGlobal64(offset));
default:
throw NotImplementedException("Atom Size {}", size);
}
}
void StoreResult(TranslatorVisitor& v, IR::Reg dest_reg, const IR::Value& result, AtomSize size) {
switch (size) {
case AtomSize::U32:
case AtomSize::S32:
case AtomSize::F16x2:
return v.X(dest_reg, IR::U32{result});
case AtomSize::U64:
case AtomSize::S64:
return v.L(dest_reg, IR::U64{result});
case AtomSize::F32:
return v.F(dest_reg, IR::F32{result});
default:
break;
}
}
IR::Value ApplyAtomOp(TranslatorVisitor& v, IR::Reg operand_reg, const IR::U64& offset,
AtomSize size, AtomOp op) {
switch (size) {
case AtomSize::U32:
case AtomSize::S32:
return ApplyIntegerAtomOp(v.ir, offset, v.X(operand_reg), op, size);
case AtomSize::U64:
case AtomSize::S64:
return ApplyIntegerAtomOp(v.ir, offset, v.L(operand_reg), op, size);
case AtomSize::F32:
return ApplyFpAtomOp(v.ir, offset, v.F(operand_reg), op, size);
case AtomSize::F16x2:
return ApplyFpAtomOp(v.ir, offset, v.ir.UnpackFloat2x16(v.X(operand_reg)), op, size);
default:
throw NotImplementedException("Atom Size {}", size);
}
}
void GlobalAtomic(TranslatorVisitor& v, IR::Reg dest_reg, IR::Reg operand_reg,
const IR::U64& offset, AtomSize size, AtomOp op, bool write_dest) {
IR::Value result = AtomOpNotApplicable(size, op)
? LoadGlobal(v.ir, offset, size)
: ApplyAtomOp(v, operand_reg, offset, size, op);
if (write_dest)
StoreResult(v, dest_reg, result, size);
}
} // Anonymous namespace
void TranslatorVisitor::ATOM(u64 insn) {
union {
u64 raw;
BitField<0, 8, IR::Reg> dest_reg;
BitField<20, 8, IR::Reg> operand_reg;
BitField<49, 3, AtomSize> size;
BitField<52, 4, AtomOp> op;
} const atom{insn};
const IR::U64 offset{AtomOffset(*this, insn)};
GlobalAtomic(*this, atom.dest_reg, atom.operand_reg, offset, atom.size, atom.op, true);
}
// (hookshot) ATOM.E.CAS Rd, [Ra + offset], Rb, Rb+1: Rb the comparator, Rb+1 the value. 32 bits only (size 0).
void TranslatorVisitor::ATOM_cas(u64 insn) {
union {
u64 raw;
BitField<0, 8, IR::Reg> dest_reg;
BitField<20, 8, IR::Reg> operand_reg;
BitField<49, 3, u64> size;
} const cas{insn};
if (cas.size != 0) {
throw NotImplementedException("ATOM.CAS size {}", cas.size.Value());
}
const IR::U64 offset{AtomOffset(*this, insn)};
const IR::U32 comparator{X(cas.operand_reg)};
const IR::U32 value{cas.operand_reg == IR::Reg::RZ ? ir.Imm32(0) : X(cas.operand_reg + 1)};
X(cas.dest_reg, ir.GlobalAtomicCompareExchange(offset, comparator, value));
}
void TranslatorVisitor::RED(u64 insn) {
union {
u64 raw;
BitField<0, 8, IR::Reg> operand_reg;
BitField<20, 3, AtomSize> size;
BitField<23, 3, AtomOp> op;
} const red{insn};
const IR::U64 offset{AtomOffset(*this, insn)};
GlobalAtomic(*this, IR::Reg::RZ, red.operand_reg, offset, red.size, red.op, true);
}
} // namespace Shader::Maxwell

View File

@@ -0,0 +1,109 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "common/bit_field.h"
#include "common/common_types.h"
#include "shader_recompiler/frontend/maxwell/translate/impl/impl.h"
namespace Shader::Maxwell {
namespace {
enum class AtomOp : u64 {
ADD,
MIN,
MAX,
INC,
DEC,
AND,
OR,
XOR,
EXCH,
};
enum class AtomsSize : u64 {
U32,
S32,
U64,
};
IR::U32U64 ApplyAtomsOp(IR::IREmitter& ir, const IR::U32& offset, const IR::U32U64& op_b, AtomOp op,
bool is_signed) {
switch (op) {
case AtomOp::ADD:
return ir.SharedAtomicIAdd(offset, op_b);
case AtomOp::MIN:
return ir.SharedAtomicIMin(offset, op_b, is_signed);
case AtomOp::MAX:
return ir.SharedAtomicIMax(offset, op_b, is_signed);
case AtomOp::INC:
return ir.SharedAtomicInc(offset, op_b);
case AtomOp::DEC:
return ir.SharedAtomicDec(offset, op_b);
case AtomOp::AND:
return ir.SharedAtomicAnd(offset, op_b);
case AtomOp::OR:
return ir.SharedAtomicOr(offset, op_b);
case AtomOp::XOR:
return ir.SharedAtomicXor(offset, op_b);
case AtomOp::EXCH:
return ir.SharedAtomicExchange(offset, op_b);
default:
throw NotImplementedException("Integer Atoms Operation {}", op);
}
}
IR::U32 AtomsOffset(TranslatorVisitor& v, u64 insn) {
union {
u64 raw;
BitField<8, 8, IR::Reg> offset_reg;
BitField<30, 22, u64> absolute_offset;
BitField<30, 22, s64> relative_offset;
} const encoding{insn};
if (encoding.offset_reg == IR::Reg::RZ) {
return v.ir.Imm32(static_cast<u32>(encoding.absolute_offset << 2));
} else {
const s32 relative{static_cast<s32>(encoding.relative_offset << 2)};
return v.ir.IAdd(v.X(encoding.offset_reg), v.ir.Imm32(relative));
}
}
void StoreResult(TranslatorVisitor& v, IR::Reg dest_reg, const IR::Value& result, AtomsSize size) {
switch (size) {
case AtomsSize::U32:
case AtomsSize::S32:
return v.X(dest_reg, IR::U32{result});
case AtomsSize::U64:
return v.L(dest_reg, IR::U64{result});
default:
break;
}
}
} // Anonymous namespace
void TranslatorVisitor::ATOMS(u64 insn) {
union {
u64 raw;
BitField<0, 8, IR::Reg> dest_reg;
BitField<8, 8, IR::Reg> addr_reg;
BitField<20, 8, IR::Reg> src_reg_b;
BitField<28, 2, AtomsSize> size;
BitField<52, 4, AtomOp> op;
} const atoms{insn};
const bool size_64{atoms.size == AtomsSize::U64};
if (size_64 && atoms.op != AtomOp::EXCH) {
throw NotImplementedException("64-bit Atoms Operation {}", atoms.op.Value());
}
const bool is_signed{atoms.size == AtomsSize::S32};
const IR::U32 offset{AtomsOffset(*this, insn)};
IR::Value result;
if (size_64) {
result = ApplyAtomsOp(ir, offset, L(atoms.src_reg_b), atoms.op, is_signed);
} else {
result = ApplyAtomsOp(ir, offset, X(atoms.src_reg_b), atoms.op, is_signed);
}
StoreResult(*this, atoms.dest_reg, result, atoms.size);
}
} // namespace Shader::Maxwell

View File

@@ -0,0 +1,33 @@
// SPDX-FileCopyrightText: Copyright 2021 yuzu Emulator Project
// SPDX-License-Identifier: GPL-2.0-or-later
#include "common/bit_field.h"
#include "common/common_types.h"
#include "shader_recompiler/frontend/maxwell/translate/impl/impl.h"
namespace Shader::Maxwell {
enum class BitSize : u64 {
B32,
B64,
B96,
B128,
};
void TranslatorVisitor::AL2P(u64 inst) {
union {
u64 raw;
BitField<0, 8, IR::Reg> result_register;
BitField<8, 8, IR::Reg> indexing_register;
BitField<20, 11, s64> offset;
BitField<47, 2, BitSize> bitsize;
} al2p{inst};
if (al2p.bitsize != BitSize::B32) {
throw NotImplementedException("BitSize {}", al2p.bitsize.Value());
}
const IR::U32 converted_offset{ir.Imm32(static_cast<u32>(al2p.offset.Value()))};
const IR::U32 result{ir.IAdd(X(al2p.indexing_register), converted_offset)};
X(al2p.result_register, result);
}
} // namespace Shader::Maxwell

Some files were not shown because too many files have changed in this diff Show More