mirror of
https://git.triforceproject.com/triforce/source
synced 2026-11-21 17:19:56 -04:00
Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
70c210cee1 | ||
|
|
043db52c62 | ||
|
|
6c7117635c | ||
|
|
b0e46adb48 | ||
|
|
fb26495475 | ||
|
|
a750a21025 | ||
|
|
72793f2231 | ||
|
|
6f338c8f56 |
2
.gitignore
vendored
2
.gitignore
vendored
@@ -18,6 +18,8 @@ gamefile/
|
|||||||
|
|
||||||
# ---- build output and caches ----
|
# ---- build output and caches ----
|
||||||
build/
|
build/
|
||||||
|
# (but launcher/Logic/Build/ is source: git on Windows ignores case, so build/ hid it and v0.5.2 was committed without it)
|
||||||
|
!launcher/Logic/Build/
|
||||||
__pycache__/
|
__pycache__/
|
||||||
*.pyc
|
*.pyc
|
||||||
*.o
|
*.o
|
||||||
|
|||||||
@@ -37,8 +37,15 @@ Open an [issue](https://github.com/ruckus7382sfgsgdf/CBeta0.2/issues) and includ
|
|||||||
- **a game problem:** `triforce_log.txt` from your game folder (the folder with `Triforce.exe`). If you started the
|
- **a game problem:** `triforce_log.txt` from your game folder (the folder with `Triforce.exe`). If you started the
|
||||||
game again after the problem, send `triforce_log.1.txt` too: each start keeps the four runs before it as
|
game again after the problem, send `triforce_log.1.txt` too: each start keeps the four runs before it as
|
||||||
`triforce_log.1.txt` (the last) to `triforce_log.4.txt`.
|
`triforce_log.1.txt` (the last) to `triforce_log.4.txt`.
|
||||||
- **a Setup or build problem:** `setup.log` from `%LOCALAPPDATA%\Programs\Triforce`, and for a failed build the text
|
- **a freeze:** `triforce_hang.txt` from the game folder as well, when it is there. After 20 seconds without a frame
|
||||||
from **Show details** on the build page.
|
the game writes down where each of its threads was (addresses in the program, no game data).
|
||||||
|
- **something seen in play:** press **F8** when you see it, and send the folder it names (`captures\<date>-<time>`
|
||||||
|
in the game folder): a picture of the frame, the end of the log, your settings and your save, with no game file
|
||||||
|
among them. The System tab's **Export save for a bug report** makes one zip of the save and the logs instead.
|
||||||
|
- **a Setup or build problem:** `setup.log` from `%LOCALAPPDATA%\Programs\Triforce`. For a build that stopped, press
|
||||||
|
**Copy report** on the build page and paste it: the Setup and tools versions, what the build was started with,
|
||||||
|
the last 200 lines of **Show details** and of `setup.log`, with your user folder written as `%USERPROFILE%` so
|
||||||
|
your name is not in it.
|
||||||
|
|
||||||
Before you report:
|
Before you report:
|
||||||
|
|
||||||
@@ -48,7 +55,7 @@ Before you report:
|
|||||||
injectors: they change what Triforce does.
|
injectors: they change what Triforce does.
|
||||||
- If graphics look wrong after an update, clearing the shader caches often helps: close the game and delete
|
- If graphics look wrong after an update, clearing the shader caches often helps: close the game and delete
|
||||||
`triforce_programs.bin`, `triforce_programs.idx`, `triforce_pipelines.bin`, `triforce_pipeline_states.bin`,
|
`triforce_programs.bin`, `triforce_programs.idx`, `triforce_pipelines.bin`, `triforce_pipeline_states.bin`,
|
||||||
`triforce_computes.bin`, `mkw_vk_pipelines.bin` and `triforce_prepared.bin` from the game folder (never
|
`triforce_computes.bin`, `mkw_vk_pipelines.bin`, `mkw_vk_pipelines_play.bin` and `triforce_prepared.bin` from the game folder (never
|
||||||
`triforce_shaders.bin`). The next start prepares them again.
|
`triforce_shaders.bin`). The next start prepares them again.
|
||||||
|
|
||||||
One topic per issue or pull request, in English, please.
|
One topic per issue or pull request, in English, please.
|
||||||
@@ -90,7 +97,70 @@ To build from a clone instead:
|
|||||||
The launcher lives in `setup/`: a .NET Framework 4.8 WPF app, built with `powershell -File setup\build.ps1` (needs the
|
The launcher lives in `setup/`: a .NET Framework 4.8 WPF app, built with `powershell -File setup\build.ps1` (needs the
|
||||||
C# compiler from Visual Studio Build Tools). `setup\pack_tools.ps1` packs the tools zip a release carries. Never run a
|
C# compiler from Visual Studio Build Tools). `setup\pack_tools.ps1` packs the tools zip a release carries. Never run a
|
||||||
development build's install, update or uninstall without `TRIFORCE_TEST_ROOT` set to a scratch folder: it would act
|
development build's install, update or uninstall without `TRIFORCE_TEST_ROOT` set to a scratch folder: it would act
|
||||||
on your real install.
|
on your real install. With it the install, the settings (a text file in place of the registry), the shortcuts, the
|
||||||
|
game folders and the build's work folder (`<root>\build`) all stay inside that folder. The network is not part of
|
||||||
|
that: such a run still asks the release site and downloads, unless `TRIFORCE_RELEASE_API` names another address.
|
||||||
|
|
||||||
|
What keeps a run of Setup from damaging an install is in `setup/Guard.cs`, written without the window so that
|
||||||
|
`tests/test_setup_*.py` compile it alone (with the same C# compiler) and run it, no Setup started:
|
||||||
|
|
||||||
|
- **One Setup at a time for an install.** The first one holds a lock named after the folder it works in (and, in a
|
||||||
|
test run, after `TRIFORCE_TEST_ROOT`, so a test never meets the real install's Setup). A second start brings the
|
||||||
|
first one's window to the front and ends; an uninstall is refused while Setup is open.
|
||||||
|
- **Room on the disk, asked first.** Before the tools are downloaded Setup asks the disk for room for the zip and for
|
||||||
|
what it unpacks to (five times the zip's size, until the zip is there and says it exactly before its first file is
|
||||||
|
unpacked), and stops with both numbers, what it takes and what is free, when there is none. With less than 60 GB on
|
||||||
|
the game folder's disk the Build button says so in the same way and still lets the player go on. An update of a game
|
||||||
|
folder is not asked about: what it takes depends on what changed.
|
||||||
|
- **Downloads that go on, and are checked.** A file is fetched into `<file>.part`. One that was cut off (the
|
||||||
|
connection, Cancel, Setup closed) is gone on from at the next try with a Range request, but only as a part of the
|
||||||
|
same file: `<file>.part.of` names the address and size it was begun for and the server's ETag (else its date) for
|
||||||
|
it, which goes back as If-Range; anything else is fetched anew. A server that sends nothing for 60 seconds is given
|
||||||
|
up. The file gets its name only once it has the size the release lists, which the server must not contradict. A
|
||||||
|
tools zip that is there whole is not fetched again; one that does not unpack is.
|
||||||
|
- **Copy report**, beside Show details after a build that stopped (not one the player cancelled): what Setup knows of
|
||||||
|
the build and the last 200 lines of its details and of `setup.log` go to the clipboard, with the account's folder
|
||||||
|
(in either slash, doubled backslashes, any case, and its short name) replaced by `%USERPROFILE%`. A name in a path
|
||||||
|
outside that folder stays.
|
||||||
|
- **Its own update, undone when the new Setup does not start.** The new exe takes the running one's name, the running
|
||||||
|
one goes on as `TriforceSetup.old.exe` and starts the new one with `--relaunched`. No second program watches: the
|
||||||
|
old Setup itself stays, out of sight, until the new one's window is up (a named event), 30 seconds at most. A new
|
||||||
|
Setup that cannot be started, or that ends with an error's exit code before that, is replaced by the old exe again in
|
||||||
|
one rename; the old window comes back and says so, and that release's Setup is not fetched again on this PC
|
||||||
|
(`SetupSkip` in `HKCU\Software\Triforce`; the tools still are). One that runs on without the sign, or ends without
|
||||||
|
an error, is left alone.
|
||||||
|
|
||||||
|
Three more parts are written the same way, each a file without the window and a test that runs it on folders of
|
||||||
|
its own:
|
||||||
|
|
||||||
|
- **The Mods page** (`setup/Mods.cs`, `tests/test_setup_mods.py`; the page is `Shell.Mods.cs`). It keeps no list:
|
||||||
|
the mods folder is read as `host/mods.c` reads it, and every change is one a player could make by hand. On and off
|
||||||
|
is the `-` in front of a folder's name; the order is a number in front (`01-`, two or three digits and a hyphen),
|
||||||
|
kept apart from the mod's own name, and moving a mod numbers the folders of the mods folder anew, all of them or
|
||||||
|
none; a removed mod moves into `mods\.removed`; an archive is unpacked into `mods\.adding-<id>` (a name with a
|
||||||
|
dot in front is not looked at by the game), checked as it lies there (no path out of the folder, no link, no
|
||||||
|
device's name, a bound on size and count), and only its mod folder moves into place. `.zip` is read with .NET,
|
||||||
|
`.7z` and `.rar` with the `tar.exe` in System32 and nothing else: do not add an unpacker to the repository. Every
|
||||||
|
write goes through one question, "is this inside the mods folder?", and no change is made while the game of that
|
||||||
|
folder runs. Code patches (`exefs`) are built in: Setup keeps a mark of them at each build (`ModsBuilt` in
|
||||||
|
`HKCU\Software\Triforce`) and says "Build again to apply" while the mods' mark is another one. Cheats are not in
|
||||||
|
that mark: the game reads them when it starts.
|
||||||
|
- **Add to Steam** (`setup/Steam.cs`, `tests/test_setup_steam.py`; the pictures are drawn in `SteamArt.cs`, the
|
||||||
|
link is `Shell.Steam.cs`). Steam's `shortcuts.vdf` is another program's file: it is read whole, other shortcuts are
|
||||||
|
copied through as bytes and never rebuilt, a file that does not read to its end is refused, the new file takes the
|
||||||
|
old one's name in one rename, and the old one is kept (`.triforce-backup`, `.before-triforce`). Nothing is
|
||||||
|
written while Steam runs, and only from the question the player answers. **No test and no test run may touch a
|
||||||
|
real Steam folder:** Steam's folder is always handed in, tests author their own in a temporary folder, and with
|
||||||
|
`TRIFORCE_TEST_ROOT` Setup looks only at `<root>\Steam` and never asks the registry.
|
||||||
|
- **Sounds** (`setup/Sound.cs`, `tests/test_setup_sound.py`; where they play is `Shell.Sounds.cs`). Setup plays the
|
||||||
|
game's own decide and cancel sounds, `Sound/Resource/Stream/SE_Sys_Do00.bwav` and `SE_Sys_Cancel00.bwav`, read from
|
||||||
|
the player's game folder (or from the mod the game itself would read them from) and decoded in memory. **No sound
|
||||||
|
is in the repository, in a release or in Setup, and none may be added:** before a game is built, and for a file that
|
||||||
|
is missing or does not decode, Setup is silent. Nothing decoded is written to the disk. The tests author their own
|
||||||
|
BWAV files and never read a game's. Sounds play only for what the player does (a click, a row chosen, a choice
|
||||||
|
changed; not a page the programme turns), not while the game runs, and never in a run without a player
|
||||||
|
(`--snapshot`, `TRIFORCE_UI_SCRIPT`, `TRIFORCE_TEST_ROOT`): such a run gets, with `TRIFORCE_SOUND_LOG=<file>`, a
|
||||||
|
line in that file for each sound that would have played, and still makes none.
|
||||||
|
|
||||||
## Building
|
## Building
|
||||||
|
|
||||||
@@ -117,6 +187,10 @@ Each replacement is named by the CRC-32 of the module the renderer's own emitter
|
|||||||
coverage line in the log to say so: dump the modules again (`RX_VK_DUMP_ALL_SPV=<folder>`) and match them by what they
|
coverage line in the log to say so: dump the modules again (`RX_VK_DUMP_ALL_SPV=<folder>`) and match them by what they
|
||||||
read. Never share a game folder built with it.
|
read. Never share a game folder built with it.
|
||||||
|
|
||||||
|
What RenoDX's mod does not have (RenoDRT as a tone mapper, the gamma correction) is in `gpu/vk/shaders/renodx_glue`:
|
||||||
|
files of Triforce's own that are compiled together with the fetched shaders and call RenoDX's library there. Keep it
|
||||||
|
that way: nothing of RenoDX's source goes into them or anywhere else in the repository.
|
||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
The game-free checks need no game files:
|
The game-free checks need no game files:
|
||||||
@@ -158,7 +232,7 @@ it builds on. By submitting a change you agree to license it under those terms.
|
|||||||
an incompatible license, and never paste decompiled game code or anything generated from the game files.
|
an incompatible license, and never paste decompiled game code or anything generated from the game files.
|
||||||
|
|
||||||
The third-party parts (Eden, suyu, Sirit, the Vulkan Memory Allocator, SDL, Opus, Dear ImGui, zstd, the Libertinus
|
The third-party parts (Eden, suyu, Sirit, the Vulkan Memory Allocator, SDL, Opus, Dear ImGui, zstd, the Libertinus
|
||||||
fonts, zbic and the a64-kit tools) keep their own notices. Preserve them.
|
fonts, SMAA, zbic and the a64-kit tools) keep their own notices. Preserve them.
|
||||||
|
|
||||||
Thanks for testing, reporting and sending fixes. Every report from a different GPU and every area played further
|
Thanks for testing, reporting and sending fixes. Every report from a different GPU and every area played further
|
||||||
moves the port forward.
|
moves the port forward.
|
||||||
|
|||||||
6
assets/triforce_assets/CREDITS.txt
Normal file
6
assets/triforce_assets/CREDITS.txt
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
Majora's Mask (majora_mask.bin, majora_alb.png, majora_emc.png)
|
||||||
|
3D model "Majora's Mask" by PigeonMage, https://sketchfab.com/3d-models/majoras-mask-e9cfcdf9875846f6b6137f0156cd4eb2
|
||||||
|
Licence: Creative Commons Attribution 4.0 (CC BY 4.0), https://creativecommons.org/licenses/by/4.0/
|
||||||
|
Changes: the mesh placed on the Happy Mask Salesman's head bone, scaled, and its textures resized to 1024 x 1024.
|
||||||
|
|
||||||
|
The Happy Mask Salesman himself is not in these files: Triforce reads his model and animations from your own game files.
|
||||||
BIN
assets/triforce_assets/majora_alb.png
Normal file
BIN
assets/triforce_assets/majora_alb.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.4 MiB |
BIN
assets/triforce_assets/majora_emc.png
Normal file
BIN
assets/triforce_assets/majora_emc.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 129 KiB |
BIN
assets/triforce_assets/majora_mask.bin
Normal file
BIN
assets/triforce_assets/majora_mask.bin
Normal file
Binary file not shown.
@@ -10,7 +10,21 @@ public partial class App : Application
|
|||||||
// Why it stopped is one "error <text>" line on stdout (the stream a launcher reads) and again on stderr
|
// Why it stopped is one "error <text>" line on stdout (the stream a launcher reads) and again on stderr
|
||||||
protected override void OnStartup(StartupEventArgs e)
|
protected override void OnStartup(StartupEventArgs e)
|
||||||
{
|
{
|
||||||
if (e.Args.Length > 0 && e.Args[0] == "--cli")
|
var cli = e.Args.Length > 0 && e.Args[0] == "--cli";
|
||||||
|
// ((Triforce) one builder at a time for its tools, Instance.cs: asked before any build or window)
|
||||||
|
if (Tools.Find() is { } tools && !Instance.Take(tools.Root))
|
||||||
|
{
|
||||||
|
if (cli)
|
||||||
|
{
|
||||||
|
var said = "error " + Instance.Refusal(tools.Root).Replace("\n\n", " ");
|
||||||
|
Console.Out.WriteLine(said);
|
||||||
|
Console.Error.WriteLine(said);
|
||||||
|
}
|
||||||
|
else MessageBox.Show(Instance.Refusal(tools.Root), "Hookshot Builder", MessageBoxButton.OK, MessageBoxImage.Information);
|
||||||
|
Environment.Exit(1);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (cli)
|
||||||
{
|
{
|
||||||
Environment.Exit(Cli(e.Args.Skip(1).ToList()));
|
Environment.Exit(Cli(e.Args.Skip(1).ToList()));
|
||||||
return;
|
return;
|
||||||
|
|||||||
53
builder/HookshotBuilder/Instance.cs
Normal file
53
builder/HookshotBuilder/Instance.cs
Normal file
@@ -0,0 +1,53 @@
|
|||||||
|
using System;
|
||||||
|
using System.IO;
|
||||||
|
using System.Threading;
|
||||||
|
|
||||||
|
namespace HookshotBuilder;
|
||||||
|
|
||||||
|
/// (Triforce, handoff §5) One Hookshot Builder at a time for its build tools. Two of them on one tools folder ran make_game.py on
|
||||||
|
/// the same work folder, and make_game.py stops a build it finds running there: the first one's build ended without a reason.
|
||||||
|
/// The first builder holds a named mutex for as long as it runs (Windows lets go of it when the process ends, however it ends);
|
||||||
|
/// a second one for the same tools says so and ends. The name comes from the tools folder (Tools.Root), so builders of other
|
||||||
|
/// folders, Triforce Setup (its lock is named after its own folder) and the game (after its game folder) never meet it.
|
||||||
|
/// HOOKSHOT_SINGLE_INSTANCE=0: no lock (as before). Plain logic: tests/test_builder_instance.py compiles this file alone
|
||||||
|
static class Instance
|
||||||
|
{
|
||||||
|
static Mutex? held;
|
||||||
|
|
||||||
|
/// The lock's name for a tools folder: one for each spelling of it (case, '/' or '\', a separator at its end), a 64-bit FNV-1a
|
||||||
|
/// of that spelling. Local\: this user's session
|
||||||
|
public static string Name(string root)
|
||||||
|
{
|
||||||
|
try { root = Path.GetFullPath(root); } catch (Exception) { }
|
||||||
|
var text = root.Replace('/', '\\').TrimEnd('\\').ToUpperInvariant();
|
||||||
|
var h = 14695981039346656037UL; // (a name, not a secret)
|
||||||
|
foreach (var c in text) { h ^= c; h *= 1099511628211UL; }
|
||||||
|
return @"Local\HookshotBuilder-" + h.ToString("x16");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// True when this process now holds the tools folder (or did already; it waited up to waitMs for a builder that was ending),
|
||||||
|
/// false when another builder does. Where no lock can be made at all nobody is refused
|
||||||
|
public static bool Take(string root, int waitMs = 5000)
|
||||||
|
{
|
||||||
|
if (held != null) return true;
|
||||||
|
if (Environment.GetEnvironmentVariable("HOOKSHOT_SINGLE_INSTANCE") == "0") return true;
|
||||||
|
Mutex m;
|
||||||
|
try { m = new Mutex(false, Name(root)); }
|
||||||
|
catch (UnauthorizedAccessException) { return false; } // (it is there, and belongs to a builder run as administrator)
|
||||||
|
catch (Exception) { return true; }
|
||||||
|
try
|
||||||
|
{
|
||||||
|
if (!m.WaitOne(waitMs)) { m.Dispose(); return false; }
|
||||||
|
}
|
||||||
|
catch (AbandonedMutexException) { } // (its holder ended without letting go: Windows hands it on, and it is ours)
|
||||||
|
catch (Exception) { m.Dispose(); return true; }
|
||||||
|
held = m;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a second builder says
|
||||||
|
public static string Refusal(string root) =>
|
||||||
|
"Hookshot Builder is already running for these build tools:\n\n" + root +
|
||||||
|
"\n\nUse the window that is open, or close it and start the builder again. (Two builders on one tools folder would build " +
|
||||||
|
"into the same work folder and stop each other's build.)";
|
||||||
|
}
|
||||||
@@ -952,6 +952,15 @@ public partial class MainWindow : Window
|
|||||||
clock.Stop();
|
clock.Stop();
|
||||||
elapsed.Stop();
|
elapsed.Stop();
|
||||||
runner = null;
|
runner = null;
|
||||||
|
// ((Triforce, 11-18) a Cancel first, whatever the run said: Runner.Cancel asks the game's shader preparation to stop
|
||||||
|
// and it ends well, so make_game.py goes on to its @@done. Taken as done, a Cancel in "Preparing shaders" said "Your
|
||||||
|
// game is ready", made the shortcuts and freed the build's files, where it went back to the files, as before)
|
||||||
|
if (failure == "cancelled")
|
||||||
|
{
|
||||||
|
SetStep(0);
|
||||||
|
ShowPage(FilesPage);
|
||||||
|
return;
|
||||||
|
}
|
||||||
if (finished.Length > 0)
|
if (finished.Length > 0)
|
||||||
{
|
{
|
||||||
foreach (var (id, r) in rows)
|
foreach (var (id, r) in rows)
|
||||||
@@ -964,12 +973,6 @@ public partial class MainWindow : Window
|
|||||||
ShowReady();
|
ShowReady();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
if (failure == "cancelled")
|
|
||||||
{
|
|
||||||
SetStep(0);
|
|
||||||
ShowPage(FilesPage);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
if (current != null && rows.TryGetValue(current, out var row)) row.Stopped();
|
if (current != null && rows.TryGetValue(current, out var row)) row.Stopped();
|
||||||
ShowProblem(failure ?? "The build tools stopped without saying why.", tools: failure?.Contains("antivirus", StringComparison.OrdinalIgnoreCase) == true);
|
ShowProblem(failure ?? "The build tools stopped without saying why.", tools: failure?.Contains("antivirus", StringComparison.OrdinalIgnoreCase) == true);
|
||||||
}
|
}
|
||||||
@@ -1231,7 +1234,7 @@ public partial class MainWindow : Window
|
|||||||
void CloseWindow(object sender, RoutedEventArgs e)
|
void CloseWindow(object sender, RoutedEventArgs e)
|
||||||
{
|
{
|
||||||
if (runner != null && MessageBox.Show(this, T(S.CloseQuestion), "Hookshot", MessageBoxButton.YesNo) != MessageBoxResult.Yes) return;
|
if (runner != null && MessageBox.Show(this, T(S.CloseQuestion), "Hookshot", MessageBoxButton.YesNo) != MessageBoxResult.Yes) return;
|
||||||
runner?.Cancel();
|
runner?.Cancel().Wait(); // (the app ends with the window: a shader preparation asked to stop is waited for, or killed)
|
||||||
Close();
|
Close();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
123
builder/HookshotBuilder/Reuse.cs
Normal file
123
builder/HookshotBuilder/Reuse.cs
Normal file
@@ -0,0 +1,123 @@
|
|||||||
|
using System;
|
||||||
|
using System.IO;
|
||||||
|
using System.Linq;
|
||||||
|
using System.Threading;
|
||||||
|
|
||||||
|
namespace HookshotBuilder;
|
||||||
|
|
||||||
|
/// (Triforce) What the setup (builder/setup/setup.c) lends from the installed version to the one it puts together, and the lock
|
||||||
|
/// every setup holds while it does. An update moves the folders of parts that did not change (Python, the compiler...) from
|
||||||
|
/// versions\<installed> into versions\<tag>.part and writes each down first in its reused.txt (the installed version's tag,
|
||||||
|
/// then a folder a line); a cut-off setup leaves them there. Free gives them back (UndoEveryReuse, setup.c undo_every_reuse,
|
||||||
|
/// the same rules) and never deletes a <tag>.part that still holds them (Lent): deleted, the installed version lost them for
|
||||||
|
/// good. Plain logic: tests/test_builder_setup_lock.py compiles this file alone
|
||||||
|
static class Reuse
|
||||||
|
{
|
||||||
|
public const string Note = "reused.txt";
|
||||||
|
|
||||||
|
/// True when folder is an unfinished version (a versions\*.part) that still holds folders of another version
|
||||||
|
public static bool Lent(string folder) =>
|
||||||
|
Path.GetFileName(folder.TrimEnd('\\', '/')).EndsWith(".part", StringComparison.OrdinalIgnoreCase) &&
|
||||||
|
File.Exists(Path.Combine(folder, Note));
|
||||||
|
|
||||||
|
/// The setup's lock for a setup folder (setup.c setup_lock_name and setup_parts.lock_name, the same): a 64-bit FNV-1a of the
|
||||||
|
/// folder's full path with '\' between its parts, none at its end, a to z in upper case. Local\: this user's session
|
||||||
|
public static string LockName(string setupBase)
|
||||||
|
{
|
||||||
|
try { setupBase = Path.GetFullPath(setupBase); } catch (Exception) { }
|
||||||
|
var text = setupBase.Replace('/', '\\').TrimEnd('\\');
|
||||||
|
var h = 14695981039346656037UL; // (a name, not a secret)
|
||||||
|
foreach (var c in text) { h ^= c >= 'a' && c <= 'z' ? (uint)(c - 32) : c; h *= 1099511628211UL; }
|
||||||
|
return @"Local\HookshotBuilder-Setup-" + h.ToString("x16");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The setup folder held by this thread until Dispose (one that holds nothing where no lock can be made at all), or null
|
||||||
|
/// when a setup held it for waitMs: one is installing now
|
||||||
|
public static Held? TakeLock(string setupBase, int waitMs = 0)
|
||||||
|
{
|
||||||
|
Mutex m;
|
||||||
|
try { m = new Mutex(false, LockName(setupBase)); }
|
||||||
|
catch (UnauthorizedAccessException) { return null; } // (it is there, and belongs to a setup run as administrator)
|
||||||
|
catch (Exception) { return new Held(null); }
|
||||||
|
try
|
||||||
|
{
|
||||||
|
if (!m.WaitOne(waitMs)) { m.Dispose(); return null; }
|
||||||
|
}
|
||||||
|
catch (AbandonedMutexException) { } // (its holder ended without letting go: Windows hands it on, and it is ours)
|
||||||
|
catch (Exception) { m.Dispose(); return new Held(null); }
|
||||||
|
return new Held(m);
|
||||||
|
}
|
||||||
|
|
||||||
|
public sealed class Held : IDisposable
|
||||||
|
{
|
||||||
|
Mutex? m;
|
||||||
|
internal Held(Mutex? m) => this.m = m;
|
||||||
|
|
||||||
|
public void Dispose()
|
||||||
|
{
|
||||||
|
if (m == null) return;
|
||||||
|
try { m.ReleaseMutex(); } catch (Exception) { }
|
||||||
|
m.Dispose();
|
||||||
|
m = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The folders a setup moved from the installed version into install (a <tag>.part), moved back: each only where that version
|
||||||
|
/// has none; the note goes once all went back. True: nothing is left to move back
|
||||||
|
public static bool Undo(string versions, string install)
|
||||||
|
{
|
||||||
|
var note = Path.Combine(install, Note);
|
||||||
|
string[] lines;
|
||||||
|
try { lines = File.ReadAllLines(note).Where(l => l.Trim().Length > 0).ToArray(); }
|
||||||
|
catch (Exception e) when (e is IOException or UnauthorizedAccessException) { return !File.Exists(note); }
|
||||||
|
if (lines.Length == 0 || !NameOk(lines[0], false)) // (no version named: nothing was moved yet)
|
||||||
|
{
|
||||||
|
try { File.Delete(note); } catch (Exception e) when (e is IOException or UnauthorizedAccessException) { }
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
var ok = true;
|
||||||
|
foreach (var rel in lines.Skip(1))
|
||||||
|
{
|
||||||
|
if (!NameOk(rel, true)) continue;
|
||||||
|
var from = Path.Combine(install, rel.Replace('/', '\\'));
|
||||||
|
var back = Path.Combine(versions, lines[0], rel.Replace('/', '\\'));
|
||||||
|
if (!(Directory.Exists(from) || File.Exists(from)) || Directory.Exists(back) || File.Exists(back)) continue;
|
||||||
|
try
|
||||||
|
{
|
||||||
|
Directory.CreateDirectory(Path.GetDirectoryName(back)!);
|
||||||
|
if (Directory.Exists(from)) Directory.Move(from, back);
|
||||||
|
else File.Move(from, back);
|
||||||
|
}
|
||||||
|
catch (Exception e) when (e is IOException or UnauthorizedAccessException) { ok = false; }
|
||||||
|
}
|
||||||
|
if (ok)
|
||||||
|
try { File.Delete(note); } catch (Exception e) when (e is IOException or UnauthorizedAccessException) { }
|
||||||
|
return ok;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Undo for every versions\*.part with a note; the first whose folders could not all go back (each is tried), null when none
|
||||||
|
public static string? UndoEveryReuse(string versions)
|
||||||
|
{
|
||||||
|
string? left = null;
|
||||||
|
string[] dirs;
|
||||||
|
try { dirs = Directory.GetDirectories(versions); }
|
||||||
|
catch (Exception e) when (e is IOException or UnauthorizedAccessException) { return null; }
|
||||||
|
foreach (var d in dirs.OrderBy(d => d, StringComparer.Ordinal))
|
||||||
|
if (Lent(d) && !Undo(versions, d)) left ??= d;
|
||||||
|
return left;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A name from the note (setup.c list_name_ok, the same): letters, digits and . _ + - only (with path, '/' or '\' between
|
||||||
|
/// its parts), no "." or ".." part, no drive, no root
|
||||||
|
static bool NameOk(string s, bool path)
|
||||||
|
{
|
||||||
|
if (s.Length == 0 || s[0] == '/' || s[0] == '\\') return false;
|
||||||
|
foreach (var c in s)
|
||||||
|
if (!(c is >= 'a' and <= 'z' or >= 'A' and <= 'Z' or >= '0' and <= '9' or '.' or '_' or '+' or '-' ||
|
||||||
|
(path && (c == '/' || c == '\\'))))
|
||||||
|
return false;
|
||||||
|
foreach (var part in path ? s.Split('/', '\\') : new[] { s })
|
||||||
|
if (part.Length == 0 || part == "." || part == "..") return false;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -81,7 +81,12 @@ sealed class Runner
|
|||||||
this.ngxSdk = ngxSdk;
|
this.ngxSdk = ngxSdk;
|
||||||
}
|
}
|
||||||
|
|
||||||
static ProcessStartInfo Info(Tools tools, IEnumerable<string> args, int? workers = null, string? ngxSdk = null)
|
/// (Triforce, 11-18) The event this run's game opens while it prepares its shaders (gpu/vk/vk_main.cpp PrepareStopWatch),
|
||||||
|
/// named for this run alone: a Cancel here never stops another builder's preparation
|
||||||
|
readonly string stopEvent = @"Local\TriforcePrepareStop." + Guid.NewGuid().ToString("N");
|
||||||
|
|
||||||
|
static ProcessStartInfo Info(Tools tools, IEnumerable<string> args, int? workers = null, string? ngxSdk = null,
|
||||||
|
string? stopEvent = null)
|
||||||
{
|
{
|
||||||
var si = new ProcessStartInfo(tools.Python)
|
var si = new ProcessStartInfo(tools.Python)
|
||||||
{
|
{
|
||||||
@@ -98,6 +103,9 @@ sealed class Runner
|
|||||||
// (NVIDIA DLSS: only when ticked. Without it, one set on the PC is not passed on either: a build is with DLSS only when asked)
|
// (NVIDIA DLSS: only when ticked. Without it, one set on the PC is not passed on either: a build is with DLSS only when asked)
|
||||||
if (ngxSdk != null) si.Environment["HOOKSHOT_NGX_SDK"] = ngxSdk;
|
if (ngxSdk != null) si.Environment["HOOKSHOT_NGX_SDK"] = ngxSdk;
|
||||||
else si.Environment.Remove("HOOKSHOT_NGX_SDK");
|
else si.Environment.Remove("HOOKSHOT_NGX_SDK");
|
||||||
|
// (not a HOOKSHOT_ name: make_game.py's game_env keeps it for the game)
|
||||||
|
if (stopEvent != null) si.Environment["TRIFORCE_PREPARE_STOP"] = stopEvent;
|
||||||
|
else si.Environment.Remove("TRIFORCE_PREPARE_STOP");
|
||||||
// (the folder's own compiler, headers and libraries, when it has them: nothing of the PC's but Windows' own)
|
// (the folder's own compiler, headers and libraries, when it has them: nothing of the PC's but Windows' own)
|
||||||
var sets = new[] { ("EMSDK", "emsdk"), ("RX_CPM", "cpm"), ("WGPU_NATIVE", "wgpu-native") };
|
var sets = new[] { ("EMSDK", "emsdk"), ("RX_CPM", "cpm"), ("WGPU_NATIVE", "wgpu-native") };
|
||||||
foreach (var (name, folder) in sets)
|
foreach (var (name, folder) in sets)
|
||||||
@@ -118,7 +126,7 @@ sealed class Runner
|
|||||||
{
|
{
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
using var p = Process.Start(Info(tools, args, workers, ngxSdk))!;
|
using var p = Process.Start(Info(tools, args, workers, ngxSdk, stopEvent))!;
|
||||||
process = p;
|
process = p;
|
||||||
p.ErrorDataReceived += (_, e) => { if (e.Data != null) Line?.Invoke(e.Data); };
|
p.ErrorDataReceived += (_, e) => { if (e.Data != null) Line?.Invoke(e.Data); };
|
||||||
p.BeginErrorReadLine();
|
p.BeginErrorReadLine();
|
||||||
@@ -138,10 +146,26 @@ sealed class Runner
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
public void Cancel()
|
/// (Triforce, 11-18) The game's --prepare-shaders run, when that is the step under way, is asked to stop first (the event
|
||||||
|
/// gpu/vk/vk_main.cpp PrepareStopWatch waits on): it ends its driver's part cleanly and keeps what it made, where a kill
|
||||||
|
/// used one of that part's three tries and left shaders out for good. Killed as before when it has not ended in 30 s,
|
||||||
|
/// and at once when no preparation is under way. The wait is not on the caller's thread
|
||||||
|
public Task Cancel()
|
||||||
{
|
{
|
||||||
cancelled = true;
|
cancelled = true;
|
||||||
try { process?.Kill(entireProcessTree: true); } catch (Exception) { }
|
var p = process;
|
||||||
|
var asked = false;
|
||||||
|
try
|
||||||
|
{
|
||||||
|
if (EventWaitHandle.TryOpenExisting(stopEvent, out var stop))
|
||||||
|
using (stop) asked = stop.Set();
|
||||||
|
}
|
||||||
|
catch (Exception) { }
|
||||||
|
return Task.Run(() =>
|
||||||
|
{
|
||||||
|
try { if (asked && p != null && p.WaitForExit(30000)) return; } catch (Exception) { }
|
||||||
|
try { p?.Kill(entireProcessTree: true); } catch (Exception) { }
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Is this the supported game and version? The answer in plain words.
|
/// Is this the supported game and version? The answer in plain words.
|
||||||
|
|||||||
@@ -75,17 +75,20 @@ static class Storage
|
|||||||
|
|
||||||
/// What Free lets go. afterBuild: what goes once the game is ready, the compiled objects of the build (objcache, which make_game.py
|
/// What Free lets go. afterBuild: what goes once the game is ready, the compiled objects of the build (objcache, which make_game.py
|
||||||
/// prunes to them) and their system headers' stamp kept, so that the next update compiles only what changed (issue 1287: every
|
/// prunes to them) and their system headers' stamp kept, so that the next update compiles only what changed (issue 1287: every
|
||||||
/// update compiled all ~600 files again, an hour and more). The Free up space button lets those go too
|
/// update compiled all ~600 files again, an hour and more). The Free up space button lets those go too. (Triforce) Never an
|
||||||
static IEnumerable<string> Freeable(bool afterBuild = false)
|
/// unfinished version that still holds folders of the installed one (Reuse.Lent): deleted, the installed version lost its
|
||||||
|
/// Python or its compiler for good. installing: a setup holds the folder now, and what it is putting together (downloads\,
|
||||||
|
/// versions\) stays. Nothing is changed here: Measure counts it
|
||||||
|
static IEnumerable<string> Freeable(bool afterBuild = false, bool installing = false)
|
||||||
{
|
{
|
||||||
foreach (var d in afterBuild ? new[] { "sa", "exefs", "probe" } : new[] { "objcache", "sa", "exefs", "probe" })
|
foreach (var d in afterBuild ? new[] { "sa", "exefs", "probe" } : new[] { "objcache", "sa", "exefs", "probe" })
|
||||||
yield return Path.Combine(Work, d);
|
yield return Path.Combine(Work, d);
|
||||||
foreach (var d in new[] { "downloads", "update" })
|
foreach (var d in installing ? new[] { "update" } : new[] { "downloads", "update" })
|
||||||
yield return Path.Combine(Base, d);
|
yield return Path.Combine(Base, d);
|
||||||
var versions = Path.Combine(Base, "versions");
|
var versions = Path.Combine(Base, "versions");
|
||||||
if (Directory.Exists(versions))
|
if (Directory.Exists(versions) && !installing)
|
||||||
foreach (var v in Directory.EnumerateDirectories(versions))
|
foreach (var v in Directory.EnumerateDirectories(versions))
|
||||||
if (!Kept(v)) yield return v;
|
if (!Kept(v) && !Reuse.Lent(v)) yield return v;
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Bytes in the builder's folder and the build files folder, and in what Free lets go (an object hard-linked in both sa\build and
|
/// Bytes in the builder's folder and the build files folder, and in what Free lets go (an object hard-linked in both sa\build and
|
||||||
@@ -103,12 +106,18 @@ static class Storage
|
|||||||
/// Lets go of what Measure counts as freeable; returns the bytes it could not delete (a file in use)
|
/// Lets go of what Measure counts as freeable; returns the bytes it could not delete (a file in use)
|
||||||
public static long Free(bool afterBuild = false)
|
public static long Free(bool afterBuild = false)
|
||||||
{
|
{
|
||||||
foreach (var d in Freeable(afterBuild)) Delete(d);
|
// ((Triforce) what a cut-off setup moved from the installed version goes back first (Reuse.UndoEveryReuse), and then its
|
||||||
|
// <tag>.part is freeable; one whose folders could not all go back stays. With the setup's folder held: a setup running now
|
||||||
|
// is not undone under it, and its folders are left alone)
|
||||||
|
using var setup = Reuse.TakeLock(Base);
|
||||||
|
var installing = setup == null;
|
||||||
|
if (!installing) Reuse.UndoEveryReuse(Path.Combine(Base, "versions"));
|
||||||
|
foreach (var d in Freeable(afterBuild, installing)) Delete(d);
|
||||||
if (!afterBuild) Delete(Path.Combine(Work, "system_headers.txt")); // (the stamp of the compiled objects' system headers goes with them)
|
if (!afterBuild) Delete(Path.Combine(Work, "system_headers.txt")); // (the stamp of the compiled objects' system headers goes with them)
|
||||||
if (!afterBuild) return Measure().freeable;
|
if (!afterBuild) return Measure().freeable;
|
||||||
var seen = new HashSet<(uint, ulong)>(); // (what of the after-build set is still there: a file in use)
|
var seen = new HashSet<(uint, ulong)>(); // (what of the after-build set is still there: a file in use)
|
||||||
long left = 0;
|
long left = 0;
|
||||||
foreach (var d in Freeable(afterBuild: true)) left += Size(d, seen);
|
foreach (var d in Freeable(afterBuild: true, installing)) left += Size(d, seen);
|
||||||
return left;
|
return left;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -302,27 +302,85 @@ static class Updater
|
|||||||
using var response = await http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, token);
|
using var response = await http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, token);
|
||||||
if (!response.IsSuccessStatusCode) throw HttpFailure(response);
|
if (!response.IsSuccessStatusCode) throw HttpFailure(response);
|
||||||
|
|
||||||
var total = response.Content.Headers.ContentLength ?? size;
|
// ((Triforce) the size the release lists, which the server must not contradict; the server's only where none is listed (the
|
||||||
await using var from = await response.Content.ReadAsStreamAsync(token);
|
// server's went first, and a short or other file passed as whole). Room for it asked before the first byte, and the bytes
|
||||||
await using var to = File.Create(path);
|
// written to <file>.part, which becomes the file only once complete: a failure removes it)
|
||||||
var buffer = new byte[1 << 20];
|
var told = response.Content.Headers.ContentLength;
|
||||||
long have = 0;
|
if (size > 0 && told is { } other && other != size)
|
||||||
var shown = DateTime.MinValue;
|
throw new IOException($"The server offers {other} bytes as {Path.GetFileName(path)}, and the release lists {size}. Nothing was downloaded.");
|
||||||
int n;
|
var total = size > 0 ? size : told ?? 0;
|
||||||
while (true)
|
if (total > 0 && NoRoom(path, total) is { } none) throw new IOException(none);
|
||||||
|
var part = path + ".part";
|
||||||
|
try
|
||||||
{
|
{
|
||||||
try { n = await from.ReadAsync(buffer, token); }
|
await using (var from = await response.Content.ReadAsStreamAsync(token))
|
||||||
catch (IOException e) { throw new HttpRequestException("The download connection ended unexpectedly.", e); }
|
await using (var to = File.Create(part))
|
||||||
if (n == 0) break;
|
|
||||||
await to.WriteAsync(buffer.AsMemory(0, n), token);
|
|
||||||
have += n;
|
|
||||||
if (progress != null && total > 0 && DateTime.UtcNow - shown > TimeSpan.FromMilliseconds(200))
|
|
||||||
{
|
{
|
||||||
shown = DateTime.UtcNow;
|
var buffer = new byte[1 << 20];
|
||||||
progress(Math.Min(1.0, (double)have / total));
|
long have = 0;
|
||||||
|
var shown = DateTime.MinValue;
|
||||||
|
int n;
|
||||||
|
while (true)
|
||||||
|
{
|
||||||
|
try { n = await from.ReadAsync(buffer, token); }
|
||||||
|
catch (IOException e) { throw new HttpRequestException("The download connection ended unexpectedly.", e); }
|
||||||
|
if (n == 0) break;
|
||||||
|
await to.WriteAsync(buffer.AsMemory(0, n), token);
|
||||||
|
have += n;
|
||||||
|
if (total > 0 && have > total) throw new IOException($"The server sent more than the {total} bytes of {Path.GetFileName(path)}.");
|
||||||
|
if (progress != null && total > 0 && DateTime.UtcNow - shown > TimeSpan.FromMilliseconds(200))
|
||||||
|
{
|
||||||
|
shown = DateTime.UtcNow;
|
||||||
|
progress(Math.Min(1.0, (double)have / total));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (total > 0 && have != total) throw new HttpRequestException($"The download ended early ({have >> 20} of {total >> 20} MB).");
|
||||||
}
|
}
|
||||||
|
File.Move(part, path, true);
|
||||||
}
|
}
|
||||||
if (total > 0 && have != total) throw new HttpRequestException($"The download ended early ({have >> 20} of {total >> 20} MB).");
|
catch (Exception)
|
||||||
|
{
|
||||||
|
try { File.Delete(part); } catch (Exception) { }
|
||||||
|
throw;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// ((Triforce) setup/Guard.cs Space, the same rule) Null when the disk of path has room for need bytes and 32 MB more (or
|
||||||
|
/// cannot be asked); else the sentence that says how much it takes and how much is free
|
||||||
|
static string? NoRoom(string path, long need)
|
||||||
|
{
|
||||||
|
long free;
|
||||||
|
try { free = new DriveInfo(Path.GetPathRoot(Path.GetFullPath(path))!).AvailableFreeSpace; }
|
||||||
|
catch (Exception) { return null; }
|
||||||
|
need += 32L << 20;
|
||||||
|
return free >= need ? null
|
||||||
|
: $"There is no room for {Path.GetFileName(path)} on {Path.GetPathRoot(Path.GetFullPath(path))}: it takes {(need + (1 << 20) - 1) >> 20} MB there, and {free >> 20} MB are free. Make room and try again.";
|
||||||
|
}
|
||||||
|
|
||||||
|
static readonly HashSet<string> Devices = new(StringComparer.OrdinalIgnoreCase) { "CON", "PRN", "AUX", "NUL", "CONIN$", "CONOUT$" };
|
||||||
|
|
||||||
|
/// ((Triforce) setup/Mods.cs Unsafe, the same rules) Why a zip's member must not be unpacked under its name, null when it may:
|
||||||
|
/// a path from the disk's root or through "..", a colon (a drive's letter, a hidden stream), a name Windows keeps for a device
|
||||||
|
/// (with or without an extension), a name that ends in a dot or a space (Windows drops them), a character Windows refuses
|
||||||
|
internal static string? UnsafeName(string name)
|
||||||
|
{
|
||||||
|
var n = name.Replace('\\', '/');
|
||||||
|
if (n.Trim().Length == 0) return "a file without a name";
|
||||||
|
if (n[0] == '/') return "a path that starts at the disk's root";
|
||||||
|
if (n.IndexOf(':') >= 0) return "a name with a colon or a drive's letter";
|
||||||
|
foreach (var p in n.Split('/'))
|
||||||
|
{
|
||||||
|
if (p.Length == 0 || p == ".") continue;
|
||||||
|
if (p == "..") return "a path that leaves its own folder (..)";
|
||||||
|
if (p[p.Length - 1] == '.' || p[p.Length - 1] == ' ') return "a name that ends in a dot or a space, which Windows drops";
|
||||||
|
var dot = p.IndexOf('.');
|
||||||
|
var stem = (dot >= 0 ? p.Substring(0, dot) : p).TrimEnd(' ');
|
||||||
|
if (Devices.Contains(stem) || stem.Length == 4 && (stem.StartsWith("COM", StringComparison.OrdinalIgnoreCase) || stem.StartsWith("LPT", StringComparison.OrdinalIgnoreCase)) &&
|
||||||
|
(stem[3] is >= '1' and <= '9' || stem[3] == (char)0xB9 || stem[3] == (char)0xB2 || stem[3] == (char)0xB3))
|
||||||
|
return "a name Windows keeps for a device (" + p + ")";
|
||||||
|
if (p.Any(c => c < 32 || "<>\"|?*".IndexOf(c) >= 0)) return "a name with a character Windows does not allow";
|
||||||
|
}
|
||||||
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
static HttpRequestException HttpFailure(HttpResponseMessage response) =>
|
static HttpRequestException HttpFailure(HttpResponseMessage response) =>
|
||||||
@@ -335,14 +393,41 @@ static class Updater
|
|||||||
error is HttpRequestException { StatusCode: { } status } ? $" (HTTP {(int)status})" : "";
|
error is HttpRequestException { StatusCode: { } status } ? $" (HTTP {(int)status})" : "";
|
||||||
|
|
||||||
/// The zip's HookshotBuilder folder, as target (an older one there replaced)
|
/// The zip's HookshotBuilder folder, as target (an older one there replaced)
|
||||||
|
/// ((Triforce) every name checked first (UnsafeName: such a zip is refused whole), a failure removes the part, and an older
|
||||||
|
/// target is moved aside and back when the new one cannot take its place: before, it was deleted first, and a move that
|
||||||
|
/// failed then left neither)
|
||||||
static async Task<string> Unpack(string zip, string target)
|
static async Task<string> Unpack(string zip, string target)
|
||||||
{
|
{
|
||||||
var part = target + ".part";
|
var part = target + ".part";
|
||||||
if (Directory.Exists(part)) Directory.Delete(part, true);
|
if (Directory.Exists(part)) Directory.Delete(part, true);
|
||||||
await Task.Run(() => ZipFile.ExtractToDirectory(zip, part));
|
try
|
||||||
|
{
|
||||||
|
await Task.Run(() =>
|
||||||
|
{
|
||||||
|
using (var z = ZipFile.OpenRead(zip))
|
||||||
|
foreach (var e in z.Entries)
|
||||||
|
if (UnsafeName(e.FullName) is { } why)
|
||||||
|
throw new InvalidDataException($"The download holds {why} (\"{e.FullName}\"): it was not unpacked.");
|
||||||
|
ZipFile.ExtractToDirectory(zip, part);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
catch (Exception)
|
||||||
|
{
|
||||||
|
try { if (Directory.Exists(part)) Directory.Delete(part, true); } catch (Exception) { }
|
||||||
|
throw;
|
||||||
|
}
|
||||||
var inner = Path.Combine(part, "HookshotBuilder");
|
var inner = Path.Combine(part, "HookshotBuilder");
|
||||||
if (Directory.Exists(target)) Directory.Delete(target, true);
|
var old = target + ".old";
|
||||||
Directory.Move(Directory.Exists(inner) ? inner : part, target);
|
if (Directory.Exists(old)) Directory.Delete(old, true);
|
||||||
|
if (Directory.Exists(target)) Directory.Move(target, old);
|
||||||
|
try { Directory.Move(Directory.Exists(inner) ? inner : part, target); }
|
||||||
|
catch (Exception)
|
||||||
|
{
|
||||||
|
try { if (!Directory.Exists(target) && Directory.Exists(old)) Directory.Move(old, target); } catch (Exception) { }
|
||||||
|
try { if (Directory.Exists(part)) Directory.Delete(part, true); } catch (Exception) { }
|
||||||
|
throw;
|
||||||
|
}
|
||||||
|
try { if (Directory.Exists(old)) Directory.Delete(old, true); } catch (Exception) { }
|
||||||
if (Directory.Exists(part)) Directory.Delete(part, true);
|
if (Directory.Exists(part)) Directory.Delete(part, true);
|
||||||
return target;
|
return target;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -11,6 +11,7 @@
|
|||||||
<application xmlns="urn:schemas-microsoft-com:asm.v3">
|
<application xmlns="urn:schemas-microsoft-com:asm.v3">
|
||||||
<windowsSettings>
|
<windowsSettings>
|
||||||
<dpiAwareness xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">PerMonitorV2</dpiAwareness>
|
<dpiAwareness xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">PerMonitorV2</dpiAwareness>
|
||||||
|
<longPathAware xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">true</longPathAware>
|
||||||
</windowsSettings>
|
</windowsSettings>
|
||||||
</application>
|
</application>
|
||||||
</assembly>
|
</assembly>
|
||||||
|
|||||||
@@ -1,7 +1,8 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# The Hookshot AppImage: the builder window (builder/linux/hookshot_builder.py) with everything it builds the game with -
|
# The Hookshot AppImage: the builder window (builder/linux/hookshot_builder.py) with everything it builds the game with -
|
||||||
# Python and the tools, clang and lld, a glibc 2.35 sysroot, FFmpeg, SDL2 and the Vulkan back end, built already.
|
# Python and the tools, clang and lld, a glibc 2.35 sysroot, FFmpeg, SDL2 and the Vulkan back end, built already.
|
||||||
# Hookshot.AppImage the builder window (Play is in the window, once a game is built)
|
# Hookshot.AppImage the launcher window (Play is in the window, once a game is built)
|
||||||
|
# Hookshot.AppImage --classic the classic builder window instead
|
||||||
# Hookshot.AppImage --play start the game the builder saved, without opening the window
|
# Hookshot.AppImage --play start the game the builder saved, without opening the window
|
||||||
# Hookshot.AppImage --play <folder> start the game in that folder
|
# Hookshot.AppImage --play <folder> start the game in that folder
|
||||||
# Hookshot.AppImage <folder> the same
|
# Hookshot.AppImage <folder> the same
|
||||||
@@ -35,4 +36,20 @@ fi
|
|||||||
if [ -n "${1:-}" ] && [ -d "$1" ]; then
|
if [ -n "${1:-}" ] && [ -d "$1" ]; then
|
||||||
exec "$HOOKSHOT_TOOLS/builder/linux/hookshot-launch" "$1"
|
exec "$HOOKSHOT_TOOLS/builder/linux/hookshot-launch" "$1"
|
||||||
fi
|
fi
|
||||||
|
# The window: the Triforce launcher (usr/bin/triforce-launcher, a WebKitGTK window). It exits 4 when this system has no
|
||||||
|
# WebKitGTK 4.1 (e.g. some Steam Deck images), and the classic builder window opens instead; --classic or
|
||||||
|
# TRIFORCE_CLASSIC=1 always opens the classic one.
|
||||||
|
LAUNCHER="$HERE/usr/bin/triforce-launcher"
|
||||||
|
if [ "${1:-}" = "--classic" ]; then
|
||||||
|
shift
|
||||||
|
TRIFORCE_CLASSIC=1
|
||||||
|
fi
|
||||||
|
if [ "${TRIFORCE_CLASSIC:-0}" != "1" ] && [ -x "$LAUNCHER" ]; then
|
||||||
|
# (WebKitGTK's GPU path breaks on NVIDIA's driver and under gamescope; its plain path is fine for a launcher)
|
||||||
|
export WEBKIT_DISABLE_DMABUF_RENDERER="${WEBKIT_DISABLE_DMABUF_RENDERER:-1}"
|
||||||
|
export TRIFORCE_APPRUN="$HERE/AppRun"
|
||||||
|
"$LAUNCHER" "$@"
|
||||||
|
code=$?
|
||||||
|
[ "$code" -ne 4 ] && exit "$code"
|
||||||
|
fi
|
||||||
exec "$PY" -I "$HOOKSHOT_TOOLS/builder/linux/hookshot_builder.py" "$@"
|
exec "$PY" -I "$HOOKSHOT_TOOLS/builder/linux/hookshot_builder.py" "$@"
|
||||||
|
|||||||
@@ -232,16 +232,21 @@ def _kept_version(d):
|
|||||||
return bool(cur) and _same(cur, d)
|
return bool(cur) and _same(cur, d)
|
||||||
|
|
||||||
|
|
||||||
def freeable(after_build=False):
|
def freeable(after_build=False, installing=False):
|
||||||
|
"""What free lets go (measure counts it; nothing is changed here). (Triforce) Never an unfinished version that still holds
|
||||||
|
folders of the installed one (setup_parts.lent: its reused.txt): deleted, the installed version lost its Python or its
|
||||||
|
compiler for good. installing: another install holds the setup's folder now, and what it is putting together
|
||||||
|
(downloads\\, versions\\) stays"""
|
||||||
|
from . import setup_parts
|
||||||
for d in (("sa", "exefs", "probe") if after_build else ("objcache", "sa", "exefs", "probe")):
|
for d in (("sa", "exefs", "probe") if after_build else ("objcache", "sa", "exefs", "probe")):
|
||||||
yield os.path.join(work(), d)
|
yield os.path.join(work(), d)
|
||||||
for d in ("downloads", "update"):
|
for d in ("update",) if installing else ("downloads", "update"):
|
||||||
yield os.path.join(paths.base(), d)
|
yield os.path.join(paths.base(), d)
|
||||||
v = paths.versions()
|
v = paths.versions()
|
||||||
if os.path.isdir(v):
|
if os.path.isdir(v) and not installing:
|
||||||
for name in os.listdir(v):
|
for name in os.listdir(v):
|
||||||
p = os.path.join(v, name)
|
p = os.path.join(v, name)
|
||||||
if os.path.isdir(p) and not _kept_version(p):
|
if os.path.isdir(p) and not _kept_version(p) and not setup_parts.lent(p):
|
||||||
yield p
|
yield p
|
||||||
|
|
||||||
|
|
||||||
@@ -275,13 +280,24 @@ def measure():
|
|||||||
|
|
||||||
def free(after_build=False):
|
def free(after_build=False):
|
||||||
"""Lets go of what measure counts as freeable; returns the bytes it could not delete (a file in use)"""
|
"""Lets go of what measure counts as freeable; returns the bytes it could not delete (a file in use)"""
|
||||||
for d in list(freeable(after_build)):
|
from . import setup_parts
|
||||||
_delete(d)
|
# ((Triforce) what a cut-off install moved from the installed version goes back first (setup_parts.undo_every_reuse), and
|
||||||
if not after_build:
|
# then its <tag>.part is freeable; one whose folders could not all go back stays. With the setup's folder held: an install
|
||||||
_delete(os.path.join(work(), "system_headers.txt"))
|
# running now is not undone under it, and its folders are left alone)
|
||||||
return measure()[1]
|
lock = setup_parts.take_lock(0)
|
||||||
seen = set()
|
try:
|
||||||
return sum(_size(d, seen) for d in freeable(True))
|
if lock:
|
||||||
|
setup_parts.undo_every_reuse()
|
||||||
|
for d in list(freeable(after_build, installing=not lock)):
|
||||||
|
_delete(d)
|
||||||
|
if not after_build:
|
||||||
|
_delete(os.path.join(work(), "system_headers.txt"))
|
||||||
|
return measure()[1]
|
||||||
|
seen = set()
|
||||||
|
return sum(_size(d, seen) for d in freeable(True, installing=not lock))
|
||||||
|
finally:
|
||||||
|
if lock:
|
||||||
|
lock.release()
|
||||||
|
|
||||||
|
|
||||||
def gb(n):
|
def gb(n):
|
||||||
|
|||||||
@@ -22,12 +22,80 @@ from . import REPO, paths
|
|||||||
UA = "HookshotBuilder-Setup/2.0"
|
UA = "HookshotBuilder-Setup/2.0"
|
||||||
WAITS = (3, 6, 12, 20, 30, 45, 60) # (8 tries over ~3 minutes: GitHub's release downloads answered 504 for hours on 2026-11-08)
|
WAITS = (3, 6, 12, 20, 30, 45, 60) # (8 tries over ~3 minutes: GitHub's release downloads answered 504 for hours on 2026-11-08)
|
||||||
OWN = ("python", "emsdk", "cpm", "third_party", "wgpu-native")
|
OWN = ("python", "emsdk", "cpm", "third_party", "wgpu-native")
|
||||||
|
MARGIN = 32 << 20 # ((Triforce) what a disk keeps free beyond what is about to be written to it: setup/Guard.cs Space.Margin)
|
||||||
|
REUSED = "reused.txt" # ((Triforce) in versions\<tag>.part: the folders moved over from the installed version, reuse_part)
|
||||||
|
DEVICES = {"CON", "PRN", "AUX", "NUL", "CONIN$", "CONOUT$"}
|
||||||
|
LOCK_WAIT = 20 # ((Triforce) seconds an install waits for another one into the same folder before it says so: setup.c SETUP_LOCK_WAIT_MS)
|
||||||
|
|
||||||
|
|
||||||
class SetupError(Exception):
|
class SetupError(Exception):
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def unsafe_member(name):
|
||||||
|
"""(Triforce) Why a zip's member must not be unpacked under its name, None when it may (setup/Mods.cs Unsafe, the same
|
||||||
|
rules): a path from the disk's root, a colon (a drive's letter, a hidden stream of another file), a path through "..", a
|
||||||
|
name Windows keeps for a device (CON, NUL, COM1, lpt2.txt: with or without an extension), a name that ends in a dot or a
|
||||||
|
space (Windows drops them, so the file would land under another name), a character Windows does not allow. zipfile's
|
||||||
|
extractall drops some of these quietly and writes the file under another name; a device's name it writes to the device"""
|
||||||
|
n = name.replace("\\", "/")
|
||||||
|
if not n.strip():
|
||||||
|
return "a member without a name"
|
||||||
|
if n.startswith("/"):
|
||||||
|
return "a path that starts at the disk's root"
|
||||||
|
if ":" in n:
|
||||||
|
return "a name with a colon or a drive's letter"
|
||||||
|
for p in n.split("/"):
|
||||||
|
if p in ("", "."):
|
||||||
|
continue
|
||||||
|
if p == "..":
|
||||||
|
return "a path that leaves its own folder (..)"
|
||||||
|
if p[-1] in ". ":
|
||||||
|
return "a name that ends in a dot or a space, which Windows drops"
|
||||||
|
stem = p.split(".", 1)[0].rstrip(" ").upper()
|
||||||
|
# (COM1 to COM9 and LPT1 to LPT9; Windows takes the raised digits 1, 2 and 3 for digits there too)
|
||||||
|
if stem in DEVICES or (len(stem) == 4 and stem[:3] in ("COM", "LPT") and stem[3] in "123456789\xb9\xb2\xb3"):
|
||||||
|
return "a name Windows keeps for a device (%s)" % p
|
||||||
|
if any(ord(c) < 32 or c in '<>"|?*' for c in p):
|
||||||
|
return "a name with a character Windows does not allow"
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def check_zip(z, dest, what):
|
||||||
|
"""(Triforce) Every member's name checked (unsafe_member) and the room for all of it asked before the first file is
|
||||||
|
written: such a zip is refused whole"""
|
||||||
|
need = 0
|
||||||
|
for info in z.infolist():
|
||||||
|
why = unsafe_member(info.filename)
|
||||||
|
if why:
|
||||||
|
raise SetupError("%s holds %s (\"%s\"): it was not unpacked." % (what, why, info.filename[:120]))
|
||||||
|
need += info.file_size + 4096 # (a file takes whole clusters)
|
||||||
|
room(dest, need, what)
|
||||||
|
|
||||||
|
|
||||||
|
def free_bytes(folder):
|
||||||
|
"""The free bytes of the disk that holds folder (asked of the nearest folder above it that is there); None when unknown"""
|
||||||
|
d = os.path.abspath(folder)
|
||||||
|
while not os.path.isdir(d):
|
||||||
|
up = os.path.dirname(d)
|
||||||
|
if up == d:
|
||||||
|
return None
|
||||||
|
d = up
|
||||||
|
try:
|
||||||
|
return shutil.disk_usage(d).free
|
||||||
|
except OSError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def room(folder, need, what):
|
||||||
|
"""(Triforce) SetupError before anything is written when the disk of folder has no room for need bytes and the margin"""
|
||||||
|
free = free_bytes(folder)
|
||||||
|
if free is not None and free < need + MARGIN:
|
||||||
|
raise SetupError("There is no room for %s on %s: it takes %d MB there, and %d MB are free. Make room and try again."
|
||||||
|
% (what, os.path.splitdrive(os.path.abspath(folder))[0] or folder, (need + MARGIN + (1 << 20) - 1) >> 20,
|
||||||
|
free >> 20))
|
||||||
|
|
||||||
|
|
||||||
class Part:
|
class Part:
|
||||||
def __init__(self, sha, size, dest, frm, to, url):
|
def __init__(self, sha, size, dest, frm, to, url):
|
||||||
self.sha, self.size, self.dest, self.frm, self.to, self.url = sha, int(size), dest, frm, to, url
|
self.sha, self.size, self.dest, self.frm, self.to, self.url = sha, int(size), dest, frm, to, url
|
||||||
@@ -182,6 +250,17 @@ def reuse_part(tag, p, install):
|
|||||||
if not os.path.exists(src):
|
if not os.path.exists(src):
|
||||||
return False
|
return False
|
||||||
os.makedirs(os.path.dirname(dst), exist_ok=True)
|
os.makedirs(os.path.dirname(dst), exist_ok=True)
|
||||||
|
# ((Triforce) written down before it moves: the installed version is the rollback point, and a cancelled or failed install
|
||||||
|
# moves it back (undo_reuse). Before, a cancel left the installed version without the folders it lent and nothing ran)
|
||||||
|
try:
|
||||||
|
journal = os.path.join(install, REUSED)
|
||||||
|
new = not os.path.isfile(journal)
|
||||||
|
with open(journal, "a", encoding="utf-8") as f:
|
||||||
|
f.write(("%s\n" % was if new else "") + rel + "\n")
|
||||||
|
f.flush()
|
||||||
|
os.fsync(f.fileno())
|
||||||
|
except OSError:
|
||||||
|
return False
|
||||||
try:
|
try:
|
||||||
os.replace(src, dst)
|
os.replace(src, dst)
|
||||||
return True
|
return True
|
||||||
@@ -189,6 +268,139 @@ def reuse_part(tag, p, install):
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def undo_reuse(work):
|
||||||
|
"""(Triforce) The folders reuse_part moved from the installed version into the unfinished one (work), moved back: what a
|
||||||
|
cancelled, failed or cut-off install leaves is the version that was installed, whole. Each goes back only where the
|
||||||
|
installed version has none; the list is removed once all went back"""
|
||||||
|
journal = os.path.join(work, REUSED)
|
||||||
|
try:
|
||||||
|
with open(journal, encoding="utf-8") as f:
|
||||||
|
lines = [l.rstrip("\n") for l in f if l.strip()]
|
||||||
|
except OSError:
|
||||||
|
return True
|
||||||
|
if not lines:
|
||||||
|
return True
|
||||||
|
was, ok = lines[0], True
|
||||||
|
for rel in lines[1:]:
|
||||||
|
if os.path.isabs(rel) or ".." in rel.replace("\\", "/").split("/"):
|
||||||
|
continue
|
||||||
|
src, back = os.path.join(work, rel), os.path.join(paths.versions(), was, rel)
|
||||||
|
if not os.path.exists(src) or os.path.exists(back):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
os.makedirs(os.path.dirname(back), exist_ok=True)
|
||||||
|
os.replace(src, back)
|
||||||
|
except OSError:
|
||||||
|
ok = False
|
||||||
|
if ok:
|
||||||
|
try:
|
||||||
|
os.remove(journal)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
return ok
|
||||||
|
|
||||||
|
|
||||||
|
def undo_every_reuse():
|
||||||
|
"""(Triforce) undo_reuse for every unfinished version (versions\\*.part) that holds a reused.txt. Before, only the <tag>.part
|
||||||
|
of the release being installed now was undone: an install of another release cut off before left the installed version's
|
||||||
|
folders in its own <tag>.part, and the installed version ran without them. Returns the first such folder whose folders
|
||||||
|
could not all go back (each is tried), None when nothing is left to move back"""
|
||||||
|
v, left = paths.versions(), None
|
||||||
|
try:
|
||||||
|
names = sorted(os.listdir(v))
|
||||||
|
except OSError:
|
||||||
|
return None
|
||||||
|
for name in names:
|
||||||
|
work = os.path.join(v, name)
|
||||||
|
if name.lower().endswith(".part") and os.path.isfile(os.path.join(work, REUSED)) and not undo_reuse(work):
|
||||||
|
left = left or work
|
||||||
|
return left
|
||||||
|
|
||||||
|
|
||||||
|
def lent(folder):
|
||||||
|
"""(Triforce) True when folder is an unfinished version (a versions\\*.part) that still holds folders of another version
|
||||||
|
(its reused.txt): only undo_reuse may let it go, never a delete (core.freeable)"""
|
||||||
|
return os.path.basename(folder.rstrip("\\/")).lower().endswith(".part") and os.path.isfile(os.path.join(folder, REUSED))
|
||||||
|
|
||||||
|
|
||||||
|
def lock_name(base=None):
|
||||||
|
"""(Triforce) The name of the lock every install into base holds (setup.c setup_lock_name and Reuse.cs LockName, the same):
|
||||||
|
Local\\HookshotBuilder-Setup-<a 64-bit FNV-1a of the folder's full path with '\\' between its parts, none at its end, a to z
|
||||||
|
in upper case, taken a UTF-16 unit at a time>. Local\\: this user's session"""
|
||||||
|
full = os.path.abspath(base or paths.base()).replace("/", "\\").rstrip("\\")
|
||||||
|
data, h = full.encode("utf-16-le", "surrogatepass"), 14695981039346656037 # (a name, not a secret)
|
||||||
|
for i in range(0, len(data), 2):
|
||||||
|
c = data[i] | data[i + 1] << 8
|
||||||
|
h = ((h ^ (c - 32 if 97 <= c <= 122 else c)) * 1099511628211) & 0xFFFFFFFFFFFFFFFF
|
||||||
|
return "Local\\HookshotBuilder-Setup-%016x" % h
|
||||||
|
|
||||||
|
|
||||||
|
class Lock:
|
||||||
|
"""(Triforce) The lock of the setup's folder, held (release() lets go of it; one that holds nothing where no lock could be
|
||||||
|
made at all). Windows' named mutex belongs to the thread that took it: released on that thread"""
|
||||||
|
|
||||||
|
def __init__(self, handle=None, file=None):
|
||||||
|
self.handle, self.file = handle, file
|
||||||
|
|
||||||
|
def release(self):
|
||||||
|
if self.handle:
|
||||||
|
k = ctypes_kernel32()
|
||||||
|
k.ReleaseMutex(self.handle)
|
||||||
|
k.CloseHandle(self.handle)
|
||||||
|
self.handle = None
|
||||||
|
if self.file:
|
||||||
|
self.file.close() # (the flock goes with the file)
|
||||||
|
self.file = None
|
||||||
|
|
||||||
|
|
||||||
|
def ctypes_kernel32():
|
||||||
|
import ctypes
|
||||||
|
from ctypes import wintypes
|
||||||
|
k = ctypes.WinDLL("kernel32", use_last_error=True)
|
||||||
|
k.CreateMutexW.argtypes = [ctypes.c_void_p, wintypes.BOOL, wintypes.LPCWSTR]
|
||||||
|
k.CreateMutexW.restype = wintypes.HANDLE
|
||||||
|
k.WaitForSingleObject.argtypes = [wintypes.HANDLE, wintypes.DWORD]
|
||||||
|
k.WaitForSingleObject.restype = wintypes.DWORD
|
||||||
|
k.ReleaseMutex.argtypes = [wintypes.HANDLE]
|
||||||
|
k.CloseHandle.argtypes = [wintypes.HANDLE]
|
||||||
|
return k
|
||||||
|
|
||||||
|
|
||||||
|
def take_lock(wait=LOCK_WAIT, base=None):
|
||||||
|
"""(Triforce) The setup's folder for this install alone: Lock, or None when another install (this app's, or setup.c's)
|
||||||
|
held it for wait seconds. Before, two installs into one folder ran at once, and one's sweep (undo_every_reuse) gave back
|
||||||
|
the folders the other had just moved into its <tag>.part: that one then finished without its Python. Windows lets go of
|
||||||
|
the mutex when its holder ends, however it ends; elsewhere a lock file (setup.lock in the folder) is flock'ed. Where no
|
||||||
|
lock can be made at all nobody is refused"""
|
||||||
|
base = base or paths.base()
|
||||||
|
if os.name == "nt":
|
||||||
|
k = ctypes_kernel32()
|
||||||
|
h = k.CreateMutexW(None, False, lock_name(base))
|
||||||
|
if not h: # (5: it is there, and belongs to a setup run as administrator)
|
||||||
|
import ctypes
|
||||||
|
return None if ctypes.get_last_error() == 5 else Lock()
|
||||||
|
if k.WaitForSingleObject(h, int(max(0, wait) * 1000)) in (0, 0x80): # (0x80: its holder ended without letting go: ours)
|
||||||
|
return Lock(handle=h)
|
||||||
|
k.CloseHandle(h)
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
import fcntl
|
||||||
|
os.makedirs(base, exist_ok=True)
|
||||||
|
f = open(os.path.join(base, "setup.lock"), "a")
|
||||||
|
except (ImportError, OSError):
|
||||||
|
return Lock()
|
||||||
|
end = time.monotonic() + max(0, wait)
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
fcntl.flock(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||||
|
return Lock(file=f)
|
||||||
|
except OSError:
|
||||||
|
if time.monotonic() >= end:
|
||||||
|
f.close()
|
||||||
|
return None
|
||||||
|
time.sleep(0.2)
|
||||||
|
|
||||||
|
|
||||||
def rmtree(p):
|
def rmtree(p):
|
||||||
shutil.rmtree(p, ignore_errors=True)
|
shutil.rmtree(p, ignore_errors=True)
|
||||||
|
|
||||||
@@ -207,6 +419,29 @@ def current():
|
|||||||
|
|
||||||
def install(status, progress=lambda f: None, tag=None, repair=False, cancel=None):
|
def install(status, progress=lambda f: None, tag=None, repair=False, cancel=None):
|
||||||
"""Installs the release (the latest, or tag) unless it is there already; returns (tag, its folder)"""
|
"""Installs the release (the latest, or tag) unless it is there already; returns (tag, its folder)"""
|
||||||
|
# ((Triforce) one install at a time into the setup's folder, its sweep included (take_lock): another one waits for it, a while)
|
||||||
|
lock = take_lock(0)
|
||||||
|
if lock is None:
|
||||||
|
status("Another Hookshot Builder setup is installing. Waiting for it to finish...")
|
||||||
|
lock = take_lock(LOCK_WAIT)
|
||||||
|
if lock is None:
|
||||||
|
raise SetupError("Another Hookshot Builder setup is installing into %s right now. Wait for it to finish (or close it), "
|
||||||
|
"then try again." % paths.base())
|
||||||
|
try:
|
||||||
|
return install_locked(status, progress, tag, repair, cancel)
|
||||||
|
finally:
|
||||||
|
lock.release()
|
||||||
|
|
||||||
|
|
||||||
|
def install_locked(status, progress, tag, repair, cancel):
|
||||||
|
"""install's work, with the setup's folder held"""
|
||||||
|
# ((Triforce) an install cut off before (the app closed), of this release or another: what it moved over from the installed
|
||||||
|
# version goes back first, before the installed version is looked at, and a part folder whose folders could not all go back
|
||||||
|
# is kept, not deleted with them)
|
||||||
|
left = undo_every_reuse()
|
||||||
|
if left:
|
||||||
|
raise SetupError("The last install could not be undone: %s still holds folders of the installed version. Close every "
|
||||||
|
"Hookshot window and try again." % left)
|
||||||
tag, parts = read_manifest(status, tag, cancel)
|
tag, parts = read_manifest(status, tag, cancel)
|
||||||
versions, downloads = paths.versions(), os.path.join(paths.base(), "downloads")
|
versions, downloads = paths.versions(), os.path.join(paths.base(), "downloads")
|
||||||
final = os.path.join(versions, tag)
|
final = os.path.join(versions, tag)
|
||||||
@@ -216,39 +451,23 @@ def install(status, progress=lambda f: None, tag=None, repair=False, cancel=None
|
|||||||
rmtree(final)
|
rmtree(final)
|
||||||
os.makedirs(work, exist_ok=True)
|
os.makedirs(work, exist_ok=True)
|
||||||
os.makedirs(downloads, exist_ok=True)
|
os.makedirs(downloads, exist_ok=True)
|
||||||
n = len(parts)
|
try:
|
||||||
for i, p in enumerate(parts, 1):
|
unpack_parts(status, progress, tag, parts, work, downloads, cancel)
|
||||||
if cancel and cancel():
|
except BaseException:
|
||||||
raise SetupError("cancelled")
|
# ((Triforce) a cancel or a failure: the installed version gets back what was moved from it, and the partial goes)
|
||||||
if reuse_part(tag, p, work):
|
if undo_reuse(work):
|
||||||
status("Part %d of %d unchanged: kept from the installed version" % (i, n))
|
rmtree(work)
|
||||||
continue
|
raise
|
||||||
zpath = os.path.join(downloads, p.sha + ".zip")
|
|
||||||
if not (os.path.isfile(zpath) and sha256_file(zpath) == p.sha):
|
|
||||||
download_part(status, progress, p, i, n, zpath, cancel)
|
|
||||||
status("Unpacking part %d of %d..." % (i, n))
|
|
||||||
progress(-1)
|
|
||||||
dest = work if p.dest == "." else os.path.join(work, p.dest.replace("/", os.sep))
|
|
||||||
os.makedirs(dest, exist_ok=True)
|
|
||||||
try:
|
|
||||||
with zipfile.ZipFile(zpath) as z:
|
|
||||||
z.extractall(dest)
|
|
||||||
except (OSError, zipfile.BadZipFile) as e:
|
|
||||||
raise SetupError("Unpacking part %d of %d failed: %s" % (i, n, e)) from None
|
|
||||||
if p.frm != "-" and p.to != "-": # (llvm-mingw-<version>-ucrt-x86_64 -> llvm-mingw)
|
|
||||||
src, dst = os.path.join(dest, p.frm), os.path.join(dest, p.to)
|
|
||||||
if not os.path.exists(dst):
|
|
||||||
try:
|
|
||||||
os.replace(src, dst)
|
|
||||||
except OSError:
|
|
||||||
raise SetupError("Could not set up part %d of %d (renaming %s)." % (i, n, p.frm)) from None
|
|
||||||
if not os.path.isfile(os.path.join(work, "HookshotBuilder", "tools", "nx2", "make_game.py")):
|
|
||||||
raise SetupError("The build tools are missing after unpacking. Your antivirus may have removed them: allow the folder %s "
|
|
||||||
"and try again." % paths.base())
|
|
||||||
try:
|
try:
|
||||||
os.replace(work, final)
|
os.replace(work, final)
|
||||||
except OSError:
|
except OSError:
|
||||||
|
if undo_reuse(work):
|
||||||
|
rmtree(work)
|
||||||
raise SetupError("Could not finish installing into %s." % final) from None
|
raise SetupError("Could not finish installing into %s." % final) from None
|
||||||
|
try:
|
||||||
|
os.remove(os.path.join(final, REUSED))
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
paths.write_small(os.path.join(final, "installed.txt"), tag)
|
paths.write_small(os.path.join(final, "installed.txt"), tag)
|
||||||
write_parts(final, parts)
|
write_parts(final, parts)
|
||||||
elif not os.path.isfile(os.path.join(final, "parts.txt")):
|
elif not os.path.isfile(os.path.join(final, "parts.txt")):
|
||||||
@@ -258,8 +477,45 @@ def install(status, progress=lambda f: None, tag=None, repair=False, cancel=None
|
|||||||
return tag, final
|
return tag, final
|
||||||
|
|
||||||
|
|
||||||
|
def unpack_parts(status, progress, tag, parts, work, downloads, cancel):
|
||||||
|
"""Each part into work: moved over from the installed version, or downloaded (when not there already), checked and unpacked"""
|
||||||
|
n = len(parts)
|
||||||
|
for i, p in enumerate(parts, 1):
|
||||||
|
if cancel and cancel():
|
||||||
|
raise SetupError("cancelled")
|
||||||
|
if reuse_part(tag, p, work):
|
||||||
|
status("Part %d of %d unchanged: kept from the installed version" % (i, n))
|
||||||
|
continue
|
||||||
|
zpath = os.path.join(downloads, p.sha + ".zip")
|
||||||
|
if not (os.path.isfile(zpath) and sha256_file(zpath) == p.sha):
|
||||||
|
download_part(status, progress, p, i, n, zpath, cancel)
|
||||||
|
status("Unpacking part %d of %d..." % (i, n))
|
||||||
|
progress(-1)
|
||||||
|
dest = work if p.dest == "." else os.path.join(work, p.dest.replace("/", os.sep))
|
||||||
|
os.makedirs(dest, exist_ok=True)
|
||||||
|
try:
|
||||||
|
with zipfile.ZipFile(zpath) as z:
|
||||||
|
check_zip(z, dest, "Part %d of %d" % (i, n)) # ((Triforce) every name, and the room, before the first file)
|
||||||
|
z.extractall(dest)
|
||||||
|
except (OSError, zipfile.BadZipFile) as e:
|
||||||
|
raise SetupError("Unpacking part %d of %d failed: %s" % (i, n, e)) from None
|
||||||
|
if p.frm != "-" and p.to != "-": # (llvm-mingw-<version>-ucrt-x86_64 -> llvm-mingw)
|
||||||
|
src, dst = os.path.join(dest, p.frm), os.path.join(dest, p.to)
|
||||||
|
if not os.path.exists(dst):
|
||||||
|
try:
|
||||||
|
os.replace(src, dst)
|
||||||
|
except OSError:
|
||||||
|
raise SetupError("Could not set up part %d of %d (renaming %s)." % (i, n, p.frm)) from None
|
||||||
|
if not os.path.isfile(os.path.join(work, "HookshotBuilder", "tools", "nx2", "make_game.py")):
|
||||||
|
raise SetupError("The build tools are missing after unpacking. Your antivirus may have removed them: allow the folder %s "
|
||||||
|
"and try again." % paths.base())
|
||||||
|
|
||||||
|
|
||||||
def download_part(status, progress, p, i, n, zpath, cancel):
|
def download_part(status, progress, p, i, n, zpath, cancel):
|
||||||
http, part = Http(), zpath + ".part"
|
http, part = Http(), zpath + ".part"
|
||||||
|
# ((Triforce) room for the size the list names, asked before the first byte: a full disk stopped the download half way with
|
||||||
|
# Windows' own words for it. The part is written from its first byte at each try, so no address or mark is kept beside it)
|
||||||
|
room(os.path.dirname(zpath), p.size, "part %d of %d" % (i, n))
|
||||||
for attempt in range(len(WAITS) + 1):
|
for attempt in range(len(WAITS) + 1):
|
||||||
if attempt:
|
if attempt:
|
||||||
status("Part %d of %d: GitHub didn't answer (%s). Trying again in %d s..." % (i, n, http.status or "no answer",
|
status("Part %d of %d: GitHub didn't answer (%s). Trying again in %d s..." % (i, n, http.status or "no answer",
|
||||||
|
|||||||
@@ -89,12 +89,21 @@ static void status(const wchar_t *fmt, ...) {
|
|||||||
PostMessageW(g_win, WM_STATUS, 0, 0);
|
PostMessageW(g_win, WM_STATUS, 0, 0);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* (Triforce) the unfinished version's folder while one is put together, and the versions folder: what fail() undoes (undo_reuse) */
|
||||||
|
static wchar_t g_part_dir[MAX_PATH], g_versions_dir[MAX_PATH];
|
||||||
|
static int undo_reuse(const wchar_t *versions, const wchar_t *install);
|
||||||
|
static void rmtree(const wchar_t *dir);
|
||||||
|
static void setup_lock_release(void);
|
||||||
|
|
||||||
static void fail(const wchar_t *fmt, ...) {
|
static void fail(const wchar_t *fmt, ...) {
|
||||||
wchar_t msg[1024];
|
wchar_t msg[1024];
|
||||||
va_list ap;
|
va_list ap;
|
||||||
va_start(ap, fmt);
|
va_start(ap, fmt);
|
||||||
vswprintf(msg, 1024, fmt, ap);
|
vswprintf(msg, 1024, fmt, ap);
|
||||||
va_end(ap);
|
va_end(ap);
|
||||||
|
/* ((Triforce) the installed version gets back the folders that were moved from it, and the unfinished one goes) */
|
||||||
|
if (g_part_dir[0] && undo_reuse(g_versions_dir, g_part_dir)) rmtree(g_part_dir);
|
||||||
|
setup_lock_release(); /* ((Triforce) undone: the next setup may begin) */
|
||||||
if (g_headless) say_error(msg), ExitProcess(1);
|
if (g_headless) say_error(msg), ExitProcess(1);
|
||||||
MessageBoxW(g_win, msg, L"Hookshot Builder setup", MB_OK | MB_ICONERROR);
|
MessageBoxW(g_win, msg, L"Hookshot Builder setup", MB_OK | MB_ICONERROR);
|
||||||
PostMessageW(g_win, WM_DONE, 1, 0);
|
PostMessageW(g_win, WM_DONE, 1, 0);
|
||||||
@@ -318,6 +327,29 @@ static int part_folder(const Part *p, wchar_t *rel, int n) {
|
|||||||
return 1;
|
return 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#define REUSED_NOTE L"reused.txt"
|
||||||
|
/* ((Triforce) undo_reuse's note: rel added to <install>\reused.txt, the installed version's tag first when it is new; written
|
||||||
|
through to the disk. 0: it could not be written (the folder is then not moved) */
|
||||||
|
static int reuse_note(const wchar_t *install, const char *was, const wchar_t *rel) {
|
||||||
|
wchar_t note[MAX_PATH];
|
||||||
|
char line[2 * MAX_PATH + 160];
|
||||||
|
swprintf(note, MAX_PATH, L"%ls\\" REUSED_NOTE, install);
|
||||||
|
const int fresh = GetFileAttributesW(note) == INVALID_FILE_ATTRIBUTES;
|
||||||
|
HANDLE f = CreateFileW(note, FILE_APPEND_DATA, 0, NULL, OPEN_ALWAYS, FILE_FLAG_WRITE_THROUGH, NULL);
|
||||||
|
if (f == INVALID_HANDLE_VALUE) return 0;
|
||||||
|
int n = fresh ? snprintf(line, sizeof line, "%s\n", was) : 0;
|
||||||
|
const int k = WideCharToMultiByte(CP_UTF8, 0, rel, -1, line + n, (int)sizeof line - n - 2, NULL, NULL);
|
||||||
|
DWORD put = 0;
|
||||||
|
int ok = k > 0;
|
||||||
|
if (ok) {
|
||||||
|
n += k - 1;
|
||||||
|
line[n++] = '\n';
|
||||||
|
ok = WriteFile(f, line, (DWORD)n, &put, NULL) && put == (DWORD)n && FlushFileBuffers(f);
|
||||||
|
}
|
||||||
|
CloseHandle(f);
|
||||||
|
return ok;
|
||||||
|
}
|
||||||
|
|
||||||
/* (players asked for less in %LOCALAPPDATA%: the downloaded zips are not kept beside the unpacked version any more. So that an update
|
/* (players asked for less in %LOCALAPPDATA%: the downloaded zips are not kept beside the unpacked version any more. So that an update
|
||||||
still downloads only what changed, a part whose checksum the installed version lists in its parts.txt (and whose folder it has)
|
still downloads only what changed, a part whose checksum the installed version lists in its parts.txt (and whose folder it has)
|
||||||
is moved over from it instead of downloaded: that version is deleted after the install anyway) */
|
is moved over from it instead of downloaded: that version is deleted after the install anyway) */
|
||||||
@@ -343,9 +375,106 @@ static int reuse_part(const wchar_t *versions, const char *tag, const Part *p, c
|
|||||||
wcscpy(parent, to);
|
wcscpy(parent, to);
|
||||||
wchar_t *slash = wcsrchr(parent, L'\\');
|
wchar_t *slash = wcsrchr(parent, L'\\');
|
||||||
if (slash) *slash = 0, mkdirs(parent);
|
if (slash) *slash = 0, mkdirs(parent);
|
||||||
|
/* ((Triforce) written down before it moves: the installed version is the rollback point, undo_reuse) */
|
||||||
|
if (!reuse_note(install, was, rel)) return 0;
|
||||||
return MoveFileW(from, to) != 0;
|
return MoveFileW(from, to) != 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* (Triforce) The folders reuse_part moved from the installed version into the unfinished one (install), moved back: what a
|
||||||
|
failed, closed or cut-off setup leaves is the version that was installed, whole. Before, they stayed in <tag>.part, which
|
||||||
|
the next run deleted with them, and the version still installed had lost them. Each goes back only where the installed
|
||||||
|
version has none; the note (<tag>.part\reused.txt: the installed version's tag, then a folder a line) is removed once all
|
||||||
|
went back. 1: nothing is left to move back */
|
||||||
|
static int undo_reuse(const wchar_t *versions, const wchar_t *install) {
|
||||||
|
wchar_t note[MAX_PATH];
|
||||||
|
swprintf(note, MAX_PATH, L"%ls\\" REUSED_NOTE, install);
|
||||||
|
HANDLE f = CreateFileW(note, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);
|
||||||
|
if (f == INVALID_HANDLE_VALUE) return 1;
|
||||||
|
char buf[8192];
|
||||||
|
DWORD got = 0;
|
||||||
|
ReadFile(f, buf, sizeof buf - 1, &got, NULL);
|
||||||
|
CloseHandle(f);
|
||||||
|
buf[got] = 0;
|
||||||
|
char *next = buf, *was = next_line(&next), *rel;
|
||||||
|
int ok = 1;
|
||||||
|
if (!was || !list_name_ok(was, 0)) return DeleteFileW(note), 1; /* (no version named: nothing was moved yet) */
|
||||||
|
while ((rel = next_line(&next)) != NULL) {
|
||||||
|
if (!list_name_ok(rel, 1)) continue;
|
||||||
|
wchar_t wrel[MAX_PATH], from[MAX_PATH], back[MAX_PATH], parent[MAX_PATH];
|
||||||
|
MultiByteToWideChar(CP_UTF8, 0, rel, -1, wrel, MAX_PATH);
|
||||||
|
swprintf(from, MAX_PATH, L"%ls\\%ls", install, wrel);
|
||||||
|
swprintf(back, MAX_PATH, L"%ls\\%hs\\%ls", versions, was, wrel);
|
||||||
|
if (GetFileAttributesW(from) == INVALID_FILE_ATTRIBUTES || GetFileAttributesW(back) != INVALID_FILE_ATTRIBUTES) continue;
|
||||||
|
wcscpy(parent, back);
|
||||||
|
wchar_t *slash = wcsrchr(parent, L'\\');
|
||||||
|
if (slash) *slash = 0, mkdirs(parent);
|
||||||
|
if (!MoveFileW(from, back)) ok = 0;
|
||||||
|
}
|
||||||
|
if (ok) DeleteFileW(note);
|
||||||
|
return ok;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* (Triforce) undo_reuse for every unfinished version (versions\*.part) that holds a note. Before, only the <tag>.part of the
|
||||||
|
release being set up now was undone: a setup of another release cut off before left the installed version's folders in its
|
||||||
|
own <tag>.part, and the installed version ran without them. Each is tried; 1: nothing is left to move back, 0: left names the
|
||||||
|
first folder whose folders could not all go back */
|
||||||
|
static int undo_every_reuse(const wchar_t *versions, wchar_t *left, size_t cap) {
|
||||||
|
wchar_t pattern[MAX_PATH], dir[MAX_PATH], note[MAX_PATH];
|
||||||
|
WIN32_FIND_DATAW fd;
|
||||||
|
swprintf(pattern, MAX_PATH, L"%ls\\*", versions);
|
||||||
|
HANDLE h = FindFirstFileW(pattern, &fd);
|
||||||
|
if (h == INVALID_HANDLE_VALUE) return 1;
|
||||||
|
int ok = 1;
|
||||||
|
do {
|
||||||
|
const size_t n = wcslen(fd.cFileName);
|
||||||
|
if (!(fd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) || n < 6 || _wcsicmp(fd.cFileName + n - 5, L".part")) continue;
|
||||||
|
swprintf(dir, MAX_PATH, L"%ls\\%ls", versions, fd.cFileName);
|
||||||
|
swprintf(note, MAX_PATH, L"%ls\\" REUSED_NOTE, dir);
|
||||||
|
if (GetFileAttributesW(note) == INVALID_FILE_ATTRIBUTES || undo_reuse(versions, dir)) continue;
|
||||||
|
if (ok) wcsncpy(left, dir, cap - 1), left[cap - 1] = 0;
|
||||||
|
ok = 0;
|
||||||
|
} while (FindNextFileW(h, &fd));
|
||||||
|
FindClose(h);
|
||||||
|
return ok;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* (Triforce) One setup at a time for a setup folder (base), its sweep (undo_every_reuse) included. Two at once each undid the
|
||||||
|
other's <tag>.part: the second one's sweep gave the installed version back the folders the first had just moved over, and
|
||||||
|
the first finished without its Python. The lock is a named mutex (Windows lets go of it when its holder ends, however it
|
||||||
|
ends), named as setup_parts.lock_name and Reuse.cs LockName name it: Local\HookshotBuilder-Setup-<a 64-bit FNV-1a of the
|
||||||
|
folder's full path with '\' between its parts, none at its end, a to z in upper case>. Local\: this user's session */
|
||||||
|
#define SETUP_LOCK_WAIT_MS 20000
|
||||||
|
static HANDLE g_setup_lock;
|
||||||
|
static void setup_lock_name(const wchar_t *base, wchar_t *name, size_t cap) {
|
||||||
|
wchar_t full[2 * MAX_PATH];
|
||||||
|
const DWORD got = GetFullPathNameW(base, 2 * MAX_PATH, full, NULL);
|
||||||
|
if (!got || got >= 2 * MAX_PATH) wcsncpy(full, base, 2 * MAX_PATH - 1), full[2 * MAX_PATH - 1] = 0;
|
||||||
|
size_t n = wcslen(full);
|
||||||
|
for (size_t i = 0; i < n; i++) if (full[i] == L'/') full[i] = L'\\';
|
||||||
|
while (n && full[n - 1] == L'\\') full[--n] = 0;
|
||||||
|
unsigned long long h = 14695981039346656037ULL; /* (a name, not a secret) */
|
||||||
|
for (size_t i = 0; i < n; i++) {
|
||||||
|
const unsigned c = full[i] >= L'a' && full[i] <= L'z' ? full[i] - 32u : full[i];
|
||||||
|
h = (h ^ c) * 1099511628211ULL;
|
||||||
|
}
|
||||||
|
swprintf(name, cap, L"Local\\HookshotBuilder-Setup-%016llx", h);
|
||||||
|
}
|
||||||
|
/* 1: this setup holds base (or no lock can be made at all: nobody is refused), 0: another setup held it for wait_ms */
|
||||||
|
static int setup_lock_take(const wchar_t *base, DWORD wait_ms) {
|
||||||
|
wchar_t name[96];
|
||||||
|
if (g_setup_lock) return 1;
|
||||||
|
setup_lock_name(base, name, 96);
|
||||||
|
HANDLE m = CreateMutexW(NULL, FALSE, name);
|
||||||
|
if (!m) return GetLastError() != ERROR_ACCESS_DENIED; /* (it is there, and belongs to a setup run as administrator) */
|
||||||
|
const DWORD r = WaitForSingleObject(m, wait_ms);
|
||||||
|
if (r == WAIT_OBJECT_0 || r == WAIT_ABANDONED) return g_setup_lock = m, 1; /* (abandoned: its holder ended, it is ours) */
|
||||||
|
CloseHandle(m);
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
static void setup_lock_release(void) {
|
||||||
|
if (g_setup_lock) ReleaseMutex(g_setup_lock), CloseHandle(g_setup_lock), g_setup_lock = NULL;
|
||||||
|
}
|
||||||
|
|
||||||
/* The checksums of the parts this version holds in folders of their own (reuse_part reads them at the next update) */
|
/* The checksums of the parts this version holds in folders of their own (reuse_part reads them at the next update) */
|
||||||
static void write_parts(const wchar_t *install, const Part *parts, int nparts) {
|
static void write_parts(const wchar_t *install, const Part *parts, int nparts) {
|
||||||
wchar_t list[MAX_PATH], rel[MAX_PATH];
|
wchar_t list[MAX_PATH], rel[MAX_PATH];
|
||||||
@@ -536,6 +665,14 @@ static DWORD WINAPI work(LPVOID unused) {
|
|||||||
const wchar_t *tb = _wgetenv(L"HOOKSHOT_SETUP_BASE");
|
const wchar_t *tb = _wgetenv(L"HOOKSHOT_SETUP_BASE");
|
||||||
if (tb && *tb) wcsncpy(base, tb, MAX_PATH - 1), base[MAX_PATH - 1] = 0;
|
if (tb && *tb) wcsncpy(base, tb, MAX_PATH - 1), base[MAX_PATH - 1] = 0;
|
||||||
}
|
}
|
||||||
|
/* ((Triforce) this folder for this setup alone until the builder starts, its sweep included (setup_lock_take): another one
|
||||||
|
waits for it, a while) */
|
||||||
|
if (!setup_lock_take(base, 0)) {
|
||||||
|
status(L"Another Hookshot Builder setup is installing. Waiting for it to finish...");
|
||||||
|
if (!setup_lock_take(base, SETUP_LOCK_WAIT_MS))
|
||||||
|
fail(L"Another Hookshot Builder setup is installing into %ls right now. Wait for it to finish (or close it), then run the "
|
||||||
|
L"setup again.", base);
|
||||||
|
}
|
||||||
swprintf(downloads, MAX_PATH, L"%ls\\downloads", base);
|
swprintf(downloads, MAX_PATH, L"%ls\\downloads", base);
|
||||||
mkdirs(downloads);
|
mkdirs(downloads);
|
||||||
|
|
||||||
@@ -618,12 +755,21 @@ static DWORD WINAPI work(LPVOID unused) {
|
|||||||
swprintf(exe, MAX_PATH, L"%ls\\%ls", install, wrun);
|
swprintf(exe, MAX_PATH, L"%ls\\%ls", install, wrun);
|
||||||
for (wchar_t *q = exe; *q; q++) if (*q == L'/') *q = L'\\';
|
for (wchar_t *q = exe; *q; q++) if (*q == L'/') *q = L'\\';
|
||||||
|
|
||||||
|
{ /* ((Triforce) a setup cut off before, of this release or another: what it moved over from the installed version goes
|
||||||
|
back first, before the installed version is looked at; a part folder whose folders could not all go back is kept, not
|
||||||
|
deleted with them) */
|
||||||
|
wchar_t left[MAX_PATH];
|
||||||
|
if (!undo_every_reuse(versions, left, MAX_PATH))
|
||||||
|
fail(L"The last setup could not be undone: %ls still holds folders of the installed version. Close every Hookshot window "
|
||||||
|
L"and run the setup again.", left);
|
||||||
|
}
|
||||||
if (repair || GetFileAttributesW(marker) == INVALID_FILE_ATTRIBUTES || GetFileAttributesW(exe) == INVALID_FILE_ATTRIBUTES) {
|
if (repair || GetFileAttributesW(marker) == INVALID_FILE_ATTRIBUTES || GetFileAttributesW(exe) == INVALID_FILE_ATTRIBUTES) {
|
||||||
/* (unpacked into <tag>.part and renamed once complete: a setup stopped half way never leaves a version that looks installed) */
|
/* (unpacked into <tag>.part and renamed once complete: a setup stopped half way never leaves a version that looks installed) */
|
||||||
swprintf(install, MAX_PATH, L"%ls\\%ls.part", versions, wtag);
|
swprintf(install, MAX_PATH, L"%ls\\%ls.part", versions, wtag);
|
||||||
rmtree(install);
|
rmtree(install);
|
||||||
rmtree(final_dir);
|
rmtree(final_dir);
|
||||||
mkdirs(install);
|
mkdirs(install);
|
||||||
|
wcscpy(g_versions_dir, versions), wcscpy(g_part_dir, install); /* (fail() undoes from here on) */
|
||||||
for (int i = 0; i < nparts; i++) {
|
for (int i = 0; i < nparts; i++) {
|
||||||
Part *p = &parts[i];
|
Part *p = &parts[i];
|
||||||
wchar_t zip[MAX_PATH], wsha[65];
|
wchar_t zip[MAX_PATH], wsha[65];
|
||||||
@@ -721,7 +867,13 @@ static DWORD WINAPI work(LPVOID unused) {
|
|||||||
if (GetFileAttributesW(staged_exe) == INVALID_FILE_ATTRIBUTES)
|
if (GetFileAttributesW(staged_exe) == INVALID_FILE_ATTRIBUTES)
|
||||||
fail(L"The builder is missing after unpacking (%ls). Your antivirus may have removed it: allow the folder %ls and run the setup again.", staged_exe, base);
|
fail(L"The builder is missing after unpacking (%ls). Your antivirus may have removed it: allow the folder %ls and run the setup again.", staged_exe, base);
|
||||||
if (!MoveFileW(install, final_dir)) fail(L"Could not finish installing into %ls.", final_dir);
|
if (!MoveFileW(install, final_dir)) fail(L"Could not finish installing into %ls.", final_dir);
|
||||||
|
g_part_dir[0] = 0; /* (installed: nothing to undo; the note goes) */
|
||||||
wcscpy(install, final_dir);
|
wcscpy(install, final_dir);
|
||||||
|
{
|
||||||
|
wchar_t note[MAX_PATH];
|
||||||
|
swprintf(note, MAX_PATH, L"%ls\\" REUSED_NOTE, final_dir);
|
||||||
|
DeleteFileW(note);
|
||||||
|
}
|
||||||
write_small(marker, tag);
|
write_small(marker, tag);
|
||||||
write_parts(install, parts, nparts);
|
write_parts(install, parts, nparts);
|
||||||
}
|
}
|
||||||
@@ -732,6 +884,7 @@ static DWORD WINAPI work(LPVOID unused) {
|
|||||||
}
|
}
|
||||||
prune(versions, downloads, tag, parts, nparts);
|
prune(versions, downloads, tag, parts, nparts);
|
||||||
rmtree(downloads); /* (installed: its zips are not kept; an update moves unchanged folders over instead, reuse_part) */
|
rmtree(downloads); /* (installed: its zips are not kept; an update moves unchanged folders over instead, reuse_part) */
|
||||||
|
setup_lock_release(); /* ((Triforce) installed: the builder this starts may start a setup of its own) */
|
||||||
|
|
||||||
status(L"Starting the Hookshot Builder...");
|
status(L"Starting the Hookshot Builder...");
|
||||||
wchar_t cmd[4096], dir[MAX_PATH];
|
wchar_t cmd[4096], dir[MAX_PATH];
|
||||||
|
|||||||
@@ -2,6 +2,6 @@
|
|||||||
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
|
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
|
||||||
<assemblyIdentity type="win32" name="Hookshot.BuilderSetup" version="1.0.0.0"/>
|
<assemblyIdentity type="win32" name="Hookshot.BuilderSetup" version="1.0.0.0"/>
|
||||||
<dependency><dependentAssembly><assemblyIdentity type="win32" name="Microsoft.Windows.Common-Controls" version="6.0.0.0" processorArchitecture="*" publicKeyToken="6595b64144ccf1df" language="*"/></dependentAssembly></dependency>
|
<dependency><dependentAssembly><assemblyIdentity type="win32" name="Microsoft.Windows.Common-Controls" version="6.0.0.0" processorArchitecture="*" publicKeyToken="6595b64144ccf1df" language="*"/></dependentAssembly></dependency>
|
||||||
<application xmlns="urn:schemas-microsoft-com:asm.v3"><windowsSettings><dpiAware xmlns="http://schemas.microsoft.com/SMI/2005/WindowsSettings">true</dpiAware></windowsSettings></application>
|
<application xmlns="urn:schemas-microsoft-com:asm.v3"><windowsSettings><dpiAware xmlns="http://schemas.microsoft.com/SMI/2005/WindowsSettings">true</dpiAware><longPathAware xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">true</longPathAware></windowsSettings></application>
|
||||||
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3"><security><requestedPrivileges><requestedExecutionLevel level="asInvoker" uiAccess="false"/></requestedPrivileges></security></trustInfo>
|
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3"><security><requestedPrivileges><requestedExecutionLevel level="asInvoker" uiAccess="false"/></requestedPrivileges></security></trustInfo>
|
||||||
</assembly>
|
</assembly>
|
||||||
|
|||||||
@@ -108,7 +108,10 @@ second game.
|
|||||||
7. **The CPU saturates first.** Ryzen 5 7600X, 6 cores / 12 threads: the game's threads, the replay, captures and
|
7. **The CPU saturates first.** Ryzen 5 7600X, 6 cores / 12 threads: the game's threads, the replay, captures and
|
||||||
pipeline compiles share it, and extra helper threads have lowered fps before. Thread counts are measured, never
|
pipeline compiles share it, and extra helper threads have lowered fps before. Thread counts are measured, never
|
||||||
raised by default (`HOS_VCORES`, `RX_VK_CAPTURE_THREADS`, `HOS_THREAD_TIMES=<seconds>`, the `HOS_SAMPLE` profiler
|
raised by default (`HOS_VCORES`, `RX_VK_CAPTURE_THREADS`, `HOS_THREAD_TIMES=<seconds>`, the `HOS_SAMPLE` profiler
|
||||||
with `scripts/prof.py`).
|
with `scripts/prof.py`). The one exception, by the owner's choice (2026-11-15): the Replay helper threads row is
|
||||||
|
Automatic unless set, none with four cores or fewer, and its threads are kept only where trials in play measure the
|
||||||
|
frames shorter, or, without a trial, where the replay is plainly the frame's limit by a margin over the GPU's time
|
||||||
|
(the rule in `gpu/vk/replay_threads.h`, fitted to the owner's logs; a trial that finds them slower holds it).
|
||||||
|
|
||||||
Also inherited, and game-specific in MKW: MKW's frame-pacer hooks at fixed guest addresses are gated on MKW's build
|
Also inherited, and game-specific in MKW: MKW's frame-pacer hooks at fixed guest addresses are gated on MKW's build
|
||||||
id (`hos_game_is_mkw`); OoT gets its own once its pacer is found, never by address coincidence.
|
id (`hos_game_is_mkw`); OoT gets its own once its pacer is found, never by address coincidence.
|
||||||
|
|||||||
@@ -76,17 +76,35 @@ screen. It costs some GPU time on cards that cull quickly, so it is off unless s
|
|||||||
| Switch | What it does |
|
| Switch | What it does |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `HOS_HEADLESS` (set) | No window: the renderer draws offscreen, for screenshots and frame dumps |
|
| `HOS_HEADLESS` (set) | No window: the renderer draws offscreen, for screenshots and frame dumps |
|
||||||
| `HOS_WINDOW_SIZE=<w>x<h>` | The window's size in pixels (1280x720 by default, wider for an ultrawide picture) |
|
| `HOS_WINDOW_SIZE=<w>x<h>` | The window's size in pixels (1280x720 by default, wider for an ultrawide picture; 64 to 16384 a side) |
|
||||||
| `HOS_WINDOW_NOACTIVATE` (set) | The window opens behind the others and does not take the focus (Windows) |
|
| `HOS_WINDOW_NOACTIVATE` (set) | The window opens behind the others and does not take the focus (Windows) |
|
||||||
| `HOS_FULLSCREEN=1` | Starts in fullscreen; the Insert menu's Fullscreen row is then left alone |
|
| `HOS_FULLSCREEN=1` | Starts in fullscreen; the Insert menu's Fullscreen row is then left alone |
|
||||||
| `HOS_AUDIO=0` | No sound device is opened (the mixer still runs); `scripts/run-vk.ps1` sets it unless `-Sound` |
|
| `HOS_AUDIO=0` | No sound device is opened (the mixer still runs); `scripts/run-vk.ps1` sets it unless `-Sound` |
|
||||||
| `HOS_LANGUAGE=<code>` | The game's language for this run, a code from `language.txt` |
|
| `HOS_LANGUAGE=<code>` | The game's language for this run, a code from `language.txt` |
|
||||||
| `HOS_OUTPUT_RES=<0..3>`, `HOS_HANDHELD=1`, `HOS_ASPECT=<ratio>` | Over the options file: the output resolution (0 2048x1152, 1 2560x1440, 2 3840x2160, 3 1920x1080), the handheld profile, a picture wider than 16:9 (2.3889 for 3440x1440) |
|
| `HOS_OUTPUT_RES=<0..3>`, `HOS_HANDHELD=1`, `HOS_ASPECT=<ratio>` | Over the options file: the output resolution (0 2048x1152, 1 2560x1440, 2 3840x2160, 3 1920x1080), the handheld profile, a picture wider than 16:9 (2.3889 for 3440x1440; the Ultrawide row's 21:9 is 2.3704, 64:27, and its 32:9 3.5556) |
|
||||||
| `HOS_DYNRES=0`, `HOS_REALTIME_STEP=0`, `HOS_TEXT_SPEED=<0..3>` | Over the options file: Resolution 100% (non-zero: Auto), Game speed Per frame (1: Real time), Text speed |
|
| `HOS_DYNRES=0`, `HOS_REALTIME_STEP=0`, `HOS_TEXT_SPEED=<0..3>` | Over the options file: Resolution 100% (non-zero: Auto), Game speed Per frame (1: Real time), Text speed |
|
||||||
| `RX_VK_FRAMEGEN=0`, `RX_VK_FRAMEGEN_MULT=<2..8>`, `RX_VK_PACE=0`, `RX_VK_UPSCALE=<0..2>` | No frame generation whatever the menu says; over the options file: its multiplier, frame pacing, the upscaling filter |
|
| `HOS_STEP_STEADY=0` | (Triforce, on by default since the lab's A/B; `=0` the mean of the last five as before) The game's real-time step kept steady, after NX2navi's steady step (`host/step_steady.h`, `host/nn_vi.c` `realtime_step`): the mean of the last 12 present intervals without their longest and shortest in place of the mean of the last five (`HOS_REALTIME_WINDOW`), the paced period itself where that mean is within 3% of it, and what the display showed and the game was not given repaid at a 64th a frame, 10% of the step at most (a quarter second owed at most). A cap the game holds steps it by the period exactly, and a hitch is paid back over most of a second instead of in the next five frames (a 50 ms frame at 240: 200 frames of at most 10% more, where the mean of five gave five steps of 13 ms). Game time still follows the clock |
|
||||||
|
| `RX_VK_FRAMEGEN=0`, `RX_VK_FRAMEGEN_MULT=<0..9>`, `RX_VK_PACE=0`, `RX_VK_UPSCALE=<0..2>` | No frame generation whatever the menu says; over the options file: the Frame generation row's value (0 Off, 1 Auto, 2 to 8 the multiplier, 9 AMD FSR frame generation: a lab run's pin for each generator; `gpu/vk/framegen_rows.h` `env_row`), frame pacing, the upscaling filter |
|
||||||
|
| `RX_VK_FG_OWN_MOTION=0` | (Triforce, FG-4) Frame generation's motion the camera's alone, as before, for comparing. Without it, while NVIDIA DLSS or AMD FSR 4 upscale, the depth and motion frame generation takes (AMD's, NVIDIA DLSS frame generation, FSR 3.1's interpolation) are made from the upscaler chain's frame, with what moves by itself drawn into the motion (`gpu/vk/vk_temporal.cpp` `fg_inputs_from_chain`, `gpu/vk/vk_fsr3_inputs.cpp` `chain_motion`): also where the gameplay network does not translate on the card and the upscaler draws it in its place (RDNA 3), where they were never made before and AMD's frame generation made no frame |
|
||||||
|
| `RX_VK_FG_HUDLESS=0` | (Triforce, FG-2) No picture without the HUD is taken for AMD's frame generation, so its library makes no frame (the game's frame alone): a test. Without it the gameplay upscale's output is taken before the HUD is drawn into it, sRGB-encoded as the game's last pass encodes it, and given as `HUDLessColor` (`gpu/vk/vk_framegen.cpp` `framegen_hudless_note`, `hudless_for`); a present without it shows the game's frame alone; with HDR Native or RenoDX's RGBA16F scene AMD's library is not asked and the renderer's own makes the frame, as at 2x |
|
||||||
|
| `RX_VK_FRAMEGEN_FINE=0` | Frame generation's own made frames (Auto, 2x to 8x) the old way: a vector per 16x16 pixels in steps of 4 pixels, a blend of four of them per pixel, a pixel alike in both frames taken as still. Without it, a vector per 8x8 pixels to a fraction of a pixel, settled among its neighbours, and one motion per pixel (`gpu/vk/vk_framegen.cpp` `fine_vectors`, `shaders/fg_fine_*.comp`), which keeps moving straight lines straight and thin lines drawn (`tests/test_framegen_lines.py` measures both) |
|
||||||
|
| `RX_VK_PACE_SMOOTH=0` | (Triforce, on by default since the lab's A/B; `=0` off) The Frame pacing row on the plain present too, without frame generation: with the row on, a picture of the game's that comes sooner than seven eighths of the recent interval after the last one waits for that, its blit on the queue and the queue's lock let go; a late one goes at once, and none waits more than 50 ms (`gpu/vk/present_smooth.h`, after NX2navi's `pace_smooth`). Without it the row spaces only frame generation's frames, as before. With `RX_VK_PRESENT_THREAD=0` the present is on the game's presenting thread, so a wait also holds that thread's next frame: what that does to the rate under the cap is for a measurement before it is on. A `vk: frame pacing` line every 1200 pictures says how many waited and how long |
|
||||||
|
| `RX_VK_PRESENT_THREAD=1` | (off by default since the lab showed no gain) (Triforce, on by default; `=0` as before) The game's presents go out from a present thread of their own (`gpu/vk/present_thread.h`, after NX2navi's present thread): the presenting thread records and submits the blit as before (the frame fence, the swapchain's image, the overlays, the bug capture's copy and the frame log's: the same commands in the same order on the queue), then hands the pacing's wait (`RX_VK_PACE_SMOOTH`) and the present call to that thread and goes on to its next frame; the next present waits first until the one handed over is out, so one is in hand at most. Not with frame generation, NVIDIA DLSS frame generation or AMD Anti-Lag 2 (presents and stages of their own); NVIDIA Reflex's present markers carry their frame's id. The `vk: a frame's present` line every 120 presents says how many went out from it, the presenting thread's time in the present and how much of it was the wait for the present before, and the pacing's wait and the present call (on whichever thread); the stall line's present is the presenting thread's, with that wait named |
|
||||||
|
| `RX_VK_REFLEX_LIMIT=1` | (Triforce, off by default) With the NVIDIA Reflex row on and no frame generation, Reflex's sleep holds the Game frame rate cap (the sleep mode's minimum interval) where each frame starts, in place of the game's own wait; and with DLSS frame generation, Streamline's Reflex holds it the same way (`gpu/vk/vk_dlssg.cpp` `pace`; options_version 7: the Reflex frame limit row is gone, and this is its only switch until that path has been played on an RTX 40/50 card; `vk: dlssg: frame limit:` lines say who holds it): one wait a frame, timed by the driver (`gpu/vk/vk_reflex.cpp` `reflex_pace`, the rule `gpu/vk/dlssg_pace.h`'s; after NX2navi's `low_latency`). The game's own wait holds a movie, a frame RivaTuner holds, a window that showed nothing for 200 ms, and the rest of the run once over half of 60 frames came sooner than three quarters of the period, or when the driver refuses the interval. `vk: NVIDIA Reflex: frame limit:` lines say who holds it; the Full overlay's Frame limit line says NVIDIA Reflex after the rate. |
|
||||||
|
| `HOS_PACE_HANDOFF=0` | The game's own wait paces every frame, with DLSS frame generation's Reflex too and whether RivaTuner Statistics Server runs: the renderer is never asked whether another limiter holds a frame, and RivaTuner gets no limit from the game (`host/nn_vi.c` `pace_handed`, `gpu/vk/vk_pace.cpp`) |
|
||||||
|
| `HOS_MEM_QUERY_USED=0` | (Triforce, 11-19) `nn::os::QueryMemoryInfo` answers the memory in use as the program and the system's part only, as before. By default it is the program, the heap and every page the game mapped beyond it, at most the total, as Horizon answers (`host/hos_mem.c` `query_used`): the game's resource cache sizes what it may still take from it, and without it kept every area it ever loaded (12.5 GB of its 9809 MB after four areas). Needs the census on (`HOS_MEM_CENSUS`). `HOS_MEM_QUERY_SLACK_MB=<MB>` (1024 by default): that much less of the game's pages answered as in use, room for its cache beyond a Switch's (our game needs ~3.2 GB of regions at a save load where a Switch leaves ~2.8; with 0 it evicted in every scene) |
|
||||||
|
| `HOS_MEM_TRACE=1` | (Triforce, lab) Logs every address region and page call of the game with its size and caller (`memtrace:` lines; a region's with its call stack) and every `QueryMemoryInfo` answer |
|
||||||
|
| `HOS_FPS_STEADY=0`, `=1`, `=2` | (Triforce, off by default: the Frame pacing row's two Steady values, over them) 1 any rate, 2 steps of the display's refresh: the game's frames paced to a period this PC holds in the scene, never shorter than the Game frame rate cap's (`host/pace_steady.h`, `host/nn_vi.c` `steady_period`). The rate is chosen by probes (`pace_steady_probed`): 32 frames at the cap, unpaced, the last 24 of them measured, at the start, 2 s after a load, at least every 12 s, and when a quarter of the last 64 frames ran late (2 s after the last probe at the soonest, twice that each time a probe changes nothing, up to 12 s); the rate becomes the first the last three probes' 90th percentile fits in with 2% to spare where that is slower, or the first it fits in with 5% to spare where that is faster. A frame paced slower does more work (the lab's Market: 11.6 ms unpaced, 15.5 ms paced at 55 fps), so the frames' own measure as they run chased the rate down; frames late now only bring a probe on, and the rate is never under what a probe shows the PC holds. `HOS_FPS_STEADY_PROBE=0`: no probes, the rule below as before. Without probes: what a frame took is its present interval less the longest of the pacing's waits in it (the acquire's, on the presenting thread or another, and the present's hold), and what it needed the interval less all of them (OoT's main thread and Presentation Thread take turns; where the waits add up to the whole interval, the longest: `pace_steady_measure`); a quarter of the last 64 frames that took more than the period lowers it at once to the first rate their 90th percentile fits in with 2% to spare, and once it has held 3 s since a raise (6 s since a lowering, doubled for each lowering within 30 s of a raise, up to 120 s, and back to 6 s at a load) it rises to the fastest rate the 90th percentile of what the last 512 frames needed fits in with 5% to spare (one rate at a time where the game acquires on another thread). A frame that took over 2.5 times the median of the last 64 (hitches among them), a present interval over 250 ms, and the frames of the 2 s after such a load (the scene is not its measure yet) are left out; the first lowering after a load waits for 128 frames of the new scene and takes their 90th percentile. The acquire's wait starts the frames a period apart (the steady step then gives the game the period as its time step) and the present holds each picture of the game's to the period after the last, less a 64th of it (`gpu/vk/present_smooth.h` `present_smooth_paced_at`, with or without the Frame pacing row; not with frame generation). Another limiter holding the frames (RivaTuner, Reflex) puts the cap back while it does. `vi: steady frame rate:` lines at each change say what the frames needed, and every 5 s what the rule was given (what the frames took and needed at p50 and p90, the present interval's p50, the waits a frame by kind, the frames kept and left out by reason, how many of the last 64 kept were late, and the probes and what the last three took unpaced); the `vk: frame pacing` line counts the present's holds |
|
||||||
|
| `HOS_FPS_STEADY_PROBE=0` | (Triforce, with `HOS_FPS_STEADY` or the Steady frame rate row) No unpaced probes: the rate by the frames' own measure as they run, as before test31 (`host/pace_steady.h` `pace_steady_probed`, `host/nn_vi.c` `steady_period`); the frames' work grows when they are paced slower, and that rule slides down |
|
||||||
|
| `HOS_PACE_SPIN=0`, `HOS_PACE_SPIN=<microseconds>` | (Triforce) The game's own frame wait on the high-resolution timer for the whole wait, as before, which wakes up to about 0.5 ms after the frame's time. By default the timer takes the wait up to 0.6 ms before the frame's time and the rest is yielded away on the clock, so the frame starts on its time (`host/pace_wait.h`, `host/nn_vi.c` `pace_sleep`); a number is another margin. Every 5 s in which frames waited a `vi: frame waits` line says how long after their time the waits ended, on average and at most |
|
||||||
|
| `HOS_RTSS=0` | RivaTuner Statistics Server is left out entirely: nothing of it is read or loaded and its profile is not touched; a line `triforce_rtss_profile.txt` holds from a run that did not close is only reported in the log (`gpu/vk/rtss_limit.cpp`, *The options file* below) |
|
||||||
|
| `HOS_RTSS_DLSSG=1` | With DLSS frame generation RivaTuner is tried as the limiter too, at the cap times the multiplier in effect, with every safeguard (off: the game's own wait or Reflex, `gpu/vk/pace_rule.h` says why) |
|
||||||
| `RX_VK_DEVICE=<n>` | The graphics card listed as `vk: device <n>` in the log, over the menu's GPU device |
|
| `RX_VK_DEVICE=<n>` | The graphics card listed as `vk: device <n>` in the log, over the menu's GPU device |
|
||||||
| `HOOKSHOT_NO_UPDATE_CHECK=1` | No look for a newer release at the start |
|
| `HOOKSHOT_NO_UPDATE_CHECK=1` | No look for a newer release at the start |
|
||||||
|
| `HOS_SINGLE_INSTANCE=0` | (Triforce) No lock on the game folder: as before, a second copy of the game starts on a folder one already runs from. By default the first holds a lock named after its game folder (`host/hos_instance.h`), taken before the log is opened; a second copy waits up to 5 s for one that is ending (20 s after the menu's Restart), then says in a message box which process holds the folder and ends (a `--prepare-shaders` run or `HOS_HEADLESS`: an `@@error` line, and the prepare ends as one with no Vulkan). Other folders (the lab's copy, the owner's game) never meet. The log's `instance:` line says what this run did |
|
||||||
| `HOS_TRACE=1` | Every Horizon call logged, as `--trace` does |
|
| `HOS_TRACE=1` | Every Horizon call logged, as `--trace` does |
|
||||||
|
| `HOS_SAVE_EXPORT_DIR=<folder>` | Where the System tab's Export save rows write their zip, in place of the Downloads folder (set in the environment; `tests/test_save_share.py` uses it) |
|
||||||
|
| `HOS_SAVE_LOCK=0`, `TRIFORCE_SAVE_LOCK=0` | (Triforce) The save's lock between the game and Setup: the game holds `<user>\save.lock` open without sharing from its first save mount to its end, and Setup's Convert saves holds it across its copy (`host/save_tx.h` `save_tx_lock`, `setup/Guard.cs` `SaveLock`). A running game makes Setup refuse the copy; a game started during one waits for it (30 s at most) before it reads its save. `HOS_SAVE_LOCK=0`: the game takes none; `TRIFORCE_SAVE_LOCK=0`: Setup takes none and only looks at the running programs, as before |
|
||||||
| `RX_BACKEND=<name>` | Unset or `vulkan`: `hookshot_vk.dll`; any other value but `eden`: the WebGPU renderer, when the exe was built with one |
|
| `RX_BACKEND=<name>` | Unset or `vulkan`: `hookshot_vk.dll`; any other value but `eden`: the WebGPU renderer, when the exe was built with one |
|
||||||
|
|
||||||
### Input for scripted runs
|
### Input for scripted runs
|
||||||
@@ -128,8 +146,12 @@ case-sensitive, and an unknown one is ignored.
|
|||||||
| `HOS_THREAD_TIMES=<seconds>` | Every interval, the threads that used over 5% of a core |
|
| `HOS_THREAD_TIMES=<seconds>` | Every interval, the threads that used over 5% of a core |
|
||||||
| `HOS_FPS_LOG` (set), `HOS_PRESENT_TRACE=<csv>` | The game's frame limiter, VRR flags and nominal rate every 300 presents; a CSV line per game present |
|
| `HOS_FPS_LOG` (set), `HOS_PRESENT_TRACE=<csv>` | The game's frame limiter, VRR flags and nominal rate every 300 presents; a CSV line per game present |
|
||||||
| `RX_VK_FRAME_LOG=<csv>` | A CSV line per present: frame, seconds, frame time in ms |
|
| `RX_VK_FRAME_LOG=<csv>` | A CSV line per present: frame, seconds, frame time in ms |
|
||||||
|
| `RX_VK_FRAME_TRACE=1` | A line a frame in `triforce_frames.csv` beside the game (the lab's scenarios keep it as `frames.csv`): present to present, the waits for shaders and pipelines, the uploads, and the replay's parts (`memsync_ms` ... `tex_lookup_ms`: the draws' parts are sampled and summed over every thread that records). (Triforce, wp20, `gpu/vk/frame_trace.h`) Then each frame split as the stall line splits a long one: `replay_ms` (the submits and the present phase), `present_ms`, `dispatch_ms`, `qsubmit_ms`, `par_wait_ms` (the replay thread's waits for its helpers' segments), `outside_ms` (the frame less the replay and the present: the game's own update), `game_wait_ms` (the game's fence waits), `captures_ms` (the captures at its submits), `wb_wait_ms` and `wb_held_ms` (the waits for write-backs, and their part after the writer's recording was done: `RX_VK_WB_WAIT_REC`), `qs_bg_ms` (the queue submits beside a background import). `python -I scripts/frame_rank.py <run folder or csv>` ranks what made the frames over the 99th percentile and over 1.5x the median by these, per window (the Market, the tour's stays, the loads) |
|
||||||
|
| `RX_VK_WB_WAIT_REC=1` | (Triforce, off by default; the handoff's §12.9, `gpu/vk/wb_rec_wait.h`) A wait for a write-back still on the GPU (a capture's late pages on the game's submitting thread, a read or binding of a page put off) ends once the recording its submission was finished in is done on the GPU, not when the submissions' timeline reaches it: that timeline is held below any submission stopped at an event wait, and the lab's Market had 20-46 ms waits of the game's thread with the GPU 4 submissions behind. Whatever the switch, the wait for the recording comes first and the rest is counted: the stall line's `N of them X ms after their writer's recording was done`, the `late:` line's `ms of the wait after the recording that finished it was done`, the frame trace's `wb_held_ms`. Off: the captures' late-page wait orders something not known yet (`RX_VK_LATE_NEW`), so it is tried with the lab's flicker count first |
|
||||||
| `RX_VK_PASS_TIMING=1` or `2` | GPU time per pass (2: per program too), logged every 600 presents |
|
| `RX_VK_PASS_TIMING=1` or `2` | GPU time per pass (2: per program too), logged every 600 presents |
|
||||||
| `RX_VK_SLOW_LOG=<ms>` | The replay's times for each frame that took longer than this |
|
| `RX_VK_SLOW_LOG=<ms>` | The replay's times for each frame that took longer than this |
|
||||||
|
| `RX_VK_STALL_LOG=<ms>` or `0` | On in every run (`gpu/vk/stall_watch.h`): every 600 of the game's frames a `vk: frame times` line (the longest frame and how many were over 33, 50, 100 and 250 ms, present to present and in the replay's own time), and for each frame of 50 ms or more a `vk: stall` line of where its time went: the replay's phases, memory sync's parts, the waits for write-backs still on the GPU (how far ahead, and whether behind a present's blit), the waits for shaders or pipelines, the uploads, the pools imported where they were bound, the GPU behind, the present (its image, fence, call and Reflex's sleep), the game's fence waits, the rest, and a pool's trial in guest memory if one is under way (`RX_VK_DEMOTE_QUICK`); and the new textures left for the next frame and whether the frame's picture was held (`RX_VK_HOLD_MS`), whose holds the 600-frame line counts too. At most 3 stall lines a second and 1500 a session. `<ms>`: the stall line's threshold; `0`: neither line |
|
||||||
|
| `RX_VK_SLOW_TAKE_MS=<ms>` or `0` | (Triforce, 20 by default) A submit-time capture take that long or longer (`gpu/vk/vk_memory.cpp` `take_capture`): a `vk: slow capture take` line with where its time went (the marks and late pages, the pools lock, the retires, the stamps, the texture shadow, the buffer kept or given back), the first 200 of them. A Linux RTX 4090 near the Market stuttered with `memory sync [capture taken 30 to 110 ms]` while the 600-take average said 1.3 ms. `0`: none |
|
||||||
| `HOS_PGO=<file>` | In a run of an `RX_PGO=gen` build: `<file>.reset` clears the counters, `<file>.dump` writes them to `<file>`. It is read before `hookshot_env.txt`, so set it in the environment |
|
| `HOS_PGO=<file>` | In a run of an `RX_PGO=gen` build: `<file>.reset` clears the counters, `<file>.dump` writes them to `<file>`. It is read before `hookshot_env.txt`, so set it in the environment |
|
||||||
|
|
||||||
### The GPU
|
### The GPU
|
||||||
@@ -137,25 +159,133 @@ case-sensitive, and an unknown one is ignored.
|
|||||||
| Switch | What it does |
|
| Switch | What it does |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `RX_VK_VALIDATION` (set) | The Khronos validation layer |
|
| `RX_VK_VALIDATION` (set) | The Khronos validation layer |
|
||||||
| `RX_VK_CHECKPOINTS=0` | No NVIDIA checkpoints (on by default; after a lost device they name the draw it stopped at) |
|
| `RX_VK_CHECKPOINTS=0` | No NVIDIA checkpoints (on by default; after a lost device they name the draw it stopped at). (Triforce) The boundaries between a frame's work have markers too: the write-back diff pass, each recording's start, uploads and end, the present's blit and its menu (`gpu/vk/checkpoint_kinds.h`); the report reads the two stages' last markers together, says what followed the last one reached, and logs the switches, the queue's timeline and the mirrors' piece binds the submits wait for. (P4) Each marking thread numbers its own markers in a ring of its own (15 of 4096; threads after the 15th share a 16th), no atomic shared by the recording threads; the report names a marker's ring and reads the markers around it in that ring |
|
||||||
| `RX_VK_HANG_DUMP=<file>` | Every thread's stack to the file once no frame was shown for `RX_VK_HANG_SEC` seconds (20) (Windows) |
|
| `RX_VK_HANG_DUMP=<file>` or `0` | The hang watch (Windows) is on in every run: once no frame was shown for `RX_VK_HANG_SEC` seconds (20) with the game's window in front, the top of every thread's stack goes into the log and the whole stacks into `triforce_hang.txt` in the game folder, once a stall and at most three times a run. `<file>`: that file instead, and every second counts whatever window is in front. `0`: no watch (Triforce) The same report can be asked for at any time: put a file named `triforce_hang.txt.now` in the game folder (the report's name with `.now`); within a second the watch takes it away and writes every thread's stack, for a game that goes on showing frames and gets nowhere (a loading screen that never ends). |
|
||||||
| `RX_VK_NOOPT` (set) | Pipelines built without the driver's optimisations |
|
| `RX_VK_NOOPT` (set) | Pipelines built without the driver's optimisations |
|
||||||
|
| `RX_VK_PASS_BARRIER=game` or `full` | The barrier a draw pass starts with (`gpu/vk/pass_barrier.h`). `game`: a pass right after another, with nothing recorded between (no barrier of the game's, no copy, clear, uniform update or dispatch), waits for the earlier work in its fragment stages only, so its vertex, task and mesh work may overlap the earlier pass's fragments; a pass the replay itself split off for a hazard (`a pass of its own` in the log) and `RX_VK_BARRIER_EVERY_DRAW`'s keep a full one. (Triforce, P4) "Nothing recorded between" is the command buffer's own count (`RX_VK_BARRIER_CONTEXT`), so beside the Replay helper threads too. `full`: a full barrier at every pass start, as before. Unset: `game` with NVIDIA's driver, `full` on every other card. The log's `vk: pass barriers:` line says which, and the frame statistics' `pass starts ordered in their fragment stages only` how many a frame |
|
||||||
|
| `RX_VK_SPLIT_BARRIER=pass` or `full` | (Triforce) A long submission goes to the GPU in parts of 120 draws and dispatches (`RX_VK_CHUNK`); `gpu/vk/barrier_rules.h`. `pass`: a pass open at a part's split goes on in the next part, begun again with the same targets behind a barrier of its attachments' outputs alone and with what its draws sampled and wrote, so the GPU does not drain between two draws of one of the game's passes; where the part's end records anything (its uniform updates, its write-backs) or an event's write is held, the full barriers as before. A parallel replay's segments are not split into parts: each begins and ends with a full barrier, so nothing goes on across a segment's edge. `full`: a full barrier at each part's end and the next one's start, as before. Unset: `pass` with NVIDIA's driver, `full` on every other card. The log's `vk: a submission's parts:` lines say which and, every 3,600 splits, how many went on in their pass and how many of those parts had uploads (whose own full barriers order them after the part before all the same) |
|
||||||
|
| `RX_VK_EVENT_BARRIER=late` or `full` | (Triforce) An event signal in the game's command stream (`gpu/vk/barrier_rules.h`). `late`: nothing is recorded at the signal and its pass goes on (the word's buffer is bound there, as before); the event's word is written right after the next full barrier on its command buffer (the game's next barrier, a dispatch's or pass start's), so after all the work before the signal, and a recording's end writes what is still held behind a full barrier before its write-backs copy the words to the game's memory; a wait the replay finds met for an event whose write is held gets a full barrier first. The game sees the word at that barrier rather than at the signal. `full`: the pass ended, a full barrier and the write at each signal, as before. Unset: with the pass barriers' game mode (`RX_VK_PASS_BARRIER`: NVIDIA's driver). Not with the parallel replay. The `vk: event signals:` line says which |
|
||||||
|
| `RX_VK_TAIL_EARLY=1` or `=0` | (Triforce, on by default since options_version 6; over the Debug tab's Start next frame early row, `tail_early=0` to turn it off; read at the start; after NX2navi's `tail_early`) The early tail (`gpu/vk/tail_early.h`, `gpu/vk/vk_replay.cpp`): the replay cuts a frame's recording at the gameplay upscale's dispatch (the game's network, or DLSS, DLAA, FSR in its place) and records the rest, the upscaler and the interface, as one command buffer; its event signals are held. At the recording's end a tail that wrote no binding (its inline uniform updates aside), bound no guest memory itself, ran no program of the game's that reaches guest memory through the page table or a texel buffer, and wrote no texture back has its event words written in its upload command buffer, after the scene's write-backs and ahead of its own commands: OoT's present poll then sees its frame's event while the GPU still runs the tail. Any other tail has the words written at its end, as before. Also with the parallel replay (its segments end at that dispatch) and with the lazy write-back. Not with `RX_VK_BENCH` or a frame dump. A `vk: the early tail` line at the start says on, and every 1200 tails how many went early and why the others did not. Not yet run in the game |
|
||||||
|
| `RX_VK_DISPATCH_BARRIER=game` | (Triforce) Off unless set, not yet run in the game: a dispatch right after another, nothing recorded between, records no full barrier of its own (not an indirect one, nor a program a stand-in may take over), as NX2navi's `barrier_game` does; the game's own barriers order its dispatches. Only the game shows whether that is always so (a one-frame band or flicker where it is not): a switch for the lab. A `vk: RX_VK_DISPATCH_BARRIER=game:` line every 20,000 dispatches it left without one. (wp14, P4) It no longer stands aside while the Replay helper threads record (most frames at the Market, so the lab's earlier run of it measured little): "nothing recorded between" is the command buffer's own count of its commands (`gpu/vk/cmd_work.h`, `RX_VK_BARRIER_CONTEXT`), a segment being a command buffer of its own |
|
||||||
|
| `RX_VK_BARRIER_CONTEXT=0` | (Triforce, P4; after NX2navi's per-context barrier state) What the barrier rules ask, "was anything recorded into this command buffer since its last full barrier, its last pass's end, its last dispatch", is answered by each command buffer's own count of its commands (`gpu/vk/cmd_work.h`), where one count of every thread's commands served before (an atomic add on every command by every recording thread). So the rules hold while the Replay helper threads record: the left-out repeats of a full barrier, `RX_VK_PASS_BARRIER`'s pass starts and `RX_VK_DISPATCH_BARRIER`. A command buffer's begin starts its state afresh; each segment begins and ends with a full barrier, so no rule relaxes across a segment's edge. `RX_VK_SPLIT_BARRIER` asks also whether anything went to the queue between two parts (every submit is counted). The rules' defaults are as they were. `0`: the one count, and the rules standing aside while the helper threads record, as before. The `vk: barrier state:` line at the start says which; the frame statistics' `full barriers recorded, ... left out` and `pass starts ordered in their fragment stages only` show the difference |
|
||||||
|
| `RX_VK_BATCH=parts`, `1` or `0` | Held submits (`gpu/vk/vk_replay.cpp` `batch_way`). `1` (the default with Mesa's RADV): every submit held until something needs the GPU's results or the queue's order. (Triforce) `parts`, off unless set: only a whole submission's recordings are held, and only while the game's submit is being replayed; a part goes at once with those held before it, as do 4 held (`RX_VK_BATCH_MAX`), and the held go when the replay of the game's submit returns. Neither changes the command buffers or their order on the queue. `0`: each submit at once (the default elsewhere). The `vk: batched submits` lines say which and how many command buffers went in each queue call |
|
||||||
|
| `RX_VK_FOLIAGE_SHADOW_CULL=<layer>[f]` | (Triforce, after NX2navi's 754ded9) On from layer 1 unless set (`off`: as the game draws them): the draws of mesh programs into the sun's shadow cascades (depth only, square, wider than 768) from that layer of the cascades on (0 to 2; NX2navi's is 1) that cull neither face are drawn with their back faces culled (the front ones with `f`, as `1f`): the foliage's double-sided cards draw half their fragments there (`gpu/vk/foliage_shadow.h`). Not quite the same picture: a lone card facing away from the sun casts no shadow in the middle and far cascades (where cards stand in pairs or clumps the others cover it; the near cascade is drawn as the game asks). The log says `vk: RX_VK_FOLIAGE_SHADOW_CULL` at the start and names the first draws it culled (`vk: foliage shadow cull:`) |
|
||||||
|
| `RX_VK_FOLIAGE_SHADOW_VRS=<layer>` | (Triforce, after NX2navi's 754ded9) On from layer 2 unless set (`off`: the extension not asked for, no pipeline changed): the same draws of native mesh programs, from that layer of the cascades on, shade one fragment per 2 x 2 pixels (`VK_KHR_fragment_shading_rate`, where the device has `pipelineFragmentShadingRate` and native mesh shaders; elsewhere nothing changes): the leaves' alpha test runs a quarter as often, depth still per texel. The rate is dynamic state only in the pipelines of native mesh programs with no colour target and a depth one (and in their two shader libraries, keyed apart), which the driver builds again once: on such a device the driver's part of the preparation makes those libraries too (and the `RX_VK_LIB_CHECK` sample asks for them), once more at the first start after the update, before the game (the other libraries come back from the cache), and the light precompile builds each depth-only pipeline of a native mesh program once more; their keys, the program cache and every other pipeline stay as they were. The log says `vk: the far shadow cascades' foliage at 2 x 2:` at the start and names the first draws (`vk: foliage shadow 2 x 2:`) |
|
||||||
|
| `RX_VK_FOG_CULL=1` | (Triforce, after NX2navi's 0379b00) Off unless set: the volumetric fog's first compute pass skips, as the game does, the froxels behind a coarse depth table of the frame before and writes them as empty, which the fog's history smears into dark trails beside near objects (worst at the screen edges, in foggy places such as the Lost Woods and Kokiri Forest). By default those froxels are computed too: the comparison is made never true in that one module, found by its shape in the SPIR-V (`gpu/vk/spv_fog.h`: a block of five image writes, four of (0,0,0,1) and last 1/255, and the one branch to it an ordered greater-than decides). It is applied where the module is made (`make_module`), outside the translation stamp: the program cache's records stay as they were translated, so nothing is translated again, and only that one compute pipeline is new to the driver's caches. With the precompile keeping none (`RX_VK_PRECOMPILE_KEEP`, less RAM), its record of the compute programs built has the patch and the matcher's version in its key (`gpu/vk/vk_draw.cpp` kFogPatch times `spv_fog::kMatcher`, raised whenever the matcher finds modules it did not: a record written by the matcher before 11-17's, which patched none of Triforce's fog modules, is built again), so the first start with the patch, or with a new matcher, builds the listed compute programs once more before the game (every one but the fog's a cache hit, the fog's compiled into the play cache, `mkw_vk_pipelines_play.bin`, or the main one where there is none, and saved before the record says built) rather than compiling the fog's at its first dispatch; with `RX_VK_FOG_CULL=1` that key is the one it was. A little more GPU time in the fog's pass. The log says `vk: fog: module <crc>, the volumetric fog's first pass:` and what was done (also for a near miss: the exit found but not its comparison) |
|
||||||
|
| `RX_VK_FIFO` (set), `RX_VK_IMMEDIATE` (set) | The swapchain's present mode for one run, over the Insert menu's VSync row: FIFO, as with vsync forced by a driver; immediate, where the surface lists it. Unset, the row decides (`gpu/vk/present_mode.h`): On is mailbox where the driver has it (immediate on Windows where it has none, FIFO elsewhere), Off immediate wherever the surface lists it |
|
||||||
| `RX_VK_SKIP_PROG=<prefix>,...` | Leaves out the draws of these programs (default `53509044401d78d2`; `-` leaves out none) |
|
| `RX_VK_SKIP_PROG=<prefix>,...` | Leaves out the draws of these programs (default `53509044401d78d2`; `-` leaves out none) |
|
||||||
| `RX_VK_PIPE_SKIP=0`, `1` or `2` | Default/`1`: wait for shader translation; a pending pipeline can be skipped only for attachment-only draws whose targets hold an earlier frame. `0`: wait for every pipeline too. `2`: restore the old skip for untranslated draws with persistent targets; this can omit writes before the shader is known. Translated memory-writing draws still wait |
|
| `RX_VK_PIPE_SKIP=0`, `1` or `2` | Default/`1`: wait for shader translation; a pending pipeline can be skipped only for attachment-only draws whose targets hold an earlier frame. `0`: wait for every pipeline too. `2`: restore the old skip for untranslated draws with persistent targets; this can omit writes before the shader is known. Translated memory-writing draws still wait |
|
||||||
|
| `RX_VK_PIPE_FRAME_WAIT_MS=<ms>`, `RX_VK_PIPE_ONESHOT_BUDGET=1`, `RX_VK_PIPE_SKIP_NEW=0` | A draw whose pipeline is not ready (`gpu/vk/pipe_wait.h`, NX2navi's rules). A full-screen draw (at most 6 vertices over half the pass or more: a lighting, fog or tone mapping pass) into targets that hold an earlier frame is left out only where the hold of the last picture takes its frame (`RX_VK_HOLD_PIPELINES`); where it cannot (at its bound, resting after one, or off) the draw waits for its pipeline up to this many ms a frame in all (50, at most 500), then 40 ms more, and is left out after both; its job goes to the front of the workers' queue meanwhile. `0`: left out, as before. `RX_VK_PIPE_ONESHOT_BUDGET=1` (off; NX2navi has it on): draws into targets drawn once and draws whose shaders write memory wait from the same budget and are left out once it is spent, instead of waiting until their pipeline is made. `RX_VK_PIPE_SKIP_NEW=0`: a pass whose colour target is drawn for the first time is kept by its depth target alone, as before (by default its draws wait, as for any target drawn once: the game's item icon capture). The log's `vk: pipelines not ready` line counts them |
|
||||||
|
| `RX_VK_PIPE_FAST_WORKERS=0`, `RX_VK_PIPE_LINK_MS=<ms>` | A pipeline whose stage libraries were cold has those libraries built by a worker first and is linked from them, ready with that link, and its whole optimised build follows as a job of its own (`gpu/vk/pipe_wait.h` `libs_first`, NX2navi's `pipe_fast_workers`); a draw that waits for it runs the libraries and the link, not the whole build. `0`: the whole build first and the cold libraries after it, as before. `RX_VK_PIPE_LINK_MS=<ms>` (0 by default, at most 100; NX2navi's 6): the links made on the draws' threads in a frame take at most this long; a new pipeline after that is linked by a worker, its draws left out or waited for meanwhile (needs the fast pipeline link and the first switch on) |
|
||||||
| `RX_VK_PRECOMPILE=0` or `1` | The shader preparation before the title off or on (on in builds made with `RX_RELEASE=1`, as the builders' are) |
|
| `RX_VK_PRECOMPILE=0` or `1` | The shader preparation before the title off or on (on in builds made with `RX_RELEASE=1`, as the builders' are) |
|
||||||
| `RX_VK_TASK_MESH_FALLBACK=0` or `1` | Task + mesh draws through the compute fallback (1) or the driver's mesh shaders (0). By default the fallback is on for AMD's Navi 21 and Navi 22 on Windows (RX 6800, 6800 XT, 6900 XT, 6950 XT, 6700 XT), where the native path timed out the driver (issue 996) |
|
| `RX_VK_PREPARE_THREADS=<n>` | The threads the shaders are translated and their driver's part made on before the title, 1 to 64. Unset: every processor, a thread past half of them for each GB of RAM free at the start, and half in a process started above normal priority (the log's "the shaders' preparation runs on" line says how many) |
|
||||||
|
| `RX_VK_TASK_MESH_FALLBACK=0` or `1` | Task + mesh draws through the compute fallback (1) or the driver's mesh shaders (0). By default the fallback is on for AMD's Windows driver for RDNA 2 (every RX 6000 card and that generation's integrated GPUs: the driver whose `vk: vendor fixes` line names the "AMD proprietary shader compiler"), where the native path timed out the driver on the RX 6700 XT and RX 6800 XT (issue 996) |
|
||||||
|
| `RX_VK_MESH_AHEAD=0` | (Triforce, on by default, after NX2navi's `mesh_ahead`; `=0` as before) An indirect mesh draw's records are copied into a ring as `{count, 1, 1}` before it, outside the pass, so each one ended its pass and the next draw began it again (OoT's foliage, grass and leaves: about 16 a frame at the Market). Now the first indirect mesh draw of a run copies the records of the indirect mesh draws after it as well, in the same three barriers, and those stay in the pass: a run is the indirect mesh draws after it with only state, draws, counter reports and render conditions between (`gpu/vk/vk_replay.cpp` `mesh_run_kind`); a dispatch, a barrier, a copy, a clear, a uniform update, new targets or an event ends it, so no work of the game's between can write the records after they were copied. A draw takes only the entry copied for it (its op, records, count and stride); at most 32 draws and 1 MB of the ring a run (`gpu/vk/mesh_ahead.h`). A `vk: indirect mesh draws` line every 20,000 conversions says how many took their records from an earlier draw |
|
||||||
| `RX_VK_NO_PROGRAM_CACHE` (set) | Neither reads nor writes `hookshot_programs.bin` and the pipeline notes |
|
| `RX_VK_NO_PROGRAM_CACHE` (set) | Neither reads nor writes `hookshot_programs.bin` and the pipeline notes |
|
||||||
|
| `RX_VK_PROGRAM_CARRY=0` | The program cache (`triforce_programs.bin`) is good for one build of the translator only, as before: any change to its files, a log line or a host option's number in `gpu/vk/vk_api.h` too, translates every program again, and the pipeline cache, the pipeline notes and the prepare's record go with it. By default the cache is carried over an update that changed nothing of what translates (`gpu/vk/program_carry.h` has the rule, `tests/test_program_carry.py` holds it): the translation stamp (`nx2/build.py` `translation_stamp`) is the translator's files without `vk_api.h`, without the parts of `vk_shader.cpp` and `vk_bank.cpp` marked as outside it, and without comments, blank lines and spaces at lines' ends. The build log's `build: translation stamp` line says whether it is the last release's (`nx2/released_translation_stamp.json`, which players' caches are for, and which `tests/test_program_carry.py` holds this source to) or which stamped files moved it against that release; a second line, in brackets, compares it with the previous build in the same folder (its record: `translation_stamp.json` in the build folder, written once the objects are compiled). Both are about the build's part of the stamp: the runtime adds its own settings to it (`disk_stamp`). The log's `vk: program cache` line at the start says whether the programs were kept, carried over an update of the translator (and from which build), or translated again (and how many the cache had); while they are translated again a line now and then counts them. Switching it either way translates every program once more |
|
||||||
|
| `RX_VK_CACHE_SET=0` | (Triforce) No earlier set of the shader caches is kept or put back (`gpu/vk/cache_set.h` has the rule, `gpu/vk/vk_cache_set.cpp` the files, `tests/test_cache_set.py`). By default, when the caches' stamp changes and the build of the translator did not (a setting of the translation: the menu's Robust buffers row, the mesh shaders row, the GPU, an `RX_VK_` switch of the translation), the set there (the program cache and its index, the pipeline cache and the play file with their `.for`, the prepare's record with its `.check`, `.warming` and `.warming.skip`, the pipeline notes and what is kept beside them by the stamp) is renamed to `<name>.<stamp>.<ext>` when the disk has twice the set and 10 GB free, and `triforce_cache_set.kept.txt` lists it. One set is kept. A start whose stamp is the kept set's puts every piece of it back (a set with a piece missing is deleted), and the set it found is kept in its place; a start of another build deletes it. `triforce_cache_set.txt` names what the set there was made for, and the log's `vk: shader caches:` lines say which part of the stamp changed. A set put back while the player's `triforce_pipeline_states.bin` is not the one it was kept with leaves `triforce_cache_set.states.txt`, and the prepare before the title then goes through the state lists, so that what was met under the other set is built before the game. Files of a kept set's names that no record names (a start that ended in a swap) are deleted at the next start. `RX_VK_CACHE_SET=0` also deletes a kept set; `RX_VK_PCACHE_KEEP_OLD` and `RX_VK_NO_PROGRAM_CACHE` leave it off too (a kept set of this build stays, another build's is deleted) |
|
||||||
|
| `RX_VK_GPL=1` or `0`, `RX_VK_GPL_KEEP_LINKS` (set) | The fast pipeline link (`VK_EXT_graphics_pipeline_library`; `gpu/vk/gpl_rules.h`), on by default since options_version 5: a new pipeline is linked from its program's two shader libraries where the driver has them compiled, and the optimised build replaces it. The Debug tab's Fast pipeline link row is the same without an environment to set (`fast_pipeline_link=1`); the switch goes over the row. With it the preparation has its driver's part ("Compiling shaders"), which now goes on after a start that died in it, leaving out the programs that start had in the driver's hands (`triforce_prepared.bin.warming` and `.warming.skip`), up to three starts in a row. `RX_VK_GPL_KEEP_LINKS`: the linked pipelines the optimised ones replaced are kept for the run, as before, instead of destroyed 600 presents and 10 s later. `RX_VK_NO_WARM` (set): no driver's part; `RX_VK_GPL_SYNC` (set): a cold library is compiled in the draw; `RX_VK_GPL_NO_CACHE` (set): the libraries are not kept in the pipeline cache |
|
||||||
|
| `RX_VK_LIB_CHECK=0` | (Triforce, after NX2navi's e10ef60) On unless `0`. At a start whose record says the driver's part is made for these shaders and this driver, a sample of 12 kept program variants has its two stage libraries made as that part makes them, asking the caches only where the device can (the answer a draw's link gets; without the flag, `RX_VK_GPL_SYNC`, their time against a second making, which comes from the cache), twice, before the game (`gpu/vk/lib_check.h`). When more than half are cold (`mkw_vk_pipelines.bin` lost or refused, or the libraries built otherwise now without `kWarmRecipe` counted up) the driver's part is made again at once, as after a driver update, with its journal and skip list; warm, nothing more happens (some 50 library lookups). `triforce_prepared.bin.check` records that it did: a start that finds them cold again right after leaves them to the draws, as before (each try holds a start for minutes; remove the file to try again). A making again whose cache could not be saved is recorded the same way. A cache file over its bound is not made again at all. The log's `vk: stage libraries:` line says what was found |
|
||||||
|
| `RX_VK_SPV_PACK=0` | Every shader module kept for the whole run, as before. By default the SPIR-V of graphics and mesh programs is kept packed in memory (`gpu/vk/spv_store.h`, to about 0.37 of its size) and their shader modules, which the driver otherwise holds for the whole run, are made only for the time a pipeline is created with them, on the thread that builds it. Compute programs, the programs the preparation lets go and the programs a draw reads from the cache itself keep their modules. The log's `vk: shader code packed` lines say what it comes to and what it costs |
|
||||||
|
| `RX_VK_PCACHE_RAM=1` | Off by default (in its one session on, v0.4.0-test9, it gave back nothing the working set showed). With it, once the PC has had under 1.5 GB of RAM available at two looks a second apart, the pipeline cache (when it is 256 MB or more) is destroyed once no pipeline creation is using it (`gpu/vk/pcache_gate.h`): the pipelines built after that go without it and it is not saved again that session, so `mkw_vk_pipelines.bin` keeps its last save. The log's `vk: RAM short ...: the pipeline cache ... is let go` line says when and what the working set did. Whatever this switch says, the cache's save in play waits while a copy of it would not fit in the RAM available with 2 GB to spare (the save at the game's close does not), and the builder of the predicted pipelines behind the game rests while under 3 GB (or a tenth of a PC of more than 30 GB) is available (`gpu/vk/mem_parts.h`) |
|
||||||
|
| `RX_VK_WRITERS_WAIT_MS=<ms>` | (Triforce) How long the game's close waits for the files the renderer is writing on threads of their own (`gpu/vk/file_writers.h`; 2000 by default, `0`: no wait and nothing refused, as before). The close ends the process through ExitProcess, which stops every other thread where it is: the pipeline cache's save in play, the order file, a bug capture's or a debug picture, the hang report, the predicted pipelines' list, the frame log. At the close (`HOS_OPT_EXITING`) no such write begins any more and the ones under way are waited for; one still going at the bound is cut, and is one of the files written whole beside the old (`.part` / `.tmp`, then put in place: the prepared record `triforce_prepared.bin` and the `.for` files are now too), or a list whose reader takes whole records only. The log's `vk: exit: the files being written ... were waited for` line says how long, and what was cut |
|
||||||
|
| `RX_VK_OPTIONS_MERGE=0` | (Triforce) The menu writes its own values over `triforce_options.ini` in place at every change, as before. By default (`gpu/vk/ini_merge.h`, `vk_menu.cpp` `save`) each line it writes is merged by its key with the file as it is then: a line changed outside the game while it runs (a text editor, a tool) and not changed in the game since the game read or wrote the file is written as the file has it, so a change made outside and one made in the menu within the same second both stay (a row changed on both sides is the game's); the game takes such a line at its next start. The file is written whole beside the old one and put in its place (in place only where another program holds it open so that it cannot be replaced). The log's `menu: options: ... changed in triforce_options.ini outside the game` line names such lines |
|
||||||
|
| `RX_VK_MEM_PARTS=0` | No `vk: memory, the parts` line. By default every 600 presents it gives the working set, the game's own memory committed and in RAM (a sample of its pages), the rest of the working set in RAM by what it is (a sample too: the program's code, the driver's code, other code, mapped files and views, write-combined or uncached memory, the heaps and other private memory), the shader modules alive and the packed stages, the pipeline cache, the pipelines and stage libraries kept (how many a draw has had, how many were given back) and the allocator's blocks in system memory (`gpu/vk/mem_parts.h`), and what its walks and samples took. It is written on a thread of its own that nothing waits for; the guest window's walk is made again every eighth line |
|
||||||
|
| `HOS_MEM_CENSUS=<seconds>` or `0` | (Triforce) The guest memory census (`host/guest_census.h`), on by default every 5 seconds, on a thread of its own: a `mem: census` line with what the runtime handed out of the guest window by kind (the program and the system region, the heap, the address regions and the game's own pages in them, what lies outside them: the physical maps and first touches), the game's own count against the 9809 MB it is told it has, the pages the game let go that stay committed (zeroed under an import, or discarded with a freed region) and what the give-back gave back, the process's commit and working set, and the PC's commit and RAM; and a `mem: census, the renderer` line with the Vulkan back end's big holders (its imports, mirrors, VRAM in use, the allocator's blocks in system memory, the pipeline cache, the pipelines and stage libraries kept, the shader modules). A count of bits a 4 KB page, kept as each place commits, maps or lets go of guest pages. `0`: no line (and, with the give-back off, nothing kept) |
|
||||||
|
| `HOS_MEM_GIVE_BACK=1` or `2` | (Triforce) Off by default, not yet run in the game: the pages the game let go that stay committed because an import held them (`FreeMemoryPages`, `UnmapPhysicalMemory`) or because they were discarded with a freed address region are decommitted once nothing of the renderer imports them, from 4 GB up (below it pages are handed out without a commit), up to 256 MB a census pass, in runs of 16 MB at most; the renderer answers and runs the decommit with its import locks held (`gpu/vk/vk_memory.cpp` `guest_release_unimported`), so no import can lock them between. `1`: every pass; `2`: only while the PC is short (its commit past 85% of its limit, or under 1.5 GB of RAM available). The census line's `given back` says how much |
|
||||||
|
| `HOS_MEM_MAP_RESULT=0` | (Triforce) `MapPhysicalMemory` answers success even where Windows refused the commit, as before. By default it answers Horizon's LimitReached (`0x10801`), which the SDK turns into its out-of-memory result, and the log says `mem: MapPhysicalMemory ...: the commit failed` |
|
||||||
|
| `RX_VK_PRECOMPILE_KEEP=1` or `0` | The precompile keeps every pipeline it builds for the session (1), or keeps none of them: they are built for the caches and let go, and a draw that first needs one builds it from there (0). Unset or `auto`: kept where the RAM available at the precompile and what the game holds already (less what reading the pipeline cache added, except on AMD's driver: `own_for_keep`) come to 22 GB or more, on a PC of 20 GB or more (`gpu/vk/mem_parts.h` keep_all_pipelines); the log's `vk: precompile:` line says which and why |
|
||||||
|
| `RX_VK_PIPE_RELEASE=0`, `RX_VK_PIPE_RELEASE_EVERY=<presents>` | Every kept pipeline stays for the session, as before. By default, while the PC's RAM is short (under 1.5 GB available at two looks in a row), every 300 presents up to 512 of the kept pipelines no draw has had yet are given back to the driver (`gpu/vk/pipe_release.h`); a draw that needs one later has it built again by a worker. The log's `vk: RAM short: ... kept pipelines no draw has had given back` lines say how many and what the working set did |
|
||||||
|
| `RX_VK_PCACHE_CARRY=0` | The pipeline cache's file carried over a stamp change (`gpu/vk/pcache_carry.h`): at an update's preparation (Setup's "Preparing shaders", `HOOKSHOT_PREPARE_ONLY`) a `mkw_vk_pipelines.bin` made for other shaders is read all the same when it is this device's and driver's, within its bound, and at most 1.25 times its base (the size of the last set built fresh, kept beside it in `mkw_vk_pipelines.bin.base` with the carries since): the pipelines of shaders the update did not change are hits instead of hours of compiling. The cap is over every carry since the base was built, not a quarter more each time, and a carried cache may never pass the file's bound either. The preparation looks at a carried cache's size every 10 s while it builds and at each save: past the cap (or the bound), nothing more is built into it, it is not written, `mkw_vk_pipelines.bin.nocarry` names the new stamp (a preparation begun again after a Cancel starts fresh), and the preparation builds the driver's part again into a fresh cache: the compute pipelines the carried run built, from the program cache, then the stage libraries, nothing translated again (from the driver's own cache for what it had just compiled), which gets a new base. A carried preparation that is stopped (a Cancel, a closed window) saves its cache under the new stamp with its history marked open in `.base`: the preparation begun again reads it as carried still, capped the same (or, when `.nocarry` names the stamp, not at all), until a carried preparation goes through to its end. The log says `pipeline cache carried over a stamp change (N MB, base M MB)`. `0`: a file for other shaders is never read, as before (the rule of 11-14). The play file keeps its own rule |
|
||||||
|
| `RX_VK_PCACHE_CARRY_PROJECT=0` | The early drop of a carried pipeline cache (`gpu/vk/pcache_carry.h`). When the stage libraries are built into a cache carried over a stamp change, every 32nd program of the warm's order is built first, the others waiting for it, and the cache's growth over that sample, per program, times all the programs, is where it ends at this rate. With at least 1500 libraries in the sample, a projected growth of at least twice the room it had when they began (its cap, or the bound, less its size then) drops the carry at once, exactly as past its limit (not written, `.nocarry`, the driver's part built again into a fresh cache), a few minutes in rather than when it gets there: an update whose file was built by an older translator misses nearly every library and filled the cache for 34 minutes before it was let go. The sample is spread over the whole order, not its head: the notes' programs go first and the rest in the program cache's file order, and hits and misses come in stretches of it, so a head that misses while the rest hits would have been dropped for nothing. The notes' programs wait for the sample (a few minutes cold). The log says `the pipeline cache carried over a stamp change would grow to about N MB at this rate`. The compute pipelines' pass and the precompile keep the look past the limit alone. `0`: dropped only once past its limit, and the warm in its own order, as before |
|
||||||
|
| `RX_VK_PCACHE_MAX_MB=<MB>` | The bound on the pipeline cache's file, `mkw_vk_pipelines.bin` (`gpu/vk/pcache_bound.h`): a file over it is not read (the cache starts empty and the next save writes a new one), and data over it is not written (the file keeps its last save). Unset: an eighth of the PC's RAM, 2048 MB at least and 8192 MB at most. `0`: no bound, as before |
|
||||||
|
| `RX_VK_PCACHE_CARRY_MB=<MB>` | (Triforce, 11-18) The carry of the pipeline cache's file over an update that changed the translation (`gpu/vk/pcache_bound.h` `carried`): a file for other shaders is read all the same when it is no larger than this (1536 MB unset), leaves room under the bound for as much again and the PC has 31 GB of RAM or more (Windows), so the driver finds what it compiled for the shaders that did not change (AMD's driver keeps no cache of its own for these). `0`: never carried, the rule of 11-14 alone |
|
||||||
|
| `RX_VK_PCACHE_PLAY_FILE=0`, `RX_VK_PCACHE_FOLD_MB=<MB>` | The pipeline cache's play file (`gpu/vk/pcache_play.h`, NX2navi's play cache): once the preparation is done, what a run compiles goes into a second cache with a file of its own, `mkw_vk_pipelines_play.bin` beside the first (a creation asks the first cache without compiling, then the play cache), and a save writes that small file; `mkw_vk_pipelines.bin` is written only when the preparation built into it. Before, a save after a few new pipelines wrote the whole file again (2.6 GB at the owner's start after its precompile). At a start whose play file has reached `RX_VK_PCACHE_FOLD_MB` (64) it is folded into the first cache, that session builds into it as before, and its first save writes it and removes the play file. Needs `pipelineCreationCacheControl`. `0`: no play file, as before |
|
||||||
|
| `RX_VK_WARM_SAVE_MIN=<minutes>` | (Triforce) Checkpoint saves of the pipeline cache during the shader preparation's long builds (`gpu/vk/warm_checkpoint.h`, `tests/test_warm_checkpoint.py`): while the driver's part (the stage libraries, about two hours cold on an RX 7900 XT) and the precompile build, `mkw_vk_pipelines.bin` is saved every 10 minutes once at least 1000 were built since the last save, so that a restart, a crash, a power cut or a kill goes on from the last checkpoint (what it saved comes back as hits) instead of from nothing. Each goes through the preparation's own save: the file's bound, a carried session's cap with its history kept open as a stopped one's (`RX_VK_PCACHE_CARRY`), the file written whole with its `.for` and `.base`; it is left out while a copy of the cache would not fit in the RAM available with 2 GB to spare or the disk lacks its size and 1 GB (asked again a minute later), fetched with room for what is built meanwhile, written to the disk before it takes the old file's place, given up when the window is closed or Setup's Cancel is pressed (the stop's own save writes the cache), due only while the part moves (a stuck one goes silent), and it does not count the part as made (the next start still runs it). Beside the file, `.own` says it holds this part's pipelines alone, so that a part over the file's bound after a crash ends as a new file over the bound, not as an older build's file made again from nothing. The log says `vk: the driver's part so far saved for a restart: N MB in X s`. Not on Intel unless `RX_VK_PCACHE_PLAY_SAVE=1` (its driver crashed fetching the cache while pipelines were built with it); `RX_VK_PCACHE_PLAY_SAVE=0`: on no card. `0`: saved at the end alone, as before |
|
||||||
| `RX_VK_FETCH_BOUNDS=0` or `1`, `RX_VK_NAN_BITS=0` or `1`, `RX_VK_GATHER_OFFSET=0` or `1` | The translation's vendor and driver workarounds (`gpu/vk/vk_vendor.h`) off or on for one run, whatever the GPU; the log's `vk: vendor fixes` line says which are on. By default: out-of-bounds texel fetches answered with 0 on every GPU but NVIDIA's; NaN tests on a value's bits on AMD's older Windows shader compiler (RX 5000 and RX 6000); gathers nudged by 1/512 of a texel on AMD's and Intel's drivers. They change the shaders' translation (one preparation of the shaders) |
|
| `RX_VK_FETCH_BOUNDS=0` or `1`, `RX_VK_NAN_BITS=0` or `1`, `RX_VK_GATHER_OFFSET=0` or `1` | The translation's vendor and driver workarounds (`gpu/vk/vk_vendor.h`) off or on for one run, whatever the GPU; the log's `vk: vendor fixes` line says which are on. By default: out-of-bounds texel fetches answered with 0 on every GPU but NVIDIA's; NaN tests on a value's bits on AMD's older Windows shader compiler (RX 5000 and RX 6000); gathers nudged by 1/512 of a texel on AMD's and Intel's drivers. They change the shaders' translation (one preparation of the shaders) |
|
||||||
|
| `RX_VK_NAN_SELECTS=0` | (Triforce, after NX2navi's e38f42a, issue #156) Where the NaN tests are made on the bits (`RX_VK_NAN_BITS`: by default AMD's older Windows shader compiler, RX 5000 and RX 6000), a float select of a constant whose bits are a NaN (an integer mask the translation types as a float) is also made a select of the bits, so that compiler cannot fold it into stray black or white blocks (`gpu/vk/spv_nan_select.h`: 32-bit selects, and 16-bit ones in a module that has a 16-bit integer type). It is applied where a module is made (`make_module`), outside the translation stamp: no program is translated again, nothing changes on any other card, and on those cards only the modules that have such a select are new to the driver's caches. So that those are compiled before the game rather than at their first draw or dispatch, the driver's part of the preparation, the light precompile's record and the predicted pipelines' record have a stamp of their own there: the first start on those cards goes through them once more, everything but those modules coming back from the caches (the variants on the driver's part's skip list are carried over). `RX_VK_NAN_SELECTS=0` leaves the selects as they are (the NaN tests on bits stay). The log says `vk: NaN selects: module <crc>, <n> float select(s) of a NaN constant made select(s) of the bits` for the first four modules and every 64th |
|
||||||
| `RX_VK_MIRROR=0`, `RX_VK_MIRROR_MB=<MB>` | No VRAM mirrors of the game's memory pools (on for discrete cards); their budget |
|
| `RX_VK_MIRROR=0`, `RX_VK_MIRROR_MB=<MB>` | No VRAM mirrors of the game's memory pools (on for discrete cards); their budget |
|
||||||
| `RX_VK_VRAM_MB=<MB>` | The texture cache's budget (default 70% of what the driver allows, at most 6 GB) |
|
| `RX_VK_DEMOTE_QUICK=0` | A pool the CPU and the GPU both write is tried in guest memory (`vk_memory.cpp` `demote_trials`); by default a trial whose GPU time a frame is over 3 times its baseline and 5 ms more, after 3 frames or more, ends then, settling or measured, and is judged as a slower one is (mirrored again, measured again: `gpu/vk/demote_watch.h`). `0`: every trial runs its 10 settling and its measured frames, as before (in the owner's logs, some twenty frames of 100 ms for a pool at 106 ms against 7.4) |
|
||||||
|
| `RX_VK_IMPORT_AHEAD=1` or `0` | On by default with NVIDIA's driver (`=0` off). Off for test12 to test14: the GPU hung at the title in 6 of 12 lab runs; the cause, the game's freed pages decommitted under an import the background thread had not finished (`bg_held`), is closed and the lab's A/B with it was clean in 10 of 10 (`gpu/vk/import_ahead.h`). The ranges of the game's memory a run imported on a binding's own thread (8 to 17 ms each on an RTX 5090), whole as they were imported (their span over the game's pools included; the first version kept 16 MB chunks, which a binding across their ends imported again as one), are kept in `triforce_imports_ahead.txt` beside the game, at most 256, and the next runs import them on the background import thread as soon as the game's pools reach them, 4 at a look, at each change of the pool table and every 8 submissions (`gpu/vk/import_ahead.h`), each range once a session, and none while the PC's RAM is low (under 3 GB available, or a tenth of the RAM: the RAM's rule gives idle imports back then, and the two took turns on the same ranges in the owner's test9 session). A file of the first version's chunks is read as it is. Unset: off (it was on with NVIDIA's driver only, as `RX_VK_ASYNC_IMPORT` is); `1`: on any card; `0`: off. (Triforce) A race that fits the hang is closed since: the host decommitted the game's freed pages under an import the background thread had locked and not yet put in the pool table (`vk_memory.cpp` `bg_held`; `RX_VK_BG_IMPORT_HOLD=0`: not, as before, for the A/B); it stays off until a lab A/B with `=1` hangs no more. The log's `vk: imports ahead` lines count them and name the ranges sent, `vk: the game's pages ... are freed under a background import` the frees that window met; the stall line's "pools imported where bound" is what is left |
|
||||||
|
| `RX_VK_IMPORT_TOUCH=0` | Off: the pages of a range of the game's memory are not read before the driver imports it. On by default, on Windows, for the imports made on the background thread (`gpu/vk/import_touch.h`): the driver locks a range's pages inside `vkAllocateMemory`, faulting in the untouched ones, and a queue submit from another thread waits for it; the ranges imported ahead are up to 1 GB. Read first, outside the driver, the allocate of the 288 to 752 MB ranges went from 16 to 93 ms to 4 to 13 ms on an RTX 5090, and the longest queue submit beside an import in play from 71 to 93 ms to 10 to 12 ms. The log's `vk: import` lines give a slow import's parts: the first 12 of a session, and the first 48 of 64 MB or more. |
|
||||||
|
| `HOS_CRASH_THREADS=0` | Off: a crash report has the faulting thread alone. On by default, on Windows (`host/crash_threads.h`): a crash or fatal error's report ends with one `CRASH: beside it, thread ...` line for every other thread, its host stack as module and offset, taken at the report's start on a helper thread; the threads running code come first, then the ones waiting in the system. For the crash while a save loads, which reads an object whose memory holds something else: the lines say what the other threads were doing. |
|
||||||
|
| `HOS_CRASH_TEST_S=<seconds>` | (the lab's) A host thread reads address 8 that long after the start: a crash report on purpose, to look at one from a run in the game (its other threads' lines above). Not for play. |
|
||||||
|
| `EDEN_NX2_GM_LEAN=1` | Only the global-memory helpers a program's instructions call are put into its SPIR-V (`gpu/vk/eden/shader_recompiler/backend/spirv/gm_lean.h`). A stage that reads or writes through a pointer no storage buffer was tracked to carried every helper, six and for a storing stage eighteen, each with a search over every storage buffer in it. Off unless set: the SPIR-V is then what it was. An `EDEN_` name: each setting has its own program cache. The translator's files are in the translation stamp, so a build with this code translates every program again once, whatever the switch says. |
|
||||||
|
| `RX_VK_UPLOAD_AGAIN_MS=<ms>` | A texture uploaded before whose memory the game wrote again, met by a draw after the frame's uploads took this long (16), is uploaded at the next frame instead; its last bytes are drawn meanwhile, at most 8 frames in a row (`gpu/vk/upload_owed.h` `later`). New textures are never left so (theirs is `RX_VK_HOLD_MS`'s rule). `0`: every one at once, as before. The log's "left for the next frame" lines and the stall line count them. `RX_VK_UPLOAD_AGAIN_FRAMES=<n>`: a texture's upload is left so at most that many frames in a row (2: its last bytes drawn a frame or two at most; `8`: as before). With 16 ms nothing was ever left in the lab's runs (the frames with re-upload bursts took 9.8 ms of uploads); `3` spreads a burst over the frames after it, its textures a frame or two late meanwhile, which is why it is not the default |
|
||||||
|
| `RX_VK_UPLOAD_TORN=0` | (Triforce, wp18-flash) A texture uploaded before, 1 MB or more, whose storage the CPU has rewritten since the copy its re-upload would read (a capture's copy of one of its whole pages, every 16th and the last, differs from guest memory at the draw's look) keeps its last bytes for that frame and is uploaded at the next frame with what that frame's capture took (`gpu/vk/upload_torn.h`): a copy taken while the game was still writing the texture was test29's white frames at the Market after a tour (BC6H bytes passing through, decoded to bright colours the bloom spread over the picture). At most one frame in a row, and not for a texture found so in the two presents before (one the CPU rewrites every frame is uploaded as before). Not a dispatch's upload. `0`: uploaded at once, as before. The log's `re-uploads of storage the CPU rewrote since its capture` lines count them, those found and those left (with `0` too: the measure of how often a copy may have been taken mid-write) |
|
||||||
|
| `RX_VK_REUPLOAD_EARLY=0` | The textures the CPU keeps rewriting (uploaded again from its writes twice or more within 120 presents, 1 MB or more, sampled in the frame or the one before, not drawn into by the GPU) are looked at once on the replay thread before a parallel replay's segments begin, after the dry pass, their uploads in the recording before the segments: not made by a segment under the recording lock, where the other recording threads queued behind them (lab S2 after a tour: 4 to 12 such uploads of 28 to 84 MB in ~13% of the frames, the descriptor time 24 ms over the threads against 3). `0`: the segments' looks alone, as before. The log's `textures the CPU keeps rewriting, looked at before` line says how many and how long (`gpu/vk/reupload_early.h`) |
|
||||||
|
| `RX_VK_UPLOAD_SAME=0` | (Triforce) A texture uploaded before whose storage the CPU wrote again is checked against the bytes its image holds (`gpu/vk/upload_same.h`): the units of its storage the writes reached (each level's and layer's GOB-block rows, as its upload reads them) whose bytes hash as at their last upload are not uploaded, and when that is all of them nothing is. The upload of the rest reads the copy the check hashed, so that what is kept is what the image got; any other upload and any GPU write forget it. Textures of 16 MB or less uploaded on the CPU (block-compressed, or under 8 MB), not sparse; one whose checks find nothing the same 4 times in a row is uploaded unchecked for its next 64 re-uploads. Lab S2 after some tours: the game rewrote a few big BC textures the Market samples with the bytes they had, 40 to 60 MB every 4 to 5 frames (the Market at 62-78 fps against 83-96, ~26 GB of uploads in its stay). `0`: every re-upload made as before. The log's `re-uploads checked against the bytes their images hold` line says what it looked at, found the same and uploaded. `RX_VK_UPLOAD_SAME_KEEP=0`: the check hashes every byte it looks at; by default a page the submit-time capture keeps a copy of is hashed once and its hash kept with the copy until a capture copies other bytes into it (about half of those textures' pages passed through other bytes between two captures) |
|
||||||
|
| `RX_VK_UPLOAD_SAME_ASTC=0` | (Triforce) ASTC textures decoded on the GPU (kept as RGBA8, BC3 or BC1; 2D only) are checked too, by `RX_VK_UPLOAD_SAME`'s check: one unit a layer, its levels from the first one the image keeps, which is what the GPU decoder stages of a layer a write reached; the staging reads the copy the check hashed. Lab S2's Market after a tour: the game rewrote such textures with the bytes they had (1536 x 1536 ASTC 5x5 of 11 levels, 768 x 768 kept as BC3, 512 x 512), staged and decoded again unchecked, about 1.4 GB in 10 s (90 fps down to 72, the 1% low 57 to 33). `0`: ASTC re-uploads made as before, every layer a write reached staged and decoded. The `re-uploads checked` line says what of it was ASTC; the `re-uploads not checked` line every 600 frames counts the re-uploads made without the check, by route and by why |
|
||||||
|
| `RX_VK_TWIN_SAME=0` | (Triforce) The parallel replay's twin (a texture an earlier segment samples that a later segment of the same submission uploaded, uploaded again in the earlier segment's uploads so that its draws see the bytes as submitted) uploads only what that submission's uploads into the image asked for (`gpu/vk/upload_same.h` `note_upload`; the whole storage when one of them was whole or the image was made again), and where every unit it reads has a hash kept by `RX_VK_UPLOAD_SAME`'s check, reads a copy of them hashed as copied: when the copy holds the bytes the hashes are of, the hashes are kept. Before, the twin uploaded the whole storage and, unchecked, forgot the hashes the later segment's checked upload had just kept, so the next rewrite of the same bytes was uploaded again, the twin with it, every few frames: lab S2's Market after a tour, about 97% of what was still uploaded unchecked in those loops, up to 6.9 GB in one stay. `0`: the twin uploads the whole storage and forgets the hashes, as before. The log's `the parallel replay's twins` line every 600 frames counts them, the bytes uploaded against the whole, and how many kept the hashes |
|
||||||
|
| `RX_VK_UPLOAD_EARLIEST=0` | (Triforce) A parallel replay's upload of a texture, made by the worker of a later segment of the submission while an earlier one is still being recorded, is recorded into the init command buffer of the earliest segment still being recorded (submitted before that segment's uploads and commands; `gpu/vk/upload_earliest.h`, `vk_texture.cpp` `UploadRedirect`). Every segment before it has ended its recording and no longer samples the texture, and every one from it on finds the upload at or before its place in the queue, so no earlier segment uploads the texture again (the twin, `RX_VK_TWIN_SAME`): it is uploaded once, the same bytes. Lab S2's heavy Market loops (round 48) uploaded 2.1 GB of twins in their last minutes, the big streamed or new textures each uploaded once by a later segment and again by an earlier one; those twins took the frames' upload budget (`RX_VK_UPLOAD_NEW_MB`, `RX_VK_UPLOAD_NEW_MS`), and the new textures after them were left for the next frame, the picture held. Only an upload staged from the CPU as it is recorded is moved: not one read on the GPU (the GPU unswizzle, from the pool) or decided from live memory (the all-zero clear), which in the earlier segment's init would run before a segment between writes that memory on the GPU. Not for a texture the GPU drew (its contents are the GPU's; the CPU's writes over it, `RX_VK_CPU_OVER_GPU`, keep their own path) or a sparse one: those keep the segment's own uploads, and the twin. The uploads' GPU time in an init command buffer is not in the segments' GPU time. `0`: every upload in its segment's own uploads, the twin as before. The log's `uploads recorded into an earlier segment's init command buffer` line every 600 frames (with `0` too) counts them and the bytes, and how often and how long a segment's end waited for its init lock, each segment's own, held while such an upload (or a new image's first layout change) is recorded into its init (its submit waits as long) |
|
||||||
|
| `RX_VK_SHADOW_KEEP=0` | (Triforce) A texture page the submit-time capture takes with the bytes its shadow copy holds (the copy the uploads read, `RX_VK_TEX_SAME`'s compare) keeps that copy as it is, not copied again, its kept hash with it (`gpu/vk/shadow_prev.h`). Lab S2's Market after some tours: about half of the ~400 texture pages a capture took, six captures a frame, held their copy's bytes. Only for a page the capture takes once; any other is copied as before, and the shadow holds what the capture holds either way. `0`: every page copied, as before. The log's `texture shadow (600 takes)` line counts the pages compared, left, given back and copied |
|
||||||
|
| `RX_VK_SHADOW_PREV=0` | (Triforce) A texture page the capture takes with other bytes keeps the copy it had, and that copy's hash, beside the new one (`gpu/vk/shadow_prev.h` `Prev`), and a capture that brings those bytes back gives it back, hash and all: no copy, and the re-upload check after it finds the hashes kept (lab S2's Market after some tours: the other half of those pages went back and forth between two versions, and a check hashed each again after every change, 0.2 to 0.3 ms a frame). A kept copy that does not come back within 8 changes of its page, or within 64 captures, goes; at most 64 MB are kept. `0`: none kept, a changed page copied over as before. The `texture shadow` line says how many are kept and how many went |
|
||||||
|
| `RX_VK_SHADOW_VERS` | (Triforce) The copies a texture page keeps beside its own in the shadow (`gpu/vk/shadow_prev.h` `Vers`): `RX_VK_SHADOW_PREV` keeps one, and a page whose captures bring more than two versions (A, B, C, A every few frames) is copied again for every version but the one before the last. `2` to `4` keep that many, newest first, and a capture that brings any kept version's bytes takes that one back, the copy it had becoming the newest, so each version a page cycles through while it is still among the kept copies is copied once: 4 KB a version, under the same 64 MB cap. Unset or `1`: one copy, as before. The `texture shadow` line counts the pages given a copy of their own, one before it, and two or more before |
|
||||||
|
| `RX_VK_SHADOW_CHECK=1` | (Triforce) Each texture page `RX_VK_SHADOW_KEEP` or `RX_VK_SHADOW_PREV` left uncopied is compared with the capture's copy again after the capture is taken, and the hashes kept with the shadow's copies are hashed again: the `texture shadow check` line every 600 captures counts the copies that differ and the hashes that are not of their bytes, both of which must be 0. Slow: for the lab's checks. Unset or `0`: not checked |
|
||||||
|
| `RX_VK_HOLD_MS=<ms>` | Hold the last picture (`gpu/vk/present_hold.h`, on by default): a new texture a draw samples after the frame's uploads took their budget (`RX_VK_UPLOAD_NEW_MS`, `RX_VK_UPLOAD_NEW_MB`) is uploaded at the next frame, the dummy drawn in its place, and that frame is not shown: the window gets the last complete picture again, the Insert menu, the counters and the notices drawn over it, until a frame is drawn with every texture complete. The game runs on underneath. No frame generation makes a frame from a held one, and the one shown after a hold starts its history anew (and the temporal upscalers theirs). `<ms>`: the hold's bound (250); at it the rest are uploaded at once and the next frame is shown whatever it holds, and no hold comes for 120 presents (twice as long after each hold at its bound that follows, up to 1920). `0`: no hold, no new texture left for later: the one frame that takes them all, as before. The log's `vk: hold:` lines, the stall line and the 600-frame line count them |
|
||||||
|
| `RX_VK_HOLD_PIPELINES=0` | A frame with draws or dispatches left out because their program, pipeline or compute pipeline was still being made is held as one with a texture still owed its bytes (`RX_VK_HOLD_MS`'s hold, with its bound and its rests; NX2navi's frame hold, `gpu/vk/pipe_wait.h`): by default the window keeps the last complete picture instead of showing one that lacks a pass. `0`: such frames are shown, as before |
|
||||||
|
| `RX_VK_UPLOAD_NEW_MS=<ms>`, `RX_VK_UPLOAD_NEW_MB=<MB>` | The new textures' budget a frame while the hold is on: once the frame's uploads took 12 ms or 64 MB, a new texture is left for the next frame (ASTC ones too; the frame's first upload always goes). 12 ms keeps an area change's frames near 16.7 ms with their draws and memory sync; 64 MB is what the GPU copies or decodes in a few milliseconds, so that no frame leaves it a backlog. `0`: that half of the budget off |
|
||||||
|
| `RX_VK_UPLOAD_SPLIT_MB=<MB>` | A new layered ASTC texture whose texels are more than this (128) has its first upload in parts while the hold is on, the texture's layers over at most 6 frames (`gpu/vk/layer_split.h`): its draws sample the dummy and its frames are held until the last part is in; for a dispatch or at the hold's bound what is left goes at once. In the owner's test7 session one 1365 x 1365 x 90 texture kept as BC3 took one frame's upload command buffer 114.8 ms of GPU time. `0`: whole, as before. The stall line counts the parts |
|
||||||
|
| `RX_VK_DUMMY_GEN=0` | A frame that left textures for later makes every texture view a draw resolved stale (`destroy_gen`), as before. By default only the looks that resolved to the dummy for want of their texture are (`texture_dummy_generation`, `vk_draw.cpp` `texture_view_of`); the stall line counts the views resolved anew |
|
||||||
|
| `RX_VK_VIEW_KEEP=0` | Any texture's end (an eviction, a dead render target let go, a texture made BC3 from BC1, the compaction moving one) makes every texture look of every recording thread stale, and each is resolved anew when next met (the slot, the cache, the view, under the parallel replay's lock), as before. By default a thread that finds the destroy generation moved keeps the looks of the textures whose views did not go: those are noted by their creation number before the generation moves (`gpu/vk/view_keep.h`, `vk_draw.cpp` `texture_view_of`); never a look of no texture or no view, nor a flat texture's 3D copy. More than 1024 gone since a thread's last pass: all resolved anew. The log's `vk: texture looks kept` lines count them, the stall line the views resolved anew. After NX2navi's `tex_gen_undo` |
|
||||||
|
| `RX_VK_DESC_KEEP=0` or `check` | The per-binding reuse of a draw's descriptors (`gpu/vk/desc_keep.h`, on by default): a draw that builds its descriptors takes, for each sampled texture, user uniform buffer, stage's constants and read-only storage buffer, what the program's last draw on that thread resolved it to, while everything it was resolved from is the same: the texture's handle, its slot's registration, the samplers', no texture destroyed or made again, no texture left for later and the same submission; the buffer's address and size, no inline uniform update, no pool moved or mirror made, the same recording. `0`: every descriptor resolved, as before (so also with `RX_VK_NO_DESC_REUSE`, `RX_VK_SAMPLER_CACHE=0` and the descriptor diagnostics). `check`: every descriptor resolved too, and one that comes out otherwise than what was kept for the same inputs is counted and logged (`vk: descriptor reuse check`): the way to try the rules. The frame statistics' `a frame's descriptors` line says how many were resolved and taken, how many pushes were whole, of the changed bindings or of nothing, and how many sets were written, through a template, or bound again |
|
||||||
|
| `RX_VK_DESC_SIG=0` or `check` | The whole-draw skip by the signature of a draw's descriptor inputs (`gpu/vk/desc_sig.h`, on by default with the per-binding reuse): a draw whose bindings moved, of the program whose descriptors its last draw pushed in that command buffer, first reads what each of them is resolved from (a texture's handle and its slot's version, a user uniform buffer's address and size, a read-only storage buffer's address and size words, a driver bank's stage generation) and the stamps the per-binding reuse gives each kind; when that equals the signature the pushed descriptors were built from, and their build kept every one, the build and the push are left out (a build would take each of them again and push nothing). Push programs only, of the kinds the per-binding reuse keeps: not those with storage buffers they write, tables, texel buffers or storage images, not the mesh fallback or RenoDX's. `0`: built, as before. `check`: built too, and one that comes out otherwise than what is pushed is counted and logged (`vk: descriptor signature check`). The frame statistics' `a frame's descriptor signatures` line says how many draws read theirs and how many were kept whole |
|
||||||
|
| `HOS_OOT_TEX_COMPACT_MS=<n>` | (the lab's) The game's texture data compaction (its `aglTextureDataMgr::Compaction` thread moves a texture's data in the game's heap, a cycle of about 18 frames; idle in a fresh Market, at work at every ask after a tour, where each move is a texture at a new address for the renderer to capture and check): at most one cycle started every n ms. `-1` (a test): none started. Unset or 0: as shipped. The cycles started are logged every 10 s either way ("oot: texture compaction in"). `host/oot_tex_compact.c`, `tests/test_oot_tex_compact.py` |
|
||||||
|
| `RX_VK_TEX_WARM=1` | Textures made and uploaded ahead, while the frames are light (`gpu/vk/tex_warm.h`; off unless set). A texture is made and uploaded where a draw first samples it, and the frame's budget for that (12 ms, 64 MB, the picture held) applies only to targets drawn in the frame before, so an area's first frame, whose targets are new, takes every texture of the area at once: 350 to 800 uploads of 250 to 390 MB, 100 to 180 ms, in the lab's tours. With the switch every registered slot is noted, and at the start of a game's submit that follows a light frame (under 400 draws and dispatches: a loading screen, a menu) the noted textures, 8 frames old at least, are made and uploaded, 6 ms a frame. Bytes the game writes afterwards are uploaded again by the write marks, as before. "vk: textures made ahead in light frames" counts them. `tests/test_tex_warm.py` |
|
||||||
|
| `HOS_MEM_LIMIT=1` or `<MB>` | (the lab's) The game's memory held to what it is told it has, as the console holds it. The game is told 9809 MB, but nothing was refused until the PC's own commit ran out, and it holds 11.7 GB by its own count in a fresh Market and 13.5 GB after a tour (the `mem: census` line), with its GPU heap within 15 MB of full and its texture compaction running every frame. `1`: `AllocateMemoryPages` and `MapPhysicalMemory` are refused past the 9809 MB, and `UsedMemorySize` is the game's count; a number above 1: refused past that many MB. Unset or 0: as before. What the game does when refused is what the lab is to find out. `host/hos_mem.c`, `tests/test_mem_limit.py` Refused, the game asks again thousands of times, each ask a region taken and freed; a region of the game's own freed with none of its pages in it is no longer discarded (`hos_region_discard_needed`), and the regions it frees with pages in them are discarded with the thread's virtual core given up. Those two were the stutter when walking with the limit on: in a three-area lab run the game's own part of the frame was over 8 ms in 24% of the frames before and in 0.3% after (as without the limit), 63.8 fps to 100.0. **`=1` does not fit this build:** the heap takes 6.9 GB of the 9.8 and an area's resources need 4.7 GB and more, so a save's load can starve and stay on its loading screen for good (the owner's, 2026-11-20, read with the hang watch's stacks: the game at 9,807 MB, its load threads idle). Use a size in MB: `12800` loaded and played four areas without reaching it. |
|
||||||
|
| `RX_VK_DESC_SET_KEEP=0`, `RX_VK_DESC_TEMPLATE=0` | A program over the driver's push limit (32 descriptors on NVIDIA's, AMD's and Intel's drivers) has a set written for each draw that builds its descriptors. By default a set whose descriptors are all as those of the last one written for that layout in the recording is bound again instead, while no texture was destroyed or made again and no pool moved, and a set written is written through the program's descriptor update template (Vulkan 1.1). (Triforce, after NX2navi's `desc_template`) A program within the push limit has its descriptors pushed whole (in a new command buffer, or after another program at its bind point) through a push descriptor template of its own (`vkCmdPushDescriptorSetWithTemplateKHR`, one a program and bind point, from the same entries and packed array as a set's), not with the write structs the driver walks one by one; a push of the changed bindings only keeps its write structs. The frame statistics' `a frame's descriptors` line counts the pushes made through it. With `RX_VK_RECORDER` the packet copies the packed array (`gpu/vk/vk_recorder.h`). `RX_VK_DESC_SET_KEEP=0`: a new set each time; `RX_VK_DESC_TEMPLATE=0`: sets written and pushes made with the write structs; both as before |
|
||||||
|
| `RX_VK_CREATE_NEW_MS=<ms>` | While the hold is on, a sampled texture seen for the first time once the frame's makes took this long (8), or its uploads their budget, is not made yet (`vk_texture.cpp` `create_later`): the draw samples the dummy, the frame is held, the next frame makes it. The frame's first make always goes; never a dispatch's. `0`: made at once, as before (a texture left for later was made all the same). The stall line counts the textures made, their time, those not made yet, and the images made |
|
||||||
|
| `RX_VK_UNSWIZZLE_MT_MB=<MB>` | A block-linear level unswizzled on the CPU is split over several threads (a thread for each 2 MB, 8 at most) from this size (4); `16`: from 16 MB, as before |
|
||||||
|
| `RX_VK_PROGRAM_READAHEAD=<n>` | After a draw reads a program from the program cache itself, or takes one read ahead, workers read the next `n` programs (32) and keep them made for the draws that come to them, 512 at most (`gpu/vk/program_readahead.h`). `0`: none, as before. The log's `vk: programs from the cache` lines and the stall line count them |
|
||||||
|
| `RX_VK_PROGRAM_READ_UNLOCKED=0` | A draw that reads a program from the program cache itself, or waits for one the read-ahead's worker is reading, lets the shaders' lock go meanwhile, so that the replay's helper threads go on picking their own programs (they queued behind it: "the program picks waited 169 ms for the shaders' lock" at the Market's arrival); a second draw of that program waits for the first's result, and the record stays listed until it is read (`gpu/vk/program_read_once.h`). A program's release waits until no pick has the lock let go. `0`: read and waited for under the lock, as before. The stall line counts the waits for another draw's read, and the picks' waits to take the lock again with their waits for it |
|
||||||
|
| `RX_VK_STAGE_CODE_CACHE=<MB>` | What a program pick made of a stage's bytes (the control section and the code area read from guest memory: its code key, the instructions lowered to Maxwell) is kept by those bytes, 64 MB at most, the least recently used going first: a program object met for the first time whose stages' bytes were met before (the game makes them again at loads) takes a copy instead of decoding them again under the shaders' lock (`gpu/vk/stage_code_cache.h`; an entry is taken only for the same bytes, byte for byte). The save's load-in frame said "the program picks read the stages' code 45.6 ms": a stage was some 60 us to read and decode, and is 3 us to find. `0`: none, each stage decoded again for each program object, as before. The log's `vk: the stages' code read for the program picks` lines count them |
|
||||||
|
| `RX_VK_STAGE_CODE_LAZY=0` | A program pick reads the stages of a program object it meets for the first time without lowering their code: their keys, NVN stages and headers (what finds the program) come from the control sections and code areas, and the code is lowered (and kept, `RX_VK_STAGE_CODE_CACHE`) only when the pick translates a variant. A program found in memory, read ahead or in the program cache was made from stages that lower (its key names them); one not found has its stages lowered before it is translated, and should one not lower, the pick starts again from those that do: the program it was before (`gpu/vk/stage_code_cache.h`). The save's load-in frame lowered every stage of each program object met, 48 of a stage's 60 us, for programs all prepared before the title. Not with `RX_VK_REJECT_STAGES`, `RX_VK_REJECT_KEYS`, `RX_VK_OLD_LOWERING`, `RX_VK_DUMP_NATIVE` or `RX_VK_DUMP_FAILED`, which act in the lowering. `0`: each stage lowered as it is read, as before. The log's `vk: the stages' code read for the program picks` lines count the stages read without lowering and those lowered since |
|
||||||
|
| `RX_VK_STAGE_CODE_UNLOCKED=0` | A program pick reads the stages of a program object it meets for the first time (guest memory, their keys, the stage code cache) with the shaders' lock let go, as a draw's own read of a program (`gpu/vk/program_read_once.h`: a program's release waits until no pick has the lock let go), so that the replay's helper threads go on picking their own programs meanwhile; two picks of one object at once each read it, and the first list put stands. `0`: read under the lock, as before. The stall line's "the program picks read the stages' code" counts the read and the lock's taking again |
|
||||||
|
| `RX_VK_PROGRAM_ORDER=0` | Each session writes the order its draws first needed their programs in (read from the program cache, or translated), the first 4096, into `triforce_program_order.bin` beside the cache now and then (this session's first, then those of the sessions before it has not met). The next session's read-ahead workers, from the first draw on (the title and the file select), read those programs in that order whenever no draw asks them for anything, the whole order, and keep them made apart from the shelf until a draw takes them (`RX_VK_PROGRAM_ORDER_KEEP` below; `gpu/vk/program_order.h`): the save's load-in frame read some 100 programs on the draws' threads before. Not with the RAM low. `0`: neither read nor written, as before. The stall line says why the draws read what they still read themselves (named by no order, read ahead and let go before, not read ahead yet) and how many they took in this order; the log says once when the whole order has been gone through, and how many programs it keeps |
|
||||||
|
| `RX_VK_PROGRAM_ORDER_KEEP=0` | A program read ahead in the last sessions' order stays until a draw takes it, whatever the draws meet meanwhile, and nothing in the order is passed by: the workers read all of it (4096 at most) while the title and the file select stand still, and a program a draw's ask read that the order names is kept with these, not on the shelf, which lets its oldest go. Only a RAM gone short lets them go (all of them). The pipelines' warm keeps its window: the first 512 kept and not taken are read with it, those beyond without, each warmed as the draws take warmed ones (the nearest first). Some 40 to 50 KB a program packed: the owner's order of 2411 programs some 100 to 120 MB, 4096 at most some 200 MB. The owner's test37 session (SDR, where the order's sessions were HDR Native) took another path through the order and its load-in frame read 115 programs itself, 64 of them read ahead and let go as behind where the draws were, 51 passed by. `0`: the rules before: 512 at most, each until the draws are 256 places past it (where the draws are: the middle place of the last 16 programs they needed first), what lies behind not read |
|
||||||
|
| `RX_VK_PROGRAM_READAHEAD_NOTES=0` | The read-ahead's next programs are those after it in the pipeline notes' order (the order earlier sessions first built their pipelines: an area's programs lie together), 4 workers, the compute programs too (in the order play first dispatched them, `triforce_computes.bin`), and a full shelf lets its oldest go; the program cache file's order only for a program neither names. `0`: the file's order (since the shaders are prepared before the title, the preparation's), one worker, no compute programs, and nothing more read once 512 wait, as before (test16's file select load took 7 of some 600 programs read ahead that way) |
|
||||||
|
| `RX_VK_PIPELINE_WARM=0` or `1` | Where the precompile keeps no pipelines (the RAM spared) and the fast pipeline link is on, the read-ahead's worker that has just read a program makes, before any draw can have it, the stage libraries of the pipelines the pipeline notes name for it (asked of the caches only: one the driver would have to compile is left to its draw) and links them; the draw takes the link made in place of linking its own, and the whole build follows as before (`gpu/vk/pipe_warm.h`). At most 32 notes a program variant, 512 links held for draws that have not come (the oldest let go beyond) and 1024 libraries no draw has had; nothing made while the RAM is low or short, and what is held goes back to the driver while it is short. `0`: none, as before; `1`: the libraries only. The log's `vk: pipelines made behind the draws` lines count them |
|
||||||
|
| `RX_VK_PIPELINE_WARM_FOLLOW=1` | Off by default (test19: the lab's tour showed no gain). `1`: the warm follows the draws: a program read within 64 places (in the notes' order) after one the draws consumed in the last 2 s has its libraries made and linked, one within 192 its libraries only, one farther (or once the draws stopped taking) nothing; and the libraries no draw has had are kept up to 2048, the oldest let go beyond (with the links made from it), so the bound never stops the warm (`gpu/vk/pipe_warm.h`, test18: 2266 links made, 276 taken, the libraries at their bound from the load on); it spends at most 250 ms of the workers' time a second (500 saved up), and never makes again a library it let go. Off: every program read is warmed whole and the libraries stop at 1024, as before |
|
||||||
|
| `RX_VK_COMPUTE_WARM=1` | Off by default (test19: the lab's tour showed no gain). `1`: where the precompile keeps no pipelines, the read-ahead's worker that has just read a compute program play dispatched (`triforce_computes.bin`) makes its compute pipeline, asked of the caches only, as its first dispatch would; the dispatch that takes the program has it. One a program while it waits on the read-ahead's shelf, destroyed when the shelf lets the program go and given back while the RAM is short; once a program variant a session. Off: each compute pipeline is made at its first dispatch, as before |
|
||||||
|
| `RX_VK_PIPELINE_ORDER=0` | Each session writes the order its draws first needed their pipelines in (a new one, or one the precompile kept that no draw had), the first 4096, into `triforce_pipeline_order.bin` beside the pipeline notes now and then, from the presents on a thread of its own (this session's first, then those of the sessions before it has not met). Where the pipeline warm runs (`RX_VK_PIPELINE_WARM`), the next session finds each one's note before play and the warm makes, for a program variant the read-ahead has read, the pipelines that order names for it and no other (32 at most a variant; the notes' other pipelines are not made then, and the warm does not follow the draws): their stage libraries asked of the caches and their links, held for the draws by their place in the order, 1024 at most (beyond, the farthest ahead goes, and none is made farther), and 2048 libraries no draw has had at most. Where the draws are: the middle place of the last 16 pipelines they needed first; a link held 512 places behind it goes with the libraries only it held, and none is made there (`gpu/vk/pipe_order.h`): the save's load-in frame made 86 to 92 ms of pipelines on the draws' threads, the warm's links from the notes let go before the load came (lab round 37: 1964 made, 54 taken). The first session after this writes the file; the next one has it. `0`: neither read nor written, the warm from the notes as before. The stall line counts the pipelines new to the draws, those linked ahead (of them in this order), and the links and stage libraries the draws made themselves; the log's `vk: pipelines made ahead in the order` lines the rest |
|
||||||
|
| `RX_VK_VRAM_MB=<MB>` | The texture cache's budget as a fixed size, whatever the card. Unset: on a card the driver allows more than 8.5 GB, no size (`RX_VK_TEX_PRESSURE`); on a smaller one 45% of what the driver allows; with `RX_VK_TEX_PRESSURE=0` 70% of it above 8.5 GB, at most 6 GB (the rule before) |
|
||||||
|
| `RX_VK_TEX_PRESSURE=0` or `=<pct>` | (Triforce, after NX2navi's `TEX_VRAM_PRESSURE`) On a card the driver allows more than 8.5 GB, with VK_EXT_memory_budget, not an integrated GPU, without `RX_VK_VRAM_MB` and without a quality preset whose budget applies, the texture budget follows the live use instead of the fixed 6 GB: textures go once the driver's count, less the free space in the blocks they can take back, is past 80% of what the driver allows now, the least recently used first, each unused 2 s or more, 64 a pass, down to 2% under that line (`gpu/vk/tex_pressure.h`); the 90% and 97% rules and the compaction's lines stay. While the system's commit is low the fixed size comes back (`RX_VK_TEX_COMMIT_FREE_MB`). `0`: the fixed size, as before; `<pct>`: that line (50 to 95). The log's `vk: texture budget` line at the start says which, and the `vk: VRAM:` line every 600 presents the budget (by live use: what the textures may hold before the line) and the line |
|
||||||
|
| `RX_VK_TEX_KEEP_TAKEN=0` | (Triforce, after NX2navi's `TEX_KEEP_TAKEN`) While VRAM is short, a texture uploaded from memory that the game let go 30 presents ago goes first (`RX_VK_LET_GO_PCT`); one the game has taken back before waits 8 times the longest it stayed let go before a take-back, when that is longer (`gpu/vk/tex_pressure.h`): made again, the frame that wants it draws the dummy (NX2navi: Kakariko's haze gone every few seconds). The eviction by age still takes it. `0`: the 30 presents alone, as before |
|
||||||
|
| `RX_VK_TEX_COME_BACK=0` | (Triforce, after NX2navi's pause for drops that come back) Where the budget follows the live use: when in 240 presents 64 or more textures the line let go went and half or more of them were made again within 600, the line stands aside for 600 presents, doubled at each such period up to 7200; a period with fewer back ends the doubling (`gpu/vk/tex_pressure.h` `ComeBack`). The 90% and 97% rules and the compaction's lines still hold meanwhile. The log says when a pause begins. `0`: no pause |
|
||||||
|
| `RX_VK_TEX_STEP=<n>` | The live textures one pass of `texture_sweep` over the budget lets go of at most (64 where the budget follows the live use; the render targets drawn over since do not count), the rest at the next look two presents on, not ten. Set, on any card, the fixed size's too; `0`: no step, one cut as before |
|
||||||
|
| `RX_VK_TEX_COMMIT_FREE_MB=<MB>` | Where the budget follows the live use: while less of the system's commit than that is free (the default: the larger of 4 GB and a tenth of the limit, NX2navi's), the fixed size is the budget again (Windows charges a discrete card's memory to the commit, and an allocation that fails there ends the game). `0`: never. Not on Linux |
|
||||||
|
| `RX_VK_SWEEP_ALL=1` | Every texture of the cache ordered by its last use for every pass of `vk_texture.cpp` `texture_sweep`, as before. By default a pass under the budget (every 10 presents, and the dead render targets' passes between them) orders only the textures it can take: those the GPU drew and those the game let go; the others it passes over all the same. Over the budget every texture is ordered. The log's "passes for the dead render targets between the looks" line says what a pass costs |
|
||||||
|
| `RX_VK_TEX_RATE_HIGH=0` | The texture eviction's waits (`vk_texture.cpp` `texture_sweep`: 60 presents unused over the budget, 30 after the game let a texture go, 600 or 3600 for one the GPU drew, the compaction's 600 and 1800) count presents above 60 a second, as before. By default they count their seconds at any rate up to 480 a second (`gpu/vk/present_rate.h`, after NX2navi's M19): at the Market's 94 a second "60 presents" were 0.64 s. Below 60 they count seconds unless `RX_VK_TEX_RATE_LOW=0`. The log says the first time a run goes above 68 a second |
|
||||||
|
| `RX_VK_TEX_DETAIL=full`, `reduced` or `low` | The texture detail for one run, over the Insert menu's Texture detail row: whole textures, or the large ones without their largest level, or without two (`gpu/vk/tex_detail.h`). Unset, the row decides, and its Automatic goes by what the driver allows: full from 10,240 MB, low under 6,144 MB, never full on an integrated GPU |
|
||||||
|
| `RX_VK_TEX_BC1=0/1` | Opaque ASTC textures kept as BC1 instead of BC3 for one run, or not, over the Debug tab's Opaque textures as BC1 row (on by default since options_version 5): a texture the renderer keeps as BC3 whose every block the GPU decoder writes opaque is encoded as BC1, half the video memory, the same texels (`gpu/vk/tex_bc1.h`, `gpu/vk/astc_opaque.h`). One whose memory later holds blocks with alpha, or that a copy on the GPU writes, becomes BC3 for good; the log counts both |
|
||||||
| `RX_VK_COMPACT=1`, `RX_VK_COMPACT_PCT=<50..90>`, `RX_VK_COMPACT_MB=<MB>` | VRAM compaction (off by default): the textures that are not render targets in a pool of their own, moved out of sparsely used blocks a few a present so that the blocks go back to the driver, and the texture budget holding the driver's count at that share of what it allows (85); the free space that starts a run (512; 192 while the count is over 90%) |
|
| `RX_VK_COMPACT=1`, `RX_VK_COMPACT_PCT=<50..90>`, `RX_VK_COMPACT_MB=<MB>` | VRAM compaction (off by default): the textures that are not render targets in a pool of their own, moved out of sparsely used blocks a few a present so that the blocks go back to the driver, and the texture budget holding the driver's count at that share of what it allows (85); the free space that starts a run (512; 192 while the count is over 90%) |
|
||||||
| `RX_VK_SKIP_COMPRESSED_OVER_RT=1` | A compressed (ASTC, BC, ETC2/EAC) texture made over storage render targets used is not uploaded, as before: bosses and new enemies all blue the first time |
|
| `RX_VK_SKIP_COMPRESSED_OVER_RT=1` | A compressed (ASTC, BC, ETC2/EAC) texture made over storage render targets used is not uploaded, as before: bosses and new enemies all blue the first time |
|
||||||
| `RX_VK_NO_WRITEBACK` (set) | GPU writes to mirrored pools are not copied back to guest memory (an experiment, for A/B) |
|
| `RX_VK_NO_WRITEBACK` (set) | GPU writes to mirrored pools are not copied back to guest memory (an experiment, for A/B) |
|
||||||
| `RX_VK_WB_DIFF=0` | Write-backs copy every byte again instead of only what changed against a VRAM shadow (the diff is on by default; v2.0.9 and v2.0.10 had it off, issue 1161) |
|
| `RX_VK_WB_DIFF=0` | Write-backs copy every byte again instead of only what changed against a VRAM shadow (the diff is on by default; v2.0.9 and v2.0.10 had it off, issue 1161) |
|
||||||
|
| `RX_VK_LAZY_WB=1`, `=2` or `=0` | The lazy write-back for one run, over the Debug tab's Lazy write-back row (`lazy_write-back=1`; on by default since options_version 5; `docs/lazy-writeback.md`, `gpu/vk/wb_lazy_state.h`): a whole page of what shaders write into a mirrored pool that the CPU leaves alone is no longer copied back at every submission; its page has no access, and whoever touches it (the game's code or the host's, by a fault; a host call that hands it to the kernel; the GPU through the import) has it copied down first. Windows only. On AMD's driver the row alone leaves it off, as NX2navi did after games ended in play there; `=1` turns it on anyway. `=2` is the self-check: every copy is still made, the pages are protected and paid as if not, and each payment compares what it would have copied with guest memory: a `MISMATCH at guest` line is a page it would have got wrong. The log says `vk: lazy write-back on` (or why off) at the start, and every 600 submits `vk: lazy write-back (600 submits)`: the MB not copied, the pages owed, the pages paid by a fault, a host call, a GPU read, a mirror given up and a recording's submit, the payments and their time, and what failed |
|
||||||
|
| `RX_VK_LAZY_PAYERS=<1..4>` | (Triforce, 4 by default; `=1` as before) How many of the lazy write-back's payments copy at once (`gpu/vk/wb_lazy_payers.h`, NX2navi's `wb_lazy_unlocked`). A payment's copy goes onto the queue after everything submitted so far and its thread waits for it (0.5 to 3 ms each in the owner's sessions, up to 21 ms at a load); with one at a time, a second thread that touched another owed page, or the replay giving a mirror up, waited for the first one's copy to come back before its own went onto the queue. Each payment holds a slot of its own (a command buffer and a fence); the pages of two payments are never the same ones (`gpu/vk/wb_lazy_state.h`), and each copy still goes onto the one queue after everything before it. The 600-submit line `vk: lazy write-back (600 submits)` ends with the payments that waited for a free slot and the most slots in use at once |
|
||||||
|
| `RX_VK_GEN_SEED=<n>` | The page write generations (`gpu/vk/write_gen.h`) start at `n` and not at 1 (decimal, or hex with `0x`; 2^62 at most). They are 64 bits; the 32-bit ones of earlier builds ran out after days of play, and textures and mirrors then kept old contents. `RX_VK_GEN_SEED=0xFFFF0000` takes a play test across 2^32 within seconds: the log says `the write generations start at` and, at the crossing, `this run has made 4294967296 write generations`, and nothing in the picture should change there |
|
||||||
| `RX_VK_DLSS=0/1/2`, `RX_VK_DLSS_PRESET=<letter>`, `RX_VK_DLSS_LOG=1`, `RX_VK_REFLEX`, `RX_VK_REFLEX_LOG=1`, `RX_VK_ANTILAG=0/1` | NVIDIA DLSS and Reflex, and AMD Anti-Lag 2, for one run (`=-` on Reflex or Anti-Lag 2: its extensions not even enabled): the readme's NVIDIA DLSS section. `RX_VK_DLSS_TRACE=<frames>` logs one frame's large draws every that many frames, `RX_VK_DLSS_DEBUG=motion` shows the motion vectors, `=alternate` the game's own upscale every other frame |
|
| `RX_VK_DLSS=0/1/2`, `RX_VK_DLSS_PRESET=<letter>`, `RX_VK_DLSS_LOG=1`, `RX_VK_REFLEX`, `RX_VK_REFLEX_LOG=1`, `RX_VK_ANTILAG=0/1` | NVIDIA DLSS and Reflex, and AMD Anti-Lag 2, for one run (`=-` on Reflex or Anti-Lag 2: its extensions not even enabled): the readme's NVIDIA DLSS section. `RX_VK_DLSS_TRACE=<frames>` logs one frame's large draws every that many frames, `RX_VK_DLSS_DEBUG=motion` shows the motion vectors, `=alternate` the game's own upscale every other frame |
|
||||||
| `RX_VK_FSR4=0/1/2`, `RX_VK_FSR_DIR=<folder>` | AMD FSR 4 for one run (0 never, 1 FSR 4, 2 its Native AA), and another folder for AMD's DLLs (not from `hookshot_env.txt`): the readme's AMD FSR 4 section. `RX_VK_FSR4=1/2` and `RX_VK_DLSS=1/2` go over the Upscaler row, a saved FSR 3.1 too where the engine they ask for runs; `RX_VK_FSR=1` goes over them (one upscaler at a time). The DLSS trace and debug switches above apply to FSR 4 too, on the frame chain they share |
|
| `RX_VK_FSR4=0/1/2`, `RX_VK_FSR_DIR=<folder>` | AMD FSR 4 for one run (0 never, 1 FSR 4, 2 its Native AA), and another folder for AMD's DLLs (not from `hookshot_env.txt`): the readme's AMD FSR 4 section. `RX_VK_FSR4=1/2` and `RX_VK_DLSS=1/2` go over the Upscaler row, a saved FSR 3.1 too where the engine they ask for runs; `RX_VK_FSR=1` goes over them (one upscaler at a time; `=2`: FSR 3.1 as its Native AA, the 3D at full size). The DLSS trace and debug switches above apply to FSR 4 too, on the frame chain they share |
|
||||||
| `RX_VK_ASYNC=1` | The replay on a thread of its own instead of the game's submitting thread |
|
| `RX_VK_FSR4_DRS=1`, `RX_VK_FSR4_AUTO_EXPOSURE=0` | (Triforce) AMD FSR 4's context as before, for comparing (`gpu/vk/vk_fsr4.cpp` `make_context`, `gpu/vk/temporal_rules.h` `plan_context`): `RX_VK_FSR4_DRS=1` one context with FSR's dynamic-resolution flag for every render size from the start, where a fixed Render resolution otherwise gets a context of its own size without it (and the game's dynamic resolution one with it once the size first changes); `RX_VK_FSR4_AUTO_EXPOSURE=0` its HDR contexts (Upscale before bloom, or an RGBA16F picture with RenoDX's HDR Native) without FSR's auto exposure. A creation or evaluation of DLSS or FSR 4 that fails leaves the game's upscale for 2 s and is tried again; the fourth failure in a row turns that engine off (the Upscaler row then shows Game and says so) until the row is set again |
|
||||||
|
| `RX_VK_SHARPEN_3D=0`, `RX_VK_FSR3_BOOST=0`, `RX_VK_UPSCALE_LOD_BIAS=0/<negative>` | (Triforce) For comparing with before (`gpu/vk/sharpen_rules.h`): the Sharpness row sharpens the 3D before the HUD with DLSS, DLAA, FSR 3.1 and FSR 4 (FSR 3.1's own RCAS, FSR 4's own sharpening, FSR 3.1's RCAS pass over DLSS's picture) and then nothing at the window; `RX_VK_SHARPEN_3D=0` the finished picture at the window for every upscaler, the HUD with it, as before. `RX_VK_FSR3_BOOST=0` FSR 3.1's sharpness the row's alone, without the share towards the most below three quarters of the output's width. `RX_VK_UPSCALE_LOD_BIAS=0` no texture LOD bias while DLSS or FSR scales the 3D with jitter (unset: log2(render / output) - 0.5 in quarters of a level, so -1.5 at Performance, -1 at Quality and -0.5 at the Native AAs, on the wrapping, mipmapped textures of the draws into the scene at the upscaler's render size (or, in the frame the dynamic resolution steps, at the size the scene's camera is drawn at), never the shadow maps, captures, other passes or dispatches; the log says `vk: texture LOD bias` at each change); a negative number, such as `RX_VK_UPSCALE_LOD_BIAS=-1` or `=-1.5`, that bias (-3 at most) in the same draws and only while an upscaler with jitter runs, for trying other values (the log says it once at the start) |
|
||||||
|
| `RX_VK_FSR4_FG=0/1/4` | AMD's frame generation, the Frame generation row's AMD FSR (`gpu/vk/vk_fsr4.cpp` `fsr4fg`): 0 never in this run (`RX_VK_FSR4=0` too, which keeps AMD's library unloaded); 1, as without it, the newest frame generation AMD's library lists for the card, FSR 4's where listed (RX 9000), else FSR 3.1's (3.1.6 on RX 6000 and 7000), the row naming it ("AMD FSR (3.1.6)") and the log the version in use; 4 FSR 4's alone, as before (FG-1) |
|
||||||
|
| `TRIFORCE_MENU_AHEAD=0` | (Triforce) What the options' rows ask of NVIDIA's and AMD's libraries (whether DLSS, FSR 4 and FSR 4's frame generation run here: NGX's start, AMD's loader) asked at the options' open on the presenting thread, as before, and not once on a thread of its own as soon as the game runs (`gpu/vk/vk_menu.cpp` `menu_ahead`; its log line says how long each took). For comparing: the first open waited about 2 s for them in test11 |
|
||||||
|
| `RX_VK_AA=<0..3>`, `RX_VK_AA_KEEP=1` | The anti-aliasing for one run, over the Insert menu's Anti-aliasing row: 0 the game's own SMAA, 1 none, 2 FXAA, 3 SMAA at its Ultra preset in its place (`gpu/vk/aa_rows.h`, `gpu/vk/vk_aa.cpp`). Unset, the row decides; with DLSS, DLAA or FSR drawing from the tone mapped picture (Upscale before bloom off) the game's SMAA is left out (none: they want jittered frames no other anti-aliasing has been over), and before the bloom it runs as the game's; the switch goes over both, for an A/B (`RX_VK_AA=0`: the game's SMAA under the upscaler). The game's SMAA is three compute programs known by their keys (game build `e0f3ccfa`): with another choice in effect its composition is replaced and its edge and weight passes are left out, and `RX_VK_AA_KEEP=1` keeps those two running. The log says `vk: aa:` when a composition is first replaced, why one was left to the game, and when none was found for a minute |
|
||||||
|
| `RX_VK_ASYNC=1` | The replay on a thread of its own instead of the game's submitting thread (off by default). The game still presents only once the GPU has run its frame, so nothing is released early. (Triforce, `gpu/vk/async_rules.h`; not yet run in the game) With it on, each with a switch of its own to put it back: a signal of the CPU's takes its capture on the game's thread only when a submission handed over before it may still wait for that event, which the host tells at the submit (`RX_VK_ASYNC_EVENT_NOTES=0`: at every signal while the thread has work, as before; the `vk: the replay thread: ... signals of the CPU's` line counts both); the submit's capture shares its copies with the copy helpers and has them all made before the hand-over (`RX_VK_ASYNC_CAPTURE_BEHIND=0`: every copy on the game's thread); the game's submit keeps its virtual core for the hand-over (`HOS_ASYNC_KEEP_CORE=0`: given up and taken again); the game's present poll waits for the replay thread asleep and looks at its event between yields only once the GPU has the frame (`HOS_PRESENT_POLL=0`: the game's own sleeps); a present runs the jobs behind it until the frame's stopped work has gone on and then makes go only what was already stopped at the present before (`RX_VK_PRESENT_DRAIN=1`: every present forces every stopped queue past its wait, as before; `=0`: none does; `RX_VK_PRESENT_WAIT_MS`, 8: how long it runs the jobs behind); and the thread looks for its next job for 100 µs before it sleeps (`RX_VK_ASYNC_SPIN_US=<0..1000>`). Still open, and why it stays off: after a ten-area tour the player's character is drawn black for one frame, 0 to 3 times a minute at the Market (`gpu/vk/async_rules.h` has what the lab ruled out). Two diagnostics for it: `RX_VK_LOG_READBACK_ZERO=1` logs each readback of GPU results that is all 0 where it was not at its readback before, and `RX_VK_REGION_TRAP=1` (with `RX_VK_PASS_TRACE=1` and the frame log) logs the passes of the last frames and writes their render conditions to `region_cond_<n>.txt` when the picture's bottom middle goes dark for a frame, eight times a run. `RX_VK_ASYNC_FORCED=1`: the thread as it was before 11-14, every catch-up forcing |
|
||||||
|
| `RX_VK_OCC_QUERY=0`, `RX_VK_OCC_QUERY_SKIP=<addr>[,...]`, `RX_VK_OCC_COND=0`, `RX_VK_OCC_LOG_COND=1` | (on by default since 11-17: +2.7 to +4.6% fps in every scene measured, 10-12% fewer draws, no flicker added; `RX_VK_OCC_QUERY=0` turns it off) The game's samples-passed reports (which objects show; a count of 0 is not drawn) get their real counts: the replay still writes each report as it passes it with 4096 (`RX_VK_OCC_FIXED`, everything visible), and the draws from its reset are counted by Vulkan occlusion queries (precise where the card has them); once the GPU is done with the recording that holds them the count is written into the report's record over the 4096, as NX2navi writes it at completion. The done recordings are found with one look at the recordings' timeline (nothing waits) as a game's fence wait ends (the game reads its counts after it), at each recording's begin and each present, at the latest when the recording's slot comes round. The game's report records rotate (a ring: an address is another object's at its next report), so a count is written only while its report is still the newest at its address, and only when it is whole (every query available, no draw left out, reset and report in one recording, read back within 8 presents); otherwise the record keeps its 4096. The game also predicates draws and clears on these words (render conditions), decided as NX2navi decides them: a word whose newest report is a samples-passed one whose count is still to come is unknown and the condition renders; a word whose count was written (or for which none comes) is read from memory as before, where a written 0 skips the draws and clears it gates. A count goes in only over the value the replay left in the word (a compare-and-write: never over the game's CPU re-arming the record), and a record rewritten in the stream after its report (a small buffer clear, a report of another counter type) gets no count. A parallel replay's segments signal the recordings' timeline and are sealed as they are submitted, so their counts are written too. The count word is written as the replay writes a report (`guest_cpu_write`, and the word kept out of the mirrors' write-backs). With hostQueryReset (Vulkan 1.2) a recording's queries are reset on the CPU once read, only those it used. `RX_VK_OCC_QUERY_SKIP`: report addresses that keep the fixed answer (the log's `occlusion query sites` lines list the first 64 addresses with their counts). The log's `occlusion queries` line every 600 presents: the reports, how many were counted whole and why not, the counts written at completion (and of them 0), those not written and why (lost, superseded by a later report at the address, late, a render condition's), the mean latency in presents and how many were written after the next present, where the done recordings were found, and the render conditions on samples-passed words (rendered because a count was pending, rendered or skipped from a written count, from memory with no count to come, as before in the knob's off phases) with the draws and clears they skipped. `RX_VK_OCC_COND=0`: a condition on a samples-passed word always renders (the counts are still written), to tell the counts from the skips. `RX_VK_OCC_LOG_COND=1`: one such condition in 499 logged (300 lines), with both words, their sources (pending, written, memory) and the present. Lab: the knob `occq` in `RX_VK_LAB_AB` runs the queries the whole run and writes the counts of the reports made in its phases (`gpu/vk/occ_answer.h`, `vk_replay.cpp` "occlusion queries") |
|
||||||
|
| `RX_VK_GATE_WAIT_MS=<1..100>` | (Triforce, off unless set, after NX2navi's `gate_wait_ms`, whose default is 10; not yet run in the game) A submission whose first command buffer begins (after state only) with a wait for an event the game signals from the CPU, unmet in the replay's order, waits at its submit for that signal, at most this long, before its capture and its replay (`gpu/vk/gate_wait.h`, `gpu/vk/vk_replay.cpp` `gate_wait_signal`): the capture then holds what the game wrote up to its signal, and the submission is replayed in the submit call on the submitting thread, instead of stopping at the wait and going on from the order thread (`sync_watch`) with a second capture once the signal came. The wait holds no lock and looks at the event's word in the game's memory. One that runs out goes as before; after 4 that ran out, once they are more than a tenth, it is off for the run (logged). A `vk: RX_VK_GATE_WAIT_MS` line every 600 waits gives their mean and how many ran out; the order line's `captures at a CPU event's wait` and `made to go by presents` say how often the old way was taken |
|
||||||
|
| `RX_VK_CAPTURE_BEHIND=0` | (Triforce, on by default; `=0` as before) The submit-time capture (`gpu/vk/vk_memory.cpp` `capture_make`) takes the write marks on the game's submitting thread and copies there only the pages read at once (the late pages and the texture storage); the rest of its copies, in parts of 32 pages, go to the capture's copy helpers while that thread replays the submission (`gpu/vk/capture_behind.h`). A read of a page as submitted copies its part itself when no helper has taken it yet, or waits for the helper; every part is done and flushed before a recording is submitted and before the thread goes back to the game. Only with the replay on the submitting thread (not `RX_VK_ASYNC=1`), with copy helpers (`RX_VK_CAPTURE_THREADS`, not `RX_VK_LOW_SPEC`), at least 128 pages behind and at most 512 ahead (a scene's load: copied whole as before). At the Market the old copy was ~0.25 ms of each submit's ~0.35 ms capture, six submits a frame. The 120-present line `a frame's captures at the game's submits` gives their time on the game's threads (the scan, the copies made there, and the copies behind made or waited for by the replay), the MB copied and behind, and who copied the parts; the stall line names the captures when they took 1 ms or more |
|
||||||
|
| `RX_VK_PARALLEL=<0..8>`, `RX_VK_PARALLEL_AUTO=<2..8>` | The parallel replay: a frame's large command buffers recorded on that many threads, always, or wherever short trials as the game runs find it faster than the replay thread alone; `RX_VK_PARALLEL=0`: never. The Debug tab's Replay helper threads row is the second without an environment to set (`replay_helper_threads=` 1 to 7 for 2 to 8 threads); either switch goes over the row. The row's Automatic (8), which it is unless the options file says otherwise, is the count `gpu/vk/replay_threads.h` gives the PC: none with four cores or fewer or on AMD's driver for RDNA 1 and 2, 2 threads with five to seven cores, 3 with eight or more. The log says at the start what was chosen (`replay helper threads:`) and at each trial which way is in use (`parallel replay, trial`). Between trials the tuner's rule (`gpu/vk/replay_threads.h` `rule`) judges every 120 frames on the replay thread alone: where the replay is plainly the frame's limit (70% of the frame or more) and the helper threads would take at least 1 ms and a tenth of the frame off it, never counting below the GPU's time (the game waits for each frame's GPU work), they are turned on without a trial; where they cannot pay (no command buffer large enough to cut, the GPU's time about as long as the replay's, or the replay under 60% of the frame: the game's own threads are its limit) no trial is made while that holds; trials decide the rest, and a trial that takes the helpers back off holds the rule's "on" for 3,600 presents. The log says so (`parallel replay, the rule`) when its verdict changes. `RX_VK_PARALLEL_RULE=0`: trials only, as before. With DLSS or FSR on, for upscaling or frame generation, the dry pass tells their hooks in the draws in stream order, and each command buffer from the first draw or dispatch that takes what they noted (the upscale, or the engine's point before the bloom) on is recorded in order after the segments before it (`gpu/vk/stream_order.h`); `RX_VK_PARALLEL_TEMPORAL=0`: the replay thread records alone then, as before. The frame statistics' `a frame's parallel replay:` line says in how many frames the threads recorded, the command buffers in segments and alone (and why), and the time in them; the next one (`its parts`) where that time went: the setup before the threads begin, how soon each command buffer's first segment reached the GPU, the helper threads' wake-up, each command buffer's longest segment, the replay thread's wait, the submits, the recording lock's waits, and the segments' GPU time (the `GPU time` line counts the segments too; it left them out). After each trial's line a `trial N's parts` line gives its three parts' frame, replay thread's time, GPU time, the rest of the frame, full barriers and draws, and the helper threads' parts. That header lists what an audit of the parallel replay found and fixed, and what is left. `RX_VK_NO_DRY_COPIES=1`: the dry pass does not mark what copies and texture clears write, as before. The replay thread records the next segment to submit itself when no helper thread has begun it (it only waited before, on the core kept for it); `RX_VK_PARALLEL_OWN=0`: it waits for the helper threads, as before. `RX_VK_PARALLEL_SEG=<32..>`: the draws and dispatches a segment is cut for (160); `RX_VK_PARALLEL_SEGS=<2..12>`: the most segments a command buffer is cut into (6), without which a smaller `RX_VK_PARALLEL_SEG` cuts no finer once a command buffer has six segments' worth |
|
||||||
|
| `RX_VK_PARALLEL_BACKOFF=0` | The tuner's trials as before: one every 1,800 presents after one that kept the way, each with 104 frames the other way. With the back-off (the default, `gpu/vk/replay_threads.h` `Tuner`) each trial that keeps the way, the other way longer than both of its own parts, puts the next one off twice as long (3,600, 7,200, then 14,400 presents at most; a margin of 10% or more counts twice), one too close to tell keeps the old interval (the first such one after a change of scene or way: 900 presents), and once the way has held so the other way is tried for 64 frames, not 104; a short trial that finds the other way shorter changes nothing itself, a full trial follows at once. The next trial comes early (900 presents after the last one at the soonest, never while the rule says no trial) when the scene clearly changes: two windows of 120 frames in a row whose draws moved by a quarter (200 at least) or whose replay or GPU time moved by 30% (1 ms at least) from the first window after the way was set, or a trial whose first and last parts did. In the lab's steady Market a trial came every ~2,090 presents, each ~100 frames 1.9 ms longer; the trial lines now end with when the next may come, and `parallel replay, the scene changed` says when one was brought forward |
|
||||||
|
| `RX_VK_RECORDER=thread`, `=defer`, `=sync` | (off unless set; not yet measured in the game) `thread`: the replay thread's `vkCmd*` calls packed with copies of their arguments into a ring and made by a recorder thread of their own, in order (`gpu/vk/vk_recorder.h`); the replay thread waits for it at every command buffer begin, end and reset, submit, wait, readback and destruction. Beside the Replay helper threads too (no longer refused with `RX_VK_PARALLEL`): a parallel replay's segments make their calls themselves, after the ring has been run (`segments_begin`), and the recorder is idle while they record. With it on, the frame statistics' `a frame's recorder thread` line says how many of the replay's calls it made a frame and in how long, and how long the replay thread waited for it; what it saves is the replay time line's `commands` with it against without it. `defer` and `sync` are its test steps (held and run on the same thread). `RX_VK_RECORDER_RING_MB` (8), `RX_VK_RECORDER_SPIN_US`, `RX_VK_RECORDER_STATS=0`, `RX_VK_RECORDER_POISON=1`, `RX_VK_RECORDER_PIN` (`next`, `sibling` or a mask) |
|
||||||
|
| `RX_VK_REPLAY_PRIO=0` or `1` | (Triforce, on by default, after NX2navi's `recorder_prio`; `=0` as before) A thread runs at the highest priority while it replays the game's frames (the submitting thread's replay and its capture, the present, the stopped work the order thread lets on: `gpu/vk/vk_replay.cpp` `ReplayPin`) and gets its own priority back after, so the game's code it runs between replays keeps the game's priority; the core kept for the replay is kept from the game's threads only, and the helper threads and the capture's copy helpers (above normal) outranked a replay at the game's priority there. A thread at that priority or above already is left alone. The replay thread of `RX_VK_ASYNC=1` and the recorder thread of `RX_VK_RECORDER=thread` take the level for good. `=1`: above normal. The log's `vk: a thread that replays runs at` line says which (`gpu/vk/replay_prio.h`) |
|
||||||
| `HOS_VCORES=0`, `HOS_CORES=<1..8>`, `HOS_RESERVE_CORE=0` | No virtual cores; the cores the game sees (6); no core kept for the replay (one is, with 8 or more logical processors) |
|
| `HOS_VCORES=0`, `HOS_CORES=<1..8>`, `HOS_RESERVE_CORE=0` | No virtual cores; the cores the game sees (6); no core kept for the replay (one is, with 8 or more logical processors) |
|
||||||
|
| `HOS_VCORE_SPIN_US=<us>`, `HOS_VCORE_SPIN_MAX=<n>` | (Triforce, on by default; `HOS_VCORE_SPIN_US=0` as before) A game thread that waits for a virtual core it asked a holder to give up spins for it, up to 250 us, before it sleeps (`host/hos_thread.c` `vc_spin`, after NX2navi's `core_take`): the holder gives the core up at its next checkpoint, mostly within microseconds, and the waiter then runs without an OS wake. Who gets a core does not change, only how soon. At most `HOS_VCORE_SPIN_MAX` threads spin at once and the rest sleep at once: by default a quarter of the logical processors beyond 8, at most 4 (none with 8 or fewer, 1 with 12, 2 with 16). The `vcores:` line (every 5 s) counts the waits that ended in the spin and those that slept, the time spun, the spins left out for the cap, and how soon a granted core ran after a spin and after a sleep; the start's `vcores: a waiter spins up to` line says what is in effect. A yield with no thread waiting for a core takes no lock (NX2navi's yield fast path) |
|
||||||
|
| `RX_VK_HELPER_SPIN_US=<us>` | (Triforce, on by default; `=0` as before) The parallel replay's spins (`gpu/vk/helper_spin.h`, after NX2navi's replay workers): a helper thread with no segment to record spins up to 100 us for the next before it sleeps, and the replay thread spins as long for a segment a helper records before it sleeps for it. The frame statistics' `its spins` line, after the parallel replay's parts, says how often each ended in the spin or slept and the time spun |
|
||||||
|
| `HOS_X3D=0` | (Windows) The game's threads and the replay's core are not kept on the die with the largest L3 cache. By default they are, on a processor whose L3 caches differ in size and whose largest serves 8 or more cores (AMD's two-die X3D parts: a 7950X3D's or 9950X3D's 3D V-Cache die; `host/hos_x3d.h`), and the log's `cpu: logical processors ... share the largest L3 cache` line says so. On every other processor nothing changes and the log has no such line |
|
||||||
|
| `HOS_POWER_THROTTLING=1` | (Windows) Windows' power throttling of a window not in front left as it is (for A/B runs). By default every run, the shader preparation's too, opts out of it at the start: full speed, not the efficiency speed (EcoQoS) Windows 11 gives such a process, and the 1 ms timer resolution kept, which Windows 11 may otherwise ignore for it (Windows 10: full speed only). The log's `power:` line says what Windows took |
|
||||||
|
| `HOS_HOST_ZSTD=1` or `2` | The game's files (`.zs`) decoded by the host's zstd (`third_party/zstd`) instead of the game's own translated decoder; off unless set, and nobody has played with it on. `1`: the host decodes what `host/host_zstd.h` takes (whole buffers, and the loading threads' streams with their content size and room for it), the game's code the rest, errors included. `2`: check, the game's code decodes everything as before and the host beside it, and the log's `zstd check: ... MISMATCH` lines name every difference (its `zstd:` lines count what was compared): the way to try it before `1`. Game build `e0f3ccfa` only. The lift needs both of its calls (`nx2/oot_patches.py` `patch_host_zstd`): a build log that says `the host's zstd decoder's calls were found at 1 of 2 places` has one only, and the switch then covers only that function |
|
||||||
|
|
||||||
### Building
|
### Building
|
||||||
|
|
||||||
@@ -165,9 +295,14 @@ A build reads and writes tens of gigabytes. When the game folder or the build fi
|
|||||||
| Switch | What it does |
|
| Switch | What it does |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `HOOKSHOT_JOBS=<n>` (or `--jobs`) | Compilers at once (default one per CPU thread, each started when the free memory holds it; `HOOKSHOT_MEM_GATE=0` turns that check off) |
|
| `HOOKSHOT_JOBS=<n>` (or `--jobs`) | Compilers at once (default one per CPU thread, each started when the free memory holds it; `HOOKSHOT_MEM_GATE=0` turns that check off) |
|
||||||
| `HOOKSHOT_LIFT_JOBS=<n>` | Processes that write the lifted C (default the cores, at most 16) |
|
| `HOOKSHOT_LIFT_JOBS=<n>` | Processes that write the lifted C. Forked on Linux and macOS: default the cores, at most 16, one per 2 GB of free memory. Spawned on Windows (`nx2/emit_pool.py`), where they also cut the big chunks and share the page marks: default half the processors, at most 8, one per GB of free memory. `1`: one process, on every system; the C is the same whatever the count |
|
||||||
| `HOOKSHOT_KEEP_BUILD_FILES=1` | Keeps the lifted C and the objects after a build, so that the next one compiles only what changed |
|
| `HOOKSHOT_KEEP_BUILD_FILES=1` | Keeps the lifted C and the objects after a build, so that the next one compiles only what changed |
|
||||||
|
| `HOOKSHOT_EARLY_COMPILE=0` | (Windows) Not compiled beside the lift: libopus and the two renderers, which need no translated code. By default `make_game.py` starts `build.py` with `RX_EARLY=1` for them while the game is translated (the lift's own processes take at most half the processors there), and the build proper finds their objects made |
|
||||||
|
| `HOOKSHOT_MEM_GIANT=1` or `0` | Whether the memory gate counts a function of 1.2 MB or more by the square of its size (a chunk that is one 5 MB function takes 6.2 GB, not the 1.9 GB it is counted at otherwise). By default only on a PC with less than 14 GB free when the compile starts: with more, counting the giants in full halves the compilers running at once for no need |
|
||||||
|
| `HOOKSHOT_MEM_LOG=1` | For each file compiled, `@@mem <file> <peak MB> <counted MB> <seconds>`: the most memory its compiler was seen holding, beside what the memory gate counted it at (`mem_need` in `nx2/build.py`). For fitting the gate after a change to the lift or the compiler |
|
||||||
| `HOOKSHOT_MINGW=<dir>`, `HOOKSHOT_MSVC=1` | The llvm-mingw to build with; the Visual Studio route instead |
|
| `HOOKSHOT_MINGW=<dir>`, `HOOKSHOT_MSVC=1` | The llvm-mingw to build with; the Visual Studio route instead |
|
||||||
|
| `HOOKSHOT_SINGLE_INSTANCE=0` | (Triforce) Hookshot Builder takes no lock on its build tools, as before. By default a second builder for the same tools folder (`builder/HookshotBuilder/Instance.cs`, named after `Tools.Root`), its `--cli` included, waits up to 5 s for one that is ending, then says so and ends: two of them built into one work folder and stopped each other's build. Triforce Setup has its own lock (`setup/Guard.cs` `Instance`) |
|
||||||
|
| `TRIFORCE_LEGACY_PATHS=1` | (Triforce) Triforce Setup keeps .NET Framework's legacy path checks, which refuse paths of 260 characters and more whatever its manifest says (an exe csc builds is taken as written for .NET 4.0). By default `setup/Setup.cs` `Main` sets the two switches 4.6.2 defaults to (`Switch.System.IO.UseLegacyPathHandling`, `Switch.System.IO.BlockLongPaths`) before any path is used, so that with Windows' own LongPathsEnabled on Setup reads and writes deep paths |
|
||||||
| `RX_CPM=<dir>`, `WGPU_NATIVE=<dir>` | The CPM cache (Vulkan and SPIR-V headers, VMA, FFmpeg) and wgpu-native |
|
| `RX_CPM=<dir>`, `WGPU_NATIVE=<dir>` | The CPM cache (Vulkan and SPIR-V headers, VMA, FFmpeg) and wgpu-native |
|
||||||
| `RX_TARGET=windows` | On Linux: the Windows game, with llvm-mingw's Linux build in `HOOKSHOT_MINGW` |
|
| `RX_TARGET=windows` | On Linux: the Windows game, with llvm-mingw's Linux build in `HOOKSHOT_MINGW` |
|
||||||
| `RX_RELEASE=1` | A release build (`make_game.py` sets it): no console window, the shaders prepared before the title |
|
| `RX_RELEASE=1` | A release build (`make_game.py` sets it): no console window, the shaders prepared before the title |
|
||||||
@@ -175,36 +310,164 @@ A build reads and writes tens of gigabytes. When the game folder or the build fi
|
|||||||
| `RX_MARCH=<level>` | The CPU level to build for (default this PC's: `x86-64-v3`, else `x86-64-v2`, else `x86-64`) |
|
| `RX_MARCH=<level>` | The CPU level to build for (default this PC's: `x86-64-v3`, else `x86-64-v2`, else `x86-64`) |
|
||||||
| `RX_VK_ONLY=1` | Only the Vulkan back end, with no lift |
|
| `RX_VK_ONLY=1` | Only the Vulkan back end, with no lift |
|
||||||
| `OOT_WORK`, `OOT_DUMP`, `OOT_EXEFS`, `OOT_ROMFS` | The work folder and the dump's folders, over `work/dump.txt` |
|
| `OOT_WORK`, `OOT_DUMP`, `OOT_EXEFS`, `OOT_ROMFS` | The work folder and the dump's folders, over `work/dump.txt` |
|
||||||
|
| `HOS_MODS=<folder>` | The mods folder, in place of `mods` in the game folder: for the running game (RomFS mods, cheat files) and for a build (the code patches it applies) |
|
||||||
|
|
||||||
|
## Mods: code patches and cheats
|
||||||
|
|
||||||
|
The game loads RomFS mods at its start (`host/mods.c`). The two kinds of mod that change what the game *does* go other
|
||||||
|
ways, because the game's code is C compiled ahead of time and cannot be patched while it runs.
|
||||||
|
|
||||||
|
**The mods folder**, for the game and for a build alike (`host/mods.c` `find_mods`, `nx2/code_mods.py` `find_mods`,
|
||||||
|
which must stay the same walk): a mod is a folder under `mods` that holds a `romfs`, `exefs` or `cheats` folder, at
|
||||||
|
most four levels down, and nothing below a mod is looked at. A folder whose name starts with `-` or `_` is off, one
|
||||||
|
that starts with `.` is skipped. Mods are ordered by their path under `mods`, ASCII letters' case ignored, and the last
|
||||||
|
one wins. Triforce Setup's mod manager can switch a mod by renaming its folder and order mods by a prefix, and needs
|
||||||
|
no other file.
|
||||||
|
|
||||||
|
**Code patches** (`<mod>/exefs/*.ips`, `*.pchtxt`), `nx2/code_mods.py`:
|
||||||
|
|
||||||
|
- `make_game.py` calls `build_step` once its copy of the game's code (`<work>/exefs`) is there and checked, after the
|
||||||
|
copy the game folder keeps for updates (`.triforce_exefs`) is made and before the lift key is taken. The patches go
|
||||||
|
into that copy, a module with a patch written again as a plain NSO, and the lift reads it: `lift.py`, `chunks.py`
|
||||||
|
and the rest of the translator know nothing of mods.
|
||||||
|
- The lift key hashes the files of that copy, so the patched bytes are in it: a patch added, taken out, switched off
|
||||||
|
or edited translates the game again, a patch that changes no byte does not, and neither does a change to
|
||||||
|
`code_mods.py` itself (`LIFT_KEY_NOT`).
|
||||||
|
- A patch names its module by build id (the `.ips` file's name, the `.pchtxt` file's `@nsobid-`, trailing zeros left
|
||||||
|
out or not) and may be for any module of the exefs. Offsets count the NSO's 0x100-byte header, as Atmosphere and the
|
||||||
|
emulators read them; `.pchtxt` knows `@flag offset_shift`, `@enabled`, `@disabled`, `@little-endian`, `@big-endian`
|
||||||
|
and `@stop`. Mods in order, a mod's files by name, the later write wins and the build says which file wrote over
|
||||||
|
which.
|
||||||
|
- A file is refused whole, with a line that says why: another build's, cut short, a value that cannot be read, a write
|
||||||
|
into the header, past the image or between the segments. The last leaves out code caves, which the translator has
|
||||||
|
not been tried on.
|
||||||
|
- The build writes `triforce_code_mods.txt` into the game folder when it is done: a line per patch file, tab
|
||||||
|
separated, `applied` or `refused`, the file's FNV-1a 64 and size, its path under the mods folder, then the module
|
||||||
|
and the bytes changed, or the reason. The next build (the game's update offer runs `--update`) reads the mods folder
|
||||||
|
again, so the same patches are built in again, and says which of the record's are gone, changed or refused now. The
|
||||||
|
game reads the record at its start (`host/mod_code.h`) and logs for each patch file whether it is built in or waits
|
||||||
|
for a build, and which built-in patches are gone from the folder.
|
||||||
|
|
||||||
|
**Cheats** (`<mod>/cheats/<first 16 digits of main's build id>.txt`), `host/mod_cheats.h` and `host/mod_cheats.c`:
|
||||||
|
|
||||||
|
- Read at the start, run by an interpreter once a game frame, at the game's present and on its thread
|
||||||
|
(`hos_mod_cheats_frame`). Nothing is compiled in, so a cheat file needs no build. NX2navi compiled in the cheats that
|
||||||
|
write instructions; Triforce's translator has no switchable patches, so such a cheat (an opcode 0 into a module's
|
||||||
|
`.text`) is refused and belongs in a code patch.
|
||||||
|
- The opcodes it runs are listed at the top of `host/mod_cheats.h`: 0 to A and C0 to C3 of Atmosphere's cheat machine.
|
||||||
|
Any other is refused when the file is read, the whole cheat, by name. Registers are shared by one file's cheats within
|
||||||
|
a frame and cleared between frames; a master code runs first.
|
||||||
|
- Memory is the program image and the heap, each access checked (`mod_cheat_readable`); the modules' code
|
||||||
|
(`hos_code` in the generated `hos_image.c`) is never written. Region 0 is the module main's base (`hos_main_base`),
|
||||||
|
1 and 2 `HOS_HEAP_BASE`, 3 `0x08000000`, as `host/hos_svc.c` tells the game.
|
||||||
|
- Each cheat that is not a heading has a row on the System tab, 96 at most (`gpu/vk/mod_cheat_rows.h`; option ids 130
|
||||||
|
to 133 in `gpu/vk/vk_api.h`). The rows are not in the options file: `triforce_cheats.txt` in the game folder has a
|
||||||
|
line for each cheat that is on, its number (FNV-1a 32 of `<mod>/<name>`) and that text, and a cheat listed there is
|
||||||
|
on at the start.
|
||||||
|
|
||||||
## The options file
|
## The options file
|
||||||
|
|
||||||
`hookshot_options.ini`, in the game folder, holds the Insert menu's rows as `<key>=<number>`: the row's name in lower
|
`hookshot_options.ini`, in the game folder, holds the Insert menu's rows as `<key>=<number>`: the row's name in lower
|
||||||
case with `_` for spaces (`frame_generation=2`, `invert_camera_x=1`), except Output resolution, which is `output_res`.
|
case with `_` for spaces (`frame_generation=2`, `invert_camera_x=1`), except Output resolution, which is `output_res`.
|
||||||
`gpu/vk/vk_menu.cpp` writes the whole file at every change (`save`), with `options_version=4`,
|
`gpu/vk/vk_menu.cpp` writes the whole file at every change (`save`; (Triforce) merged by key with a change made outside the game meanwhile, and written whole: `RX_VK_OPTIONS_MERGE` above), with `options_version=8`,
|
||||||
`microphone_device=<name>` when a microphone is chosen by name, and the controls (`pad_<button>` and `key_<button>`:
|
`microphone_device=<name>` when a microphone is chosen by name, and the controls (`pad_<button>` and `key_<button>`:
|
||||||
the README's [Controls](../readme.md#controls)). A value outside a row's range is clamped when it is read. Where the
|
the README's [Controls](../readme.md#controls)). A value outside a row's range is clamped when it is read. Where the
|
||||||
number is not the obvious one:
|
number is not the obvious one:
|
||||||
|
|
||||||
| Key | Values |
|
| Key | Values |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `quality_preset` | 0 Custom, 1 Minimum (4GB), 2 Low (6GB), 3 Balanced (8GB), 4 High (12GB), 5 Ultra (16GB+) |
|
|
||||||
| `gpu_device` | 0 Auto, n the n-th card with Vulkan 1.3 as the row lists them |
|
| `gpu_device` | 0 Auto, n the n-th card with Vulkan 1.3 as the row lists them |
|
||||||
| `frame_generation` | 0 Off, 2 to 8 the multiplier |
|
| `frame_generation` | 0 Off, 1 Auto, 2 to 8 the multiplier, 9 AMD FSR frame generation (the made frame by AMD's frame generation library, FSR 4's or FSR 3.1's by the card, with the picture without the HUD; paused with HDR Native or RenoDX's RGBA16F scene, where the renderer's own makes the frame as at 2x: `gpu/vk/vk_fsr4.cpp` `fsr4fg`; where that cannot run the value stays in the file and the row shows and runs 2x: `gpu/vk/framegen_rows.h`) |
|
||||||
| `resolution` | 0 Auto, 1 100%, 2 75%, 3 67%, 4 50% |
|
| `dlss_frame_generation` | 0 Off, 1 2x, 2 3x, 3 4x: NVIDIA DLSS frame generation, the Frame generation row's NVIDIA DLSS values (`gpu/vk/merged_rows.h`; read at the start, `gpu/vk/vk_dlssg.cpp`). The row sets `frame_generation` to 0 with it |
|
||||||
| `output_res` | 0 2048x1152, 1 2560x1440, 2 3840x2160, 3 1920x1080, 4 5120x1440, 5 5120x2880 |
|
| `frame_rate_cap` | 0 60, 1 90, 2 120, 3 144, 4 165, 5 240, 6 the monitor's refresh rate; -1 45, -2 40, -3 30 |
|
||||||
| `upscaling` | 0 Off, 1 Catmull-Rom, 2 Lanczos |
|
| `frame_pacing` | 0 Off, 1 frames on an even line. With `steady_frame_rate`, the Frame pacing row (`gpu/vk/merged_rows.h`): Off is 0 and 0, Even 1 and 0, Steady (any rate) 1 and 1, Steady (refresh steps) 1 and 2 |
|
||||||
|
| `steady_frame_rate` | 0 Off, 1 any rate (the cap, then 200, 180, 160, 144, 120, 110, 100, 90, 80, 72, 66, 60, 55, 50, 45, 40, 36, 30 under it), 2 whole divisions of the display's refresh rate under the cap, from 30 up (`host/pace_steady.h`) |
|
||||||
|
| `resolution` | 0 Auto, 1 100%, 2 75%, 3 67%, 4 50%, 5 58%, 6 33% (with DLSS and FSR: 3 Quality, 5 Balanced, 4 Performance, 6 Ultra Performance; their Native (100%) is `dlss` 2, 5 or 6) |
|
||||||
|
| `output_res` | 0 2048x1152, 1 2560x1440, 2 3840x2160, 3 1920x1080, 4 5120x1440, 5 5120x2880, 6 5120x2160, 7 1536x864 (the handheld profile; before `options_version` 8 it was `handheld_profile=1`, read as 7) |
|
||||||
|
| `ultrawide` | 0 16:9, 1 16:10, 2 21:9, 3 32:9 (before `options_version` 6, 1 was the monitor's shape: read as the nearest of the four, `gpu/vk/ultrawide_rows.h`) |
|
||||||
|
| `hdr`, `renodx_hdr` | the HDR row (`gpu/vk/merged_rows.h`): `hdr` 0 Off, 1 Expanded, 2 Native (the game's own); `renodx_hdr` 1 with `hdr` 2 is Native + RenoDX |
|
||||||
|
| `nvidia_reflex`, `amd_anti-lag_2` | the Low latency row by the card's maker: 0 Off, 1 On, 2 On + Boost (NVIDIA alone) |
|
||||||
|
| `hdr_paper_white`, `hdr_peak_brightness` | nits: 80 to 400, and 400 to 10,000 |
|
||||||
|
| `renodx_tonemapper` | 0 Vanilla, 1 PsychoV35, 2 RenoDRT |
|
||||||
|
| `renodx_gamma_correction` | 0 Off, 1 2.2, 2 BT.1886 (2.4) |
|
||||||
|
| `renodx_match_tone_slope` | 0 Off, 1 On (as out of the box): PsychoV35's rows, with `renodx_tone_slope` and `renodx_hue_clip` |
|
||||||
|
| `renodx_tone_slope`, `renodx_highlight_saturation` | 0 to 100, 50 unchanged: the first PsychoV35's, the second RenoDRT's |
|
||||||
|
| `renodx_blowout`, `renodx_flare` | 0 to 100, 0 none: RenoDRT's |
|
||||||
|
| `renodx_hue_clip`, `renodx_hue_correction`, `renodx_color_grading` | percent, 0 to 100: 50, 100 and 100 out of the box; the first PsychoV35's, the others RenoDRT's |
|
||||||
|
| `dlss` | the Upscaler: 0 the game's own, 1 NVIDIA DLSS, 2 NVIDIA DLAA, 3 AMD FSR 3.1, 4 AMD FSR 4, 5 AMD FSR 4 Native AA, 6 AMD FSR 3.1 Native AA. The row names the engine; 2, 5 and 6 are its Native (100%) on the Render resolution row (`gpu/vk/upscaler_rows.h`) |
|
||||||
|
| `anti-aliasing` | 0 the game's own SMAA, 1 Off, 2 FXAA, 3 SMAA Ultra |
|
||||||
|
| `texture_detail` | 0 Automatic, 1 Full, 2 Reduced, 3 Low |
|
||||||
|
| `fps_counter` | 0 Off, 1 On, 2 Full (NX2navi's full performance overlay in the counter's place: `vk_menu.cpp` `draw_perf`, the frame times and the figures per frame `gpu/vk/perf_stats.h`, the machine's numbers sampled twice a second on a thread of their own `gpu/vk/perf_machine.cpp`) |
|
||||||
|
| `opaque_textures_as_bc1` | 0 Off, 1 On |
|
||||||
| `sharpness`, `rumble_strength`, `motion_sensitivity_x`, `motion_sensitivity_y` | percent |
|
| `sharpness`, `rumble_strength`, `motion_sensitivity_x`, `motion_sensitivity_y` | percent |
|
||||||
| `mouse_sensitivity` | hundredths: 100 is 1.00 |
|
| `mouse_sensitivity` | hundredths: 100 is 1.00 |
|
||||||
|
| `left_stick_dead_zone`, `right_stick_dead_zone` | -1 Default (XInput's ~24%), 0 to 50 percent (before `options_version` 8 a `stick_dead_zone` of 1 to 5, 5, 10, 15, 20 or 30% for both sticks, is read into each stick's own where it had none) |
|
||||||
|
| `vibration_curve` | hundredths, 20 to 150: the power on the motors' rumble, 100 is 1.00 (none) |
|
||||||
|
| `walk_key` | 0 Off, 1 Caps Lock, 2 Shift, 3 Ctrl, 4 Tab |
|
||||||
|
| `walk_scale` | hundredths, 20 to 90: 51 is 0.51 of the stick's travel |
|
||||||
| `robust_buffers` | 0 Auto (on), 1 Off, 2 On |
|
| `robust_buffers` | 0 Auto (on), 1 Off, 2 On |
|
||||||
| `game_speed` | 0 Per frame, 1 Real time |
|
| `game_speed` | 0 Per frame, 1 Real time |
|
||||||
| `text_speed` | 0 Normal, 1 2x, 2 4x, 3 Instant |
|
| `text_speed` | 0 Normal, 1 2x, 2 4x, 3 Instant |
|
||||||
|
| `menu_sounds` | percent, 0 to 100: 80 unless set, 0 silent. The settings' own sounds, the game's interface sounds from its RomFS (`host/menu_sound.c`, `gpu/vk/menu_sound_events.h`; option ids 135 and 136), under `master_volume` |
|
||||||
| `microphone` | 0 Off, 1 Default, 2 and up a microphone (the name in `microphone_device` decides which) |
|
| `microphone` | 0 Off, 1 Default, 2 and up a microphone (the name in `microphone_device` decides which) |
|
||||||
| `mouse_camera` | 0 As the stick, 1 Native |
|
| `mouse_camera` | 0 As the stick, 1 Native |
|
||||||
| `layout_preset` | 0 Nintendo, 1 Xbox / PC, 2 Swap Face, 3 Custom |
|
| `layout_preset` | 0 Nintendo, 1 Xbox / PC, 2 Swap Face, 3 Custom |
|
||||||
|
| `button_prompts` | 0 Auto (the controller there at the start, XInput's without a name the Xbox's; none: the device used last in the run before, `<user>/prompts_auto.txt`, else the keyboard and mouse), 1 Switch, 2 Xbox, 3 PlayStation 4, 4 PlayStation 5, 5 Steam Deck, 6 Keyboard & mouse (`host/prompts.c`, `nx2/prompt_keys.py`) |
|
||||||
|
| `replay_helper_threads` | 0 Off, 1 to 7 the parallel replay tried on 2 to 8 threads, 8 Automatic (the default; the Debug tab). A file that says 0 stays Off: delete the line, or choose Automatic on the Debug tab, for the default |
|
||||||
|
|
||||||
|
The Game language row is not in this file: its choice is the code line of `language.txt` in the game folder, which the
|
||||||
|
menu rewrites in place (`gpu/vk/game_language.h`: the code alone on that line, the rest of the file as it was, and only a
|
||||||
|
code the file itself names) and the host reads at the start (`host/nn_applet.c`). Nor are the Mod cheats rows: which
|
||||||
|
of a mod's cheats are on is the host's to keep, in `triforce_cheats.txt` (*Mods: code patches and cheats* above).
|
||||||
|
|
||||||
|
(Triforce) `options_version=8` grouped the rows and merged those that chose one thing twice (`gpu/vk/merged_rows.h`; the
|
||||||
|
436f9ac-based builds wrote 7 for it, which v0.5.1 had already given to Frame pacing's default: its 7 still has the old rows'
|
||||||
|
lines, so a file before 8 is moved, and one of those builds' 7, without them, keeps what it has). A file
|
||||||
|
from before keeps its values; these lines are read once and not written again, their rows' jobs done otherwise:
|
||||||
|
`quality_preset` (the textures' and the mirrors' memory budgets are the card's own, from its video memory), `upscaling`
|
||||||
|
(the window's filter follows the sizes: `gpu/vk/upscaler_rows.h` `auto_filter`, AMD FSR 1 when the picture is enlarged, the
|
||||||
|
plain copy otherwise and always with HDR Native), `reflex_frame_limit` (off, the old default, until Reflex holding the cap
|
||||||
|
with DLSS frame generation has been played: `RX_VK_REFLEX_LIMIT=1` for a run), `handheld_profile` (an `output_res`) and `stick_dead_zone` (each
|
||||||
|
stick's own).
|
||||||
|
|
||||||
The other rows are 0 Off and 1 On. Some are read at the start, before the menu exists, by the code that needs them:
|
The other rows are 0 Off and 1 On. Some are read at the start, before the menu exists, by the code that needs them:
|
||||||
`handheld_profile` and `output_res` (`host/nn_applet.c`), `ultrawide` (`host/hos_main.c`), `robust_buffers`
|
`output_res` (`host/nn_applet.c`; 7 is the handheld profile), `ultrawide` (`host/hos_main.c`), `frame_rate_cap`
|
||||||
(`gpu/vk/vk_main.cpp`) and the controls (`host/nn_hid.c`).
|
(`host/nn_vi.c`; the monitor's rate is the display's the window opens on, rounded, 30 to 240, and 60 when it cannot be read), `robust_buffers`
|
||||||
|
(`gpu/vk/vk_main.cpp`), `texture_detail` (`gpu/vk/vk_texture.cpp`, at the first texture) and the controls (`host/nn_hid.c`).
|
||||||
|
|
||||||
|
The `renodx_` rows other than `renodx_hdr` are read at every frame (`gpu/vk/vk_renodx.cpp` `fill_settings`), and the
|
||||||
|
menu shows the rows of one tonemapper while it is the one chosen. RenoDX's mod for this game has Vanilla and PsychoV35;
|
||||||
|
RenoDRT and the gamma correction come from `gpu/vk/shaders/renodx_glue`, three files of Triforce's own that
|
||||||
|
`scripts/renodx_shaders.py` compiles together with RenoDX's scene shaders and its output encoder as the build fetched them
|
||||||
|
(nothing of RenoDX is in the repository or changed). RenoDRT calls RenoDX's library; the gamma correction is on the
|
||||||
|
finished picture, HUD included, and the tone mappers are given a peak with it taken out, so that the brightest light
|
||||||
|
still lands on the HDR peak brightness. A change to a glue file changes the stamp of `renodx_spv.h`, which the next build
|
||||||
|
then compiles again.
|
||||||
|
|
||||||
|
RivaTuner Statistics Server has no row: where it is installed and running it is the game's frame limiter, and an options
|
||||||
|
file that still has the test builds' `rivatuner_limit=` is read by nothing (the next save drops the line). A thread of
|
||||||
|
`gpu/vk/rtss_limit.cpp` looks twice a second: it puts the limit (`gpu/vk/pace_rule.h` `pace_rtss_want`: the cap in
|
||||||
|
effect, Monitor's rate and the caps under 60 included, times the frames shown for each of the game's with frame
|
||||||
|
generation 2x to 8x or AMD FSR 4) into RTSS's profile for the executable through RTSS's own hooks library
|
||||||
|
(`RTSSHooks64.dll` from RTSS's folder: `FramerateLimit` and `FramerateLimitDenominator`, nothing else, no other profile),
|
||||||
|
reads it back, and once RTSS is seen holding the game to it (`gpu/vk/rtss_decide.h` `rtss_decide`) the host's wait steps
|
||||||
|
aside (`gpu/vk/vk_pace.cpp`), the in-house frame generation's display thread leaves its own spacing of made frames to
|
||||||
|
RTSS, and Reflex is given no frame limit. A limit lower than the profile's waits 5 s before it is written, a higher one
|
||||||
|
1 s, so a multiplier that changes in play does not rewrite the profile each time. What the profile held is kept in
|
||||||
|
`triforce_rtss_profile.txt` in the game folder before the first change (one line per executable, the others' lines
|
||||||
|
kept, a file that does not read left alone) and goes back at the exit; the next start puts it back after a kill or a
|
||||||
|
crash, unless the player changed the profile since. The game's own wait holds the frames instead with frame generation
|
||||||
|
on Auto (its presents follow the screen's refresh, not the game, so no limit on them holds the game) and with DLSS frame
|
||||||
|
generation (Streamline presents on a pace of its own: `HOS_RTSS_DLSSG=1` tries RTSS there), during movies (the profile
|
||||||
|
keeps the play limit), while no picture went out for 200 ms (a minimised window), while RTSS is not installed, not
|
||||||
|
running or has not hooked the game, and when the profile does not keep the limit (no administrator rights) or RTSS
|
||||||
|
refuses it, unless a limit the player set in RTSS, at or under the game's, holds the game (then RTSS holds movies too).
|
||||||
|
A hold needs a present since the frame start two frames back, so a window that stops showing gives the frames back
|
||||||
|
within two frames. RTSS's count past 1.05 times the limit three
|
||||||
|
periods running, 30 s hooked with the limit and no hold, or the game's frames coming sooner than the cap give the frames
|
||||||
|
back; RTSS is tried again a minute later (three times), and at once when RTSS starts again or another limit is wanted.
|
||||||
|
The game never starts RTSS, installs nothing and asks for no rights. `vk: pace:` lines in the log say which limiter is in
|
||||||
|
force and why; the Full overlay's Frame limit line names RivaTuner or NVIDIA Reflex after the rate when either holds the frames.
|
||||||
|
|
||||||
## Layout
|
## Layout
|
||||||
|
|
||||||
@@ -216,9 +479,10 @@ The other rows are 0 Off and 1 On. Some are read at the start, before the menu e
|
|||||||
| `gpu/vk/` | the Vulkan back end (`hookshot_vk.dll`, `hookshot_vk.so`), with Eden's shader recompiler and sirit |
|
| `gpu/vk/` | the Vulkan back end (`hookshot_vk.dll`, `hookshot_vk.so`), with Eden's shader recompiler and sirit |
|
||||||
| `gpu/render/`, `gpu/third_party/` | the WebGPU renderer the Windows exe falls back to when it was built with wgpu-native, its op stream (`rx_nvn.h`), suyu's decoders |
|
| `gpu/render/`, `gpu/third_party/` | the WebGPU renderer the Windows exe falls back to when it was built with wgpu-native, its op stream (`rx_nvn.h`), suyu's decoders |
|
||||||
| `builder/` | Hookshot Builder: the Windows app (.NET 8 WPF), its setup and MSI, and the Linux builder and launcher scripts |
|
| `builder/` | Hookshot Builder: the Windows app (.NET 8 WPF), its setup and MSI, and the Linux builder and launcher scripts |
|
||||||
|
| `setup/` | Triforce Setup, the program players run (.NET Framework 4.8, WPF): `Setup.cs` installs, updates and runs `make_game.py`, `Shell*.cs` and `Main.xaml` are its window, `Guard.cs` is what keeps a run from damaging an install, `Mods.cs` the mods folder as the Mods page lists and changes it, `Steam.cs` and `SteamArt.cs` the Add to Steam shortcut and its pictures, `Sound.cs` the game's own decide and cancel sounds as Setup plays them (decoded in memory from the player's game folder: no sound is shipped; `TRIFORCE_SOUND_LOG=<file>` writes down, in a test run, the sounds that would have played) ([CONTRIBUTING.md](../CONTRIBUTING.md#triforce-setup)) |
|
||||||
| `scripts/` | test runs, the profile and log readers, the save copier (`oot_saves.py`), release tooling, `scripts/linux` (dependencies, AppImage) |
|
| `scripts/` | test runs, the profile and log readers, the save copier (`oot_saves.py`), release tooling, `scripts/linux` (dependencies, AppImage) |
|
||||||
| `tests/` | the game-free checks, all run by `tests/run_all.py` |
|
| `tests/` | the game-free checks, all run by `tests/run_all.py` |
|
||||||
| `third_party/` | a64-kit (Hookshot's few changes marked `(hookshot)`), libopus, zbic (Atmosphere's zstd with ZBIC, for unpacking only), SDL2's licence, Dear ImGui and its Vulkan backend, zstd's decoder, the Libertinus fonts |
|
| `third_party/` | a64-kit (Hookshot's few changes marked `(hookshot)`), libopus, zbic (Atmosphere's zstd with ZBIC, for unpacking only), SDL2's licence, Dear ImGui and its Vulkan backend, zstd's decoder, the Libertinus fonts, SMAA's reference shader and lookup textures (the Anti-aliasing row's SMAA Ultra) |
|
||||||
|
|
||||||
License: GPL-3.0-or-later, as mkw-recomp: a64-kit's translator is suyu's (GPL-2.0-or-later) and Eden's code is
|
License: GPL-3.0-or-later, as mkw-recomp: a64-kit's translator is suyu's (GPL-2.0-or-later) and Eden's code is
|
||||||
GPL-3.0.
|
GPL-3.0.
|
||||||
|
|||||||
@@ -29,3 +29,24 @@ It needs Vulkan headers but no GPU, SDK vendor runtime or game assets.
|
|||||||
Injected failures in these tests do not prove actual driver OOM recovery or
|
Injected failures in these tests do not prove actual driver OOM recovery or
|
||||||
device-loss handling. A successful game smoke validates integration, not a fault
|
device-loss handling. A successful game smoke validates integration, not a fault
|
||||||
injection into the player's GPU.
|
injection into the player's GPU.
|
||||||
|
|
||||||
|
## AMD FSR 4's frame generation
|
||||||
|
|
||||||
|
The Frame generation row's AMD FSR 4 (`vk_fsr4.cpp`, `fsr4fg`) holds one frame generation context of AMD's library and
|
||||||
|
one image the made frame is written into. They follow the same rules:
|
||||||
|
|
||||||
|
- The image is made first, then the context; both are published together, only when both exist. A failure destroys
|
||||||
|
what was made, turns AMD's frame generation off for the session (the row then shows what runs, 2x by the renderer's
|
||||||
|
own interpolation) and is logged once. Nothing is retried per frame.
|
||||||
|
- A context is replaced (another picture size, a scene larger than it was made for, another kind of depth range) before
|
||||||
|
anything of the new frame is recorded: the queue and the device idle, AMD's context destroyed, then the image.
|
||||||
|
- A context that failed while a frame was being recorded stays until the second present after it, since that frame's
|
||||||
|
commands name it and are not on the queue yet; a context the row has left stays 120 presents. Both go at the
|
||||||
|
present's tick (`framegen_present_tick`), in the same order.
|
||||||
|
- At the process's exit (`HOS_OPT_EXITING`) nothing of AMD's is destroyed and nothing of it is called again: there is no
|
||||||
|
swapchain or thread of AMD's to stop, and the context goes with the process, as the upscaler's does.
|
||||||
|
|
||||||
|
The order is one function, `framegen_rows.h` `teardown`, which the real code and `tests/test_framegen_fsr4.py` both
|
||||||
|
run; the test also runs the real section against a stand-in for AMD's library and checks each of the cases above. It
|
||||||
|
does not prove what AMD's library or a driver does with these calls: nobody has run them on a card with FSR 4's frame
|
||||||
|
generation.
|
||||||
|
|||||||
67
docs/launcher-color-scheme.md
Normal file
67
docs/launcher-color-scheme.md
Normal file
@@ -0,0 +1,67 @@
|
|||||||
|
# The launcher's color scheme
|
||||||
|
|
||||||
|
Settings > Launcher has a **Color scheme** row under Music and Sounds. Its value is a diamond swatch and the hex
|
||||||
|
(`#E0B23E`). Selecting it (mouse, Enter, A) drops a picker open under the row, with the launcher's own .25 s height
|
||||||
|
transition; selecting the row again, or Escape / B, closes it. The picker has a saturation / brightness square, a hue bar
|
||||||
|
(one of the launcher's sliders), **Hex**, **R**, **G** and **B** fields checked as typed (six hex digits with or without
|
||||||
|
`#`, whole numbers 0-255; a wrong value gets a red border and a line saying what is allowed), eight ready colors with
|
||||||
|
Triforce gold first, and **Reset to gold**.
|
||||||
|
|
||||||
|
The color takes the place of the gold everywhere the launcher uses it: text and links, tab underlines, section titles,
|
||||||
|
sliders and their diamonds, borders, hairlines, outlines, glows, the focus ring, the build line and the wordmark. The
|
||||||
|
polished bars (Play, "Find my best settings", "Build the game", Apply, ...) keep their gradient, inner hairline, sheen and
|
||||||
|
emboss, all made from the new color, and their text takes dark or light ink, whichever has more contrast. Triforce gold
|
||||||
|
gives the look exactly as it was: with it the page sets nothing, and the stylesheets' own values draw it.
|
||||||
|
|
||||||
|
Two limits keep any color usable, both far from gold:
|
||||||
|
|
||||||
|
- **Text and lines never go too dark.** What is drawn in the color on the dark ground is lightened until it reads at
|
||||||
|
4.5:1 or better on `#0a0908` (black draws them in `#787878`).
|
||||||
|
- **Polished bars never go too light.** They are shaded no lighter than 78% lightness, so a near-white color gives a
|
||||||
|
silver bar, not a flat white one.
|
||||||
|
|
||||||
|
## How it works
|
||||||
|
|
||||||
|
- **theme.css** holds the gold as a family of `r, g, b` variables (`--gold-rgb`, `--gold-hi-rgb`, ... and the bars'
|
||||||
|
own `--gold-bar-*-rgb`), each with today's value; `v3.css`, `proto.css` and `app.css` use them in place of every gold
|
||||||
|
literal, alphas kept.
|
||||||
|
- **app.js** ("settings: Color scheme") moves each of today's shades from gold to the chosen color: the hue turns by as
|
||||||
|
much, the saturation scales by as much, the lightness keeps its share of the way to white or to black. At gold it
|
||||||
|
gives back every literal exactly. `paintColor()` sets the variables on `<html>` with the class `tint`, and removes them
|
||||||
|
all for gold. The page adds the row itself after `sfx`; a row the launcher sends with `id: 'color'` takes its place.
|
||||||
|
- **Pictures.** The crest (`img/crest.png`) is redrawn under `html.tint` with `hue-rotate / saturate / brightness`,
|
||||||
|
masked by `img/crest-gold.png` (its metal only), so the three stones stay. The title ornaments and glows get the same
|
||||||
|
filter chain from `--tint-h/-s/-b`. The game's own logo and its glow are the game's art and are not tinted. CSS
|
||||||
|
`hue-rotate` is an approximation: far turns (blue, violet) come close but not exact.
|
||||||
|
|
||||||
|
## How it is kept
|
||||||
|
|
||||||
|
1. The page sends `{t:'cmd', c:'color', hex:'#rrggbb'}`, 400 ms after the last change (as `sound`), and keeps the same
|
||||||
|
value in `localStorage['triforce.color']`, which it reads at start before the first screen.
|
||||||
|
2. The launcher takes only a `#rrggbb` (`Logic/LauncherColor.cs`, `FromCommand` / `Parse`), in lower case. It takes it
|
||||||
|
as the message arrives (`App/Backend.Core.cs`, `ColorArrived`, called from `OnMessage` before the command goes to a
|
||||||
|
worker), into `LauncherColorChoice`, which holds the player's last color while the launcher runs. The command's
|
||||||
|
worker (`On("color")`, `Keep`) then keeps the newest one as `Settings.Color`: `HKCU\Software\Triforce\Color` on
|
||||||
|
Windows, `Color=` in `builder.ini` on Linux, `registry.txt` in a test run. Anything else is left out, with one log
|
||||||
|
line a run.
|
||||||
|
3. The launcher sends back `LauncherColorChoice.Current` (the player's last color, else the kept one) as `state.color`
|
||||||
|
at start (`OnReady`, beside `sound`) and with the list the Settings page asks for each time it opens
|
||||||
|
(`settings.list`). The page then uses it, and it wins over the page's own storage, unless a choice of the player's is
|
||||||
|
still waiting to be sent. Because the launcher sends what it holds and not what it has written, a `--snapshot` run
|
||||||
|
(which writes nothing) and a `settings.list` whose worker runs before the `color` sent just ahead of it both send
|
||||||
|
the new color, not the old one.
|
||||||
|
4. A settings file from before the color has no `Color`: it reads as Triforce gold (`#e0b23e`), which is also what
|
||||||
|
**Reset to gold** sends. An uninstall removes it with the other settings.
|
||||||
|
|
||||||
|
## Checks
|
||||||
|
|
||||||
|
- `dotnet run --project launcher/LogicTests color`: the parsing of the page's message, the keeping and reading back in
|
||||||
|
`registry.txt` and `builder.ini`, old files, a bad kept value, `--snapshot`'s no writes, and the color sent back
|
||||||
|
under `--snapshot` and when the commands' workers run out of order.
|
||||||
|
- `python -I -B tests/test_launcher_color.py`: the page and the launcher agree on the message, the default and the image,
|
||||||
|
the image rides in the exe (the `ui/**` embed), and every word the page shows spells "color": the page's own
|
||||||
|
(`app.js`, `demo.js`, `index.html`) and every string in the launcher's C# (`App`, `Logic`, `Core`), the game options'
|
||||||
|
help the Settings rows show as their tooltips among them. That help is the game menu's own words
|
||||||
|
(`gpu/vk/vk_menu.cpp`, which "game options: schema matches vk_menu.cpp" compares); the menu's own table is spelled
|
||||||
|
"color" by the in-game change set, not by this one.
|
||||||
|
- `node --check launcher/ui/app.js`.
|
||||||
43
docs/launcher-music.md
Normal file
43
docs/launcher-music.md
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
# The launcher's music
|
||||||
|
|
||||||
|
The launcher plays the player's own game's music: Saria's Song on the reveal, the Temple of Time while it builds, the
|
||||||
|
item fanfare when the build is done, and the Fairy Fountain on the main screens (`MUSIC` in `launcher/ui/app.js`).
|
||||||
|
Nothing of it is in the repository or the exe: `launcher/Core/GameFiles` decodes the game's BWAV files into WAV files
|
||||||
|
in `%LOCALAPPDATA%\Triforce\launcher\media` (`<track>-<key>.wav`), and the page asks for them as `audio/<track>.wav`.
|
||||||
|
|
||||||
|
## The loop
|
||||||
|
|
||||||
|
Each of the three songs plays an intro once and then loops from a sample after it to its end, as in the game:
|
||||||
|
|
||||||
|
| Track | File | Loop |
|
||||||
|
|---|---|---|
|
||||||
|
| Saria's Song | the eight `BGM_LostWood_0` .. `_7` layers, mixed | after a short intro, to the end (every layer the same) |
|
||||||
|
| Fairy Fountain | `BGM_GreatFairyFountain` | after a long intro, to the end |
|
||||||
|
| Temple of Time | `BGM_ShrineOfTime_Child` | from a fraction of a second in, to the end |
|
||||||
|
| Item get | `BGM_ItemGet_Large` | none: it plays once |
|
||||||
|
|
||||||
|
The BWAV header gives the loop (each channel's end at 0x3C, 0xFFFFFFFF for none, and start at 0x40).
|
||||||
|
`Bwav.Decode` reads it, `Bwav.Mix` keeps it when every layer has the same one, and `Bwav.ToWav` writes it as a `smpl`
|
||||||
|
chunk after the samples: one forward loop, its last sample counted in, as the format has it. The samples stay at byte
|
||||||
|
44, so a reader that knows only the plain header still reads the file.
|
||||||
|
|
||||||
|
The page plays music through Web Audio: a track is fetched and decoded once into an `AudioBuffer`, and an
|
||||||
|
`AudioBufferSourceNode` loops it between `loopStart` and `loopEnd`, sample for sample (the context runs at the game's
|
||||||
|
48 kHz, so nothing is resampled). Its volume goes through a `GainNode` with the same fades, volume and mute as before. A
|
||||||
|
WAV without a `smpl` chunk (made by an older launcher) loops whole. Without Web Audio, or for a file it cannot decode,
|
||||||
|
an `<audio>` element plays it as before. The item fanfare starts at its full level (a fade-in would swallow its first
|
||||||
|
notes); a track faded out before it is read never starts, and a track that stops lets go of its decoded samples.
|
||||||
|
|
||||||
|
Before, an `<audio>` element looped the whole file: the intro came back on every loop, the end jumped to the start with
|
||||||
|
a step in the wave, and the element paused between the end and the start.
|
||||||
|
|
||||||
|
A launcher with this change decodes the sounds again once (the WAV cache's key has `SoundVersion` in it, and the files
|
||||||
|
made before are removed); the title art keeps its files.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
- `dotnet run --project launcher/LogicTests media`: the loop read from a BWAV the test writes, kept by the mix, written
|
||||||
|
as `smpl` and read back, and the cache made again.
|
||||||
|
- `python -I -B tests/test_launcher_music.py`: the page's music code run in node with stand-ins for Web Audio, fetch and
|
||||||
|
`<audio>` (`tests/launcher_music_test.js`), and the page and the launcher checked to read and write the loop at the
|
||||||
|
same place.
|
||||||
185
docs/lazy-writeback.md
Normal file
185
docs/lazy-writeback.md
Normal file
@@ -0,0 +1,185 @@
|
|||||||
|
# Lazy write-back: built, on by default
|
||||||
|
|
||||||
|
**Since options_version 5 (2026-11-16): on by default, at the owner's word, with the row and `RX_VK_LAZY_WB` to
|
||||||
|
turn it off; still never on AMD's driver by the row alone. The status below is from before.**
|
||||||
|
|
||||||
|
**Status (2026-11-15): built behind the hidden Debug tab's Lazy write-back row, off unless set, and `RX_VK_LAZY_WB`
|
||||||
|
(`=1` on, `=0` off, `=2` the self-check) over it. Nobody has run it in the game.** The design below is what was built;
|
||||||
|
"As built" at the end says how, and what only the game can show. The note was first written when it was studied for the
|
||||||
|
port of NX2navi's features and left out, because in this code base it cannot be proven correct without running the
|
||||||
|
game; the owner then asked for it to be built all the same, off by default, so that it can be turned on and compared.
|
||||||
|
The reasons in "Why it is not built" still stand as the things to watch for.
|
||||||
|
|
||||||
|
## What it is
|
||||||
|
|
||||||
|
What a shader writes into a mirrored pool lives in the pool's VRAM mirror. At the end of every submission the renderer
|
||||||
|
copies it back into guest memory, in case the CPU reads it (`gpu/vk/vk_memory.cpp`, the end of
|
||||||
|
`memory_end_submission`). The game's CPU reads very little of it. NX2navi stopped copying a range once it had gone
|
||||||
|
back one time, and copies it only when something touches it: "about a third more frames" in open areas, with 66 MiB a
|
||||||
|
frame no longer crossing PCIe (7 to 9 ms of a frame's 13 to 15 ms of GPU time there).
|
||||||
|
|
||||||
|
How NX2navi knows about a touch: it takes the access away from the range's pages (no read, no write). The first access
|
||||||
|
faults, the fault handler has the renderer copy the range down, gives the pages back and runs the access again
|
||||||
|
(NX2navi's `runtime/memory.c` "Lazy write-back", `runtime/vk_memory.c` `lazy_fetch`). Around that sit a thaw before every
|
||||||
|
kernel call that is handed guest memory, a list of ranges in flight, a window between the replay's last check and its
|
||||||
|
hand-over, a hold after a touch, and a choice by driver (`wb_lazy_choice.h`: off on AMD's driver after two reports of
|
||||||
|
the game ending in play, off outside Windows). It took several releases of play tests to get there, and one of its
|
||||||
|
refinements still loses the device on some loads.
|
||||||
|
|
||||||
|
## What is different here
|
||||||
|
|
||||||
|
- **CPU writes are known late, CPU reads never.** Lifted stores mark their page in a table (`A64_MARK`, the soft
|
||||||
|
write watch) and the renderer collects the marks at the next submission (`memory_collect_written`). A load is a
|
||||||
|
plain pointer access (`A64_PTR`) with no hook. So "copy it when the CPU touches it" has only two possible sources
|
||||||
|
here: page protection, as in NX2navi, or the calls the game makes before it reads.
|
||||||
|
- **Most of the gain is already taken another way.** Since 2026-11-09 a write-back is a compute pass that compares
|
||||||
|
the mirror with a VRAM shadow of guest memory and writes only the words that differ (`wb_diff`, on by default).
|
||||||
|
Measured where NX2navi measured (Hyrule Field, about 69 MB a frame of written ranges): the whole copies took 4.4 ms
|
||||||
|
of the GPU's 13, the pass takes about 0.7 ms. What a lazy write-back could still save here is that pass, the pages
|
||||||
|
it reads into the shadows, and the waits of captures for write-backs still on the GPU (`wb_pages`, the "late"
|
||||||
|
pages of `take_capture`). Nobody has measured how much that is.
|
||||||
|
- **The order in which the GPU's words reach guest memory is tuned by play.** The exposure meter is a sum the GPU
|
||||||
|
adds to and the CPU reads and clears; the readback waits in `vk_replay.cpp` `invalidate`, the late pages, the
|
||||||
|
deferred syncs and the report words cut out of write-backs (`memory_cpu_owned`) each fixed a flicker that only
|
||||||
|
showed in the game (white flashes, blinking cells, a dark frame every 15 to 30). A lazy write-back moves every one
|
||||||
|
of those moments.
|
||||||
|
|
||||||
|
## Who reads what is written back
|
||||||
|
|
||||||
|
Everything below would see old bytes while a copy is owed, unless it is made to pay first.
|
||||||
|
|
||||||
|
| Reader | Where | Can it be caught? |
|
||||||
|
|---|---|---|
|
||||||
|
| The game's own code | lifted loads, any thread | Only by a fault. `nvnBufferInvalidateMappedRange` and `nvnMemoryPoolInvalidateMappedRange` reach the renderer (`host/nvn_native.c`, `rx_nvn_invalidate`), and it already waits there for the range's write-back. Whether the game calls one before every read of GPU results is not known; `HOS_NVN_LOG_FLUSH` logs the calls |
|
||||||
|
| NVN event words | `nvnEventGetValue`, `hos_nvn_event_value` (the present poll, `host/hos_thread.c`) read guest memory with no call into the renderer | A hook in the host |
|
||||||
|
| Host code given game buffers | `memcpy` and the like through `GP()`: audio voices on the mixer thread, the movie's packets, the codec, `nn_libc.c` | By a fault, on that thread |
|
||||||
|
| The kernel | `ReadFile` and `WriteFile` straight on guest pointers (`host/nn_fs.c` `read_at` and the write beside it: every file read, every save); the same as `pread`/`pwrite` on Linux | **No.** A call into a page without access fails with an error and no fault (`host/mkw_glue.c` says so of reserved pages). Each needs a thaw before it, as NX2navi has |
|
||||||
|
| The renderer's CPU reads | captures (`capture_make` on the game's thread and its helpers, `guarded_copy`, which drops a page that faults on Windows and is a plain `memcpy` on Linux), `memory_read_as_submitted`, texture uploads (`vk_texture.cpp` `guest_bytes`), shader code, the bank's descriptors | By a fault, but on threads that hold `pools_mu` or `replay_mu`, which a copy must not need |
|
||||||
|
| The GPU through the import | every `vkCmdCopyBuffer` from `Pool::buffer` (`mirror_make`, `mirror_pieces_make`, `mirror_sync`'s live pages, `memory_rb_refresh`, `late_deferred_bound`, the shadows' fill) and every binding `gpu_buffer_bind` or `gpu_buffer` gives the import for | Never by a fault: each site has to pay first or bind the mirror |
|
||||||
|
| Code that takes guest memory for complete | a mirror given back or remade (`mirror_free`, `mirrors_give_back`, `demote_trials`, `pieces_set_direct`, `retire_pool` with its inheritance), `texture_flush_to_guest` | Each has to pay first |
|
||||||
|
|
||||||
|
And three things undo a page's protection behind the renderer's back on Windows: `import_pool` commits its whole
|
||||||
|
range again (`VirtualAlloc(MEM_COMMIT, PAGE_READWRITE)`, which its own comment says resets protection), `hos_commit`
|
||||||
|
does the same for the game's mappings, and `hos_mem.c` discards and decommits pages.
|
||||||
|
|
||||||
|
## A design for Triforce
|
||||||
|
|
||||||
|
The states of a guest page of a mirrored pool, as a write-back sees them:
|
||||||
|
|
||||||
|
| State | Guest memory holds | How it is left |
|
||||||
|
|---|---|---|
|
||||||
|
| current | everything the GPU wrote | a written binding of a later submission: **written** |
|
||||||
|
| written | all but this submission's writes | the submission's end: copied (the diff pass), then current; or, when the page went back once before and nothing forbids it, no copy: **owed** |
|
||||||
|
| owed | an older version; the mirror alone has the GPU's | a reader: **paying**; a later written binding: stays owed, the newer write joins the same debt (one copy pays for all of them, mirror to guest, so they land in order by construction) |
|
||||||
|
| paying | being copied by one thread | the copy done and waited for: current. Every other reader of the page waits for that copy, none starts a second |
|
||||||
|
|
||||||
|
Rules that keep it correct:
|
||||||
|
|
||||||
|
1. A page is owed only whole, only in a pool with a mirror in memory there (not a piece left in guest memory), only
|
||||||
|
when the mirror holds every CPU write to it (`page_gen(page) <= synced_gen`: a copy from the mirror would
|
||||||
|
otherwise put older bytes over the CPU's), and never where the replay writes words itself (the report words).
|
||||||
|
2. Every reader in the table above pays before it reads. For CPU reads that means the page has no access while it is
|
||||||
|
owed and the fault handler pays; for the kernel a thaw before the call; for the GPU's reads and for code that gives
|
||||||
|
a mirror up, a call that pays the range first.
|
||||||
|
3. A CPU write to an owed page pays first, like a read (the write faults). It does not drop the debt: the write
|
||||||
|
covers part of the page and the rest is the GPU's. Only a write of the whole page, which nothing here can
|
||||||
|
recognise, could drop it. The page then goes the usual way (marked, collected, synced into the mirror), and is not
|
||||||
|
owed again for a while (NX2navi holds it 1 s, doubling to 32 s while it keeps being touched).
|
||||||
|
4. A payment copies after everything submitted so far, in a command buffer of its own that the paying thread waits
|
||||||
|
for; the part being recorded checks at its end whether a range it left owed was paid meanwhile and copies it then.
|
||||||
|
5. The shadows of the write-back diff stay true without change: an owed page was not written by the pass, so the
|
||||||
|
shadow still equals guest memory, and a payment is the same pass run later.
|
||||||
|
6. A capture's wait (`wb_pages`) has nothing to wait for on an owed page: no submission writes it back. The wait
|
||||||
|
moves into the payment.
|
||||||
|
|
||||||
|
The page-state logic belongs in a header of its own with tests for the rules above (each reader pays first; a CPU
|
||||||
|
write pays and does not drop; debts merge; one payer at a time), as the write generations have theirs
|
||||||
|
(`gpu/vk/write_gen_table.h`).
|
||||||
|
|
||||||
|
## Why it is not built
|
||||||
|
|
||||||
|
None of these can be settled by a test without the game:
|
||||||
|
|
||||||
|
- **The set of kernel accesses.** One missed `ReadFile` into an owed page is a short read handed to the game, with no
|
||||||
|
fault and no log. The list above is what a search of `host/` found, not a proof; libraries and drivers that are
|
||||||
|
handed guest pointers (the movie decoder, audio) were read, not run.
|
||||||
|
- **The driver.** The pages are imported into Vulkan as host memory (`VK_EXT_external_memory_host`; a `udmabuf` on
|
||||||
|
Linux). Taking their access away while the import lives is what NX2navi's AMD reports point at. Importing a pool
|
||||||
|
whose pages have no access is likely to fail outright.
|
||||||
|
- **Locks.** The fault can come on any thread, including the replay thread and the capture helpers while they hold
|
||||||
|
the renderer's locks, and on Linux inside a signal handler. NX2navi's rules for this (ranges in flight, the
|
||||||
|
window, fetches outside its freeze lock) were found by profiles and stalls in play.
|
||||||
|
- **The game's timing.** See "tuned by play" above. Paying at the first touch instead of at the submission's end
|
||||||
|
changes when the exposure meter, the event words and the culling's results appear to the CPU.
|
||||||
|
- **Whether it is worth it.** With the diff pass in place the saving is unmeasured and may be small.
|
||||||
|
|
||||||
|
What was considered and rejected as a shortcut: paying only at `nvn*InvalidateMappedRange` and the host's own reads,
|
||||||
|
with no page protection. It needs no fault handler and no thaw, but it is right only if the game announces every read,
|
||||||
|
which nobody knows; a read that is not announced gets old data silently. `RX_VK_WB_MAX_KB` is the same bet made by
|
||||||
|
size, and is off by default for that reason.
|
||||||
|
|
||||||
|
## What to do first, in the game
|
||||||
|
|
||||||
|
1. Measure what is left to gain: `RX_VK_PASS_TIMING` gives the GPU time of "wb diff" and "wb copies" a frame, and the
|
||||||
|
log's "write-back diff (600 submits)" and "a capture taken" lines give the bytes compared and the takes that waited
|
||||||
|
for late pages. `RX_VK_NO_WRITEBACK=1` is the upper bound on frame rate (its picture is wrong wherever the CPU reads
|
||||||
|
results: do not judge the picture with it).
|
||||||
|
2. If that is worth the work: log which written-back ranges the game announces before reading
|
||||||
|
(`HOS_NVN_LOG_FLUSH`, `RX_VK_LOG_INVALIDATE`, `RX_VK_LOG_WRITEBACK`), and build the page probe NX2navi used before
|
||||||
|
its lazy write-back (protect a written-back range, log the first access with its thread and address, give the page
|
||||||
|
back): that turns "the CPU does not read most of it" from NX2navi's game build into a fact about this one, and
|
||||||
|
finds the kernel accesses on the way.
|
||||||
|
3. Then the host side (the fault hook, a thaw before `read_at` and the save writes, the commits that reset
|
||||||
|
protection), the renderer side (rules 1 to 6), a Debug tab row that is off by default, and the same choice by
|
||||||
|
driver NX2navi ended with.
|
||||||
|
|
||||||
|
## As built (2026-11-15)
|
||||||
|
|
||||||
|
Steps 1 and 2 above were not done first: the owner asked for it built behind a row, so that the first run with it on
|
||||||
|
is the measurement. What is where:
|
||||||
|
|
||||||
|
- **The page states**, `gpu/vk/wb_lazy_state.h`, with the rules and a threaded stress test in
|
||||||
|
`tests/test_wb_lazy_state.py`. A page is not owed at once: it is first WATCHED (read-only, still copied at every
|
||||||
|
submission) until a collection of the write watch and a capture, both numbered after the watch began, have come
|
||||||
|
and named no CPU write to it. That is rule 1 for the writes the renderer learns of late; the renderer adds the rest
|
||||||
|
of it: the CPU's newest write generation for the page no newer than its pool's `synced_gen` (`lazy_page_takes`).
|
||||||
|
That asks `cpu_gens`, the CPU's writes alone, and not the one table of generations: the first build asked that one,
|
||||||
|
where every storage buffer a draw or a dispatch writes is stamped while it is recorded (`memory_stamp_gpu`) and
|
||||||
|
`synced_gen` is raised only at the submission's end, so every page being written back looked newer than its sync
|
||||||
|
and the owner's first session with the row on watched and owed nothing (`tests/test_lazy_wb_cpu_gens.py`). A CPU write to a watched page faults and ends the watch with no copy; to an owed
|
||||||
|
page it pays first. A payment between a recording's end and its submit leaves the rest PENDING until that submit
|
||||||
|
(`memory_lazy_submitted`, called from `submit_recording` under the queue's lock). A touch holds its 64 KB block
|
||||||
|
for a second, doubling to 32 s, as NX2navi does. Whether it runs: `wb_lazy::choose`.
|
||||||
|
- **The renderer**, `gpu/vk/vk_memory.cpp` "lazy write-back": `lazy_runs` at each write-back (after the CPU's words
|
||||||
|
are cut and the runs clipped to a mirror's resident pieces; never a page with event words), the payment (`lazy_pay`:
|
||||||
|
its own command buffer after everything submitted, the held submits flushed first, waited for; the shadows told of
|
||||||
|
the pages, rule 5), `lazy_settle` before every copy out of an import and every binding given the import (rule 2 for
|
||||||
|
the GPU), and before a mirror, a piece, a retired pool or an import is given up; `import_pool` holds the pages
|
||||||
|
while it commits and imports. The fault handler is the process's first vectored handler, added when the renderer
|
||||||
|
starts, ahead of host/mkw_glue.c's. Locks: see the section's comment (a payment needs neither `pools_mu` nor
|
||||||
|
`replay_mu`, and logs nothing). `tests/test_lazy_wb_hooks.py` holds these hooks in place.
|
||||||
|
- **Several payments at once** (2026-11-16, NX2navi's `wb_lazy_unlocked`): each payment holds one of four slots, a
|
||||||
|
command buffer and a fence of its own (`gpu/vk/wb_lazy_payers.h`); it was one payer at a time, so a thread that
|
||||||
|
touched another owed page, or the replay giving a mirror up, waited for the first one's copy before its own went onto
|
||||||
|
the queue. Two payments never have the same pages (the table's PAYING has one payer), and each copy still goes onto
|
||||||
|
the one queue after everything before it. `RX_VK_LAZY_PAYERS=1`: one at a time again. The threaded stress test runs
|
||||||
|
with four slots and with one. What it does not change: a payment for a mirror given up still runs on the replay
|
||||||
|
thread under `pools_mu` (NX2navi moved its own out of its pool lock as well); moving it would mean reworking every
|
||||||
|
give-up path, for payments of which the owner's sessions have a few a minute.
|
||||||
|
- **The host**: `hos_guest_io_begin` / `hos_guest_io_end` (`host/rx_backend.c`) around every kernel call on guest
|
||||||
|
memory and every change of guest pages' state: `read_at` and the file writes (the saves too) in `host/nn_fs.c`, the
|
||||||
|
image's read, `hos_commit`, the discards and decommits of `host/hos_mem.c` and `host/hos_svc.c`, the host zstd
|
||||||
|
decoder's one-shot, and the debug dumps. `tests/test_lazy_wb_host.py` reads `host/*.c` for kernel calls that do
|
||||||
|
neither, and every exception is in its list with a reason. Audio, the movie and the codec reach guest memory with
|
||||||
|
the CPU only (`memcpy` and the like): their touches fault and pay.
|
||||||
|
- **Linux**: off (`wb_lazy::choose`), the row not shown. **AMD's driver**: the row alone leaves it off;
|
||||||
|
`RX_VK_LAZY_WB=1` turns it on there.
|
||||||
|
- **The self-check**, `RX_VK_LAZY_WB=2`: guest memory is the eager copy (every copy still made); a payment reads the
|
||||||
|
mirror's pages, what it would have copied, and compares them with guest memory. A difference is logged with its
|
||||||
|
address at the next 600-submit line (`MISMATCH at guest`), not from inside the fault.
|
||||||
|
|
||||||
|
What only the game can show: whether the driver keeps an import whose pages have no access (NX2navi's AMD reports),
|
||||||
|
whether the list of kernel calls is whole in practice (a missed one is a short read with no fault), how often the game
|
||||||
|
touches what is owed (a touch costs a GPU round trip on the touching thread), and the timing of the exposure meter and
|
||||||
|
the event words. The 600-submit line `vk: lazy write-back (600 submits)` says what it saved and who paid; with
|
||||||
|
`RX_VK_LAZY_WB=2` the `check` line says whether it would have been right.
|
||||||
@@ -25,3 +25,12 @@ These checks use no driver or real cache and do not reproduce issue #1022's repo
|
|||||||
its root cause, validate saving a multi-gigabyte driver cache, force an OS thread-resource failure, or change cache
|
its root cause, validate saving a multi-gigabyte driver cache, force an OS thread-resource failure, or change cache
|
||||||
size/retry policy. The existing wait-then-store protocol for concurrent forced callers is unchanged; the current
|
size/retry policy. The existing wait-then-store protocol for concurrent forced callers is unchanged; the current
|
||||||
force call is in prepare-only startup. Driver/native faults and thread-detach failures are outside this change.
|
force call is in prepare-only startup. Driver/native faults and thread-detach failures are outside this change.
|
||||||
|
|
||||||
|
## The file's bound
|
||||||
|
|
||||||
|
The cache's size has a rule since: `gpu/vk/pcache_bound.h`. A file over the bound is not read (the cache starts
|
||||||
|
empty), and data over the bound is neither fetched nor written, so the file keeps its last save
|
||||||
|
(`RX_VK_PCACHE_MAX_MB`, in the development guide's switches). The fixture's `over-bound` case runs the save's side
|
||||||
|
against the actual function: under and at the bound the data is written as ever, over it the old file stays, in a
|
||||||
|
forced save and on the worker, and with the bound lifted the same data is written. The read's side is
|
||||||
|
`tests/test_pcache_bound.py`. A read that ends early is refused whole by the header; no test makes a read end early.
|
||||||
|
|||||||
@@ -109,6 +109,31 @@ that only exists on that side does not stay mixed in with the new save.
|
|||||||
- **Backups.** The script keeps one for every copy. Delete old ones when you no longer need them.
|
- **Backups.** The script keeps one for every copy. Delete old ones when you no longer need them.
|
||||||
- Start each program once and save in it before the first copy, so that its save folder exists.
|
- Start each program once and save in it before the first copy, so that its save folder exists.
|
||||||
|
|
||||||
|
## A save as one zip, from the game itself
|
||||||
|
|
||||||
|
The System tab of the game's Options has **Export save**, **Export save for a bug report** and **Import a save**
|
||||||
|
(`host/save_share.c`; the readme's *Export and import a save*). They move one zip, not a folder:
|
||||||
|
|
||||||
|
```
|
||||||
|
manifest.json "format": "triforce-save", "format_version": 1, the game's title id, the Triforce version, the export time (UTC)
|
||||||
|
save/ account.sav, option.sav, slot_00/ ... as above
|
||||||
|
bug-report/ in a bug report: triforce_log.txt, triforce_log.1.txt, triforce_options.ini, triforce_hang.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
- The export is the save the next start would load, read with no save in progress (`host/nn_fs.c` `hos_fs_save_read`):
|
||||||
|
a finished commit still waiting in `user/save-tx/<user id>/ready-<n>`, else `user/save/<user id>`. It goes to the
|
||||||
|
Downloads folder (`HOS_SAVE_EXPORT_DIR=<folder>`: there instead), or beside the game when that cannot be written.
|
||||||
|
- The import takes the newest `.zip` of the game folder's `save-import` folder: this format or NX2navi's own
|
||||||
|
(`"format": "nx2navi-save"`, the same layout). It is checked whole first (`host/save_pack.c`: the manifest, every path
|
||||||
|
inside the save, no links, sizes, each file's CRC-32) and refused otherwise. Its save then waits in
|
||||||
|
`user/save-tx/<user id>/import` and the zip moves to `save-import/imported`.
|
||||||
|
- The next start copies the save about to be replaced to `user/save-backup/<user id>/import-<UTC date>-<time>` (kept until
|
||||||
|
you delete it: the five rolling backups beside it are the saves' own), then makes the import the save
|
||||||
|
(`host/save_tx_share.h`). Choosing the row again before that start cancels it.
|
||||||
|
- Triforce Setup's Convert saves, used after an import was left waiting, drops the waiting import: what was converted last
|
||||||
|
is the save.
|
||||||
|
- With `HOS_SAVE_TX=0` the game writes its save in place, and nothing is exported or imported in that run.
|
||||||
|
|
||||||
## What was tested
|
## What was tested
|
||||||
|
|
||||||
- Linux (Arch-based), Hookshot v2.0.5 and Lichii built from source on the same day.
|
- Linux (Arch-based), Hookshot v2.0.5 and Lichii built from source on the same day.
|
||||||
|
|||||||
@@ -38,8 +38,8 @@ Agents may add and re-score rows; a person reviews the change like any other PR.
|
|||||||
| 0.29 | Amd RX 580 (Polaris, no mesh shaders): hair renders white and bright, and the game crashes when Link moves; a second RX 580 8 GB user reports the same | S1 | 0.145 | 0.4 | | #bug-reporting "Amd rx 580 crashes" (no log yet) |
|
| 0.29 | Amd RX 580 (Polaris, no mesh shaders): hair renders white and bright, and the game crashes when Link moves; a second RX 580 8 GB user reports the same | S1 | 0.145 | 0.4 | | #bug-reporting "Amd rx 580 crashes" (no log yet) |
|
||||||
| 0.24 | Handheld with no keyboard (Asus ROG Xbox Ally X, controller only): fullscreen and the F1 options menu cannot be reached; no log yet | S1 | 0.12 | 0.4 | | #bug-reporting "Xinput devices cannot enter fullscreen or menu" |
|
| 0.24 | Handheld with no keyboard (Asus ROG Xbox Ally X, controller only): fullscreen and the F1 options menu cannot be reached; no log yet | S1 | 0.12 | 0.4 | | #bug-reporting "Xinput devices cannot enter fullscreen or menu" |
|
||||||
| 0.12 | AMD RX 9070 XT: texture glitch that appears after some play (since v2.0.5); same as the tracked AMD texture artifact | S2 | 0.10 | 0.4 | | #bug-reporting "texture glitch on 9070 XT aris"; #57, #1006 |
|
| 0.12 | AMD RX 9070 XT: texture glitch that appears after some play (since v2.0.5); same as the tracked AMD texture artifact | S2 | 0.10 | 0.4 | | #bug-reporting "texture glitch on 9070 XT aris"; #57, #1006 |
|
||||||
| 0.11 | Intel GPU: black screen after loading a save (or on the Deku Tree), sound only, very low fps; three reports, no logs yet; tracked in the GitHub issue linked in #bug-reporting replies (#999) | S0 | 0.035 | 0.4 | | #bug-reporting "Intel G3E. Title and intro loads...", "intel arc b580 problems", "game gets stuck on a black screen when loading a save (intel gpu... gulp...)" |
|
| 0.11 | Intel GPU: black screen after loading a save (or on the Deku Tree), sound only, very low fps; three reports, no logs yet; tracked in the GitHub issue linked in #bug-reporting replies (#999). What a log would settle: `vk: pool ... not imported (VkResult ...)` lines (the driver refusing host memory, whose bindings then draw nothing) against a `device lost` report. Not a case for running without the host-memory import: the renderer has no such path (`memory_init` in `gpu/vk/vk_memory.cpp` stops without it), and NX2navi's Intel branch, which tried one, found it drew the 3D black and went back | S0 | 0.035 | 0.4 | | #bug-reporting "Intel G3E. Title and intro loads...", "intel arc b580 problems", "game gets stuck on a black screen when loading a save (intel gpu... gulp...)" |
|
||||||
| 0.11 | Crash at the door after the owl intro (Navi wakes Link, 2.0.13): access violation in `igvk64.dll` at start-up; GPU not given yet, reach uses Intel's 3.5% until it is | S0 | 0.035 | 0.4 | | #bug-reporting "2.0.13 Crash after you see the owl during intro cutscene" (5 logs) |
|
| 0.11 | Crash at the door after the owl intro (Navi wakes Link, 2.0.13): access violation in `igvk64.dll` at start-up; GPU not given yet, reach uses Intel's 3.5% until it is. To check in the logs: the address. `igvk64.dll+0x79ddf3` (read of 0x190) is the driver's `vkGetPipelineCacheData` beside a pipeline build on the same cache, which `pipeline_cache_save` (`gpu/vk/vk_draw.cpp`) has kept apart for the saves in play since 11-12, on Intel; the saves asked for outright (the prepare, the stage libraries' pass, the exit) still fetch the cache whatever else builds. It is not the host-memory import: NX2navi's Intel branch took its crash for that first and found the same address | S0 | 0.035 | 0.4 | | #bug-reporting "2.0.13 Crash after you see the owl during intro cutscene" (5 logs) |
|
||||||
|
|
||||||
|
|
||||||
## Fixed, waiting for a release
|
## Fixed, waiting for a release
|
||||||
|
|||||||
@@ -227,6 +227,11 @@ int nvn_depth_minus_w(void);
|
|||||||
void rx_nvn_submit(const uint64_t *handles, int n);
|
void rx_nvn_submit(const uint64_t *handles, int n);
|
||||||
/* (the same with the queue it came from: the game's queues have their own state and order each other with events) */
|
/* (the same with the queue it came from: the game's queues have their own state and order each other with events) */
|
||||||
void rx_nvn_submit_q(const uint64_t *handles, int n, uint64_t queue);
|
void rx_nvn_submit_q(const uint64_t *handles, int n, uint64_t queue);
|
||||||
|
/* (Triforce, gpu/vk/async_rules.h; host/rx_backend.c) the replay on a thread of the back end's own (RX_VK_ASYNC=1): whether it
|
||||||
|
is on; the events the submit this thread makes next waits for; the present poll's wait for that thread (1: nothing left) */
|
||||||
|
int rx_async_replay(void);
|
||||||
|
void rx_nvn_submit_waits(const uint64_t *events, int n);
|
||||||
|
int rx_nvn_replay_idle_wait(uint64_t timeout_ns);
|
||||||
/* fence value of everything submitted so far; waits until the GPU reached a fence (1: reached, 0: timed out) */
|
/* fence value of everything submitted so far; waits until the GPU reached a fence (1: reached, 0: timed out) */
|
||||||
uint64_t rx_nvn_fence(void);
|
uint64_t rx_nvn_fence(void);
|
||||||
uint64_t rx_nvn_fence_q(uint64_t queue); /* (hookshot) a fence of one queue */
|
uint64_t rx_nvn_fence_q(uint64_t queue); /* (hookshot) a fence of one queue */
|
||||||
|
|||||||
@@ -146,7 +146,9 @@ Designed after totk-pc-recomp's runtime/vk_*.c (MIT) and Eden's renderer_vulkan,
|
|||||||
- **Mesh shading**: `NOP_DRAW_MESH` records NVN2's DrawMeshTasks*; programs whose code writes the internal stage
|
- **Mesh shading**: `NOP_DRAW_MESH` records NVN2's DrawMeshTasks*; programs whose code writes the internal stage
|
||||||
buffer are the task / mesh stages (marked in their headers as Eden's GraphicsEnvironment does). Unverified.
|
buffer are the task / mesh stages (marked in their headers as Eden's GraphicsEnvironment does). Unverified.
|
||||||
- **Frame interpolation** (`vk_framegen.cpp`, `RX_VK_FRAMEGEN=1`): the WebGPU renderer's, ported to GLSL
|
- **Frame interpolation** (`vk_framegen.cpp`, `RX_VK_FRAMEGEN=1`): the WebGPU renderer's, ported to GLSL
|
||||||
(`shaders/`, compiled with glslang): made frames between the game's, shown evenly by a display thread.
|
(`shaders/`, compiled with glslang): made frames between the game's, shown evenly by a display thread. (Triforce) Its
|
||||||
|
vectors are refined to one per 8x8 pixels, to a fraction of a pixel, and each made pixel takes one motion
|
||||||
|
(`fg_half`, `fg_fine_flow`, `fg_fine_pick`, `fg_fine_synth`; `RX_VK_FRAMEGEN_FINE=0`: the 16x16 blocks as ported).
|
||||||
- **Headless**: without a window presents render offscreen; `RX_VK_SHOTS=<dir>` writes every `RX_VK_SHOT_EVERY`-th
|
- **Headless**: without a window presents render offscreen; `RX_VK_SHOTS=<dir>` writes every `RX_VK_SHOT_EVERY`-th
|
||||||
as a PNG. `work/run-vk.ps1` runs it (refuses while Siege runs, stops past `-MaxGB` of working set);
|
as a PNG. `work/run-vk.ps1` runs it (refuses while Siege runs, stops past `-MaxGB` of working set);
|
||||||
`work/vksum.py <run>` summarises the log.
|
`work/vksum.py <run>` summarises the log.
|
||||||
|
|||||||
149
gpu/vk/aa_rows.h
Normal file
149
gpu/vk/aa_rows.h
Normal file
@@ -0,0 +1,149 @@
|
|||||||
|
/* (Triforce) The Insert menu's Anti-aliasing row (vk_menu.cpp), and the rule that finds the game's own anti-aliasing passes
|
||||||
|
among a frame's compute dispatches (vk_draw.cpp aa_stand_in, vk_aa.cpp), as plain functions. No Vulkan, no menu:
|
||||||
|
tests/test_aa_row.py compiles and runs them. After NX2navi's aa setting (its runtime/vk_aa.c and vk_exec_draw.c
|
||||||
|
aa_dispatch).
|
||||||
|
|
||||||
|
OoT anti-aliases every frame of the 3D with SMAA in three compute passes, right after the draw that adds the bloom and
|
||||||
|
tone maps: the edges (4 x 4 pixel tiles), the blending weights from them (an indirect dispatch over the tiles with edges)
|
||||||
|
and the composition, which samples the tone mapped picture and the weights and stores the smoothed picture into the
|
||||||
|
target the gameplay upscale then takes. It stores each channel's square root, the three packed as one RGB9E5 word, through
|
||||||
|
an integer view of that B10G11R11 target (shaders/aa_common.glsl). */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
#include "upscaler_rows.h"
|
||||||
|
|
||||||
|
namespace mkwvk::aa_rows {
|
||||||
|
|
||||||
|
/// The Anti-aliasing row (opts.aa, saved as anti-aliasing; RX_VK_AA over it): the game's own SMAA as it is, none (the tone
|
||||||
|
/// mapped picture stored as it is), FXAA over it (shaders/aa_fxaa.glsl), or SMAA at its Ultra preset in place of the game's
|
||||||
|
/// (shaders/smaa_*.comp). The saved values keep their meaning: a new choice comes after these
|
||||||
|
enum { GAME = 0, OFF = 1, FXAA = 2, SMAA = 3, kValues = 4 };
|
||||||
|
inline const char* name(int v) {
|
||||||
|
switch (v) {
|
||||||
|
case OFF: return "Off";
|
||||||
|
case FXAA: return "FXAA";
|
||||||
|
case SMAA: return "SMAA Ultra";
|
||||||
|
default: return "Game SMAA"; // (also for a value past the row; "Game", the owner's word, 11-16)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/// The row's value as one of its choices, RX_VK_AA=<0..3> over it (env: its number, -1 unset). Anything else is the game's own
|
||||||
|
inline int chosen(int row, int env) {
|
||||||
|
const int v = env >= 0 ? env : row;
|
||||||
|
return v >= GAME && v < kValues ? v : GAME;
|
||||||
|
}
|
||||||
|
/// DLSS, DLAA, FSR 3.1, FSR 4 and their native modes anti-alias the 3D themselves: while one of them draws (upscaler: what
|
||||||
|
/// runs, upscaler_rows.h) the row does not apply
|
||||||
|
inline bool upscaler_own(int upscaler) { return upscaler != upscaler_rows::GAME; }
|
||||||
|
/// What is in effect: the choice with the game's own upscaler. ((Triforce, FSR3-06, FSR4-5) While such an upscaler draws from
|
||||||
|
/// the tone mapped picture (Upscale before bloom off), none: the game's SMAA is left out, as AMD and NVIDIA ask for jittered
|
||||||
|
/// frames no other anti-aliasing has been over (SMAA before them softened what they gather, and its edges, found anew in
|
||||||
|
/// each jittered frame, flickered). It was the game's own SMAA before. Before the bloom (before_bloom) the upscaler takes
|
||||||
|
/// the scene, which the SMAA has not been over: the game's own then, as before. env_set (RX_VK_AA for the run): its choice
|
||||||
|
/// there too, for an A/B (RX_VK_AA=0: the game's SMAA under the upscaler)
|
||||||
|
inline int effective(int choice, int upscaler, bool before_bloom = false, bool env_set = false) {
|
||||||
|
if (!upscaler_own(upscaler) || env_set) return choice;
|
||||||
|
return before_bloom ? (int)GAME : (int)OFF;
|
||||||
|
}
|
||||||
|
/// The row has nothing to choose then (shown without arrows; Left and Right leave the saved value alone), nor with
|
||||||
|
/// RX_VK_AA set for the run
|
||||||
|
inline bool fixed(int upscaler, bool env_set) { return upscaler_own(upscaler) || env_set; }
|
||||||
|
|
||||||
|
// ---- the game's passes ----
|
||||||
|
|
||||||
|
/// The game's three SMAA programs, by their keys (vk_shader.cpp code_key: a hash of the compute stage's control section and
|
||||||
|
/// code as the game hands them over, the same on every PC and with every setting). Game build e0f3ccfa. NX2navi knows the
|
||||||
|
/// same three by another hash of the same code (fcb21f2afa88378c, e44377cbfafc1101, 470d661afb1cb73a in its vk_aa.c): these
|
||||||
|
/// are the programs of a build's shader list (triforce_shaders.bin) with those hashes. Another game version has other keys:
|
||||||
|
/// nothing is then found, and the frame is drawn as before the row (vk_aa.cpp says so in the log)
|
||||||
|
enum { NONE = 0, EDGES = 1, WEIGHTS = 2, COMPOSE = 3 };
|
||||||
|
constexpr uint64_t kEdgesKey = 0x26fe0c70ab9fc547ull, kWeightsKey = 0x685adb41edc19de7ull, kComposeKey = 0x6637ac1d052d22d4ull;
|
||||||
|
constexpr int role(uint64_t key) {
|
||||||
|
return key == kEdgesKey ? EDGES : key == kWeightsKey ? WEIGHTS : key == kComposeKey ? COMPOSE : NONE;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a dispatch has at one of its program's texture or image bindings, as far as the rule looks
|
||||||
|
enum { OTHER = 0, R11 = 1, E5 = 2 }; // (the image's own format: B10G11R11, the scene's and the picture's; E5B9G9R9; anything else)
|
||||||
|
struct Bound {
|
||||||
|
uint64_t image = 0; // (which image: any number that tells two apart; 0 nothing bound)
|
||||||
|
int format = OTHER;
|
||||||
|
uint32_t width = 0, height = 0;
|
||||||
|
bool flat = false; // (a plain 2D image, one sample, seen as 2D at its first level)
|
||||||
|
bool written = false; // (an image binding the program stores into)
|
||||||
|
bool integer = false; // (the program reads or stores it as integer words, not as colours: the variant of the program
|
||||||
|
// that runs is the one translated for the views bound, so this is the view's kind)
|
||||||
|
};
|
||||||
|
/// The composition's two ends among what its dispatch has bound (the program's own bindings, in its order): the target, the one
|
||||||
|
/// flat B10G11R11 (or E5B9G9R9: the same words) image it stores integer words into, and the tone mapped picture, the one flat
|
||||||
|
/// B10G11R11 texture of the target's size it samples as colours that is another image (the weights it samples too are read
|
||||||
|
/// as integer words). src and dst: their places in the lists. why: nullptr when both are found, else what was not as expected;
|
||||||
|
/// the game's own pass then runs, nothing is guessed
|
||||||
|
struct Ends {
|
||||||
|
int src = -1, dst = -1;
|
||||||
|
const char* why = nullptr;
|
||||||
|
};
|
||||||
|
inline Ends compose_ends(const Bound* textures, int ntex, const Bound* images, int nimg) {
|
||||||
|
Ends e;
|
||||||
|
int found = 0;
|
||||||
|
for (int i = 0; i < nimg; i++) {
|
||||||
|
const Bound& b = images[i];
|
||||||
|
if (!b.image || !b.written || !b.integer || !b.flat || (b.format != R11 && b.format != E5) || !b.width || !b.height) continue;
|
||||||
|
if (!found++) e.dst = i;
|
||||||
|
}
|
||||||
|
if (found != 1) return e.dst = -1, e.why = found ? "it stores into more than one such target" : "it stores into no B10G11R11 target as integer words", e;
|
||||||
|
const Bound& d = images[e.dst];
|
||||||
|
found = 0;
|
||||||
|
for (int i = 0; i < ntex; i++) {
|
||||||
|
const Bound& b = textures[i];
|
||||||
|
if (!b.image || b.image == d.image || b.integer || !b.flat || b.format != R11 || b.width != d.width || b.height != d.height) continue;
|
||||||
|
if (!found++) e.src = i;
|
||||||
|
}
|
||||||
|
if (found != 1) return e.src = e.dst = -1, e.why = found ? "it samples more than one picture of its target's size" : "it samples no B10G11R11 picture of its target's size", e;
|
||||||
|
return e;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a dispatch of the game's edge or weight pass is left out: only the composition reads what they make, so they go
|
||||||
|
/// while another choice than the game's own is in effect and the composition before this one was replaced. One that could
|
||||||
|
/// not be replaced runs as the game's, and these two run again from the next frame on, until one is replaced again.
|
||||||
|
/// keep (RX_VK_AA_KEEP=1): never left out
|
||||||
|
constexpr bool leave_out(int role, int mode, bool last_replaced, bool keep) {
|
||||||
|
return (role == EDGES || role == WEIGHTS) && mode != GAME && last_replaced && !keep;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the row says under its help about the passes (vk_aa.cpp aa_state): nothing asked or nothing seen yet, the game's
|
||||||
|
/// composition replaced, or none replaced for a while with another choice than the game's own in effect
|
||||||
|
enum { QUIET = 0, REPLACED = 1, MISSING = 2 };
|
||||||
|
/// The frames with a choice in effect and no composition replaced after which that is said (a minute at 60 a second: a
|
||||||
|
/// loading screen or a menu without the 3D that lasts longer says it too)
|
||||||
|
constexpr uint64_t kMissingFrames = 3600;
|
||||||
|
|
||||||
|
/// What vk_aa.cpp keeps about the game's passes from frame to frame (one for the run, under its lock)
|
||||||
|
struct Track {
|
||||||
|
int mode = GAME; // (the choice in effect in the frame before)
|
||||||
|
bool last_replaced = false; // (the last composition met was replaced: what leave_out asks)
|
||||||
|
int state = QUIET;
|
||||||
|
uint64_t since = 0; // (the frame the choice came into effect, or a composition was last replaced)
|
||||||
|
/// A frame begins with `want` in effect. true: the row's line became MISSING just now (said once in the log)
|
||||||
|
constexpr bool begin(uint64_t frame, int want) {
|
||||||
|
bool missing = false;
|
||||||
|
if (want == GAME) {
|
||||||
|
// (the game's own passes all run from here on: a composition replaced before says nothing about the next choice's
|
||||||
|
// first frame, whose edge and weight passes must run until one is replaced again)
|
||||||
|
last_replaced = false, state = QUIET;
|
||||||
|
} else if (mode != want) {
|
||||||
|
since = frame, state = QUIET; // (another choice: counted anew)
|
||||||
|
} else if (state != MISSING && frame - since >= kMissingFrames) {
|
||||||
|
state = MISSING, missing = true;
|
||||||
|
}
|
||||||
|
mode = want;
|
||||||
|
return missing;
|
||||||
|
}
|
||||||
|
/// A composition met with a choice in effect: replaced, or left to the game
|
||||||
|
constexpr void composed(uint64_t frame, bool replaced) {
|
||||||
|
last_replaced = replaced;
|
||||||
|
if (replaced) since = frame, state = REPLACED;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
} // namespace mkwvk::aa_rows
|
||||||
111
gpu/vk/astc_opaque.h
Normal file
111
gpu/vk/astc_opaque.h
Normal file
@@ -0,0 +1,111 @@
|
|||||||
|
// (Triforce) Which ASTC blocks the GPU decoder (host_shaders/astc_decoder_comp_spv.h, Eden's) writes with alpha 255 in every
|
||||||
|
// texel, told from the block's 16 bytes alone: what lets a decoded ASTC texture be kept as BC1 instead of BC3 (vk_texture.cpp,
|
||||||
|
// tex_bc1.h). tests/test_tex_bc1.py
|
||||||
|
//
|
||||||
|
// The answer has to be the decoder's, not the ASTC specification's: a block the decoder takes for an error is written as
|
||||||
|
// (0, 0, 0, 0), alpha 0, and its endpoint modes with alpha may still decode to 255 everywhere. So this says yes only where the
|
||||||
|
// decoder's own code leaves no other outcome, and no everywhere else (a no keeps the texture BC3, as it was):
|
||||||
|
//
|
||||||
|
// - a void extent: the decoder's IsError takes one for an error when it is the HDR kind (bit 9) or bits 10 and 11 are not both
|
||||||
|
// set, and FillVoidExtentLDR writes the constant's alpha, bits 112 to 127, divided by 65535: 255 only for 0xFFFF. It does
|
||||||
|
// not look at the extent's coordinates, so neither does this.
|
||||||
|
// - any other block, the errors in DecompressBlock's order: the reserved block modes (bits 0 to 3 zero; bits 0 and 1 zero
|
||||||
|
// with bits 6 to 8 set), a weight grid larger than the block, four partitions with two planes, more than 64 weights, the
|
||||||
|
// weights' bits under 24 or over 96, and no room left for the colour data (its count of the remaining bits wraps).
|
||||||
|
// - then the colour endpoint modes. Modes 0, 1, 6, 8 and 9 (luminance and RGB, low dynamic range) give both endpoints the
|
||||||
|
// alpha 255 as a constant (ComputeEndpoints; BlueContract and ClampByte pass it through), and the decoder's interpolation
|
||||||
|
// of two equal 16-bit values, (65535 * (64 - w) + 65535 * w + 32) / 64, is 65535 for every weight, which it writes as
|
||||||
|
// 65535 / 65535 = 1. Modes 4, 5, 10, 12 and 13 carry alpha, and 2, 3, 7, 11, 14 and 15 are the HDR ones, which take the
|
||||||
|
// decoder's other path: no for all of those, without decoding them (such a block may be opaque all the same).
|
||||||
|
// One partition has its mode in bits 13 to 16. Several partitions share one mode when bits 23 and 24 are zero (the mode is
|
||||||
|
// bits 25 to 28); when they do not, the modes' other bits sit below the weights and are not looked for: no.
|
||||||
|
#pragma once
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace astc_opaque {
|
||||||
|
|
||||||
|
/// `n` bits (16 at most) of a block from bit `at`, the lowest first, as the decoder's StreamBits reads them
|
||||||
|
inline uint32_t bits(const uint8_t* b, unsigned at, unsigned n) {
|
||||||
|
uint32_t v = 0;
|
||||||
|
for (unsigned i = 0; i < 3 && at / 8 + i < 16; i++) v |= (uint32_t)b[at / 8 + i] << (8 * i);
|
||||||
|
return (v >> (at & 7)) & ((1u << n) - 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The bits `n` weights take in the weight range `index` (0 to 11: 2, 3, 4, 5, 6, 8, 10, 12, 16, 20, 24 and 32 levels), as
|
||||||
|
/// the decoder's GetBitLength counts them: the plain bits, and a trit for every 8/5 of a bit or a quint for every 7/3
|
||||||
|
inline unsigned weight_bits(unsigned n, unsigned index) {
|
||||||
|
static const uint8_t plain[12] = {1, 0, 2, 0, 1, 3, 1, 2, 4, 2, 3, 5};
|
||||||
|
static const uint8_t kind[12] = {0, 1, 0, 2, 1, 0, 2, 1, 0, 2, 1, 0}; // (1: with a trit, 2: with a quint)
|
||||||
|
unsigned total = n * plain[index];
|
||||||
|
if (kind[index] == 1) total += (n * 8 + 4) / 5;
|
||||||
|
if (kind[index] == 2) total += (n * 7 + 2) / 3;
|
||||||
|
return total;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether the decoder writes alpha 255 into every texel of this block of bw x bh texels (see the top of the file). False
|
||||||
|
/// for a block size the decoder is not given (under 4 or over 12 a side)
|
||||||
|
inline bool block(const uint8_t b[16], unsigned bw, unsigned bh) {
|
||||||
|
if (bw < 4 || bh < 4 || bw > 12 || bh > 12) return false;
|
||||||
|
const uint32_t mode = bits(b, 0, 11);
|
||||||
|
if ((mode & 0x1FF) == 0x1FC) { // (a void extent)
|
||||||
|
if (mode & 0x200) return false;
|
||||||
|
if (!(mode & 0x400) || !bits(b, 11, 1)) return false;
|
||||||
|
return b[14] == 0xFF && b[15] == 0xFF;
|
||||||
|
}
|
||||||
|
if (!(mode & 0xF)) return false;
|
||||||
|
if (!(mode & 3) && (mode & 0x1C0) == 0x1C0) return false;
|
||||||
|
// (the weight grid, by the decoder's FindLayout and DecodeBlockSize)
|
||||||
|
const unsigned a = (mode >> 5) & 3, b2 = (mode >> 7) & 3, b1 = (mode >> 7) & 1;
|
||||||
|
unsigned layout, w, h;
|
||||||
|
if (mode & 3) {
|
||||||
|
if (mode & 8) {
|
||||||
|
if (!(mode & 4)) layout = 2, w = a + 2, h = b2 + 8;
|
||||||
|
else if (mode & 0x100) layout = 4, w = b1 + 2, h = a + 2;
|
||||||
|
else layout = 3, w = a + 2, h = b1 + 6;
|
||||||
|
} else if (mode & 4) {
|
||||||
|
layout = 1, w = b2 + 8, h = a + 2;
|
||||||
|
} else {
|
||||||
|
layout = 0, w = b2 + 4, h = a + 2;
|
||||||
|
}
|
||||||
|
} else if (mode & 0x100) {
|
||||||
|
if (!(mode & 0x80)) layout = 9, w = a + 6, h = ((mode >> 9) & 3) + 6;
|
||||||
|
else if (mode & 0x20) layout = 8, w = 10, h = 6;
|
||||||
|
else layout = 7, w = 6, h = 10;
|
||||||
|
} else if (mode & 0x80) {
|
||||||
|
layout = 6, w = a + 2, h = 12;
|
||||||
|
} else {
|
||||||
|
layout = 5, w = 12, h = a + 2;
|
||||||
|
}
|
||||||
|
if (w > bw || h > bh) return false;
|
||||||
|
const bool dual = layout != 9 && (mode & 0x400);
|
||||||
|
const unsigned parts = bits(b, 11, 2) + 1;
|
||||||
|
if (parts == 4 && dual) return false;
|
||||||
|
const unsigned weights = w * h * (dual ? 2 : 1);
|
||||||
|
if (weights > 64) return false;
|
||||||
|
// (the weights' range, by the decoder's DecodeMaxWeight: bit 4 and two more bits, less 2, six further with bit 9)
|
||||||
|
unsigned range = ((mode >> 4) & 1) | (layout < 5 ? (mode & 3) << 1 : (mode & 0xC) >> 1);
|
||||||
|
if (range < 2) return false; // (cannot be: the reserved modes above)
|
||||||
|
range = range - 2 + (layout != 9 && (mode & 0x200) ? 6 : 0);
|
||||||
|
const unsigned wbits = weight_bits(weights, range);
|
||||||
|
if (wbits < 24 || wbits > 96) return false;
|
||||||
|
unsigned read, cem;
|
||||||
|
if (parts == 1) {
|
||||||
|
cem = bits(b, 13, 4), read = 17;
|
||||||
|
} else {
|
||||||
|
const uint32_t base = bits(b, 23, 6);
|
||||||
|
if (base & 3) return false; // (a mode a partition: not looked for)
|
||||||
|
cem = base >> 2, read = 29;
|
||||||
|
}
|
||||||
|
// (the decoder's last error, its count of the remaining bits wrapping. With one shared mode it cannot: 96 + 29 + 2)
|
||||||
|
if (wbits + read + (dual ? 2u : 0u) > 128) return false;
|
||||||
|
return cem == 0 || cem == 1 || cem == 6 || cem == 8 || cem == 9;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether every one of `n` blocks (16 bytes each, in any order) is one the decoder writes opaque
|
||||||
|
inline bool all(const uint8_t* blocks, uint64_t n, unsigned bw, unsigned bh) {
|
||||||
|
for (uint64_t i = 0; i < n; i++)
|
||||||
|
if (!block(blocks + 16 * i, bw, bh)) return false;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace astc_opaque
|
||||||
137
gpu/vk/async_rules.h
Normal file
137
gpu/vk/async_rules.h
Normal file
@@ -0,0 +1,137 @@
|
|||||||
|
// (Triforce) The replay thread's rules (RX_VK_ASYNC=1, off by default): what it took to make the thread a place frames can be
|
||||||
|
// replayed on without the faults the lab's runs of it showed. Nothing here releases the game early: the game still presents only
|
||||||
|
// once the GPU has run its frame (its own event poll), so the picture and its order are the submitting thread's replay's.
|
||||||
|
//
|
||||||
|
// What the runs showed (rounds 22 and 23, vk_replay.cpp's own counters): the thread took the replay of the game's main thread's
|
||||||
|
// two queues off the part of the frame both of the game's chains wait for (about 1.3 ms at the Market), and the frame got only 0.4
|
||||||
|
// to 0.8 ms shorter, because
|
||||||
|
// 1. a capture was taken at every signal of the CPU's while the thread had any work (26,654 in 26,400 frames, 0.25 ms each on
|
||||||
|
// the game's thread), where the submitting thread's replay needs one only when a queue really goes on from a wait for that
|
||||||
|
// event (126 to 1,028 a run);
|
||||||
|
// 2. the submit's capture made every copy on the game's thread, where the submitting thread's replay leaves them to the copy
|
||||||
|
// helpers;
|
||||||
|
// 3. the game's submit gave its virtual core up and took it again around a hand-over of a quarter millisecond;
|
||||||
|
// 4. the game's present poll slept in steps of the system's timer, about 0.6 ms late on average;
|
||||||
|
// 5. every present made the stopped queues go past their waits (5,649 to 15,166 waits gone past a run against 0), which is
|
||||||
|
// also the first suspect of the flicker after a tour.
|
||||||
|
// The rules below are 1 and 5, apart from Vulkan and from the replay's state so that tests/test_async_rules.py can run them; 2 to
|
||||||
|
// 4 are in vk_replay.cpp submit_q, host/nvn_native.c nvnQueueSubmitCommands and host/hos_thread.c present_poll. Each has a switch
|
||||||
|
// of its own, and all of it is behind RX_VK_ASYNC=1.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <deque>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace mkwvk::async_rules {
|
||||||
|
|
||||||
|
/// A switch that is on unless set to 0 (RX_VK_ASYNC_EVENT_NOTES, RX_VK_ASYNC_CAPTURE_BEHIND)
|
||||||
|
inline bool on_unless_zero(const char* env) {
|
||||||
|
return !env || !*env || std::atoi(env) != 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 1. the capture at a signal of the CPU's ----
|
||||||
|
// The game writes what a queue's work reads after it submitted that work, up to its signal of the event the work waits for; the
|
||||||
|
// console's GPU reads memory after its wait. With the replay on its own thread that capture has to be taken at the signal, on the
|
||||||
|
// game's thread (later the game has written on). It is needed only when a submission handed over before the signal holds a wait
|
||||||
|
// for that event that the replay has not passed: every submission handed over after the signal has its own capture, taken after
|
||||||
|
// the game's writes. The host tells the events a submission waits for (its walk at the submit sees the wait ops:
|
||||||
|
// host/nvn_native.c resolve_deferred); a submission is noted from its hand-over until it is replayed whole.
|
||||||
|
|
||||||
|
constexpr uint64_t kAnyEvent = ~0ull; // (a submission whose waits the host did not tell: it may wait for any event)
|
||||||
|
constexpr size_t kMaxNotes = 256; // (more than this pending: every signal takes its capture, as before, until those past it are done)
|
||||||
|
|
||||||
|
struct Notes {
|
||||||
|
struct Note {
|
||||||
|
uint64_t value; // (the submission's timeline value: its hand-over's)
|
||||||
|
std::vector<uint64_t> events; // (the events it waits for; kAnyEvent: not told)
|
||||||
|
};
|
||||||
|
std::deque<Note> pending; // (in hand-over order)
|
||||||
|
bool over = false; // (a submission past kMaxNotes is not replayed whole yet: waits_for answers yes)
|
||||||
|
// ((Triforce) the submissions handed over past the bound, by value: their events are not known, so every event counts until
|
||||||
|
// each of them is done. The flag fell once `pending` was empty, which could be while one past the bound was still stopped
|
||||||
|
// at its wait, and its signal then took no capture)
|
||||||
|
std::vector<uint64_t> unnoted;
|
||||||
|
|
||||||
|
/// A submission handed over. told: the host gave its waits (events, n; n 0: it waits for none, and nothing is noted)
|
||||||
|
void add(uint64_t value, bool told, const uint64_t* events, int n) {
|
||||||
|
if (told && n <= 0) return;
|
||||||
|
if (pending.size() >= kMaxNotes) {
|
||||||
|
over = true;
|
||||||
|
unnoted.push_back(value);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
Note note{value, {}};
|
||||||
|
if (!told) note.events.push_back(kAnyEvent);
|
||||||
|
else
|
||||||
|
for (int i = 0; i < n; i++)
|
||||||
|
if (std::find(note.events.begin(), note.events.end(), events[i]) == note.events.end()) note.events.push_back(events[i]);
|
||||||
|
pending.push_back(std::move(note));
|
||||||
|
}
|
||||||
|
/// The submission of that value replayed whole (none of it stopped at a wait any more)
|
||||||
|
void done(uint64_t value) {
|
||||||
|
for (auto it = pending.begin(); it != pending.end(); ++it)
|
||||||
|
if (it->value == value) {
|
||||||
|
pending.erase(it);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
for (auto it = unnoted.begin(); it != unnoted.end(); ++it)
|
||||||
|
if (*it == value) {
|
||||||
|
unnoted.erase(it);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
if (unnoted.empty()) over = false;
|
||||||
|
}
|
||||||
|
/// Whether a submission handed over and not replayed whole may hold a wait for the event
|
||||||
|
bool waits_for(uint64_t event) const {
|
||||||
|
if (over) return true;
|
||||||
|
for (const Note& note : pending)
|
||||||
|
for (const uint64_t e : note.events)
|
||||||
|
if (e == event || e == kAnyEvent) return true;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Whether a signal of the CPU's takes a capture on the game's thread. behind: the replay thread has work handed over and not
|
||||||
|
/// recorded. notes_on: RX_VK_ASYNC_EVENT_NOTES (off: every signal with the thread behind, as before)
|
||||||
|
inline bool capture_at_signal(bool behind, bool notes_on, const Notes& notes, uint64_t event) {
|
||||||
|
if (!behind) return false;
|
||||||
|
return !notes_on || notes.waits_for(event);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 5. what a present on the replay thread does with the queues stopped at a wait ----
|
||||||
|
// Force: every stopped queue past its wait (as it was by default: the waits gone past). Keep: nothing forced (a load stood for
|
||||||
|
// good with it: only a present makes a load's stopped work go). Stale: the jobs behind the present are run first, a few
|
||||||
|
// milliseconds at most, until the frame's stopped work has gone on, and then only what was already stopped at the present before
|
||||||
|
// is made to go (vk_replay.cpp drain_stale). RX_VK_PRESENT_DRAIN: unset or 2 stale, 1 force, 0 keep
|
||||||
|
enum class PresentDrain { Keep = 0, Force = 1, Stale = 2 };
|
||||||
|
inline PresentDrain present_drain(const char* env) {
|
||||||
|
if (!env || !*env) return PresentDrain::Stale;
|
||||||
|
const int v = std::atoi(env);
|
||||||
|
return v == 0 ? PresentDrain::Keep : v == 1 ? PresentDrain::Force : PresentDrain::Stale;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- what is still open (2026-11-19, lab rounds 52 to 60; the thread stays off by default) ----
|
||||||
|
// Standing at the Market the thread gave 104.3 fps for 97.5 and a 1% low of 81.7 for 75.2, with no flicker (3 runs each). After
|
||||||
|
// a ten-area tour the player's character is drawn black for one frame, 0 to 3 times a minute at the Market (the default: none in
|
||||||
|
// 2 tours); with the presents forcing again (RX_VK_PRESENT_DRAIN=1) 37 to 111 times, so rule 5 is what keeps it rare. Every pass
|
||||||
|
// and dispatch of such a frame is there with its draws: the character is lit from wrong inputs, in a frame after one whose replay
|
||||||
|
// ran late. Ruled out: these rules (the thread without them does it too), texture uploads (none in three of four such frames),
|
||||||
|
// a readback handing the game a word it had set to 0 (none at 43 such frames: RX_VK_LOG_READBACK_ZERO), the conditions on
|
||||||
|
// occlusion counts and the occlusion answers (RX_VK_OCC_COND=0, RX_VK_OCC_QUERY=0: still there), and the game's request flags
|
||||||
|
// (all read set around eight such frames). Not settled: whether the lab's frame log, whose readback and disk writes are on this
|
||||||
|
// thread, is what makes the replay late there (one tour each: 3 such frames with its writes, none with RX_VK_FRAMELOG_DROP=1).
|
||||||
|
// RX_VK_REGION_TRAP=1 with RX_VK_PASS_TRACE=1 and the frame log logs the passes and conditions around the first eight.
|
||||||
|
|
||||||
|
// ---- the replay thread's look for its next job ----
|
||||||
|
/// RX_VK_ASYNC_SPIN_US: how long the thread looks for a job before it sleeps, in nanoseconds: 100 us unset, 0 none, 1000 at most.
|
||||||
|
/// The thread is on the core kept from the game's threads, and a wake of a sleeping thread is tens of microseconds for each of a
|
||||||
|
/// frame's hand-overs
|
||||||
|
inline int64_t spin_ns(const char* env) {
|
||||||
|
if (!env || !*env) return 100 * 1000;
|
||||||
|
return (int64_t)std::clamp(std::atoi(env), 0, 1000) * 1000;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::async_rules
|
||||||
143
gpu/vk/barrier_rules.h
Normal file
143
gpu/vk/barrier_rules.h
Normal file
@@ -0,0 +1,143 @@
|
|||||||
|
// (Triforce) Where the replay records less than a full barrier between work the game left unordered: a part's split inside a
|
||||||
|
// pass, an event's signal, a dispatch right after another; and when held submits go to the queue (RX_VK_BATCH). Apart from
|
||||||
|
// Vulkan so that tests/test_barrier_rules.py can run the rules over recorded command streams and check every hazard the game's
|
||||||
|
// ordering asks for against what the recorded barriers order (and, a rule dropped at a time, that a hazard is then left open).
|
||||||
|
//
|
||||||
|
// What the game's ordering asks for (NVN's, as the replay keeps it): work after one of the game's barriers after the work before
|
||||||
|
// it; a draw after a dispatch and a dispatch after a draw (the console's 3D and compute work share the engine and drain at the
|
||||||
|
// switch); copies, clears and uniform updates after and before what they touch; the colour, depth and stencil outputs of draws
|
||||||
|
// in the order they were drawn, from pass to pass too; a draw that samples a target its pass wrote, or writes one its pass
|
||||||
|
// sampled, after the draws before (the replay ends the pass there itself: vk_draw.cpp draw_hazards); an event's word written
|
||||||
|
// after the work before its signal, and a wait's work after the work before the signal it waits for. A signal is no barrier on
|
||||||
|
// the console: the work after it in the same queue is not ordered after the work before it.
|
||||||
|
//
|
||||||
|
// 1. A part's split (RX_VK_SPLIT_BARRIER; vk_replay.cpp maybe_split). A long submission goes to the GPU in parts of 120 draws and
|
||||||
|
// dispatches (RX_VK_CHUNK) so that the GPU starts early. Each part ended its pass with a full barrier, and the next began with
|
||||||
|
// another: the GPU drained between two draws of one of the game's passes, about 27 times a frame. With the rule the open pass
|
||||||
|
// goes on in the next part: the part ends it, and the next begins it again with the same targets behind a barrier of the
|
||||||
|
// attachments' outputs alone (colour output and the depth and stencil tests, their writes before their reads and writes), and
|
||||||
|
// keeps what the pass's draws sampled and wrote, so that draw_hazards ends the pass where it would have without the split. That
|
||||||
|
// orders all that the same draws in one pass ordered (RX_VK_CHUNK=0): within a pass Vulkan orders the attachments' accesses
|
||||||
|
// alone. Only where nothing else was recorded at the split (the part's uniform updates, its write-backs, an event's write held:
|
||||||
|
// then the full barriers as before), and a pass open. A part's uploads keep their own full barriers (vk_texture.cpp upload_cmd,
|
||||||
|
// texture_uploads_recorded), and anything recorded or submitted between the part's submit and the pass's start again (another
|
||||||
|
// submitter's work on the queue) gives the next part a full barrier first. Unset: on with NVIDIA's driver; =pass: on any card;
|
||||||
|
// =full (or any other value): as before. ((Triforce, P4) A parallel replay's segments are never split: they are the parts there,
|
||||||
|
// each begun and ended with a full barrier (vk_replay.cpp record_segment), so nothing goes on across a segment's edge; the rule
|
||||||
|
// acts in the serial replay's parts, as before. "Recorded between" is the replaying thread's own commands and every submit to the
|
||||||
|
// queue (cmd_work.h work_between); RX_VK_BARRIER_CONTEXT=0: every thread's commands, as before)
|
||||||
|
// 2. An event's signal (RX_VK_EVENT_BARRIER; vk_replay.cpp NOP_SIGNAL_EVENT). Each signal ended the pass and drained the GPU before
|
||||||
|
// its word's write. With the rule nothing is recorded at the signal: its word's buffer is bound there (so its page is among the
|
||||||
|
// recording's write-backs, as before), and the write waits for the next full barrier on its command buffer (the game's next
|
||||||
|
// barrier, the next dispatch's or pass start's) and goes right after it, so it still follows all the work before the signal;
|
||||||
|
// a recording's end writes what is still held behind a full barrier first, before its write-backs copy the words to the game's
|
||||||
|
// memory (that barrier is recorded under the pools' lock, so the writes take none: their buffers were bound at the signals). The
|
||||||
|
// writes count as work for the next barrier's look, so the write-backs' barrier is never left out after them. The writes held
|
||||||
|
// for one barrier go in the order of their signals, a barrier of the transfers between each two, so the game sees them done in
|
||||||
|
// that order (two signals of one word too). A wait met in the replay's order whose event's write is still held gets a full
|
||||||
|
// barrier first: the waiting queue's work is then after the signalling queue's work before the signal, as the drain at the
|
||||||
|
// signal made it. The game sees the word later (at that barrier, not at the signal). NX2navi records nothing at a signal at all
|
||||||
|
// (its CPU writes the word once the submission is done). Unset: with the pass barrier's game mode (NVIDIA's driver,
|
||||||
|
// pass_barrier.h); =late: on any card; =full (or any other value): as before. Not with the parallel replay.
|
||||||
|
// 3. A dispatch right after another (RX_VK_DISPATCH_BARRIER; vk_draw.cpp dispatch). Off unless asked for: =game leaves out the
|
||||||
|
// full barrier of a dispatch whose command buffer's last command was the dispatch before it (no barrier of the game's, no copy,
|
||||||
|
// no draw between; not an indirect one, whose grid another dispatch may have written), as NX2navi's barrier_game does. The game
|
||||||
|
// orders its dependent dispatches with its own barriers on the console; whether it always does, only the game can show (a
|
||||||
|
// one-frame band or flicker where it does not), so it is a switch for the lab. (Triforce, wp14, P4) Beside the parallel replay too:
|
||||||
|
// "nothing between" is the command buffer's own count of its commands (cmd_work.h), and a segment is a command buffer of its own;
|
||||||
|
// with the global count the rule stood aside while the helper threads recorded, which at the Market is most frames
|
||||||
|
// (RX_VK_BARRIER_CONTEXT=0: so again). The event rule above still does (event_hold).
|
||||||
|
// 4. Held submits (RX_VK_BATCH; vk_replay.cpp submit_recording, submit_job). =1 (on by default with Mesa's RADV) holds every submit
|
||||||
|
// until something needs the GPU's results or the queue's order. =parts holds a whole submission's recording only while the game's
|
||||||
|
// submit is being replayed: a part (split off to start the GPU early) goes to the queue at once with what is held, so does the
|
||||||
|
// held when there are RX_VK_BATCH_MAX (4 here) of them, and what is held goes when the replay of the game's submit returns.
|
||||||
|
// Holding changes neither the command buffers nor their order on the queue, only how many vkQueueSubmit2 calls carry them.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstddef>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <cstring>
|
||||||
|
|
||||||
|
namespace mkwvk::barrier_rules {
|
||||||
|
|
||||||
|
/// RX_VK_SPLIT_BARRIER: "pass" the rule on any card; "full" (or any other value) a full barrier at each split, as before; unset or
|
||||||
|
/// empty: on with NVIDIA's driver (vendor 0x10de). every_draw: RX_VK_BARRIER_EVERY_DRAW (an experiment that ends the pass at each
|
||||||
|
/// draw) keeps the old way
|
||||||
|
inline bool split_in_pass(const char* env, uint32_t vendor, bool every_draw) {
|
||||||
|
if (every_draw) return false;
|
||||||
|
if (env && *env) return std::strcmp(env, "pass") == 0;
|
||||||
|
(void)vendor; // ((Triforce, 11-16) off unless asked: the lab's Market had the same ~500 full barriers a frame with it on (the
|
||||||
|
return false; // helper threads record most frames there, and the rule stands aside for them) and no gain (90.6 / 89.2 fps
|
||||||
|
// against 92.8 / 90.3 with the old barriers)
|
||||||
|
}
|
||||||
|
/// Whether a part's split goes on in the same pass: the rule on, a pass open at the split, nothing recorded at the part's end (quiet:
|
||||||
|
/// the work count as it was after the pass's end) and no event's write held then (written behind a full barrier before the part's
|
||||||
|
/// write-backs, which copy its word to the game's memory: the GPU drained there all the same). ((Triforce, P4) no parallel replay's
|
||||||
|
/// argument: its segments are never split, maybe_split runs in the serial replay only)
|
||||||
|
inline bool split_carry(bool on, bool pass_open, bool quiet, bool writes_held) {
|
||||||
|
return on && pass_open && quiet && !writes_held;
|
||||||
|
}
|
||||||
|
/// Whether the next part starts with a full barrier: always when the pass is not carried, and when it is but something was recorded
|
||||||
|
/// between the part's submit and the pass's start again (into its command buffer, by this thread into another, or submitted to the
|
||||||
|
/// queue by any thread: cmd_work.h work_between)
|
||||||
|
inline bool split_start_full(bool carry, bool recorded_between) {
|
||||||
|
return !carry || recorded_between;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// RX_VK_EVENT_BARRIER: "late" the rule on any card; "full" (or any other value) as before; unset or empty: with the pass barrier's
|
||||||
|
/// game mode (pass_barrier::game: NVIDIA's driver unless RX_VK_PASS_BARRIER says otherwise)
|
||||||
|
inline bool event_late(const char* env, bool pass_game) {
|
||||||
|
if (env && *env) return std::strcmp(env, "late") == 0;
|
||||||
|
(void)pass_game; // ((Triforce, 11-16) off unless asked, as the split rule above: no gain measured)
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
/// Whether a signal's write is held for the next full barrier: the rule on, no parallel replay (its segments' threads share the
|
||||||
|
/// command buffers), and room for it (most: the held writes a command buffer may have)
|
||||||
|
inline bool event_hold(bool on, bool parallel, int held, int most) {
|
||||||
|
return on && !parallel && held < most;
|
||||||
|
}
|
||||||
|
/// A wait the replay finds met: a full barrier first when the write of its event's signal is still held
|
||||||
|
inline bool wait_barrier(bool its_write_held) {
|
||||||
|
return its_write_held;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// RX_VK_DISPATCH_BARRIER: "game" the barrier of a dispatch right after another left out; anything else, or unset: as before
|
||||||
|
inline bool dispatch_game(const char* env) {
|
||||||
|
return env && std::strcmp(env, "game") == 0;
|
||||||
|
}
|
||||||
|
/// Whether a dispatch records its full barrier. work_since: GPU work recorded on the command buffer since its start (vk_draw.cpp
|
||||||
|
/// work_since_barrier); no_order: RX_VK_NO_DISPATCH_ORDER; right_after: the command buffer's last command was a dispatch's (by the
|
||||||
|
/// command buffer's own count, cmd_work.h: with the parallel replay too)
|
||||||
|
inline bool dispatch_barrier(bool work_since, bool no_order, bool game, bool right_after, bool indirect) {
|
||||||
|
if (!work_since || no_order) return false;
|
||||||
|
return !(game && right_after && !indirect);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// RX_VK_BATCH
|
||||||
|
enum class Batch { Off, Hold, Parts };
|
||||||
|
/// "parts": Parts; otherwise set: Hold where its number is not 0, Off where it is (as RX_VK_BATCH was read); unset: Hold with RADV,
|
||||||
|
/// Off elsewhere
|
||||||
|
inline Batch batch_mode(const char* env, bool radv) {
|
||||||
|
if (env) {
|
||||||
|
if (std::strcmp(env, "parts") == 0) return Batch::Parts;
|
||||||
|
return std::atoi(env) != 0 ? Batch::Hold : Batch::Off;
|
||||||
|
}
|
||||||
|
return radv ? Batch::Hold : Batch::Off;
|
||||||
|
}
|
||||||
|
/// Whether a recording is held rather than handed to the queue: Hold always; Parts only a whole submission's (not a part's), and
|
||||||
|
/// only inside the replay of a game's submit (in_submit: submit_job's)
|
||||||
|
inline bool batch_hold(Batch m, bool part, bool in_submit) {
|
||||||
|
if (m == Batch::Hold) return true;
|
||||||
|
return m == Batch::Parts && !part && in_submit;
|
||||||
|
}
|
||||||
|
/// Whether what is held goes to the queue now, after a submit was added to it (held: recordings held now, the added one too): Hold at
|
||||||
|
/// cap (RX_VK_BATCH_MAX, 32) as before; Parts at most (RX_VK_BATCH_MAX, 4) or when the added one was not held (a part: it goes with
|
||||||
|
/// those before it, in order)
|
||||||
|
inline bool batch_flush(Batch m, bool added_held, size_t held, size_t cap) {
|
||||||
|
if (m == Batch::Off) return true;
|
||||||
|
if (m == Batch::Parts && !added_held) return true;
|
||||||
|
return held >= cap;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::barrier_rules
|
||||||
87
gpu/vk/bugshot_pixels.h
Normal file
87
gpu/vk/bugshot_pixels.h
Normal file
@@ -0,0 +1,87 @@
|
|||||||
|
// (Triforce) The pixels of a bug capture's picture (vk_bugshot.cpp): the window's image as the GPU copied it out, into the
|
||||||
|
// R, G, B, 255 bytes a PNG is written from. Apart from Vulkan, so tests/test_bug_capture.py compiles it alone. The formats
|
||||||
|
// are the ones create_swapchain picks (vk_main.cpp): 8-bit RGBA or BGRA, UNORM or sRGB (the bytes are sRGB codes either
|
||||||
|
// way), and with the Insert menu's HDR a 10-bit HDR10 image, ST 2084 codes of Rec. 2020 light. That one is brought to
|
||||||
|
// sRGB with the picture's paper white as white, so a PNG viewer shows what an SDR screen would show of the game:
|
||||||
|
// highlights above white and colours outside sRGB are cut off.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cmath>
|
||||||
|
#include <cstddef>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace bugshot {
|
||||||
|
|
||||||
|
enum Kind { NONE, RGBA8, BGRA8, A2B10G10R10, A2R10G10B10 };
|
||||||
|
|
||||||
|
/// What a window image's format is to this conversion, by its VkFormat number (vk_bugshot.cpp checks the numbers against
|
||||||
|
/// Vulkan's names); NONE: not one it reads
|
||||||
|
inline Kind kind_of(int vk_format) {
|
||||||
|
switch (vk_format) {
|
||||||
|
case 37: case 43: return RGBA8; // (R8G8B8A8_UNORM, R8G8B8A8_SRGB)
|
||||||
|
case 44: case 50: return BGRA8; // (B8G8R8A8_UNORM, B8G8R8A8_SRGB)
|
||||||
|
case 64: return A2B10G10R10; // (A2B10G10R10_UNORM_PACK32: red in the low ten bits)
|
||||||
|
case 58: return A2R10G10B10; // (A2R10G10B10_UNORM_PACK32: blue in the low ten bits)
|
||||||
|
default: return NONE;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// ST 2084's curve the display's way: a code 0 to 1 as nits, 0 to 10000
|
||||||
|
inline double pq_to_nits(double code) {
|
||||||
|
const double m1 = 2610.0 / 16384.0, m2 = 2523.0 / 4096.0 * 128.0;
|
||||||
|
const double c1 = 3424.0 / 4096.0, c2 = 2413.0 / 4096.0 * 32.0, c3 = 2392.0 / 4096.0 * 32.0;
|
||||||
|
const double p = std::pow(code < 0 ? 0 : code > 1 ? 1 : code, 1.0 / m2);
|
||||||
|
return 10000.0 * std::pow((p > c1 ? p - c1 : 0) / (c2 - c3 * p), 1.0 / m1);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Linear light 0 to 1 as an sRGB code 0 to 255
|
||||||
|
inline uint8_t srgb8(double linear) {
|
||||||
|
const double x = linear < 0 ? 0 : linear > 1 ? 1 : linear;
|
||||||
|
const double e = x <= 0.0031308 ? 12.92 * x : 1.055 * std::pow(x, 1.0 / 2.4) - 0.055;
|
||||||
|
return (uint8_t)(e * 255.0 + 0.5);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// n texels at px, four bytes each as the GPU wrote them, to R, G, B, 255 in place. hdr10: the image holds ST 2084 codes of
|
||||||
|
/// Rec. 2020 light (a 10-bit format only), and white_nits is the picture's paper white. False: not a format this reads
|
||||||
|
inline bool to_rgba8(uint8_t* px, size_t n, Kind kind, bool hdr10, double white_nits) {
|
||||||
|
if (kind == NONE) return false;
|
||||||
|
if (kind == RGBA8 || kind == BGRA8) {
|
||||||
|
for (size_t i = 0; i < n; i++) {
|
||||||
|
uint8_t* q = px + i * 4;
|
||||||
|
if (kind == BGRA8) {
|
||||||
|
const uint8_t b = q[0];
|
||||||
|
q[0] = q[2], q[2] = b;
|
||||||
|
}
|
||||||
|
q[3] = 255;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
// (ten bits a channel. HDR: a table from a code to its light, white at 1, and one from light to an sRGB code, so a 4K
|
||||||
|
// picture is a few table reads a texel and not nine pow calls)
|
||||||
|
constexpr int kSteps = 4096;
|
||||||
|
std::vector<float> light(hdr10 ? 1024 : 0);
|
||||||
|
std::vector<uint8_t> code8(hdr10 ? kSteps + 1 : 0);
|
||||||
|
const double white = white_nits > 1 ? white_nits : 1;
|
||||||
|
for (size_t v = 0; v < light.size(); v++) light[v] = (float)(pq_to_nits((double)v / 1023.0) / white);
|
||||||
|
for (size_t v = 0; v < code8.size(); v++) code8[v] = srgb8((double)v / kSteps);
|
||||||
|
const auto out8 = [&](float x) { return code8[x <= 0 ? 0 : x >= 1 ? kSteps : (int)(x * kSteps + 0.5f)]; };
|
||||||
|
for (size_t i = 0; i < n; i++) {
|
||||||
|
uint8_t* q = px + i * 4;
|
||||||
|
const uint32_t v = (uint32_t)q[0] | (uint32_t)q[1] << 8 | (uint32_t)q[2] << 16 | (uint32_t)q[3] << 24;
|
||||||
|
const uint32_t lo = v & 1023, mid = (v >> 10) & 1023, hi = (v >> 20) & 1023;
|
||||||
|
const uint32_t r = kind == A2B10G10R10 ? lo : hi, g = mid, b = kind == A2B10G10R10 ? hi : lo;
|
||||||
|
if (!hdr10) { // (sRGB codes with ten bits: the nearest of eight)
|
||||||
|
q[0] = (uint8_t)((r * 255 + 511) / 1023), q[1] = (uint8_t)((g * 255 + 511) / 1023), q[2] = (uint8_t)((b * 255 + 511) / 1023);
|
||||||
|
} else { // (Rec. 2020 light to Rec. 709's primaries, ITU-R BT.2087's matrix, then sRGB's curve)
|
||||||
|
const float lr = light[r], lg = light[g], lb = light[b];
|
||||||
|
q[0] = out8(1.6605f * lr - 0.5876f * lg - 0.0728f * lb);
|
||||||
|
q[1] = out8(-0.1246f * lr + 1.1329f * lg - 0.0083f * lb);
|
||||||
|
q[2] = out8(-0.0182f * lr - 0.1006f * lg + 1.1187f * lb);
|
||||||
|
}
|
||||||
|
q[3] = 255;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace bugshot
|
||||||
275
gpu/vk/cache_set.h
Normal file
275
gpu/vk/cache_set.h
Normal file
@@ -0,0 +1,275 @@
|
|||||||
|
// (Triforce) One earlier set of the shader caches, for a change of a setting: the rule (vk_cache_set.cpp does the files).
|
||||||
|
//
|
||||||
|
// Why. Everything the game keeps of its shaders is kept for one stamp (vk_shader.cpp disk_stamp, program_cache_stamp): the
|
||||||
|
// program cache (triforce_programs.bin), the pipeline cache (mkw_vk_pipelines.bin by its .for, and the play file), the
|
||||||
|
// pipeline notes and the prepare's record. A start of another stamp drops all of them whole, and prepares every shader again:
|
||||||
|
// on AMD's driver 1.5 to 2 hours (91,600 stage libraries at 16 a second, a cold precompile, the translation). Most stamps
|
||||||
|
// change with the translator, and then nothing of the old set is any use again. Some change with a setting: the menu's Robust
|
||||||
|
// buffers row (the translation's out-of-bounds buffer reads, vk_coopmat.cpp fetch_bounds_buffers), the mesh shaders row, a GPU
|
||||||
|
// of a laptop's two, an RX_VK_ switch of the translation. Switching such a setting back made the same two hours again.
|
||||||
|
// So: when the stamp changed and the build part did not (the translator and its C++ library, vk_shader.cpp
|
||||||
|
// program_cache_build_part: what only a new build changes), the set there is kept, renamed to <name>.<stamp>.<ext>, when the
|
||||||
|
// disk has room for it and for the new one (twice the set and 10 GB free). One set is kept, no more. A start whose stamp is the
|
||||||
|
// kept set's puts it back (every piece, or none: a set with a piece missing or of another size is refused and deleted), and
|
||||||
|
// the set it found takes its place as the kept one. A start of another build deletes the kept set: it is of no use to it.
|
||||||
|
// What a start found, and what changed (triforce_cache_set.txt beside the caches, the parts of the stamp), is in the log.
|
||||||
|
// The driver matches what it is handed by its own hash of the whole creation: a set put back is the very data the driver made
|
||||||
|
// for that stamp, and the caches' own checks (the .for files, the program cache's head, the pipeline cache's device and UUID
|
||||||
|
// header) still read it as they read any start's.
|
||||||
|
// A set put back has the pipeline notes of its own time: what the player met while the other set was there is in the
|
||||||
|
// player's state list (triforce_pipeline_states.bin, not a piece: it is good for any stamp). When that list is not the one
|
||||||
|
// the set was kept with, kStatesPending asks the prepare before the title to go through the lists (vk_draw.cpp prepare_run,
|
||||||
|
// state_list_prepare), as a prepare of every shader would have: those pipelines are built then, not at their first draw.
|
||||||
|
// Files of a kept set's names that no record names (a start that ended in a swap, a record that cannot be read) are deleted
|
||||||
|
// at the next start: no set is left on the disk that nothing knows of.
|
||||||
|
// RX_VK_CACHE_SET=0: no set kept or put back (a change of stamp drops the caches, as before), and a kept set is deleted;
|
||||||
|
// RX_VK_PCACHE_KEEP_OLD and RX_VK_NO_PROGRAM_CACHE leave it off too (a kept set of this build stays, another build's is
|
||||||
|
// deleted). tests/test_cache_set.py (cache_set_files.h does the files)
|
||||||
|
#pragma once
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdio>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <cstring>
|
||||||
|
#include <string>
|
||||||
|
#include <utility>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace cache_set {
|
||||||
|
|
||||||
|
// (every file kept for one stamp, as the game folder names it: vk_shader.cpp disk_init's two, vk_draw.cpp pcache_init's (the
|
||||||
|
// carry's .base too, pcache_carry.h) and pcache_play_open's, the prepare's record and what prepare_run keeps beside it, the notes
|
||||||
|
// and what vk_draw.cpp keeps beside them by the stamp. The first is the set's own: a set is the one its program cache's head names)
|
||||||
|
inline constexpr const char* kPieces[] = {
|
||||||
|
"triforce_programs.bin", "triforce_programs.idx",
|
||||||
|
"mkw_vk_pipelines.bin", "mkw_vk_pipelines.bin.for",
|
||||||
|
"mkw_vk_pipelines.bin.base", // (the carry's history of that file, by its stamp: pcache_carry.h)
|
||||||
|
"mkw_vk_pipelines.bin.own", // ((Triforce) that file holds the driver's part's own alone: vk_draw.cpp pcache_own_mark)
|
||||||
|
"mkw_vk_pipelines_play.bin", "mkw_vk_pipelines_play.bin.for",
|
||||||
|
"triforce_prepared.bin", "triforce_prepared.bin.check",
|
||||||
|
"triforce_prepared.bin.warming", "triforce_prepared.bin.warming.skip",
|
||||||
|
"triforce_pipelines.bin", "triforce_pipelines.bin.predicted",
|
||||||
|
"triforce_pipelines.bin.plan", "triforce_pipelines.bin.built",
|
||||||
|
"triforce_states_applied.bin",
|
||||||
|
};
|
||||||
|
inline constexpr size_t kPieceCount = sizeof kPieces / sizeof kPieces[0];
|
||||||
|
inline constexpr const char* kRecord = "triforce_cache_set.txt"; // (what the set there was made for)
|
||||||
|
inline constexpr const char* kKept = "triforce_cache_set.kept.txt"; // (the kept set: what it was made for, and its pieces)
|
||||||
|
inline constexpr const char* kStates = "triforce_pipeline_states.bin"; // (the player's state list, vk_draw.cpp: not a piece)
|
||||||
|
inline constexpr const char* kStatesPending = "triforce_cache_set.states.txt"; // (a set put back: the lists to go through)
|
||||||
|
inline constexpr uint32_t kProgramsMagic = 0x31505348; // (vk_shader.cpp kDiskMagic: the program cache's head)
|
||||||
|
inline constexpr uint64_t kMargin = 10ull << 30; // (free beyond twice the set)
|
||||||
|
|
||||||
|
/// RX_VK_CACHE_SET: on unless it is 0
|
||||||
|
inline bool on(const char* env) { return !(env && env[0] == '0' && !env[1]); }
|
||||||
|
|
||||||
|
/// A piece's name in the kept set: its name to the first '.', the stamp's 16 digits, the rest
|
||||||
|
/// (mkw_vk_pipelines.bin.for: mkw_vk_pipelines.0123456789abcdef.bin.for)
|
||||||
|
inline std::string kept_name(const std::string& live, uint64_t stamp) {
|
||||||
|
char hex[20];
|
||||||
|
std::snprintf(hex, sizeof hex, "%016llx", (unsigned long long)stamp);
|
||||||
|
const size_t dot = live.find('.');
|
||||||
|
if (dot == std::string::npos) return live + "." + hex;
|
||||||
|
return live.substr(0, dot) + "." + hex + live.substr(dot);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a file's name is a piece's name in a kept set (kept_name's), and the stamp it names
|
||||||
|
inline bool kept_piece_of(const std::string& file, uint64_t* stamp) {
|
||||||
|
for (const char* p : kPieces) {
|
||||||
|
const std::string live = p;
|
||||||
|
const size_t dot = live.find('.');
|
||||||
|
const std::string stem = live.substr(0, dot), rest = live.substr(dot);
|
||||||
|
if (file.size() != stem.size() + 17 + rest.size() || file.compare(0, stem.size(), stem) || file[stem.size()] != '.' ||
|
||||||
|
file.compare(file.size() - rest.size(), rest.size(), rest))
|
||||||
|
continue;
|
||||||
|
uint64_t v = 0;
|
||||||
|
bool hex = true;
|
||||||
|
for (size_t i = stem.size() + 1; i < stem.size() + 17 && hex; i++) {
|
||||||
|
const char c = file[i];
|
||||||
|
hex = (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f');
|
||||||
|
v = v << 4 | (uint64_t)(c <= '9' ? c - '0' : c - 'a' + 10);
|
||||||
|
}
|
||||||
|
if (!hex) continue;
|
||||||
|
*stamp = v;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The stamp a program cache's first 16 bytes name (magic, the build that wrote it, the stamp); false: no program cache's head
|
||||||
|
inline bool programs_head(const uint8_t* b, size_t n, uint64_t* stamp) {
|
||||||
|
if (n < 16) return false;
|
||||||
|
uint32_t magic;
|
||||||
|
std::memcpy(&magic, b, 4);
|
||||||
|
if (magic != kProgramsMagic) return false;
|
||||||
|
std::memcpy(stamp, b + 8, 8);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a set was made for (the stamp, the build part, and the parts of the stamp by name for the log) and, for the kept
|
||||||
|
/// one, its pieces as they were renamed: each one's name in the game folder and its size
|
||||||
|
struct Record {
|
||||||
|
uint64_t stamp = 0, build = 0;
|
||||||
|
std::vector<std::pair<std::string, uint64_t>> parts;
|
||||||
|
std::string env; // (the stamp's RX_VK_ and EDEN_ switches that were set, by name)
|
||||||
|
uint64_t states = 0; // (the kept one: the player's state list as it was when kept, cache_set_files.h states_id; 0: none)
|
||||||
|
std::vector<std::pair<std::string, uint64_t>> pieces;
|
||||||
|
uint64_t bytes() const {
|
||||||
|
uint64_t n = 0;
|
||||||
|
for (const auto& p : pieces) n += p.second;
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
inline std::string format(const Record& r) {
|
||||||
|
std::string s = "triforce-cache-set 1\n";
|
||||||
|
char line[160];
|
||||||
|
std::snprintf(line, sizeof line, "stamp %016llx\nbuild %016llx\n", (unsigned long long)r.stamp, (unsigned long long)r.build);
|
||||||
|
s += line;
|
||||||
|
for (const auto& [name, v] : r.parts) {
|
||||||
|
std::snprintf(line, sizeof line, "part %s %016llx\n", name.c_str(), (unsigned long long)v);
|
||||||
|
s += line;
|
||||||
|
}
|
||||||
|
s += "env " + (r.env.empty() ? std::string("-") : r.env) + "\n";
|
||||||
|
if (r.states) {
|
||||||
|
std::snprintf(line, sizeof line, "states %016llx\n", (unsigned long long)r.states);
|
||||||
|
s += line;
|
||||||
|
}
|
||||||
|
for (const auto& [name, size] : r.pieces) {
|
||||||
|
std::snprintf(line, sizeof line, "piece %s %llu\n", name.c_str(), (unsigned long long)size);
|
||||||
|
s += line;
|
||||||
|
}
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// format's text read back; false when it is not one (no head, no stamp or build, a piece that is not one of kPieces)
|
||||||
|
inline bool parse(const std::string& text, Record* out) {
|
||||||
|
Record r;
|
||||||
|
bool head = false, stamp = false, build = false;
|
||||||
|
size_t at = 0;
|
||||||
|
while (at < text.size()) {
|
||||||
|
size_t end = text.find('\n', at);
|
||||||
|
if (end == std::string::npos) end = text.size();
|
||||||
|
std::string line = text.substr(at, end - at);
|
||||||
|
at = end + 1;
|
||||||
|
if (!line.empty() && line.back() == '\r') line.pop_back();
|
||||||
|
if (line.empty()) continue;
|
||||||
|
const size_t sp = line.find(' ');
|
||||||
|
const std::string key = line.substr(0, sp), rest = sp == std::string::npos ? "" : line.substr(sp + 1);
|
||||||
|
if (!head) {
|
||||||
|
if (line != "triforce-cache-set 1") return false;
|
||||||
|
head = true;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
unsigned long long v = 0;
|
||||||
|
char name[96];
|
||||||
|
if (key == "stamp" && std::sscanf(rest.c_str(), "%16llx", &v) == 1) {
|
||||||
|
r.stamp = v, stamp = true;
|
||||||
|
} else if (key == "build" && std::sscanf(rest.c_str(), "%16llx", &v) == 1) {
|
||||||
|
r.build = v, build = true;
|
||||||
|
} else if (key == "part" && std::sscanf(rest.c_str(), "%95s %16llx", name, &v) == 2) {
|
||||||
|
r.parts.push_back({name, v});
|
||||||
|
} else if (key == "states" && std::sscanf(rest.c_str(), "%16llx", &v) == 1) {
|
||||||
|
r.states = v;
|
||||||
|
} else if (key == "env") {
|
||||||
|
r.env = rest == "-" ? "" : rest;
|
||||||
|
} else if (key == "piece" && std::sscanf(rest.c_str(), "%95s %llu", name, &v) == 2) {
|
||||||
|
bool known = false;
|
||||||
|
for (const char* p : kPieces) known = known || !std::strcmp(p, name);
|
||||||
|
if (!known) return false;
|
||||||
|
r.pieces.push_back({name, v});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!head || !stamp || !build) return false;
|
||||||
|
*out = std::move(r);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The parts by name whose value is not the same in a and b (a part only one of them has counts as changed), for the log
|
||||||
|
inline std::vector<std::string> changed(const Record& a, const Record& b) {
|
||||||
|
std::vector<std::string> out;
|
||||||
|
if (a.build != b.build) out.push_back("build");
|
||||||
|
const auto find = [](const Record& r, const std::string& name) -> const uint64_t* {
|
||||||
|
for (const auto& p : r.parts)
|
||||||
|
if (p.first == name) return &p.second;
|
||||||
|
return nullptr;
|
||||||
|
};
|
||||||
|
for (const auto& p : a.parts) {
|
||||||
|
const uint64_t* v = find(b, p.first);
|
||||||
|
if (!v || *v != p.second) out.push_back(p.first);
|
||||||
|
}
|
||||||
|
for (const auto& p : b.parts)
|
||||||
|
if (!find(a, p.first)) out.push_back(p.first);
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Room for a set kept beside the new one: twice the set and kMargin free, counting what deleting the old kept set frees
|
||||||
|
inline bool room(uint64_t free_bytes, uint64_t freed, uint64_t set_bytes) {
|
||||||
|
const uint64_t have = free_bytes + freed < free_bytes ? ~0ull : free_bytes + freed;
|
||||||
|
if (set_bytes > (~0ull - kMargin) / 2) return false;
|
||||||
|
return have >= 2 * set_bytes + kMargin;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a start knows: its stamp and build part; the set there (its program cache's head, and the record of what it was made
|
||||||
|
/// for when the record names that stamp); the kept set (its record, and whether every piece is there whole); the disk
|
||||||
|
struct Facts {
|
||||||
|
bool on = false; // (keeping and putting back allowed)
|
||||||
|
bool off_forgets = false; // (off by RX_VK_CACHE_SET=0: a kept set of this build is deleted too, for its space)
|
||||||
|
uint64_t stamp = 0, build = 0; // (this start's)
|
||||||
|
bool live = false; // (a program cache there, its head read)
|
||||||
|
uint64_t live_stamp = 0; // (what its head names)
|
||||||
|
bool live_named = false; // (the record names live_stamp: live_build is known)
|
||||||
|
uint64_t live_build = 0, live_bytes = 0;
|
||||||
|
bool kept = false; // (a kept set's record)
|
||||||
|
uint64_t kept_stamp = 0, kept_build = 0, kept_bytes = 0;
|
||||||
|
bool kept_whole = false; // (each of its pieces there at its size, its program cache's head naming kept_stamp)
|
||||||
|
bool space_known = false;
|
||||||
|
uint64_t free_bytes = 0;
|
||||||
|
};
|
||||||
|
enum class Do {
|
||||||
|
Same, // (the set there is this stamp's: nothing to do)
|
||||||
|
Restore, // (the kept set is this stamp's: put back; keep_live: the set there becomes the kept one, else it is deleted)
|
||||||
|
Keep, // (the set there is kept, renamed; the caches start empty for this stamp)
|
||||||
|
Drop, // (nothing renamed: the caches drop the set there, as before)
|
||||||
|
};
|
||||||
|
struct Plan {
|
||||||
|
Do what = Do::Drop;
|
||||||
|
bool forget = false; // (the kept set is deleted first)
|
||||||
|
bool keep_live = false; // (Restore: the set there is kept in its place)
|
||||||
|
};
|
||||||
|
|
||||||
|
inline Plan decide(const Facts& f) {
|
||||||
|
Plan p;
|
||||||
|
const bool same = f.live && f.live_stamp == f.stamp;
|
||||||
|
if (!f.on) {
|
||||||
|
// (off: nothing kept or put back. A kept set of another build is of no use to any build again: deleted; one of this
|
||||||
|
// build too when the off is RX_VK_CACHE_SET=0's, the switch that frees its space)
|
||||||
|
p.what = same ? Do::Same : Do::Drop;
|
||||||
|
p.forget = f.kept && (f.kept_build != f.build || f.off_forgets);
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
bool kept_ours = f.kept && f.kept_build == f.build;
|
||||||
|
// (another build's kept set is of no use to this one, nor to any later: deleted. A kept set that names the set there
|
||||||
|
// is what a start that ended while it swapped the two left: its pieces are not where it says)
|
||||||
|
p.forget = f.kept && (!kept_ours || (same && f.kept_stamp == f.stamp));
|
||||||
|
if (same) {
|
||||||
|
p.what = Do::Same;
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
const bool live_ours = f.live && f.live_named && f.live_build == f.build;
|
||||||
|
if (kept_ours && f.kept_stamp == f.stamp) {
|
||||||
|
if (f.kept_whole) {
|
||||||
|
p.what = Do::Restore;
|
||||||
|
p.keep_live = live_ours;
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
p.forget = true; // (a piece missing or cut: refused)
|
||||||
|
}
|
||||||
|
if (live_ours && f.space_known && room(f.free_bytes, f.kept ? f.kept_bytes : 0, f.live_bytes)) {
|
||||||
|
p.what = Do::Keep;
|
||||||
|
p.forget = f.kept; // (one set kept: the one there takes the old one's place)
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace cache_set
|
||||||
306
gpu/vk/cache_set_files.h
Normal file
306
gpu/vk/cache_set_files.h
Normal file
@@ -0,0 +1,306 @@
|
|||||||
|
// (Triforce) The kept set of the shader caches in the game folder: what a start finds there, and what it renames, deletes and
|
||||||
|
// writes for cache_set.h's rule. The standard library and file_whole.h alone, nothing of the device: vk_cache_set.cpp gives it
|
||||||
|
// this start's record (the stamp and its parts) and the switches, and tests/test_cache_set.py runs it on a folder of its own.
|
||||||
|
#pragma once
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdio>
|
||||||
|
#include <filesystem>
|
||||||
|
#include <string>
|
||||||
|
#include <utility>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
#include "cache_set.h"
|
||||||
|
#include "file_whole.h"
|
||||||
|
|
||||||
|
namespace cache_set {
|
||||||
|
|
||||||
|
using Log = void (*)(const char* fmt, ...);
|
||||||
|
|
||||||
|
inline std::string stamp_hex(uint64_t s) {
|
||||||
|
char hex[20];
|
||||||
|
std::snprintf(hex, sizeof hex, "%016llx", (unsigned long long)s);
|
||||||
|
return hex;
|
||||||
|
}
|
||||||
|
inline bool read_text(const std::string& path, std::string* out) {
|
||||||
|
FILE* f = std::fopen(path.c_str(), "rb");
|
||||||
|
if (!f) return false;
|
||||||
|
char buf[4096];
|
||||||
|
out->clear();
|
||||||
|
for (size_t n; (n = std::fread(buf, 1, sizeof buf, f)) > 0 && out->size() < (1u << 20);) out->append(buf, n);
|
||||||
|
std::fclose(f);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
inline bool read_record(const std::string& path, Record* r) {
|
||||||
|
std::string text;
|
||||||
|
return read_text(path, &text) && parse(text, r);
|
||||||
|
}
|
||||||
|
/// The stamp a program cache's head names
|
||||||
|
inline bool head_of(const std::string& path, uint64_t* stamp) {
|
||||||
|
FILE* f = std::fopen(path.c_str(), "rb");
|
||||||
|
if (!f) return false;
|
||||||
|
uint8_t b[16];
|
||||||
|
const size_t n = std::fread(b, 1, sizeof b, f);
|
||||||
|
std::fclose(f);
|
||||||
|
return programs_head(b, n, stamp);
|
||||||
|
}
|
||||||
|
inline bool size_of(const std::string& path, uint64_t* size) {
|
||||||
|
std::error_code ec;
|
||||||
|
if (!std::filesystem::is_regular_file(path, ec) || ec) return false;
|
||||||
|
const auto n = std::filesystem::file_size(path, ec);
|
||||||
|
if (ec) return false;
|
||||||
|
*size = (uint64_t)n;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
/// The pieces in the game folder by their names there, and their sizes
|
||||||
|
inline std::vector<std::pair<std::string, uint64_t>> pieces_there(const std::string& dir) {
|
||||||
|
std::vector<std::pair<std::string, uint64_t>> out;
|
||||||
|
for (const char* p : kPieces) {
|
||||||
|
uint64_t n = 0;
|
||||||
|
if (size_of(dir + p, &n)) out.push_back({p, n});
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
/// The player's state list (kStates) as it is now: its bytes hashed, with its size; 0 when there is none, or no record in it
|
||||||
|
inline uint64_t states_id(const std::string& dir) {
|
||||||
|
FILE* f = std::fopen((dir + kStates).c_str(), "rb");
|
||||||
|
if (!f) return 0;
|
||||||
|
uint64_t h = 0xCBF29CE484222325ull, total = 0;
|
||||||
|
std::vector<uint8_t> buf(1 << 16);
|
||||||
|
for (size_t n; (n = std::fread(buf.data(), 1, buf.size(), f)) > 0; total += n)
|
||||||
|
for (size_t i = 0; i < n; i++) h = (h ^ buf[i]) * 1099511628211ull;
|
||||||
|
std::fclose(f);
|
||||||
|
for (int i = 0; i < 8; i++) h = (h ^ (uint8_t)(total >> (8 * i))) * 1099511628211ull;
|
||||||
|
return total > 12 ? (h | 1) : 0; // (12: the list's head alone)
|
||||||
|
}
|
||||||
|
/// Every piece of the kept set there, at its size, and its program cache's head naming its stamp
|
||||||
|
inline bool kept_whole(const std::string& dir, const Record& kept) {
|
||||||
|
bool programs = false;
|
||||||
|
for (const auto& [name, size] : kept.pieces) {
|
||||||
|
const std::string at = dir + kept_name(name, kept.stamp);
|
||||||
|
uint64_t n = 0;
|
||||||
|
if (!size_of(at, &n) || n != size) return false;
|
||||||
|
if (name == kPieces[0]) {
|
||||||
|
uint64_t stamp = 0;
|
||||||
|
if (!head_of(at, &stamp) || stamp != kept.stamp) return false;
|
||||||
|
programs = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return programs;
|
||||||
|
}
|
||||||
|
/// The kept set deleted: every piece its stamp names (what its record lists, and what a start cut short left), then its record
|
||||||
|
inline void forget(const std::string& dir, const Record& kept) {
|
||||||
|
for (const char* p : kPieces) {
|
||||||
|
std::error_code ec;
|
||||||
|
std::filesystem::remove(dir + kept_name(p, kept.stamp), ec);
|
||||||
|
}
|
||||||
|
std::remove((dir + kKept).c_str());
|
||||||
|
}
|
||||||
|
/// The files of a kept set's names (kept_piece_of) that are not the kept set's (kept: its record read; kept_stamp: its stamp)
|
||||||
|
/// deleted: what a start that ended in a swap left, or a set whose record cannot be read. Their count and bytes
|
||||||
|
inline std::pair<size_t, uint64_t> sweep(const std::string& dir, bool kept, uint64_t kept_stamp) {
|
||||||
|
std::pair<size_t, uint64_t> gone{0, 0};
|
||||||
|
std::error_code ec;
|
||||||
|
std::vector<std::filesystem::path> drop;
|
||||||
|
for (std::filesystem::directory_iterator it(dir.empty() ? std::string(".") : dir, ec), end; !ec && it != end; it.increment(ec)) {
|
||||||
|
std::error_code e2;
|
||||||
|
if (!it->is_regular_file(e2) || e2) continue;
|
||||||
|
uint64_t stamp = 0;
|
||||||
|
if (!kept_piece_of(it->path().filename().string(), &stamp) || (kept && stamp == kept_stamp)) continue;
|
||||||
|
drop.push_back(it->path());
|
||||||
|
}
|
||||||
|
for (const auto& p : drop) {
|
||||||
|
std::error_code sized, removed;
|
||||||
|
const auto n = std::filesystem::file_size(p, sized);
|
||||||
|
if (std::filesystem::remove(p, removed) && !removed) gone.first++, gone.second += sized ? 0 : (uint64_t)n;
|
||||||
|
}
|
||||||
|
return gone;
|
||||||
|
}
|
||||||
|
/// Each file renamed (a file there in the way replaced), or none: one that cannot be leaves those done renamed back
|
||||||
|
inline bool move_all(const std::vector<std::pair<std::string, std::string>>& moves, Log log) {
|
||||||
|
for (size_t i = 0; i < moves.size(); i++) {
|
||||||
|
if (file_writers::replace_file(moves[i].first, moves[i].second)) continue;
|
||||||
|
log("vk: shader caches: %s cannot be renamed to %s: the set is left as it was", moves[i].first.c_str(),
|
||||||
|
moves[i].second.c_str());
|
||||||
|
for (size_t j = i; j-- > 0;) file_writers::replace_file(moves[j].second, moves[j].first);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
inline std::string joined(const std::vector<std::string>& v) {
|
||||||
|
std::string s;
|
||||||
|
for (const auto& x : v) s += (s.empty() ? "" : ", ") + (x == "robust" ? std::string("Robust buffers") : x);
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The switches a start has (vk_cache_set.cpp reads them from the environment)
|
||||||
|
struct Switches {
|
||||||
|
bool on = true; // (RX_VK_CACHE_SET not 0, RX_VK_PCACHE_KEEP_OLD not set)
|
||||||
|
bool off_forgets = false; // (off by RX_VK_CACHE_SET=0: a kept set is deleted)
|
||||||
|
bool space_given = false; // (tests: the disk's free space as given, not as the disk says)
|
||||||
|
uint64_t free_bytes = 0;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The start's kept set decided and done in dir (its name ends in a separator, or is empty): now is this start's record (the
|
||||||
|
/// stamp, the build part, the parts). What a start does before anything opens a cache
|
||||||
|
inline Plan run(const std::string& dir, const Record& now, const Switches& sw, Log log) {
|
||||||
|
Record had, kept;
|
||||||
|
const bool had_read = read_record(dir + kRecord, &had);
|
||||||
|
Facts f;
|
||||||
|
f.on = sw.on, f.off_forgets = sw.off_forgets;
|
||||||
|
f.stamp = now.stamp, f.build = now.build;
|
||||||
|
f.kept = read_record(dir + kKept, &kept);
|
||||||
|
// (no set is left there that nothing knows of: before the free space is measured, which it frees)
|
||||||
|
const auto [swept, swept_bytes] = sweep(dir, f.kept, kept.stamp);
|
||||||
|
if (swept)
|
||||||
|
log("vk: shader caches: %zu files of a kept set no record names deleted (%llu MB: a start that ended while it swapped "
|
||||||
|
"the sets, or a record that cannot be read)", swept, (unsigned long long)(swept_bytes >> 20));
|
||||||
|
f.live = head_of(dir + kPieces[0], &f.live_stamp);
|
||||||
|
f.live_named = f.live && had_read && had.stamp == f.live_stamp;
|
||||||
|
f.live_build = had.build;
|
||||||
|
const std::vector<std::pair<std::string, uint64_t>> live = pieces_there(dir);
|
||||||
|
for (const auto& p : live) f.live_bytes += p.second;
|
||||||
|
if (f.kept) {
|
||||||
|
f.kept_stamp = kept.stamp, f.kept_build = kept.build, f.kept_bytes = kept.bytes();
|
||||||
|
f.kept_whole = kept_whole(dir, kept);
|
||||||
|
}
|
||||||
|
if (sw.space_given) {
|
||||||
|
f.space_known = true, f.free_bytes = sw.free_bytes;
|
||||||
|
} else {
|
||||||
|
std::error_code ec;
|
||||||
|
const auto space = std::filesystem::space(dir.empty() ? std::string(".") : dir, ec);
|
||||||
|
f.space_known = !ec;
|
||||||
|
f.free_bytes = ec ? 0 : (uint64_t)space.available;
|
||||||
|
}
|
||||||
|
const Plan plan = decide(f);
|
||||||
|
|
||||||
|
if (f.live && f.live_stamp != f.stamp) { // (what changed, when it is known)
|
||||||
|
std::string what = "not known: the set there has no record of what it was made for (a build before this rule)";
|
||||||
|
if (f.live_named) {
|
||||||
|
const std::vector<std::string> parts = changed(had, now);
|
||||||
|
what = parts.empty() ? "a part of the translation's settings not named here (its per-GPU settings)" : joined(parts);
|
||||||
|
if (had.env != now.env) what += "; the switches set: " + (had.env.empty() ? "none" : had.env) + " -> " +
|
||||||
|
(now.env.empty() ? "none" : now.env);
|
||||||
|
}
|
||||||
|
log("vk: shader caches: made for %s, this start is %s: changed: %s", stamp_hex(f.live_stamp).c_str(),
|
||||||
|
stamp_hex(f.stamp).c_str(), what.c_str());
|
||||||
|
}
|
||||||
|
if (plan.forget && f.kept) {
|
||||||
|
forget(dir, kept);
|
||||||
|
log("vk: shader caches: the kept set for %s (%llu MB) deleted: %s", stamp_hex(kept.stamp).c_str(),
|
||||||
|
(unsigned long long)(kept.bytes() >> 20),
|
||||||
|
kept.build != now.build ? "it is another build's"
|
||||||
|
: !f.on ? "RX_VK_CACHE_SET=0"
|
||||||
|
: !f.kept_whole && kept.stamp == now.stamp ? "a piece of it is missing or not its size"
|
||||||
|
: "one set is kept, and this one takes its place");
|
||||||
|
} else if (f.kept && !f.on) {
|
||||||
|
log("vk: shader caches: the kept set for %s (%llu MB, as <name>.%s.<ext>) stays while it is off (RX_VK_PCACHE_KEEP_OLD; "
|
||||||
|
"RX_VK_CACHE_SET=0 deletes it)", stamp_hex(kept.stamp).c_str(), (unsigned long long)(kept.bytes() >> 20),
|
||||||
|
stamp_hex(kept.stamp).c_str());
|
||||||
|
}
|
||||||
|
const auto renames = [&](const std::vector<std::pair<std::string, uint64_t>>& pieces, uint64_t stamp, bool to_kept) {
|
||||||
|
std::vector<std::pair<std::string, std::string>> m;
|
||||||
|
for (const auto& p : pieces) {
|
||||||
|
const std::string a = dir + p.first, b = dir + kept_name(p.first, stamp);
|
||||||
|
m.push_back(to_kept ? std::make_pair(a, b) : std::make_pair(b, a));
|
||||||
|
}
|
||||||
|
return m;
|
||||||
|
};
|
||||||
|
if (plan.what == Do::Keep) {
|
||||||
|
// (its record first: a start that ends while the pieces are renamed leaves a set the next one refuses and deletes)
|
||||||
|
Record k = had;
|
||||||
|
k.pieces = live;
|
||||||
|
k.states = states_id(dir);
|
||||||
|
if (!file_writers::write_whole(dir + kKept, format(k))) {
|
||||||
|
log("vk: shader caches: %s cannot be written: the set there is not kept", kKept);
|
||||||
|
} else if (!move_all(renames(live, f.live_stamp, true), log)) {
|
||||||
|
std::remove((dir + kKept).c_str());
|
||||||
|
} else {
|
||||||
|
log("vk: shader caches: the set for %s kept (%llu MB, %zu files, as <name>.%s.<ext>): a start for it again puts it "
|
||||||
|
"back, and its shaders are not prepared again (RX_VK_CACHE_SET=0: not kept). The caches start empty for "
|
||||||
|
"this stamp", stamp_hex(f.live_stamp).c_str(), (unsigned long long)(f.live_bytes >> 20), live.size(),
|
||||||
|
stamp_hex(f.live_stamp).c_str());
|
||||||
|
}
|
||||||
|
} else if (plan.what == Do::Restore) {
|
||||||
|
bool ok = true;
|
||||||
|
std::vector<std::pair<std::string, std::string>> aside;
|
||||||
|
if (plan.keep_live) {
|
||||||
|
aside = renames(live, f.live_stamp, true);
|
||||||
|
ok = move_all(aside, log);
|
||||||
|
}
|
||||||
|
if (ok && !move_all(renames(kept.pieces, kept.stamp, false), log)) {
|
||||||
|
ok = false;
|
||||||
|
for (size_t j = aside.size(); j-- > 0;) file_writers::replace_file(aside[j].second, aside[j].first);
|
||||||
|
}
|
||||||
|
if (!ok) {
|
||||||
|
log("vk: shader caches: the kept set for %s is not put back: the caches drop the set there, as before",
|
||||||
|
stamp_hex(kept.stamp).c_str());
|
||||||
|
} else {
|
||||||
|
// (the pieces the kept set did not have are of the set there: theirs are another stamp's, or none)
|
||||||
|
if (!plan.keep_live)
|
||||||
|
for (const auto& p : live) {
|
||||||
|
bool theirs = false;
|
||||||
|
for (const auto& q : kept.pieces) theirs = theirs || q.first == p.first;
|
||||||
|
if (!theirs) std::remove((dir + p.first).c_str());
|
||||||
|
}
|
||||||
|
const uint64_t states = states_id(dir);
|
||||||
|
bool kept_now = false;
|
||||||
|
if (plan.keep_live) {
|
||||||
|
Record k = had;
|
||||||
|
k.pieces = live;
|
||||||
|
k.states = states;
|
||||||
|
kept_now = file_writers::write_whole(dir + kKept, format(k));
|
||||||
|
if (!kept_now) {
|
||||||
|
for (const auto& p : live) std::remove((dir + kept_name(p.first, f.live_stamp)).c_str());
|
||||||
|
std::remove((dir + kKept).c_str());
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
std::remove((dir + kKept).c_str());
|
||||||
|
}
|
||||||
|
// (the player met pipelines while the set put back was not there: its notes lack them. The prepare before the
|
||||||
|
// title goes through the state lists, and they are built before the game, as a prepare of every shader would)
|
||||||
|
const bool lists = states && states != kept.states;
|
||||||
|
if (lists && !file_writers::write_whole(dir + kStatesPending, "put back " + stamp_hex(kept.stamp) + "\n"))
|
||||||
|
log("vk: shader caches: %s cannot be written: the pipelines met since the set was kept are built in play",
|
||||||
|
kStatesPending);
|
||||||
|
log("vk: shader caches: the kept set for %s put back (%llu MB, %zu files): its shaders are not prepared again%s%s",
|
||||||
|
stamp_hex(kept.stamp).c_str(), (unsigned long long)(kept.bytes() >> 20), kept.pieces.size(),
|
||||||
|
kept_now ? (", and the set for " + stamp_hex(f.live_stamp) + " is kept in its place").c_str()
|
||||||
|
: plan.keep_live ? ", and the set there could not be kept: deleted" : "",
|
||||||
|
lists ? "; the pipeline state list changed since it was kept: its pipelines are noted before the title" : "");
|
||||||
|
}
|
||||||
|
} else if (plan.what == Do::Drop && f.live && f.live_stamp != f.stamp) {
|
||||||
|
const char* why = !f.on ? "RX_VK_CACHE_SET=0 or RX_VK_PCACHE_KEEP_OLD"
|
||||||
|
: !f.live_named ? "what it was made for is not known"
|
||||||
|
: f.live_build != f.build ? "another build of the translator made it: it is of no use again"
|
||||||
|
: "the disk has no room for it beside a new one (twice the set and 10 GB free)";
|
||||||
|
log("vk: shader caches: the set for %s (%llu MB) is not kept: %s; free %llu MB", stamp_hex(f.live_stamp).c_str(),
|
||||||
|
(unsigned long long)(f.live_bytes >> 20), why, (unsigned long long)(f.free_bytes >> 20));
|
||||||
|
}
|
||||||
|
// (what the set there is made for from now on: this start's caches are written for its stamp)
|
||||||
|
std::string text;
|
||||||
|
const std::string want = format(now);
|
||||||
|
if (!read_text(dir + kRecord, &text) || text != want)
|
||||||
|
if (!file_writers::write_whole(dir + kRecord, want)) log("vk: shader caches: %s cannot be written", kRecord);
|
||||||
|
return plan;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A start that leaves the caches alone (RX_VK_NO_PROGRAM_CACHE: no program cache to say which set is there): a kept set of
|
||||||
|
/// another build is still deleted, and files of a kept set's names that no record names
|
||||||
|
inline void tidy(const std::string& dir, uint64_t build, Log log) {
|
||||||
|
Record kept;
|
||||||
|
const bool known = read_record(dir + kKept, &kept);
|
||||||
|
const auto [swept, swept_bytes] = sweep(dir, known, kept.stamp);
|
||||||
|
if (swept)
|
||||||
|
log("vk: shader caches: %zu files of a kept set no record names deleted (%llu MB)", swept,
|
||||||
|
(unsigned long long)(swept_bytes >> 20));
|
||||||
|
if (known && kept.build != build) {
|
||||||
|
forget(dir, kept);
|
||||||
|
log("vk: shader caches: the kept set for %s (%llu MB) deleted: it is another build's", stamp_hex(kept.stamp).c_str(),
|
||||||
|
(unsigned long long)(kept.bytes() >> 20));
|
||||||
|
} else if (known) {
|
||||||
|
log("vk: shader caches: the kept set for %s (%llu MB) stays: RX_VK_NO_PROGRAM_CACHE", stamp_hex(kept.stamp).c_str(),
|
||||||
|
(unsigned long long)(kept.bytes() >> 20));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace cache_set
|
||||||
129
gpu/vk/capture_behind.h
Normal file
129
gpu/vk/capture_behind.h
Normal file
@@ -0,0 +1,129 @@
|
|||||||
|
/* (Triforce) A submit-time capture's copies behind the game's ordering point (vk_memory.cpp capture_make, RX_VK_CAPTURE_BEHIND).
|
||||||
|
With the replay on the submitting thread (RX_VK_ASYNC off, the default) the game's thread was held for the whole capture before
|
||||||
|
its replay began: the scan of the write marks, then the copy of every page they named. At the Castle Town Market that was
|
||||||
|
~0.35 ms a submit, six submits a frame: 2.1 ms of a 10.6 ms frame, outside the replay (the frame trace of 2026-11-15: scan 0.10
|
||||||
|
ms over ~1,240 ranges, ~2,000 pages copied, 7.8 MB). The scan is the ordering point: the pages written before the submit, their
|
||||||
|
marks taken. The copy need not be on the game's thread before the replay: the submitting thread replays the submission before it
|
||||||
|
returns to the game, so nothing it writes after its submit can reach the copy, and what reads the copy comes after the scan: the
|
||||||
|
replay's reads as submitted, the late pages' second copy and the texture shadow (take_capture), and the GPU's copies into the
|
||||||
|
mirrors once a recording is submitted. So the copy is cut in parts of consecutive pages that the capture's copy helpers take as
|
||||||
|
soon as the scan is done, while the submitting thread replays. A reader of a page whose part nobody has taken yet copies that part
|
||||||
|
itself; one whose part is being copied waits for it. Before a recording goes to the GPU, and before the submitting thread goes
|
||||||
|
back to the game, every part is done. Each page is copied once, by whoever claims its part.
|
||||||
|
Plain C++, no Vulkan: tests/test_capture_behind.py compiles it with stand-ins. */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <atomic>
|
||||||
|
#include <condition_variable>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <memory>
|
||||||
|
#include <mutex>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace mkwvk::capture_behind {
|
||||||
|
|
||||||
|
constexpr int64_t kPartPages = 32; // (128 KB a part: what a reader that needs one of its pages copies itself, ~10 us)
|
||||||
|
|
||||||
|
struct Part {
|
||||||
|
int64_t i0 = 0, i1 = 0; // (the capture's indices [i0, i1): consecutive, none of them copied ahead)
|
||||||
|
std::atomic<uint8_t> state{0}; // (0 nobody's yet, 1 being copied, 2 copied)
|
||||||
|
};
|
||||||
|
|
||||||
|
/// A capture's pages left to copy, in parts. An index is the page's place in the capture (its slot in the capture's buffer).
|
||||||
|
/// copy(i0, i1) is the caller's: it copies indices [i0, i1), and is run once for each part, by the thread that claims it
|
||||||
|
struct Work {
|
||||||
|
std::unique_ptr<Part[]> parts;
|
||||||
|
int nparts = 0;
|
||||||
|
std::vector<int32_t> part_of; // (by index: its part; -1: copied ahead, not one of this work's)
|
||||||
|
std::atomic<int> left{0}; // (the parts not copied yet)
|
||||||
|
std::mutex mu; // (the waits for a part another thread copies)
|
||||||
|
std::condition_variable cv;
|
||||||
|
|
||||||
|
/// The parts: the indices i < n with ahead[i] == 0 (ahead: null for none), in runs of at most part_pages cut at every index
|
||||||
|
/// copied ahead
|
||||||
|
void plan(const uint8_t* ahead, int64_t n, int64_t part_pages = kPartPages) {
|
||||||
|
if (part_pages < 1) part_pages = 1;
|
||||||
|
part_of.assign((size_t)(n > 0 ? n : 0), -1);
|
||||||
|
std::vector<std::pair<int64_t, int64_t>> runs;
|
||||||
|
for (int64_t i = 0; i < n;) {
|
||||||
|
if (ahead && ahead[i]) {
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
int64_t j = i + 1;
|
||||||
|
while (j < n && !(ahead && ahead[j]) && j - i < part_pages) j++;
|
||||||
|
runs.push_back({i, j});
|
||||||
|
i = j;
|
||||||
|
}
|
||||||
|
nparts = (int)runs.size();
|
||||||
|
parts.reset(new Part[(size_t)nparts]);
|
||||||
|
for (int k = 0; k < nparts; k++) {
|
||||||
|
parts[k].i0 = runs[(size_t)k].first, parts[k].i1 = runs[(size_t)k].second;
|
||||||
|
for (int64_t i = parts[k].i0; i < parts[k].i1; i++) part_of[(size_t)i] = k;
|
||||||
|
}
|
||||||
|
left.store(nparts, std::memory_order_release);
|
||||||
|
}
|
||||||
|
/// Part k is this thread's to copy (nobody had taken it)
|
||||||
|
bool claim(int k) {
|
||||||
|
uint8_t free = 0;
|
||||||
|
return parts[k].state.compare_exchange_strong(free, 1, std::memory_order_acq_rel);
|
||||||
|
}
|
||||||
|
/// Part k copied, by the thread that claimed it: its bytes are there for every thread that sees it so
|
||||||
|
void copied(int k) {
|
||||||
|
{
|
||||||
|
std::lock_guard lk(mu);
|
||||||
|
parts[k].state.store(2, std::memory_order_release);
|
||||||
|
left.fetch_sub(1, std::memory_order_acq_rel);
|
||||||
|
}
|
||||||
|
cv.notify_all();
|
||||||
|
}
|
||||||
|
bool done() const { return left.load(std::memory_order_acquire) == 0; }
|
||||||
|
/// Index i is copied (or was copied ahead)
|
||||||
|
bool ready(int64_t i) const {
|
||||||
|
if (i < 0 || (size_t)i >= part_of.size()) return true;
|
||||||
|
const int32_t k = part_of[(size_t)i];
|
||||||
|
return k < 0 || parts[k].state.load(std::memory_order_acquire) == 2;
|
||||||
|
}
|
||||||
|
/// Part k copied on return: by this thread when nobody had taken it, else by whoever took it, waited for. 1: this thread
|
||||||
|
/// copied it, 2: it waited, 0: it was copied already
|
||||||
|
template <class Copy> int make_ready(int k, Copy&& copy) {
|
||||||
|
if (parts[k].state.load(std::memory_order_acquire) == 2) return 0;
|
||||||
|
if (claim(k)) {
|
||||||
|
copy(parts[k].i0, parts[k].i1);
|
||||||
|
copied(k);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
std::unique_lock lk(mu);
|
||||||
|
cv.wait(lk, [&] { return parts[k].state.load(std::memory_order_acquire) == 2; });
|
||||||
|
return 2;
|
||||||
|
}
|
||||||
|
/// Index i copied on return (make_ready of its part)
|
||||||
|
template <class Copy> int ensure(int64_t i, Copy&& copy) {
|
||||||
|
if (i < 0 || (size_t)i >= part_of.size()) return 0;
|
||||||
|
const int32_t k = part_of[(size_t)i];
|
||||||
|
return k < 0 ? 0 : make_ready(k, copy);
|
||||||
|
}
|
||||||
|
/// Every part copied on return: those nobody had taken by this thread, in order, then the others waited for. The parts this
|
||||||
|
/// thread copied
|
||||||
|
template <class Copy> int finish(Copy&& copy) {
|
||||||
|
int made = 0;
|
||||||
|
for (int k = 0; k < nparts; k++)
|
||||||
|
if (parts[k].state.load(std::memory_order_acquire) == 0 && claim(k)) {
|
||||||
|
copy(parts[k].i0, parts[k].i1);
|
||||||
|
copied(k);
|
||||||
|
made++;
|
||||||
|
}
|
||||||
|
std::unique_lock lk(mu);
|
||||||
|
cv.wait(lk, [&] { return done(); });
|
||||||
|
return made;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Whether a capture's copies go behind: helpers to take them, enough pages for them (copy_split's own threshold: under it the
|
||||||
|
/// submitting thread copies all at once, cheaper than waking a helper), and few to copy ahead (the late pages and the texture
|
||||||
|
/// storage: take_capture reads those at once; many of them, as at a scene's load, and the capture is copied whole as before)
|
||||||
|
inline bool worth_behind(int helpers, int64_t pages, int64_t ahead, int64_t min_pages = 128, int64_t max_ahead = 512) {
|
||||||
|
return helpers > 0 && pages - ahead >= min_pages && ahead <= max_ahead;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::capture_behind
|
||||||
81
gpu/vk/checkpoint_kinds.h
Normal file
81
gpu/vk/checkpoint_kinds.h
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
/* (Triforce) The kinds of RX_VK_CHECKPOINTS markers (vk_draw.cpp checkpoint, checkpoint_mark) and what lies after each, for the
|
||||||
|
device lost's report. The owner's test11 and test12 and two lab runs of five lost the device at the title the same way: the
|
||||||
|
top and the bottom of the pipe both at "write-back of pool ...", the next marker the next frame's first dispatch, never
|
||||||
|
reached. Between those two markers there was a pool's write-back copies, the write-back diff pass, the end of that recording,
|
||||||
|
the whole present (its recording, the blit to the window's image after its acquire, the menu) and the next recording's start
|
||||||
|
and uploads: none of them had a marker, and the report could not say which the GPU stopped in. Now each of those has one, and
|
||||||
|
the report says what the work after the last marker reached is (gap). Plain functions: tests/test_checkpoint_kinds.py. */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace mkwvk::checkpoint_kinds {
|
||||||
|
|
||||||
|
/// A marker's kind as the report names it; its key follows the name in the report (a program, a pool, a texture, a recording's
|
||||||
|
/// timeline value, a swapchain image's index or a chunk count, as the name says)
|
||||||
|
inline const char* name(char c) {
|
||||||
|
switch (c) {
|
||||||
|
case 'd': return "draw of program";
|
||||||
|
case 'c': return "dispatch of program";
|
||||||
|
case 'i': return "INDIRECT dispatch of program";
|
||||||
|
case 'm': return "mirror sync of pool";
|
||||||
|
case 'b': return "write-back of pool";
|
||||||
|
case 'u': return "upload of texture";
|
||||||
|
case 'g': return "GPU unswizzle of texture";
|
||||||
|
case 'w': return "write-back diff pass, chunks";
|
||||||
|
case 'r': return "recording begun, value";
|
||||||
|
case 'v': return "uploads of the recording, value";
|
||||||
|
case 'z': return "recording ended, value";
|
||||||
|
case 'p': return "present blit begun, image";
|
||||||
|
case 'M': return "menu drawn over the present, image";
|
||||||
|
case 'P': return "present blit ended, image";
|
||||||
|
default: return "?";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the GPU had to do after a marker of this kind, up to the next marker recorded: where a device lost whose last marker
|
||||||
|
/// reached at the top of the pipe is this one stopped
|
||||||
|
inline const char* gap(char c) {
|
||||||
|
switch (c) {
|
||||||
|
case 'd': case 'c': case 'i':
|
||||||
|
return "that draw or dispatch itself (its marker is set just before it), or what its command buffer records after it before the "
|
||||||
|
"next marker: barriers, copies, clears, queries";
|
||||||
|
case 'm': return "that mirror sync's copies (guest memory -> the pool's VRAM mirror), in the recording's uploads";
|
||||||
|
case 'b':
|
||||||
|
return "that pool's write-back copies (its mirror -> guest memory, and the shadow's fill), then the write-back diff pass if "
|
||||||
|
"one follows (w): with no w after it, the end of the recording (z)";
|
||||||
|
case 'u': case 'g': return "that texture's upload or unswizzle itself";
|
||||||
|
case 'w':
|
||||||
|
return "the write-back diff pass (one compute dispatch over every pool's chunks, reading mirrors and shadows and writing guest "
|
||||||
|
"memory by device address), then its barrier and the end of the recording (z)";
|
||||||
|
case 'r': return "the recording's first commands: its barrier, and the replay's work before its first marked draw or dispatch";
|
||||||
|
case 'v': return "the recording's uploads: mirror pieces filled, mirror syncs, the page table's changes, texture uploads";
|
||||||
|
case 'z':
|
||||||
|
return "the next submission: its waits on the queue (the mirrors' new pieces bound, a swapchain image acquired) or its first "
|
||||||
|
"commands (r, v, or the present's p next)";
|
||||||
|
case 'p': return "the present's blit: the game's picture scaled into the window's image, the HDR pass and the bars, before the menu (M)";
|
||||||
|
case 'M': return "the menu's drawing (Dear ImGui) over the present, then its last barrier (P)";
|
||||||
|
case 'P': return "the present of that image, then the next submission's waits and first commands";
|
||||||
|
default: return "not known (a marker of no named kind)";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The two stages' last markers (sequence numbers; 0: none reached): one sentence of what they mean together
|
||||||
|
inline const char* reading(uint32_t top, uint32_t bottom) {
|
||||||
|
if (!top) return "no marker was reached at the top of the pipe";
|
||||||
|
if (bottom == top)
|
||||||
|
return "every command up to the last marker reached is done and nothing after it began: the GPU stopped in what follows that "
|
||||||
|
"marker (below), or in a wait before the next one";
|
||||||
|
if (bottom < top)
|
||||||
|
return "the work between the bottom's last marker and the top's was running: the GPU stopped in it (a draw or dispatch that "
|
||||||
|
"did not end, or a fault)";
|
||||||
|
return "the bottom of the pipe is past the top: the markers are not in order (several queues or threads), read them with care";
|
||||||
|
}
|
||||||
|
/// (Triforce, P4) The two stages' last markers are of two threads' rings (vk_draw.cpp checkpoint_next: each recording thread
|
||||||
|
/// numbers its own): their numbers say nothing of each other's order
|
||||||
|
inline const char* apart() {
|
||||||
|
return "the two are of two threads' recordings (each numbers its own markers: the replay thread's, a helper thread's, the "
|
||||||
|
"present's): their numbers say nothing of their order, read each with the markers around it (below)";
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::checkpoint_kinds
|
||||||
128
gpu/vk/cmd_work.h
Normal file
128
gpu/vk/cmd_work.h
Normal file
@@ -0,0 +1,128 @@
|
|||||||
|
// (Triforce) What the replay's barrier rules ask of the command buffer they record into, kept per command buffer: how many
|
||||||
|
// commands went into it, and where its last full barrier, its last draw pass's end and its last dispatch were (vk_work.h counts
|
||||||
|
// the commands; vk_texture.cpp full_barrier, pass_start_barrier and pass_resume_barrier, vk_draw.cpp dispatch read and set the
|
||||||
|
// marks). Apart from Vulkan's calls so that tests/test_barrier_rules.py, test_pass_barrier.py and test_dispatch_rule_threads.py can
|
||||||
|
// run it.
|
||||||
|
//
|
||||||
|
// After NX2navi's per-context state (runtime/vk_exec.h VkExec.recorded, barrier_at, bar_dirty, bar_passes, bar_dispatches;
|
||||||
|
// vk_exec.c barrier_to_why, chunk_record, group_put): there a recording context counts its own commands, so its rules hold inside
|
||||||
|
// the worker threads' chunks. Here one count served every thread (work_epoch, an atomic add on every command from every recording
|
||||||
|
// thread), so each rule stood aside while the Replay helper threads recorded (most frames at the Market: about 500 full barriers a
|
||||||
|
// frame, no pass start in its fragment stages only). A command buffer is recorded by one thread at a time (Vulkan asks for that),
|
||||||
|
// and its state here is only touched by whoever records into it, so nothing in it needs to be atomic; the table that finds it is.
|
||||||
|
// Another thread's commands go into its own command buffers and leave this one's marks alone; a command buffer two threads take
|
||||||
|
// turns at (a parallel replay's segment init command buffer, vk_texture.cpp InitRedirect) has every turn's commands counted in it.
|
||||||
|
//
|
||||||
|
// RX_VK_BARRIER_CONTEXT=0: the one count of every thread's commands, and the rules standing aside while the helper threads record,
|
||||||
|
// as before (vk_texture.cpp rules_aside).
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <vulkan/vulkan_core.h>
|
||||||
|
|
||||||
|
#include <atomic>
|
||||||
|
#include <cstddef>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <cstring>
|
||||||
|
|
||||||
|
namespace mkwvk {
|
||||||
|
|
||||||
|
extern std::atomic<uint64_t> work_epoch; // (vk_texture.cpp: RX_VK_BARRIER_CONTEXT=0's count, every thread's commands)
|
||||||
|
inline thread_local uint64_t work_mine = 0; // (this thread's commands, into any command buffer: work_between)
|
||||||
|
inline std::atomic<uint64_t> work_submits{0}; // (every queue submit through vk_work.h, any thread's: work_between)
|
||||||
|
|
||||||
|
/// RX_VK_BARRIER_CONTEXT=0: the global count, as before; anything else, or unset: per command buffer. Read at the first command
|
||||||
|
/// (0: not read yet, 1: per command buffer, 2: global)
|
||||||
|
inline std::atomic<int> work_way{0};
|
||||||
|
inline bool work_global() {
|
||||||
|
int w = work_way.load(std::memory_order_relaxed);
|
||||||
|
if (!w) [[unlikely]] {
|
||||||
|
const char* e = std::getenv("RX_VK_BARRIER_CONTEXT");
|
||||||
|
w = e && std::strcmp(e, "0") == 0 ? 2 : 1;
|
||||||
|
work_way.store(w, std::memory_order_relaxed);
|
||||||
|
}
|
||||||
|
return w == 2;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One command buffer's state. The marks are counts of it (work_at) taken right after the command they mark; a mark equal to the
|
||||||
|
/// count now: nothing was recorded into it since
|
||||||
|
struct CmdWork {
|
||||||
|
uint64_t n = 0; // (its commands, and its begins: work_begun)
|
||||||
|
uint64_t barrier_at = UINT64_MAX; // (right after its last full barrier: full_barrier leaves out a second one there)
|
||||||
|
uint64_t pass_end_at = UINT64_MAX; // (right after its last draw pass ended, unless the replay split that pass for a hazard)
|
||||||
|
uint64_t dispatch_at = UINT64_MAX; // (right after its last dispatch: RX_VK_DISPATCH_BARRIER)
|
||||||
|
bool known = true; // (false: the stand-in for a command buffer the table had no room for, never right after anything)
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The table: a slot a command buffer, taken at its first command or begin by whichever thread comes first and kept for good (the
|
||||||
|
/// replay makes its command buffers once: a few hundred at most). Full: cmd_work_none, which no rule relaxes on
|
||||||
|
constexpr size_t kCmdWorkSlots = 1024;
|
||||||
|
struct alignas(64) CmdWorkSlot { // (a cache line each: two threads recording two command buffers share none)
|
||||||
|
std::atomic<VkCommandBuffer> cmd{VK_NULL_HANDLE};
|
||||||
|
CmdWork w;
|
||||||
|
};
|
||||||
|
inline CmdWorkSlot cmd_work_slots[kCmdWorkSlots];
|
||||||
|
inline thread_local CmdWork cmd_work_none{0, UINT64_MAX, UINT64_MAX, UINT64_MAX, false}; // (vk_texture.cpp full_barrier tells it once)
|
||||||
|
inline CmdWork* cmd_work_find(VkCommandBuffer cmd) {
|
||||||
|
if (cmd == VK_NULL_HANDLE) return &cmd_work_none;
|
||||||
|
const size_t h = (size_t)(((uint64_t)(uintptr_t)cmd * 0x9E3779B97F4A7C15ull) >> 54); // (10 bits: kCmdWorkSlots)
|
||||||
|
for (size_t k = 0; k < kCmdWorkSlots; k++) {
|
||||||
|
CmdWorkSlot& s = cmd_work_slots[(h + k) % kCmdWorkSlots];
|
||||||
|
VkCommandBuffer at = s.cmd.load(std::memory_order_acquire);
|
||||||
|
if (at == VK_NULL_HANDLE && s.cmd.compare_exchange_strong(at, cmd, std::memory_order_acq_rel)) return &s.w;
|
||||||
|
if (at == cmd) return &s.w; // (its own, or taken by another thread just now)
|
||||||
|
}
|
||||||
|
return &cmd_work_none;
|
||||||
|
}
|
||||||
|
/// cmd's state (the last one this thread asked for is at hand: most commands go into the command buffer of the one before)
|
||||||
|
inline thread_local VkCommandBuffer cmd_work_last = VK_NULL_HANDLE;
|
||||||
|
inline thread_local CmdWork* cmd_work_last_w = nullptr;
|
||||||
|
inline CmdWork& cmd_work(VkCommandBuffer cmd) {
|
||||||
|
if (cmd != cmd_work_last || !cmd_work_last_w) [[unlikely]] cmd_work_last_w = cmd_work_find(cmd), cmd_work_last = cmd;
|
||||||
|
return *cmd_work_last_w;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The count a mark is taken in: the command buffer's own; RX_VK_BARRIER_CONTEXT=0, every thread's (work_epoch), as before
|
||||||
|
inline uint64_t work_at(const CmdWork& w) {
|
||||||
|
return work_global() ? work_epoch.load(std::memory_order_relaxed) : w.n;
|
||||||
|
}
|
||||||
|
/// Whether nothing was recorded into the command buffer since the mark was taken
|
||||||
|
inline bool work_right_after(const CmdWork& w, uint64_t mark) {
|
||||||
|
return w.known && mark == work_at(w);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A command that does or orders GPU work, into cmd (vk_work.h's macros; by hand after a library recorded into cmd itself)
|
||||||
|
inline void note_work(VkCommandBuffer cmd) {
|
||||||
|
// (RX_VK_BARRIER_CONTEXT=0: an add, not a load and a store: a thread preempted between the two would store a stale count over
|
||||||
|
// another thread's, which could make the count equal the one a barrier was recorded at with work recorded since)
|
||||||
|
if (work_global()) work_epoch.fetch_add(1, std::memory_order_relaxed);
|
||||||
|
else cmd_work(cmd).n++;
|
||||||
|
work_mine++;
|
||||||
|
}
|
||||||
|
inline VkCommandBuffer work_on(VkCommandBuffer cmd) {
|
||||||
|
note_work(cmd);
|
||||||
|
return cmd;
|
||||||
|
}
|
||||||
|
/// cmd begun (vkBeginCommandBuffer): a new recording starts afresh, whatever its handle held before (its last command may have
|
||||||
|
/// been a full barrier, and the work submitted since then is not after it)
|
||||||
|
inline VkCommandBuffer work_begun(VkCommandBuffer cmd) {
|
||||||
|
CmdWork& w = cmd_work(cmd);
|
||||||
|
w.n++;
|
||||||
|
w.barrier_at = w.pass_end_at = w.dispatch_at = UINT64_MAX;
|
||||||
|
return cmd;
|
||||||
|
}
|
||||||
|
/// A submit to the queue (vkQueueSubmit2), any thread's
|
||||||
|
inline VkQueue work_submitted(VkQueue q) {
|
||||||
|
work_submits.fetch_add(1, std::memory_order_relaxed);
|
||||||
|
return q;
|
||||||
|
}
|
||||||
|
/// Whether anything was recorded or submitted between two points of this thread's recording: equal before and after, nothing.
|
||||||
|
/// What crosses command buffers asks this (vk_replay.cpp maybe_split: a pass going on in the next part; finish_recording: the
|
||||||
|
/// early tail's write-back): this thread's own commands, into any command buffer, and every submit to the queue, any thread's
|
||||||
|
/// (another thread's recording reaches the queue between two of this thread's command buffers only by a submit).
|
||||||
|
/// RX_VK_BARRIER_CONTEXT=0: every thread's commands, as before
|
||||||
|
inline uint64_t work_between() {
|
||||||
|
return work_global() ? work_epoch.load(std::memory_order_relaxed) : work_mine + work_submits.load(std::memory_order_relaxed);
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk
|
||||||
333
gpu/vk/debug_names.h
Normal file
333
gpu/vk/debug_names.h
Normal file
@@ -0,0 +1,333 @@
|
|||||||
|
// (Triforce) The names on the Debug tab's Game flags, Cutscenes and Scan amiibo pages (vk_menu.cpp draws them), and what
|
||||||
|
// a page keeps: the list in groups, the choice, the cards shown. No Vulkan, no ImGui and no file here:
|
||||||
|
// tests/test_debug_names.py compiles this header and runs it on bytes of its own making.
|
||||||
|
//
|
||||||
|
// The game knows a flag (a Bool of its GameData) by the MurmurHash3 of its name and nothing else; its files list the
|
||||||
|
// Bools by hash (GameData/GameDataList.Product.<n>.byml.zs) and use their names in the event flows
|
||||||
|
// (Event/EventFlow/<event>.bfevfl.zs). So the flags' names are every text of the event flows whose hash is a Bool's, and
|
||||||
|
// the events are the event flows that hold a script (a flowchart: the others are a cutscene's timelines, which a script
|
||||||
|
// plays). After NX2navi's F9 panel, which found its names the same way; no line of it is here, and no name: both lists
|
||||||
|
// are made in memory from the player's own game files each run, and written nowhere.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstring>
|
||||||
|
#include <string>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
#include "oot_debug_link.h"
|
||||||
|
|
||||||
|
namespace mkwvk::debug_names {
|
||||||
|
|
||||||
|
/// A BYML file's Data.Bool[i].Hash values, sorted, with no number twice: the Bools of a GameDataList (either byte order;
|
||||||
|
/// the game's is the big-endian one). false when the bytes are not such a file. Nothing outside the bytes is read
|
||||||
|
inline bool gamedata_bools(const std::vector<unsigned char>& b, std::vector<uint32_t>& out) {
|
||||||
|
out.clear();
|
||||||
|
const size_t n = b.size();
|
||||||
|
if (n < 16 || !((b[0] == 'B' && b[1] == 'Y') || (b[0] == 'Y' && b[1] == 'B'))) return false;
|
||||||
|
const bool big = b[0] == 'B';
|
||||||
|
bool bad = false; // (a read outside the file: everything read after it is 0, and the file is refused)
|
||||||
|
auto u8 = [&](size_t o) -> uint32_t { return o < n ? b[o] : (bad = true, 0u); };
|
||||||
|
auto u24 = [&](size_t o) -> uint32_t {
|
||||||
|
if (o > n || n - o < 3) return bad = true, 0u;
|
||||||
|
return big ? (uint32_t)b[o] << 16 | (uint32_t)b[o + 1] << 8 | b[o + 2] : (uint32_t)b[o + 2] << 16 | (uint32_t)b[o + 1] << 8 | b[o];
|
||||||
|
};
|
||||||
|
auto u32 = [&](size_t o) -> uint32_t {
|
||||||
|
if (o > n || n - o < 4) return bad = true, 0u;
|
||||||
|
return big ? (uint32_t)b[o] << 24 | (uint32_t)b[o + 1] << 16 | (uint32_t)b[o + 2] << 8 | b[o + 3]
|
||||||
|
: (uint32_t)b[o + 3] << 24 | (uint32_t)b[o + 2] << 16 | (uint32_t)b[o + 1] << 8 | b[o];
|
||||||
|
};
|
||||||
|
// the keys' table: 0xc2, how many, an offset each (and one past the last) from the table's start, the texts
|
||||||
|
const size_t keys = u32(4), root = u32(12);
|
||||||
|
if (bad || !keys || u8(keys) != 0xc2) return false;
|
||||||
|
const uint32_t key_count = u24(keys + 1);
|
||||||
|
auto key_index = [&](const char* name) -> int64_t {
|
||||||
|
const size_t len = strlen(name);
|
||||||
|
for (uint32_t i = 0; i < key_count && !bad; i++) {
|
||||||
|
const size_t at = keys + (size_t)u32(keys + 4 + 4 * (size_t)i);
|
||||||
|
if (!bad && at < n && n - at > len && !memcmp(b.data() + at, name, len + 1)) return i;
|
||||||
|
}
|
||||||
|
return -1;
|
||||||
|
};
|
||||||
|
const int64_t k_data = key_index("Data"), k_bool = key_index("Bool"), k_hash = key_index("Hash");
|
||||||
|
if (bad || k_data < 0 || k_bool < 0 || k_hash < 0) return false;
|
||||||
|
// a dictionary: 0xc1, how many, then eight bytes each: the key's index (three bytes), the value's type, the value
|
||||||
|
auto member = [&](size_t dict, int64_t key, uint32_t* type) -> uint32_t {
|
||||||
|
if (u8(dict) != 0xc1) return bad = true, 0u;
|
||||||
|
const uint32_t count = u24(dict + 1);
|
||||||
|
for (uint32_t i = 0; i < count && !bad; i++) {
|
||||||
|
const size_t e = dict + 4 + 8 * (size_t)i;
|
||||||
|
if (u24(e) == (uint32_t)key) return *type = u8(e + 3), u32(e + 4);
|
||||||
|
}
|
||||||
|
return *type = 0, 0u;
|
||||||
|
};
|
||||||
|
uint32_t type = 0;
|
||||||
|
const size_t data = member(root, k_data, &type);
|
||||||
|
if (bad || type != 0xc1) return false;
|
||||||
|
const size_t bools = member(data, k_bool, &type);
|
||||||
|
if (bad || type != 0xc0 || u8(bools) != 0xc0) return false;
|
||||||
|
// an array: 0xc0, how many, a type byte each (to a multiple of four), then four bytes each
|
||||||
|
const uint32_t count = u24(bools + 1);
|
||||||
|
const size_t values = bools + 4 + (((size_t)count + 3) & ~(size_t)3);
|
||||||
|
if (bad || count > n / 4) return false;
|
||||||
|
out.reserve(count);
|
||||||
|
for (uint32_t i = 0; i < count; i++) {
|
||||||
|
if (u8(bools + 4 + i) != 0xc1) return out.clear(), false;
|
||||||
|
const uint32_t hash = member(u32(values + 4 * (size_t)i), k_hash, &type);
|
||||||
|
if (bad || (type != 0xd3 && type != 0xd1)) return out.clear(), false; // (an unsigned number, or a signed one)
|
||||||
|
if (hash) out.push_back(hash);
|
||||||
|
}
|
||||||
|
std::sort(out.begin(), out.end());
|
||||||
|
out.erase(std::unique(out.begin(), out.end()), out.end());
|
||||||
|
return !out.empty();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An event flow that holds a script: a little-endian BFEVFL whose header counts a flowchart at least (0x20)
|
||||||
|
inline bool event_has_script(const std::vector<unsigned char>& f) {
|
||||||
|
return f.size() >= 0x24 && !memcmp(f.data(), "BFEVFL\0\0", 8) && f[0xc] == 0xff && f[0xd] == 0xfe && (f[0x20] | f[0x21] << 8) >= 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The texts of a file that are flags' names: every run of a name's characters, 3 to OOT_NAME_MAX of them with a 0 after,
|
||||||
|
/// whose hash is one of bools (sorted). Added to out (the same name more than once: make_list sorts that out)
|
||||||
|
inline void flag_names(const std::vector<unsigned char>& f, const std::vector<uint32_t>& bools, std::vector<std::string>& out) {
|
||||||
|
const size_t n = f.size();
|
||||||
|
for (size_t i = 0; i < n;) {
|
||||||
|
if (!oot_name_char(f[i])) {
|
||||||
|
i++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
size_t end = i;
|
||||||
|
while (end < n && oot_name_char(f[end])) end++;
|
||||||
|
const size_t len = end - i;
|
||||||
|
if (end < n && !f[end] && len >= 3 && len <= OOT_NAME_MAX &&
|
||||||
|
std::binary_search(bools.begin(), bools.end(), oot_name_hash((const char*)f.data() + i, len)))
|
||||||
|
out.emplace_back((const char*)f.data() + i, len);
|
||||||
|
i = end;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The event an event flow's file is: its name without ".bfevfl.zs" and without a version before that (".110"); "" when
|
||||||
|
/// the file is not an event flow's or its name not an event's
|
||||||
|
inline std::string event_of_file(const std::string& file) {
|
||||||
|
static const char kEnd[] = ".bfevfl.zs";
|
||||||
|
const size_t end = sizeof kEnd - 1;
|
||||||
|
if (file.size() <= end || file.compare(file.size() - end, end, kEnd)) return "";
|
||||||
|
std::string name = file.substr(0, file.size() - end);
|
||||||
|
const size_t dot = name.rfind('.');
|
||||||
|
if (dot != std::string::npos && dot + 1 < name.size() && name.find_first_not_of("0123456789", dot + 1) == std::string::npos) name.resize(dot);
|
||||||
|
return oot_name_ok(name.c_str()) ? name : "";
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- a page's list: the names sorted, each once, in groups by what they begin with ----
|
||||||
|
struct Group {
|
||||||
|
std::string title;
|
||||||
|
int first = 0, count = 0; // (its names: [first, first + count) of the list)
|
||||||
|
bool begins = false; // (its names all begin with its title: not Other, and not a list without groups)
|
||||||
|
};
|
||||||
|
struct List {
|
||||||
|
std::vector<std::string> names;
|
||||||
|
std::vector<uint32_t> hashes; // (each name's, for the flags)
|
||||||
|
std::vector<Group> groups; // (never empty while there are names)
|
||||||
|
};
|
||||||
|
constexpr int kGroupMin = 6; // (a beginning that fewer names share has no group of its own: they are in "Other")
|
||||||
|
/// What a name begins with: its text before the first _ or . (the whole name when it has neither)
|
||||||
|
inline std::string name_prefix(const std::string& name) {
|
||||||
|
const size_t cut = name.find_first_of("_.");
|
||||||
|
return cut == std::string::npos || cut == 0 ? name : name.substr(0, cut);
|
||||||
|
}
|
||||||
|
/// The list of some names (any order, any name any number of times). grouped false: one group of them all (the amiibo
|
||||||
|
/// folder's files)
|
||||||
|
inline List make_list(std::vector<std::string> names, bool grouped = true, const char* one_title = "All") {
|
||||||
|
List list;
|
||||||
|
std::sort(names.begin(), names.end());
|
||||||
|
names.erase(std::unique(names.begin(), names.end()), names.end());
|
||||||
|
names.erase(std::remove(names.begin(), names.end(), std::string()), names.end());
|
||||||
|
if (names.empty()) return list;
|
||||||
|
std::vector<std::pair<std::string, std::string>> keyed; // (the group's title, the name)
|
||||||
|
keyed.reserve(names.size());
|
||||||
|
if (!grouped) {
|
||||||
|
for (std::string& name : names) keyed.emplace_back(one_title, std::move(name));
|
||||||
|
} else {
|
||||||
|
// (sorted, so the names of one beginning are together: counted in one pass)
|
||||||
|
std::vector<std::string> prefixes;
|
||||||
|
prefixes.reserve(names.size());
|
||||||
|
for (const std::string& name : names) prefixes.push_back(name_prefix(name));
|
||||||
|
std::vector<std::string> counted = prefixes;
|
||||||
|
std::sort(counted.begin(), counted.end());
|
||||||
|
for (size_t i = 0; i < names.size(); i++) {
|
||||||
|
const auto range = std::equal_range(counted.begin(), counted.end(), prefixes[i]);
|
||||||
|
const bool own = range.second - range.first >= kGroupMin;
|
||||||
|
// ("\x7f" sorts Other after every group of a beginning: a name's characters are all below it)
|
||||||
|
keyed.emplace_back(own ? prefixes[i] : std::string("\x7f"), std::move(names[i]));
|
||||||
|
}
|
||||||
|
std::sort(keyed.begin(), keyed.end());
|
||||||
|
}
|
||||||
|
for (size_t i = 0; i < keyed.size(); i++) {
|
||||||
|
if (list.groups.empty() || keyed[i].first != keyed[i - 1].first) {
|
||||||
|
Group g;
|
||||||
|
g.title = keyed[i].first == "\x7f" ? "Other" : keyed[i].first;
|
||||||
|
g.begins = grouped && keyed[i].first != "\x7f";
|
||||||
|
g.first = (int)i;
|
||||||
|
list.groups.push_back(std::move(g));
|
||||||
|
}
|
||||||
|
list.groups.back().count++;
|
||||||
|
list.hashes.push_back(oot_name_hash(keyed[i].second.data(), keyed[i].second.size()));
|
||||||
|
list.names.push_back(std::move(keyed[i].second));
|
||||||
|
}
|
||||||
|
return list;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A name as its card shows it: without the beginning its group is named for, and the _ or . after that (the group's
|
||||||
|
/// title says it once for all of them, and the room goes to where the names differ). The whole name in Other, in a list
|
||||||
|
/// without groups, and where nothing would be left
|
||||||
|
inline std::string card_label(const List& list, int group, int index) {
|
||||||
|
const std::string& name = list.names[index];
|
||||||
|
const Group& g = list.groups[group];
|
||||||
|
if (!g.begins || name.size() <= g.title.size() + 1) return name;
|
||||||
|
return name.substr(g.title.size() + 1);
|
||||||
|
}
|
||||||
|
/// A text that does not fit, cut in its middle: a third of what fits from its start, "..." and the rest from its end (the
|
||||||
|
/// game's names tell each other apart at their ends). fits(text): whether a text fits; it must hold for a shorter one when
|
||||||
|
/// it holds for a longer. The text itself when it fits, "..." when nothing more does. UTF-8 is cut between characters
|
||||||
|
template <class Fits>
|
||||||
|
std::string elide_middle(const std::string& text, Fits fits) {
|
||||||
|
if (fits(text)) return text;
|
||||||
|
const size_t n = text.size();
|
||||||
|
auto cut = [&](size_t keep) { // (keep bytes of the text at most: a head and a tail on character boundaries)
|
||||||
|
size_t head = keep / 3, tail = n - (keep - head);
|
||||||
|
while (head > 0 && ((unsigned char)text[head] & 0xc0) == 0x80) head--;
|
||||||
|
while (tail < n && ((unsigned char)text[tail] & 0xc0) == 0x80) tail++;
|
||||||
|
return text.substr(0, head) + "..." + text.substr(tail);
|
||||||
|
};
|
||||||
|
size_t lo = 0, hi = n ? n - 1 : 0; // (the most bytes kept that fit: lo fits or is 0, more than hi does not)
|
||||||
|
while (lo < hi) {
|
||||||
|
const size_t mid = (lo + hi + 1) / 2;
|
||||||
|
if (fits(cut(mid))) lo = mid;
|
||||||
|
else hi = mid - 1;
|
||||||
|
}
|
||||||
|
return cut(lo);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- a page: a list, the group shown, the name chosen in it, and the cards as last drawn ----
|
||||||
|
struct Page {
|
||||||
|
List list;
|
||||||
|
int group = 0; // (the group shown)
|
||||||
|
int choice = 0; // (the name chosen: its place in list.names, inside the group)
|
||||||
|
int columns = 1, capacity = 1; // (as last drawn: the cards in a row, on a page)
|
||||||
|
int shown = 0; // (the page of cards shown: the choice's, or another the wheel turned to)
|
||||||
|
|
||||||
|
bool empty() const { return list.names.empty(); }
|
||||||
|
int begin() const { return empty() ? 0 : list.groups[group].first; }
|
||||||
|
int end() const { return empty() ? 0 : list.groups[group].first + list.groups[group].count; }
|
||||||
|
int pages() const { return std::max(1, (end() - begin() + std::max(1, capacity) - 1) / std::max(1, capacity)); }
|
||||||
|
/// The cards shown: [first, last) of list.names
|
||||||
|
int first() const { return begin() + std::clamp(shown, 0, pages() - 1) * std::max(1, capacity); }
|
||||||
|
int last() const { return std::min(end(), first() + std::max(1, capacity)); }
|
||||||
|
/// The page of cards the choice is on
|
||||||
|
void reveal() { shown = empty() ? 0 : (choice - begin()) / std::max(1, capacity); }
|
||||||
|
/// A new list (the files were read, or the folder listed again): the choice kept by its name when the new list has it
|
||||||
|
void set_list(List next) {
|
||||||
|
const std::string was = !empty() && choice >= 0 && choice < (int)list.names.size() ? list.names[choice] : std::string();
|
||||||
|
list = std::move(next);
|
||||||
|
group = 0, choice = 0, shown = 0;
|
||||||
|
if (empty()) return;
|
||||||
|
const auto at = std::find(list.names.begin(), list.names.end(), was);
|
||||||
|
if (!was.empty() && at != list.names.end()) {
|
||||||
|
choice = (int)(at - list.names.begin());
|
||||||
|
for (int g = 0; g < (int)list.groups.size(); g++)
|
||||||
|
if (choice >= list.groups[g].first && choice < list.groups[g].first + list.groups[g].count) group = g;
|
||||||
|
}
|
||||||
|
reveal();
|
||||||
|
}
|
||||||
|
/// Another group: dir -1 the one before, 1 the one after, round the ends; its first name chosen
|
||||||
|
void turn_group(int dir) {
|
||||||
|
if (empty()) return;
|
||||||
|
const int n = (int)list.groups.size();
|
||||||
|
group = ((group + dir) % n + n) % n;
|
||||||
|
choice = begin(), shown = 0;
|
||||||
|
}
|
||||||
|
/// The choice moved: by cards (Left and Right), rows (Up and Down), pages, or to the group's first or last name. It
|
||||||
|
/// stays in the group; the page shown follows it
|
||||||
|
void move(int cards, int rows, int by_pages, bool home, bool to_end) {
|
||||||
|
if (empty()) return;
|
||||||
|
const int step = cards + rows * std::max(1, columns) + by_pages * std::max(1, capacity);
|
||||||
|
choice = home ? begin() : to_end ? end() - 1 : std::clamp(choice + step, begin(), end() - 1);
|
||||||
|
reveal();
|
||||||
|
}
|
||||||
|
/// A card clicked: chosen, when it is one of the group's
|
||||||
|
bool choose(int index) {
|
||||||
|
if (empty() || index < begin() || index >= end()) return false;
|
||||||
|
choice = index;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
/// The wheel: the page of cards shown, the choice where it is
|
||||||
|
void turn_shown(int by) { shown = std::clamp(shown + by, 0, pages() - 1); }
|
||||||
|
/// The hashes of the names shown now, and of the chosen one when the wheel turned its page away, at most OOT_WATCH_MAX:
|
||||||
|
/// the flags whose states the page asks for
|
||||||
|
std::vector<uint32_t> watched() const {
|
||||||
|
std::vector<uint32_t> hashes;
|
||||||
|
for (int i = first(); i < last() && (int)hashes.size() < OOT_WATCH_MAX; i++) hashes.push_back(list.hashes[i]);
|
||||||
|
if (!empty() && (choice < first() || choice >= last()) && (int)hashes.size() < OOT_WATCH_MAX) hashes.push_back(list.hashes[choice]);
|
||||||
|
return hashes;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The flags' states as the game's side last said them, for the cards shown
|
||||||
|
struct States {
|
||||||
|
std::vector<uint32_t> hashes; // (the list asked for)
|
||||||
|
std::string states; // (a character each: 1, 0, - or ?; empty until an answer for this list came)
|
||||||
|
/// The list asked for changed: true when a new W line is to be sent. The states known are forgotten then
|
||||||
|
bool ask(const std::vector<uint32_t>& now) {
|
||||||
|
if (now == hashes) return false;
|
||||||
|
hashes = now, states.clear();
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
/// A line of states: kept when it answers the list asked for
|
||||||
|
bool take(const char* line) {
|
||||||
|
std::string got(hashes.size(), '?');
|
||||||
|
if (!oot_states_parse(line, oot_watch_sum(hashes.data(), (int)hashes.size()), (int)hashes.size(), got.data())) return false;
|
||||||
|
states = std::move(got);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
/// A flag's state: 1, 0, - (the save data has no such flag) or ? (not known: no save loaded, or no answer yet)
|
||||||
|
char of(uint32_t hash) const {
|
||||||
|
for (size_t i = 0; i < hashes.size() && i < states.size(); i++)
|
||||||
|
if (hashes[i] == hash) return states[i];
|
||||||
|
return '?';
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The lines a page waits to send, a byte a read (HOS_OPT_DEBUG_ASK, HOS_OPT_AMIIBO_ASK): each ends with a line feed,
|
||||||
|
/// 0 while there is none. The game's side reads them only while the game runs, so of the lines that only say what the
|
||||||
|
/// page shows now (those beginning with `latest`: W) the newest alone is kept, and no more than kLinesMax wait
|
||||||
|
struct Lines {
|
||||||
|
static constexpr int kLinesMax = 32;
|
||||||
|
std::vector<std::string> waiting;
|
||||||
|
size_t at = 0; // (how much of waiting[0] was read)
|
||||||
|
void push(const std::string& line, char latest = 0) {
|
||||||
|
for (size_t i = at ? 1 : 0; latest && i < waiting.size();) { // (an older one not yet begun: dropped)
|
||||||
|
if (!waiting[i].empty() && waiting[i][0] == latest) waiting.erase(waiting.begin() + (long)i);
|
||||||
|
else i++;
|
||||||
|
}
|
||||||
|
if ((int)waiting.size() < kLinesMax) waiting.push_back(line);
|
||||||
|
}
|
||||||
|
int take() {
|
||||||
|
if (waiting.empty()) return 0;
|
||||||
|
if (at < waiting[0].size()) return (unsigned char)waiting[0][at++];
|
||||||
|
waiting.erase(waiting.begin()), at = 0;
|
||||||
|
return '\n';
|
||||||
|
}
|
||||||
|
/// The same line as the last one waiting: a key held on a request repeats it, and one of them is enough
|
||||||
|
bool repeats(const std::string& line) const { return !waiting.empty() && waiting.back() == line; }
|
||||||
|
/// The lines waiting are given up but those beginning with `keep` and one already half read (the Debug tools were
|
||||||
|
/// turned off: what was asked of the game and not yet taken is not done after all; a W still says what is shown)
|
||||||
|
void drop_but(char keep) {
|
||||||
|
for (size_t i = at ? 1 : 0; i < waiting.size();) {
|
||||||
|
if (waiting[i].empty() || waiting[i][0] != keep) waiting.erase(waiting.begin() + (long)i);
|
||||||
|
else i++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
} // namespace mkwvk::debug_names
|
||||||
42
gpu/vk/debug_pending.h
Normal file
42
gpu/vk/debug_pending.h
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
// (Triforce) The Inventory page's change asked of the game and not yet taken by it (vk_menu.cpp keeps it in cheat_action;
|
||||||
|
// host/hos_main.c takes it at Link's update, which does not run while the game's own Options screen has the game paused).
|
||||||
|
// The arithmetic is here so that tests/test_debug_pending.py can run it.
|
||||||
|
//
|
||||||
|
// An action is OOT_CHEAT_ITEM_ACTION + item * 256 + op: op 0..127 adds op - 64 to the item's amount, op 200 unlocks it (1)
|
||||||
|
// or relocks it (0). Only one action waits at a time. Before, each press replaced the last: ten presses of Right on an
|
||||||
|
// amount in the paused Options screen made it one more, and the card showed the amount as it was; a second press of Unlock
|
||||||
|
// unlocked it all the same. Now a press adds to the same item's waiting change, and the card shows what it will be.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
|
||||||
|
#include "oot_cheat_items.h"
|
||||||
|
|
||||||
|
namespace mkwvk::debug_pending {
|
||||||
|
|
||||||
|
/// The change to item's amount the waiting action asks (0: none, or the action is not that item's amount)
|
||||||
|
inline int amount_delta(int pending, int item) {
|
||||||
|
const int base = OOT_CHEAT_ITEM_ACTION + item * 256;
|
||||||
|
return pending >= base && pending < base + 128 ? pending - base - 64 : 0;
|
||||||
|
}
|
||||||
|
/// Whether the waiting action is item's unlock or relock
|
||||||
|
inline bool toggles(int pending, int item) { return pending == OOT_CHEAT_ITEM_ACTION + item * 256 + 200; }
|
||||||
|
|
||||||
|
/// The action after a step (dir 1 or -1) on item's amount, count what the game last said it has (0 or more): the item's
|
||||||
|
/// waiting change and the step together, kept to what the game allows (0 to 999) and to what one action says (-64 to 63).
|
||||||
|
/// Another item's waiting action is replaced, as before
|
||||||
|
inline int amount_step(int pending, int item, int count, int dir) {
|
||||||
|
const int delta = std::clamp(amount_delta(pending, item) + dir, std::max(-64, -count), std::min(63, 999 - count));
|
||||||
|
return OOT_CHEAT_ITEM_ACTION + item * 256 + 64 + delta;
|
||||||
|
}
|
||||||
|
/// The action after a press of Unlock or Relock on item: a second press before the game took the first takes it back (0)
|
||||||
|
inline int toggle(int pending, int item) { return toggles(pending, item) ? 0 : OOT_CHEAT_ITEM_ACTION + item * 256 + 200; }
|
||||||
|
|
||||||
|
/// What item's card shows: count, what the game last said, with the waiting change made (count < 0, unknown: as it is)
|
||||||
|
inline int shown(int pending, int item, int count) {
|
||||||
|
if (count < 0) return count;
|
||||||
|
if (toggles(pending, item)) return count > 0 ? 0 : 1;
|
||||||
|
return std::clamp(count + amount_delta(pending, item), 0, 999);
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::debug_pending
|
||||||
36
gpu/vk/demote_watch.h
Normal file
36
gpu/vk/demote_watch.h
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
/* (Triforce) When a pool's trial in guest memory ends early (vk_memory.cpp demote_trials). A pool the CPU and the GPU both write
|
||||||
|
is tried out of the mirrors: its pieces (or the whole pool) are bound in guest memory, 10 frames to settle, then up to 90
|
||||||
|
measured, and the GPU's time a frame is compared with the window before. Only the measured frames were judged, and only after
|
||||||
|
ten of them: in the owner's log (RTX 5090, 2026-11-15, test4) a 16 MB pool took the GPU 106 ms a frame in guest memory
|
||||||
|
against 7.4 before, and the game's thread waited for that GPU in every capture (memory sync 99 ms a submit, 18 ms a frame
|
||||||
|
over 120 presents): some twenty frames of 100 ms, the same again at its second trial, and a 32 MB pool's two trials at 39 ms
|
||||||
|
against 7.7. A fresh install tries every such pool (a pool that failed twice is kept in triforce_mirror_keep.txt for later
|
||||||
|
runs), which is what a benchmark plays. So a trial far over its baseline ends as soon as a few frames say so, settling or
|
||||||
|
measured, the way a slower one ends: mirrored again and measured once more that way, and judged against that (phase 4), so
|
||||||
|
an early end that was wrong costs a second measure, not the pool. Of the 84 trials in the owner's five logs the four costly
|
||||||
|
ones were 4.7 to 19 times their baseline; every other was under 2.3 times but one, 3.6 times, whose mirrored-again frames were
|
||||||
|
as slow (its baseline was off; it stayed in guest memory): under this rule's line too. RX_VK_DEMOTE_QUICK=0: as before. Plain
|
||||||
|
functions: tests/test_demote_watch.cpp checks them. */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdlib>
|
||||||
|
|
||||||
|
namespace mkwvk::demote_watch {
|
||||||
|
|
||||||
|
constexpr uint64_t kFrames = 3; // (the frames since the trial's start, or its measure's, before one is judged: the GPU's time comes
|
||||||
|
// in as submissions finish, a frame or two late, so the first frames weigh less than they cost)
|
||||||
|
constexpr double kTimes = 3.0; // (over kTimes the baseline and kOverMs more: see above. The trial's own rule, twice the
|
||||||
|
constexpr double kOverMs = 5.0; // baseline and 1 ms after 10 measured frames, stays as it was)
|
||||||
|
|
||||||
|
/// Whether a trial whose frames so far took ms_a_frame of GPU time each, over frames of them, ends now: far slower than base_ms
|
||||||
|
inline bool gross(double base_ms, double ms_a_frame, uint64_t frames) {
|
||||||
|
return base_ms > 0 && frames >= kFrames && ms_a_frame > base_ms * kTimes + kOverMs;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// RX_VK_DEMOTE_QUICK: on unless "0"
|
||||||
|
inline bool quick_on(const char* env) {
|
||||||
|
return !(env && *env && std::atoi(env) == 0 && env[0] == '0');
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::demote_watch
|
||||||
113
gpu/vk/desc_keep.h
Normal file
113
gpu/vk/desc_keep.h
Normal file
@@ -0,0 +1,113 @@
|
|||||||
|
// (Triforce) Per-binding descriptor reuse (vk_draw.cpp build_descriptors), apart from Vulkan so that tests/test_desc_keep.py can run
|
||||||
|
// it. NX2navi's per-binding reuse (runtime/vk_exec_draw.c pb_take and pb_keep, chrg6's) is the model; the rules below are this
|
||||||
|
// renderer's own, from what each of its descriptors is resolved from.
|
||||||
|
//
|
||||||
|
// A draw whose program, bindings, texture slots and uniform updates are all as the draw before it keeps that draw's descriptors whole
|
||||||
|
// (vk_draw.cpp same_desc). In the owner's test7 session that was 0 to 349 draws a frame of 2,300 to 2,870, and "descriptors" was 1.5
|
||||||
|
// to 2.2 ms of a replay of 7.4 to 9.8 ms: a draw that binds one uniform buffer of its own, or anything bound anywhere, built every
|
||||||
|
// descriptor again, each texture through the views' map and the sampler table, each buffer through its pool. Now each descriptor of
|
||||||
|
// a graphics program keeps its last result beside what it was resolved from, for the last kSlots programs a recording thread drew,
|
||||||
|
// and a descriptor whose inputs and stamps are all the same takes that result instead of resolving again. What a result stands on,
|
||||||
|
// by its kind (stamps_of: the stamps are read before the descriptor is resolved, so that one moved by the resolution itself, or by
|
||||||
|
// another thread meanwhile, makes the next one resolve again):
|
||||||
|
// - a sampled texture (TEX): its handle, view type and depth compare as inputs; the slot's version (the slot registered anew), the
|
||||||
|
// samplers' generation (a sampler registration's contents changed), the destroy generation (a texture destroyed or made again:
|
||||||
|
// a kept view would name freed memory, issue 1126), the dummy generation (a frame left a texture for later: a dummy is looked
|
||||||
|
// up again) and the texture epoch (a view's texture is touched once a submission: texture_view_of does it on its first look);
|
||||||
|
// - a user uniform buffer (UBO, c3 on): its address and size; the recording, the stream's inline uniform updates (ub_generation:
|
||||||
|
// an update into the range is a copy with it in) and the bind stamp (gpu_bind_stamp: a pool gone or imported, a mirror made, the
|
||||||
|
// next submission). Only a pool's or a mirror's range is kept: a copy into the recording's stream (an overlay, an unaligned range,
|
||||||
|
// a range read where it was bound) is made again each time, as it is today;
|
||||||
|
// - a stage's own constants (C1): the recording (they are uploaded once a recording, RecordingRes::c1_at);
|
||||||
|
// - a storage buffer the program only reads (SSBO, a single one, not a table): its address and size; the recording and the bind
|
||||||
|
// stamp. A pool's or a mirror's range only, as for UBO.
|
||||||
|
// The rest is resolved every time as before: the driver's banks (c0, c2: kept by their stages' generations already), storage buffers
|
||||||
|
// the program writes (each binding stamps its pages for the write-back), storage buffer tables, texel buffers and storage images.
|
||||||
|
// A failed resolution (no sampler, no buffer) is never kept: the next draw tries again.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <array>
|
||||||
|
#include <cstddef>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace desc_keep {
|
||||||
|
|
||||||
|
enum Kind : uint8_t { kNone = 0, kTex, kUbo, kC1, kSsbo };
|
||||||
|
|
||||||
|
/// The stamps results stand on, read where a descriptor is about to be resolved (only those of its kind are read: stamps_of)
|
||||||
|
struct Stamps {
|
||||||
|
uint64_t rec = 0; // the recording (vk_draw.cpp desc_rec: moved at each recording's begin)
|
||||||
|
uint64_t bind = 0; // gpu_bind_stamp
|
||||||
|
uint64_t ub = 0; // ub_generation
|
||||||
|
uint64_t samp = 0; // sampler_registrations_generation
|
||||||
|
uint64_t destroy = 0; // texture_destroy_generation
|
||||||
|
uint64_t dummy = 0; // texture_dummy_generation
|
||||||
|
uint64_t epoch = 0; // texture_epoch
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The stamps a result of that kind stands on, in a fixed order (unused ones 0)
|
||||||
|
inline std::array<uint64_t, 4> stamps_of(Kind k, const Stamps& s) {
|
||||||
|
switch (k) {
|
||||||
|
case kTex: return {s.samp, s.destroy, s.dummy, s.epoch};
|
||||||
|
case kUbo: return {s.rec, s.bind, s.ub, 0};
|
||||||
|
case kC1: return {s.rec, 0, 0, 0};
|
||||||
|
case kSsbo: return {s.rec, s.bind, 0, 0};
|
||||||
|
default: return {~0ull, ~0ull, ~0ull, ~0ull};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One descriptor's last resolution: what it was resolved from, and its result (a buffer: handle, offset, range; a texture:
|
||||||
|
/// sampler, view as texture_view_of gave it (null: the dummy is drawn), the Texture). flags: a TEX's slot was a registered one
|
||||||
|
struct Elem {
|
||||||
|
uint8_t kind = kNone;
|
||||||
|
uint8_t flags = 0;
|
||||||
|
uint32_t version = 0; // (TEX: the slot's version)
|
||||||
|
uint64_t in[2]{};
|
||||||
|
std::array<uint64_t, 4> at{};
|
||||||
|
uint64_t out[3]{};
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Whether e's result serves a descriptor of kind k with those inputs (version: a TEX's slot version, 0 for the rest), now
|
||||||
|
inline bool take(const Elem& e, Kind k, uint64_t in0, uint64_t in1, uint32_t version, const Stamps& now) {
|
||||||
|
return k != kNone && e.kind == k && e.in[0] == in0 && e.in[1] == in1 && e.version == version && e.at == stamps_of(k, now);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A descriptor resolved: kept with its inputs and the stamps read before it was resolved (before)
|
||||||
|
inline void keep(Elem& e, Kind k, uint64_t in0, uint64_t in1, uint32_t version, const Stamps& before, uint64_t out0,
|
||||||
|
uint64_t out1, uint64_t out2, uint8_t flags = 0) {
|
||||||
|
e.kind = k, e.flags = flags, e.version = version;
|
||||||
|
e.in[0] = in0, e.in[1] = in1;
|
||||||
|
e.at = stamps_of(k, before);
|
||||||
|
e.out[0] = out0, e.out[1] = out1, e.out[2] = out2;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Not kept (a failed resolution, or a result that is not to be taken again)
|
||||||
|
inline void drop(Elem& e) { e.kind = kNone; }
|
||||||
|
|
||||||
|
/// (RX_VK_DESC_KEEP=check) Whether a descriptor resolved again came out as its kept result
|
||||||
|
inline bool same(const Elem& e, uint64_t out0, uint64_t out1, uint64_t out2, uint8_t flags = 0) {
|
||||||
|
return e.out[0] == out0 && e.out[1] == out1 && e.out[2] == out2 && e.flags == flags;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A recording thread's kept results: the last programs drawn, one a slot by the program's address
|
||||||
|
inline constexpr size_t kSlots = 16;
|
||||||
|
struct Prog {
|
||||||
|
const void* prog = nullptr;
|
||||||
|
std::vector<Elem> e; // (by descriptor, in build_descriptors' order)
|
||||||
|
};
|
||||||
|
inline size_t slot_of(const void* prog) {
|
||||||
|
const uintptr_t p = (uintptr_t)prog;
|
||||||
|
return (size_t)((p >> 6) ^ (p >> 13)) % kSlots;
|
||||||
|
}
|
||||||
|
/// prog's kept results, n descriptors: emptied when its slot held another program's, or another count of them
|
||||||
|
inline Prog& prog_of(std::array<Prog, kSlots>& t, const void* prog, size_t n) {
|
||||||
|
Prog& p = t[slot_of(prog)];
|
||||||
|
if (p.prog != prog || p.e.size() != n) {
|
||||||
|
p.prog = prog;
|
||||||
|
p.e.assign(n, Elem{});
|
||||||
|
}
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace desc_keep
|
||||||
95
gpu/vk/desc_sig.h
Normal file
95
gpu/vk/desc_sig.h
Normal file
@@ -0,0 +1,95 @@
|
|||||||
|
// (Triforce) The whole-draw skip by the signature of a draw's descriptor inputs (vk_draw.cpp desc_sig_walk and draw), apart from
|
||||||
|
// Vulkan so that tests/test_desc_sig.py can run it. NX2navi's bind_sig (runtime/vk_exec_draw.c) is the model; its code is not
|
||||||
|
// chrg6's, so this is written anew from what it does, on this renderer's own rules (desc_keep.h).
|
||||||
|
//
|
||||||
|
// A draw keeps the descriptors the draw before it pushed only while nothing at all was bound anew, no inline uniform update came and
|
||||||
|
// no texture slot changed (vk_draw.cpp same_desc): 381 of 2,830 draws a frame in the owner's test18. The others build every
|
||||||
|
// descriptor again, even when what they read is what the draw before them read: a uniform buffer bound for another stage, a texture
|
||||||
|
// bound at a slot the program does not sample. Through the per-binding reuse (desc_keep.h) each of their descriptors then takes the
|
||||||
|
// result it had, and the push finds nothing changed: the build and the compare were the work, 2.0 to 3.3 ms a frame.
|
||||||
|
//
|
||||||
|
// Now such a draw first reads its signature: what each of its descriptors would be resolved from (desc_keep.h's inputs: a texture's
|
||||||
|
// handle and its slot's version, a user uniform buffer's address and size, a read-only storage buffer's address and size words, a
|
||||||
|
// driver bank's stage generation), and the stamps desc_keep.h's rules give each kind it has (stamps_of), read before the inputs.
|
||||||
|
// When it equals the signature of the descriptors at the graphics bind point, which are this program's, the build and the push are
|
||||||
|
// left out: a build would take every one of them from what was kept (each input and stamp is the same) and push nothing.
|
||||||
|
// That holds only where every descriptor of the last build was kept, so the signature stands for
|
||||||
|
// - push programs only (a set's bind point state is not followed), not the mesh fallback, not RenoDX's;
|
||||||
|
// - programs whose every descriptor is of a kind the reuse keeps (shape_of): sampled textures, user uniform buffers (c3 to c18),
|
||||||
|
// a stage's constants (c1), single read-only storage buffers, and the driver's banks (c0, c2), which their own cache keeps by
|
||||||
|
// the stage's generations. Not storage buffers a program writes (each build stamps their pages for the write-back), tables,
|
||||||
|
// texel buffers or storage images;
|
||||||
|
// - builds that dropped nothing (armed): a failed resolution, a copy in the recording's stream (an inline update's overlay, an
|
||||||
|
// unaligned range) or an unbound buffer make the next draw build again.
|
||||||
|
// The textures a hit samples that the GPU wrote (pass hazards: gfx_sampled) are read again from the last build's textures, as a
|
||||||
|
// build would. A signature is the bind point's only once that draw's descriptors were bound: a draw left out between its build and
|
||||||
|
// its bind leaves the last one in place.
|
||||||
|
// RX_VK_DESC_SIG=0: every such draw builds, as before. =check: a hit builds too, and one whose descriptors or sampled textures come
|
||||||
|
// out otherwise than what is pushed is counted and logged ("vk: descriptor signature check").
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <array>
|
||||||
|
#include <cstddef>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
#include "desc_keep.h"
|
||||||
|
|
||||||
|
namespace desc_sig {
|
||||||
|
|
||||||
|
/// The kinds of descriptor a program has, one bit each by desc_keep::Kind (bit kTex, kUbo, kC1, kSsbo); kBank its driver banks
|
||||||
|
enum : uint8_t { kBank = 1u << 7 };
|
||||||
|
constexpr uint8_t bit(desc_keep::Kind k) { return (uint8_t)(1u << k); }
|
||||||
|
|
||||||
|
/// What a program's descriptors are, from its stages' lists (made once a program: they do not change once it is translated)
|
||||||
|
struct Shape {
|
||||||
|
const void* prog = nullptr;
|
||||||
|
bool ok = false; // (every descriptor of a kind the signature stands for)
|
||||||
|
uint8_t kinds = 0;
|
||||||
|
};
|
||||||
|
inline constexpr size_t kShapes = 64;
|
||||||
|
inline size_t shape_slot(const void* prog) {
|
||||||
|
const uintptr_t p = (uintptr_t)prog;
|
||||||
|
return (size_t)((p >> 6) ^ (p >> 14)) % kShapes;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The signature's head: the program, the recording (c1 and the banks live in its streams), and the stamps of each kind it has, in
|
||||||
|
/// desc_keep.h's order (the same stamps a kept result of that kind stands on)
|
||||||
|
inline void put_head(std::vector<uint64_t>& out, const void* prog, uint8_t kinds, const desc_keep::Stamps& now) {
|
||||||
|
out.push_back((uint64_t)(uintptr_t)prog);
|
||||||
|
out.push_back(now.rec);
|
||||||
|
out.push_back(kinds);
|
||||||
|
for (desc_keep::Kind k : {desc_keep::kTex, desc_keep::kUbo, desc_keep::kC1, desc_keep::kSsbo})
|
||||||
|
if (kinds & bit(k))
|
||||||
|
for (uint64_t v : desc_keep::stamps_of(k, now)) out.push_back(v);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A signature, and what its build left besides the descriptors: the textures it sampled through a view (draw_sampled keeps those
|
||||||
|
/// the GPU wrote) and the dummy counts it added (the frame statistics')
|
||||||
|
struct Sig {
|
||||||
|
const void* prog = nullptr;
|
||||||
|
bool armed = false; // (every descriptor of that build was kept: a draw with the same signature may take them all)
|
||||||
|
std::vector<uint64_t> w;
|
||||||
|
std::vector<const void*> tex;
|
||||||
|
int64_t dummies = 0, dummies_known = 0;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Whether a draw of prog whose signature is now may keep the descriptors bound, which bound's draw built
|
||||||
|
inline bool hit(const Sig& bound, const Sig& now, const void* prog) {
|
||||||
|
return bound.armed && bound.prog == prog && now.prog == prog && !now.w.empty() && bound.w == now.w;
|
||||||
|
}
|
||||||
|
/// now's draw built its descriptors and they were bound: its signature is the bind point's (armed: none of them dropped)
|
||||||
|
inline void commit(Sig& bound, Sig& now, bool armed) {
|
||||||
|
std::swap(bound.w, now.w);
|
||||||
|
std::swap(bound.tex, now.tex);
|
||||||
|
bound.prog = now.prog, bound.armed = armed;
|
||||||
|
bound.dummies = now.dummies, bound.dummies_known = now.dummies_known;
|
||||||
|
now.prog = nullptr, now.w.clear();
|
||||||
|
}
|
||||||
|
/// Descriptors bound that no signature stands for (a program the skip does not take, a build that dropped one)
|
||||||
|
inline void forget(Sig& bound) {
|
||||||
|
bound.prog = nullptr, bound.armed = false;
|
||||||
|
bound.w.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace desc_sig
|
||||||
74
gpu/vk/dlssg_pace.h
Normal file
74
gpu/vk/dlssg_pace.h
Normal file
@@ -0,0 +1,74 @@
|
|||||||
|
// (Triforce) Who holds the frame limit while NVIDIA DLSS frame generation runs: Reflex's sleep, where the game's frame
|
||||||
|
// starts, or the game's own wait (host/nn_vi.c hos_vi_native_acquire). After NX2navi's runtime/dlssg_pace.h, whose rule
|
||||||
|
// this is. A function of what the game's presenting thread sees and nothing else, so tests/test_dlssg_pace.py drives
|
||||||
|
// it without a GPU (plain C: the test compiles it as C and as C++).
|
||||||
|
//
|
||||||
|
// The period counts the frames the game draws (the Game frame rate cap's, the host's own). Reflex holds it only while
|
||||||
|
// every one of these is so, and the game's own wait holds the whole period otherwise: the caller allows it (the
|
||||||
|
// Reflex frame limit row on, frames being generated now, a Streamline whose Reflex sleeps here and refused no limit);
|
||||||
|
// there is a period (the caller passes 0 while no frame is generated: a menu, a load, a movie); a present went out,
|
||||||
|
// and within DLSSG_HOLD_IDLE_NS (a minimized window gives the driver no present to time a frame by: Reflex's sleep is
|
||||||
|
// left out then, vk_dlssg.cpp frame_start); the limit Reflex should hold is the one Streamline last took; and the
|
||||||
|
// frames have not shown that Reflex does not hold it (for the rest of the run then).
|
||||||
|
//
|
||||||
|
// The frames are checked over windows of DLSSG_PACE_WINDOW presents Reflex held to one period and one multiplier.
|
||||||
|
// Over half of them sooner than three quarters of the period: nothing holds the game, and the game's own wait takes
|
||||||
|
// the limit for the run. Over half of them about the multiplier times the period apart (within half a period): the
|
||||||
|
// driver counted the frames on screen, the generated ones too, so Reflex is given the period over the multiplier from
|
||||||
|
// then on; the same again (a GPU that only managed that rate) gives the limit to the game's own wait for the run. A
|
||||||
|
// frame a load or a hitch held counts as neither.
|
||||||
|
#pragma once
|
||||||
|
#include <stdint.h>
|
||||||
|
|
||||||
|
#define DLSSG_HOLD_IDLE_NS 200000000ull // no present for this long: Reflex's sleep is left out
|
||||||
|
#define DLSSG_PACE_WINDOW 60 // presents over which Reflex's hold of the limit is checked
|
||||||
|
|
||||||
|
enum { DLSSG_PACE_SAME, DLSSG_PACE_FAST, DLSSG_PACE_SHOWN, DLSSG_PACE_SLOW }; // what the last window found
|
||||||
|
|
||||||
|
typedef struct {
|
||||||
|
uint64_t last_ns; // the newest frame start Reflex held, 0 when the game's own wait held it
|
||||||
|
uint64_t last_period; // its period
|
||||||
|
int last_mult; // and its multiplier
|
||||||
|
int window, fast, slow; // frames counted in the window, those too soon, those about the multiplier periods apart
|
||||||
|
int shown; // Reflex counts the frames on screen: it is given the period over the multiplier
|
||||||
|
int bad; // Reflex did not hold the limit: the game's own wait holds it for the rest of the run
|
||||||
|
} DlssgPace;
|
||||||
|
|
||||||
|
// now: the clock where the game's frame starts. period_ns: the period of the cap in effect (0: no frame is generated
|
||||||
|
// now, or no limit). mult: frames on screen per drawn frame now. hosted_ns: the clock after the newest present, 0
|
||||||
|
// before the first. allowed: Reflex may hold the limit. applied_us: the limit Streamline last took. *want_us: the
|
||||||
|
// limit Reflex should hold, 0 none. *found: what a window that ended here found (DLSSG_PACE_*). Returns 1 when Reflex
|
||||||
|
// holds this frame's limit: the game's own wait steps aside, and only then.
|
||||||
|
static inline int dlssg_pace_step(DlssgPace *p, uint64_t now, uint64_t period_ns, int mult, uint64_t hosted_ns,
|
||||||
|
int allowed, uint32_t applied_us, uint32_t *want_us, int *found) {
|
||||||
|
*found = DLSSG_PACE_SAME;
|
||||||
|
if (mult < 1) mult = 1;
|
||||||
|
if (!p->bad && p->last_ns && now >= p->last_ns && p->last_period == period_ns && p->last_mult == mult) {
|
||||||
|
uint64_t d = now - p->last_ns, m = (uint64_t)mult;
|
||||||
|
if (d < period_ns / 4 * 3) p->fast++;
|
||||||
|
else if (m > 1 && d * 2 > period_ns * (2 * m - 1) && d * 2 < period_ns * (2 * m + 1)) p->slow++;
|
||||||
|
if (++p->window >= DLSSG_PACE_WINDOW) {
|
||||||
|
if (p->fast > DLSSG_PACE_WINDOW / 2) p->bad = 1, *found = DLSSG_PACE_FAST;
|
||||||
|
else if (p->slow > DLSSG_PACE_WINDOW / 2 && p->shown) p->bad = 1, *found = DLSSG_PACE_SLOW;
|
||||||
|
else if (p->slow > DLSSG_PACE_WINDOW / 2) p->shown = 1, *found = DLSSG_PACE_SHOWN;
|
||||||
|
p->window = p->fast = p->slow = 0;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
p->window = p->fast = p->slow = 0; // a window counts only frames Reflex held to one period and multiplier
|
||||||
|
}
|
||||||
|
uint64_t us = 0;
|
||||||
|
// a present stamped after now went out while this thread read the clock: as recent as one can be
|
||||||
|
int live = hosted_ns && (hosted_ns >= now || now - hosted_ns < DLSSG_HOLD_IDLE_NS);
|
||||||
|
if (allowed && period_ns && !p->bad && live) {
|
||||||
|
uint64_t per = p->shown ? period_ns / (uint64_t)mult : period_ns;
|
||||||
|
us = (per + 500) / 1000;
|
||||||
|
if (!us) us = 1;
|
||||||
|
if (us > 0x7fffffffu) us = 0; // no limit Reflex could be given: the game's own wait keeps it
|
||||||
|
}
|
||||||
|
*want_us = (uint32_t)us;
|
||||||
|
int held = us && applied_us == (uint32_t)us;
|
||||||
|
p->last_ns = held ? now : 0;
|
||||||
|
p->last_period = held ? period_ns : 0;
|
||||||
|
p->last_mult = mult;
|
||||||
|
return held;
|
||||||
|
}
|
||||||
@@ -440,13 +440,6 @@ void EmitSetAttribute(EmitContext& ctx, IR::Attribute attr, Id value, [[maybe_un
|
|||||||
value = ctx.OpBitcast(output->type, value);
|
value = ctx.OpBitcast(output->type, value);
|
||||||
}
|
}
|
||||||
|
|
||||||
static constexpr IR::Attribute cd0 = IR::Attribute::ClipDistance0;
|
|
||||||
static constexpr IR::Attribute cd7 = IR::Attribute::ClipDistance7;
|
|
||||||
|
|
||||||
if (attr >= cd0 && attr <= cd7) {
|
|
||||||
const u32 idx = (u32) attr - (u32) cd0;
|
|
||||||
clip_distance_written.set(idx);
|
|
||||||
}
|
|
||||||
ctx.OpStore(output->pointer, value);
|
ctx.OpStore(output->pointer, value);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -197,11 +197,9 @@ void EmitPrologue(EmitContext& ctx) {
|
|||||||
|
|
||||||
if (Sirit::ValidId(ctx.clip_distances)) {
|
if (Sirit::ValidId(ctx.clip_distances)) {
|
||||||
for (u32 i = 0; i < ctx.profile.max_user_clip_distances; ++i) {
|
for (u32 i = 0; i < ctx.profile.max_user_clip_distances; ++i) {
|
||||||
if (!clip_distance_written.test(i)) {
|
const Id idx = ctx.Const(i);
|
||||||
const Id idx = ctx.Const(i);
|
const Id element = ctx.OpAccessChain(ctx.output_f32, ctx.clip_distances, idx);
|
||||||
const Id element = ctx.OpAccessChain(ctx.output_f32, ctx.clip_distances, idx);
|
ctx.OpStore(element, ctx.Const(0.0f));
|
||||||
ctx.OpStore(element, ctx.Const(0.0f));
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2312,7 +2312,11 @@ void EmitContext::DefineMesh(const IR::Program& program) {
|
|||||||
mesh_attribute_words = (std::max)(
|
mesh_attribute_words = (std::max)(
|
||||||
mesh_primitive_base + Common::DivCeil(mesh_info.max_primitives, 32U) * primitive_lines * 32,
|
mesh_primitive_base + Common::DivCeil(mesh_info.max_primitives, 32U) * primitive_lines * 32,
|
||||||
1U);
|
1U);
|
||||||
mesh_map_bytes = 4 + mesh_info.Corners() * mesh_info.max_primitives;
|
// Whole blocks of 32 primitives, as the attributes: the program writes the index bytes of every primitive block, lanes past
|
||||||
|
// max_primitives too (a count of 0 wraps its min(count - 1, lane) to every lane). Sized to max_primitives, those writes were
|
||||||
|
// clamped onto the last byte, the last primitive's last corner: with 126 primitives (most of OoT's) a stray vertex, flashing
|
||||||
|
// triangles over Kokiri's foliage and ground (as nxnative found: "primitive index bytes for every primitive block")
|
||||||
|
mesh_map_bytes = 4 + mesh_info.Corners() * Common::DivCeil(mesh_info.max_primitives, 32U) * 32;
|
||||||
mesh_shared_u32 = TypePointer(spv::StorageClass::Workgroup, U32[1]);
|
mesh_shared_u32 = TypePointer(spv::StorageClass::Workgroup, U32[1]);
|
||||||
mesh_attributes = DefineVariable(*this, TypeArray(U32[1], Const(mesh_attribute_words)),
|
mesh_attributes = DefineVariable(*this, TypeArray(U32[1], Const(mesh_attribute_words)),
|
||||||
std::nullopt, spv::StorageClass::Workgroup);
|
std::nullopt, spv::StorageClass::Workgroup);
|
||||||
|
|||||||
@@ -19,8 +19,6 @@
|
|||||||
|
|
||||||
namespace Shader::Backend::SPIRV {
|
namespace Shader::Backend::SPIRV {
|
||||||
|
|
||||||
static std::bitset<8> clip_distance_written;
|
|
||||||
|
|
||||||
using Sirit::Id;
|
using Sirit::Id;
|
||||||
|
|
||||||
class VectorTypes {
|
class VectorTypes {
|
||||||
|
|||||||
@@ -284,6 +284,9 @@ inline LoweredProgram LowerToMaxwellWith(std::span<const Instruction> code, u32
|
|||||||
if (!entered)
|
if (!entered)
|
||||||
for (u32 pc = index + 1; pc < target; ++pc) dead_instructions[pc] = true;
|
for (u32 pc = index + 1; pc < target; ++pc) dead_instructions[pc] = true;
|
||||||
}
|
}
|
||||||
|
// (hookshot) A shared memory store (STS) not yet followed by a barrier: an LDS after it may read what another lane of the
|
||||||
|
// warp stored (see LDS below)
|
||||||
|
bool shared_store_pending = false;
|
||||||
for (u32 index = 0; index < code.size(); ++index) {
|
for (u32 index = 0; index < code.size(); ++index) {
|
||||||
auto insn = code[index];
|
auto insn = code[index];
|
||||||
result.source_instruction_offsets.push_back(static_cast<u32>(result.instructions.size()));
|
result.source_instruction_offsets.push_back(static_cast<u32>(result.instructions.size()));
|
||||||
@@ -1701,6 +1704,7 @@ inline LoweredProgram LowerToMaxwellWith(std::span<const Instruction> code, u32
|
|||||||
case 0xb1d: // BAR.SYNC 0 (bit 80: .DEFER_BLOCKING), full workgroup.
|
case 0xb1d: // BAR.SYNC 0 (bit 80: .DEFER_BLOCKING), full workgroup.
|
||||||
if (insn.words[1] || (insn.words[2] & ~0x10000U) || native_predicate != 7)
|
if (insn.words[1] || (insn.words[2] & ~0x10000U) || native_predicate != 7)
|
||||||
return fail(index, "native workgroup barrier mode");
|
return fail(index, "native workgroup barrier mode");
|
||||||
|
shared_store_pending = false;
|
||||||
emit(0xf0a8000000000000ULL | predicate | (u64{1} << 43) |
|
emit(0xf0a8000000000000ULL | predicate | (u64{1} << 43) |
|
||||||
(u64{1} << 44) | (u64{1} << 39) | (u64{7} << 39));
|
(u64{1} << 44) | (u64{1} << 39) | (u64{7} << 39));
|
||||||
break;
|
break;
|
||||||
@@ -1757,6 +1761,23 @@ inline LoweredProgram LowerToMaxwellWith(std::span<const Instruction> code, u32
|
|||||||
}
|
}
|
||||||
const bool store = opcode != 0x984;
|
const bool store = opcode != 0x984;
|
||||||
const u32 data = store ? insn.Bits(32, 8) : static_cast<u32>(dest);
|
const u32 data = store ? insn.Bits(32, 8) : static_cast<u32>(dest);
|
||||||
|
// (hookshot) A warp stores to shared memory (STS) and reads it back (LDS) with no barrier between: the Switch runs a
|
||||||
|
// warp's shared memory accesses in order, Vulkan orders another lane's store and this lane's load only through a
|
||||||
|
// barrier, and the driver's compiler may move them past each other (OoT's 4/3 gameplay network on NVIDIA's cooperative
|
||||||
|
// matrix path: 16x2 bright bars, Hookshot issue 2455). As for LDSM: a subgroup barrier before the first LDS after an
|
||||||
|
// STS, only where the LDS is not predicated (every lane of the warp reaches it). RX_VK_SHARED_NO_LDS_BARRIER=1: none, as before
|
||||||
|
// ((Triforce) both our STS forms, 0x388 and 0x988 with a uniform base, count as the store; the switch read here, as
|
||||||
|
// this file's others are: upstream's env_on is in a header we do not have)
|
||||||
|
static const bool lds_barrier = [] {
|
||||||
|
const char* e = std::getenv("RX_VK_SHARED_NO_LDS_BARRIER");
|
||||||
|
return !(e && *e && *e != '0');
|
||||||
|
}();
|
||||||
|
if (store) {
|
||||||
|
shared_store_pending = true;
|
||||||
|
} else if (shared_store_pending && lds_barrier && native_predicate == 7) {
|
||||||
|
emit(0xef98000000000000ULL | predicate | (u64{1} << 20)); // (MEMBAR, bit 20: barrier_operations.cpp)
|
||||||
|
shared_store_pending = false;
|
||||||
|
}
|
||||||
emit((store ? 0xef58000000000000ULL : 0xef48000000000000ULL) |
|
emit((store ? 0xef58000000000000ULL : 0xef48000000000000ULL) |
|
||||||
predicate | data | (u64{address} << 8) | (u64{insn.Bits(40, 24)} << 20) |
|
predicate | data | (u64{address} << 8) | (u64{insn.Bits(40, 24)} << 20) |
|
||||||
(u64{width} << 48));
|
(u64{width} << 48));
|
||||||
@@ -1877,6 +1898,9 @@ inline LoweredProgram LowerToMaxwellWith(std::span<const Instruction> code, u32
|
|||||||
return fail(index, "native matrix load mode");
|
return fail(index, "native matrix load mode");
|
||||||
u32 address = insn.Bits(24, 8);
|
u32 address = insn.Bits(24, 8);
|
||||||
const u64 offset = insn.Bits(40, 24);
|
const u64 offset = insn.Bits(40, 24);
|
||||||
|
// (Fritchii f130f0c50; ours had it already) An address the results overwrite is copied first, to R254: the paths below
|
||||||
|
// use R246-R253, and R253 is where each matrix's row address is shuffled to, so with the copy there matrices 1-3 of
|
||||||
|
// an .x2/.x4 read matrix 0's rows (OoT has one such LDSM.x4, gpu/vk/tests/ldsm_address_test.cpp)
|
||||||
if (uniform_offset) {
|
if (uniform_offset) {
|
||||||
const u32 uniform = uniform_register(insn.Bits(32, 8));
|
const u32 uniform = uniform_register(insn.Bits(32, 8));
|
||||||
if (uniform == ~0U) return fail(index, "native matrix load uniform offset");
|
if (uniform == ~0U) return fail(index, "native matrix load uniform offset");
|
||||||
|
|||||||
@@ -146,6 +146,10 @@ void TranslateF2I(TranslatorVisitor& v, u64 insn, const IR::F16F32F64& src_a) {
|
|||||||
}()};
|
}()};
|
||||||
const bool is_signed{f2i.is_signed != 0};
|
const bool is_signed{f2i.is_signed != 0};
|
||||||
const auto [max_bound, min_bound] = ClampBounds(f2i.dest_format, is_signed);
|
const auto [max_bound, min_bound] = ClampBounds(f2i.dest_format, is_signed);
|
||||||
|
// (Fritchii eba45b151, zortos293/Lichii#1051 by myshadowww) INT32_MAX rounds up to 2^31 as an F32 bound, so an input at or
|
||||||
|
// above it reached OpConvertFToS out of range, which is undefined. The clamp stops below 2^31 and the exact INT32_MAX is
|
||||||
|
// selected after the conversion
|
||||||
|
const bool signed_f32_i32{f2i.dest_format == DestFormat::I32 && f2i.src_format == SrcFormat::F32 && is_signed};
|
||||||
|
|
||||||
IR::F16F32F64 intermediate;
|
IR::F16F32F64 intermediate;
|
||||||
switch (f2i.src_format) {
|
switch (f2i.src_format) {
|
||||||
@@ -156,7 +160,7 @@ void TranslateF2I(TranslatorVisitor& v, u64 insn, const IR::F16F32F64& src_a) {
|
|||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
case SrcFormat::F32: {
|
case SrcFormat::F32: {
|
||||||
const IR::F32 max_val{v.ir.Imm32(static_cast<f32>(max_bound))};
|
const IR::F32 max_val{v.ir.Imm32(signed_f32_i32 ? 0x1.fffffep30f : static_cast<f32>(max_bound))};
|
||||||
const IR::F32 min_val{v.ir.Imm32(static_cast<f32>(min_bound))};
|
const IR::F32 min_val{v.ir.Imm32(static_cast<f32>(min_bound))};
|
||||||
intermediate = v.ir.FPClamp(rounded_value, min_val, max_val);
|
intermediate = v.ir.FPClamp(rounded_value, min_val, max_val);
|
||||||
break;
|
break;
|
||||||
@@ -173,6 +177,10 @@ void TranslateF2I(TranslatorVisitor& v, u64 insn, const IR::F16F32F64& src_a) {
|
|||||||
|
|
||||||
const size_t bitsize{std::max<size_t>(32, BitSize(f2i.dest_format))};
|
const size_t bitsize{std::max<size_t>(32, BitSize(f2i.dest_format))};
|
||||||
IR::U16U32U64 result{v.ir.ConvertFToI(bitsize, is_signed, intermediate)};
|
IR::U16U32U64 result{v.ir.ConvertFToI(bitsize, is_signed, intermediate)};
|
||||||
|
if (signed_f32_i32) {
|
||||||
|
result = IR::U32{v.ir.Select(v.ir.FPGreaterThanEqual(op_a, v.ir.Imm32(0x1p31f)),
|
||||||
|
v.ir.Imm32(static_cast<u32>((std::numeric_limits<s32>::max)())), result)};
|
||||||
|
}
|
||||||
|
|
||||||
bool handled_special_case = false;
|
bool handled_special_case = false;
|
||||||
const bool special_nan_cases =
|
const bool special_nan_cases =
|
||||||
|
|||||||
@@ -61,6 +61,20 @@ IR::U32 SmallAbs(TranslatorVisitor& v, const IR::U32& value, int bitsize) {
|
|||||||
return IR::U32{v.ir.Select(is_least, value, absolute)};
|
return IR::U32{v.ir.Select(is_least, value, absolute)};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// (Fritchii, zortos293 f6a8f5132) U32 -> F32 rounded toward zero. OpConvertUToF rounds to nearest whatever the IR's rounding
|
||||||
|
// asks, so an input of 2^24 or more could round up: 0xffffff80..0xffffffff became 2^32 instead of 0x4f7fffff (a hash times 2^-32
|
||||||
|
// reached 1.0). The top 24 significant bits make the float, the rest are dropped; an input below 2^24 converts exactly
|
||||||
|
IR::F32 ConvertUnsigned32ToFloatRZ(TranslatorVisitor& v, const IR::U32& source) {
|
||||||
|
auto& ir = v.ir;
|
||||||
|
const IR::U32 msb{ir.Select(ir.INotEqual(source, ir.Imm32(0)), ir.FindUMsb(source), ir.Imm32(0))};
|
||||||
|
const IR::U32 shift{ir.ISub(ir.UMax(msb, ir.Imm32(23)), ir.Imm32(23))};
|
||||||
|
const IR::U32 retained{ir.ShiftRightLogical(source, shift)};
|
||||||
|
const IR::U32 bits{ir.BitwiseOr(ir.ShiftLeftLogical(ir.IAdd(msb, ir.Imm32(127)), ir.Imm32(23)),
|
||||||
|
ir.BitwiseAnd(retained, ir.Imm32(0x7fffff)))};
|
||||||
|
const IR::F32 exact{ir.ConvertIToF(32, 32, false, ir.BitwiseAnd(source, ir.Imm32(0xffffff)))};
|
||||||
|
return IR::F32{ir.Select(ir.IGreaterThan(msb, ir.Imm32(23), false), ir.BitCast<IR::F32>(bits), exact)};
|
||||||
|
}
|
||||||
|
|
||||||
void I2F(TranslatorVisitor& v, u64 insn, IR::U32U64 src) {
|
void I2F(TranslatorVisitor& v, u64 insn, IR::U32U64 src) {
|
||||||
const Encoding i2f{insn};
|
const Encoding i2f{insn};
|
||||||
if (i2f.cc != 0) {
|
if (i2f.cc != 0) {
|
||||||
@@ -99,7 +113,12 @@ void I2F(TranslatorVisitor& v, u64 insn, IR::U32U64 src) {
|
|||||||
.rounding = CastFpRounding(i2f.fp_rounding),
|
.rounding = CastFpRounding(i2f.fp_rounding),
|
||||||
.fmz_mode = IR::FmzMode::DontCare,
|
.fmz_mode = IR::FmzMode::DontCare,
|
||||||
};
|
};
|
||||||
auto value{v.ir.ConvertIToF(size_t(dst_bitsize), size_t(conversion_src_bitsize), is_signed, src, fp_control)};
|
IR::F16F32F64 value;
|
||||||
|
if (!is_signed && conversion_src_bitsize == 32 && dst_bitsize == 32 && i2f.fp_rounding == FpRounding::RZ) {
|
||||||
|
value = ConvertUnsigned32ToFloatRZ(v, IR::U32{src});
|
||||||
|
} else {
|
||||||
|
value = v.ir.ConvertIToF(size_t(dst_bitsize), size_t(conversion_src_bitsize), is_signed, src, fp_control);
|
||||||
|
}
|
||||||
if (i2f.neg != 0) {
|
if (i2f.neg != 0) {
|
||||||
if (i2f.abs != 0 || !is_signed) {
|
if (i2f.abs != 0 || !is_signed) {
|
||||||
// We know the value is positive
|
// We know the value is positive
|
||||||
|
|||||||
42
gpu/vk/file_whole.h
Normal file
42
gpu/vk/file_whole.h
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
// (Triforce, handoff §8) A whole file at once, for file_writers.h: written as <path>.part and put in the old one's place, so
|
||||||
|
// that a write cut short (the process ended in it, a full disk) leaves the old file or the new one, never part of one
|
||||||
|
#pragma once
|
||||||
|
#include "file_writers.h"
|
||||||
|
#include <cstdio>
|
||||||
|
#include <string>
|
||||||
|
#ifdef _WIN32
|
||||||
|
#ifndef NOMINMAX
|
||||||
|
#define NOMINMAX
|
||||||
|
#endif
|
||||||
|
#ifndef WIN32_LEAN_AND_MEAN
|
||||||
|
#define WIN32_LEAN_AND_MEAN
|
||||||
|
#endif
|
||||||
|
#include <windows.h>
|
||||||
|
#endif
|
||||||
|
|
||||||
|
namespace file_writers {
|
||||||
|
/// rename over an existing file, as each platform does it
|
||||||
|
inline bool replace_file(const std::string& from, const std::string& to) {
|
||||||
|
#ifdef _WIN32
|
||||||
|
return MoveFileExA(from.c_str(), to.c_str(), MOVEFILE_REPLACE_EXISTING) != 0;
|
||||||
|
#else
|
||||||
|
return std::rename(from.c_str(), to.c_str()) == 0;
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The whole file: <path>.part written and closed, then put in path's place; false (and no .part left) when any of it fails,
|
||||||
|
/// and then path is as it was. Inside a Writing of its own (refused once the end has begun)
|
||||||
|
inline bool write_whole(const std::string& path, const void* data, size_t n) {
|
||||||
|
Writing w;
|
||||||
|
if (!w) return false;
|
||||||
|
const std::string part = path + ".part";
|
||||||
|
FILE* f = std::fopen(part.c_str(), "wb");
|
||||||
|
if (!f) return false;
|
||||||
|
bool ok = !n || std::fwrite(data, 1, n, f) == n;
|
||||||
|
ok = std::fclose(f) == 0 && ok;
|
||||||
|
ok = ok && replace_file(part, path);
|
||||||
|
if (!ok) std::remove(part.c_str());
|
||||||
|
return ok;
|
||||||
|
}
|
||||||
|
inline bool write_whole(const std::string& path, const std::string& text) { return write_whole(path, text.data(), text.size()); }
|
||||||
|
} // namespace file_writers
|
||||||
96
gpu/vk/file_writers.h
Normal file
96
gpu/vk/file_writers.h
Normal file
@@ -0,0 +1,96 @@
|
|||||||
|
// (Triforce, handoff §8) The renderer's file writes made on threads of their own, and the process's end. The window's close ends
|
||||||
|
// the game through ExitProcess (host/hos_relaunch.c), which stops every other thread wherever it is: a thread in the middle of
|
||||||
|
// a file (a bug capture's picture, the pipeline cache's save in play, the order of the pipelines, a hang report, the prepared
|
||||||
|
// record, the predicted pipelines' list, a debug picture or the frame log) left part of it. Now:
|
||||||
|
// Writing one write, from its open to its close or rename, as a scope: counted while it runs, and refused once the end has
|
||||||
|
// begun (on any thread but the one ending the process: its own last saves go on)
|
||||||
|
// stop the process is about to end (HOS_OPT_EXITING, vk_replay.cpp option_set): no write begins from here on, and the
|
||||||
|
// ones under way are waited for, up to a bound (2 s; RX_VK_WRITERS_WAIT_MS=<ms>, 0: no wait, as before). A write
|
||||||
|
// that is still going then is cut by the exit as before; each such file is one write_whole makes, or one
|
||||||
|
// whose readers take its whole records only, so it is the old file or the new one, or a list one record short
|
||||||
|
// write_whole (file_whole.h) a whole file written as <path>.part and put in the old one's place: the old file or the new
|
||||||
|
// one, never part of one, whenever the process ends
|
||||||
|
// The threads themselves are not joined: they are detached and keep no state that outlives the process; what is waited for
|
||||||
|
// is their write, the only part of them an end can damage. The standard library alone (file_whole.h has the
|
||||||
|
// Windows part): tests/test_file_writers.py compiles it alone
|
||||||
|
#pragma once
|
||||||
|
#include <algorithm>
|
||||||
|
#include <atomic>
|
||||||
|
#include <chrono>
|
||||||
|
#include <cstdio>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <string>
|
||||||
|
#include <thread>
|
||||||
|
|
||||||
|
namespace file_writers {
|
||||||
|
inline std::atomic<bool> g_stopping{false};
|
||||||
|
inline std::atomic<int> g_busy{0}; // (writes under way on other threads than the ending one)
|
||||||
|
inline std::atomic<int> g_refused{0}; // (writes not begun because the end had)
|
||||||
|
inline thread_local bool t_ending = false; // (the thread that called stop: its writes are the exit's own)
|
||||||
|
inline thread_local int t_inside = 0; // (Writings this thread holds: one inside another is part of it, never refused)
|
||||||
|
|
||||||
|
/// One write: `file_writers::Writing w; if (!w) return;` before the file is opened, and the scope ends after its close (and
|
||||||
|
/// rename). The busy count goes up before the stop flag is read and stop sets the flag before it reads the count (both
|
||||||
|
/// sequentially consistent): a write either sees the stop and does not begin, or stop sees it and waits for it. One begun
|
||||||
|
/// inside another on the same thread (a save's own small files) goes on: the outer one is waited for whole
|
||||||
|
class Writing {
|
||||||
|
public:
|
||||||
|
Writing() {
|
||||||
|
if (t_ending || t_inside > 0) {
|
||||||
|
on_ = true;
|
||||||
|
t_inside++, nested_ = true;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
g_busy.fetch_add(1);
|
||||||
|
if (g_stopping.load()) {
|
||||||
|
g_busy.fetch_sub(1);
|
||||||
|
g_refused.fetch_add(1, std::memory_order_relaxed);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
on_ = counted_ = true;
|
||||||
|
t_inside++;
|
||||||
|
}
|
||||||
|
~Writing() {
|
||||||
|
if (counted_ || nested_) t_inside--;
|
||||||
|
if (counted_) g_busy.fetch_sub(1);
|
||||||
|
}
|
||||||
|
Writing(const Writing&) = delete;
|
||||||
|
Writing& operator=(const Writing&) = delete;
|
||||||
|
explicit operator bool() const { return on_; }
|
||||||
|
|
||||||
|
private:
|
||||||
|
bool on_ = false, counted_ = false, nested_ = false;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The end has begun: a loop that writes ends at its next turn
|
||||||
|
inline bool stopping() { return g_stopping.load(std::memory_order_relaxed); }
|
||||||
|
|
||||||
|
/// The bound of stop's wait: 2 s, or RX_VK_WRITERS_WAIT_MS (0: no wait at all, no write refused: as before this)
|
||||||
|
inline int exit_wait_ms() {
|
||||||
|
const char* e = getenv("RX_VK_WRITERS_WAIT_MS");
|
||||||
|
return e && *e ? std::max(0, atoi(e)) : 2000;
|
||||||
|
}
|
||||||
|
|
||||||
|
struct Stopped {
|
||||||
|
int left = 0; // (writes still under way when the wait ended: cut by the exit)
|
||||||
|
double waited_ms = 0; // (how long the wait took)
|
||||||
|
int refused = 0; // (writes not begun since the end began, so far)
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The process is about to end, on this thread: no write begins on another thread from here on, and the ones under way are
|
||||||
|
/// waited for, up to ms. ms 0: nothing is stopped (the old behaviour)
|
||||||
|
inline Stopped stop(int ms) {
|
||||||
|
Stopped s;
|
||||||
|
if (ms <= 0) return s;
|
||||||
|
t_ending = true;
|
||||||
|
g_stopping.store(true);
|
||||||
|
const auto t0 = std::chrono::steady_clock::now();
|
||||||
|
const auto until = t0 + std::chrono::milliseconds(ms);
|
||||||
|
while (g_busy.load() > 0 && std::chrono::steady_clock::now() < until) std::this_thread::sleep_for(std::chrono::milliseconds(1));
|
||||||
|
s.left = g_busy.load();
|
||||||
|
s.waited_ms = std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - t0).count();
|
||||||
|
s.refused = g_refused.load();
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace file_writers
|
||||||
94
gpu/vk/flat_map.h
Normal file
94
gpu/vk/flat_map.h
Normal file
@@ -0,0 +1,94 @@
|
|||||||
|
// (hookshot) From Hookshot eda64e895 (dukenheimer; FlatMap by Corentin Bonjour, Hookshot PR 2774). tests/test_flat_map.py
|
||||||
|
#pragma once
|
||||||
|
#include <cstddef>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <utility>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
/// An integer key's value, kept in one array: open addressing, linear probing, at most 3/4 full, the slot from a
|
||||||
|
/// multiplicative hash. For the replay's per-thread caches, searched on every draw: std::unordered_map divided by a prime
|
||||||
|
/// and followed a node pointer per lookup. No erase of one key (those caches are only cleared whole). A clear keeps a small
|
||||||
|
/// array and gives a large one back. An insert can move every entry: no reference or pointer is held across one.
|
||||||
|
template <class K, class V>
|
||||||
|
class FlatMap {
|
||||||
|
public:
|
||||||
|
struct Slot {
|
||||||
|
K first{};
|
||||||
|
bool used = false;
|
||||||
|
V second{};
|
||||||
|
};
|
||||||
|
|
||||||
|
Slot* find(K key) {
|
||||||
|
if (!count_) return nullptr;
|
||||||
|
for (size_t i = index(key);; i = (i + 1) & mask_) {
|
||||||
|
Slot& s = slots_[i];
|
||||||
|
if (!s.used) return nullptr;
|
||||||
|
if (s.first == key) return &s;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Slot* end() const { return nullptr; }
|
||||||
|
|
||||||
|
V& operator[](K key) {
|
||||||
|
if (Slot* s = find(key)) return s->second;
|
||||||
|
if ((count_ + 1) * 4 > slots_.size() * 3) grow();
|
||||||
|
return place(key).second;
|
||||||
|
}
|
||||||
|
|
||||||
|
size_t size() const { return count_; }
|
||||||
|
size_t capacity() const { return slots_.size(); }
|
||||||
|
|
||||||
|
/// (Triforce) every entry, as f(key, value&): no insert inside f (it can move every entry)
|
||||||
|
template <class F>
|
||||||
|
void each(F&& f) {
|
||||||
|
for (Slot& s : slots_)
|
||||||
|
if (s.used) f(s.first, s.second);
|
||||||
|
}
|
||||||
|
|
||||||
|
void clear() {
|
||||||
|
if (count_ && slots_.size() > kKeptSlots) { // (a cache cleared at its size limit: its memory goes back too)
|
||||||
|
count_ = 0;
|
||||||
|
std::vector<Slot>().swap(slots_);
|
||||||
|
mask_ = 0, shift_ = 64;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
reset();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Empty, the array kept whatever its size: for a map emptied often that fills again to about the same size
|
||||||
|
/// (RecordingRes::c1_at, at every recording's start), so it isn't given back and grown again from 16 each time
|
||||||
|
void reset() {
|
||||||
|
if (!count_) return;
|
||||||
|
count_ = 0;
|
||||||
|
for (Slot& s : slots_)
|
||||||
|
if (s.used) s = Slot{};
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
static constexpr size_t kKeptSlots = 4096;
|
||||||
|
|
||||||
|
size_t index(K key) const { return (size_t)((uint64_t)key * 0x9E3779B97F4A7C15ull >> shift_); }
|
||||||
|
|
||||||
|
Slot& place(K key) {
|
||||||
|
size_t i = index(key);
|
||||||
|
while (slots_[i].used) i = (i + 1) & mask_;
|
||||||
|
Slot& s = slots_[i];
|
||||||
|
s.first = key, s.used = true;
|
||||||
|
count_++;
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
void grow() {
|
||||||
|
std::vector<Slot> old(slots_.empty() ? 16 : slots_.size() * 2);
|
||||||
|
old.swap(slots_);
|
||||||
|
mask_ = slots_.size() - 1;
|
||||||
|
shift_ = 64;
|
||||||
|
for (size_t n = slots_.size(); n > 1; n >>= 1) shift_--;
|
||||||
|
count_ = 0;
|
||||||
|
for (Slot& s : old)
|
||||||
|
if (s.used) place(s.first).second = std::move(s.second);
|
||||||
|
}
|
||||||
|
|
||||||
|
std::vector<Slot> slots_;
|
||||||
|
size_t count_ = 0, mask_ = 0;
|
||||||
|
unsigned shift_ = 64;
|
||||||
|
};
|
||||||
78
gpu/vk/focus_anim.h
Normal file
78
gpu/vk/focus_anim.h
Normal file
@@ -0,0 +1,78 @@
|
|||||||
|
// (Triforce, 11-19) The Mods and Quit entries under the pause menu's System list, and Quit's dialog, animated as the game
|
||||||
|
// animates its own System page buttons and its own dialogs. His "the mods and quit tabs ... have no animations either" and
|
||||||
|
// "the quit menu submenu also has no animations". Each button of the System list (and of the game's dialogs) is the part
|
||||||
|
// PaCommonBtnThin_00 of the Common archive's layout; a dialog is SystemWindow_00 with three of them, 188 units apart. The
|
||||||
|
// parts play their own animations; here are their times and amounts, read from the layouts' curves (the keys: a frame, a
|
||||||
|
// value and a slope, Hermite between them, 60 frames a second, as mods_entry_anim.h reads the title's):
|
||||||
|
// Select 8 frames: the cursor (N_Cursor_00) shown from the first, its alpha from 0 to 1 by frame 4
|
||||||
|
// UnSelect 14 frames: its alpha from 1 to 0 by frame 8, hidden at 14
|
||||||
|
// In 20 frames: the window's question and buttons (N_In_00) from alpha 0 to 1 by frame 15 (slow to start: its
|
||||||
|
// slopes 0.02 and 0.14), and rising into place from 38 units under it (122 to 142 at frame 5, 160 at 20)
|
||||||
|
// Out 2 frames: their alpha from 1 to 0, straight
|
||||||
|
// The buttons' cursor does not slide from one to the next: the button let go fades its cursor out while the new one fades
|
||||||
|
// its in. No ImGui and no Vulkan: tests/test_pause_mods_anim.py compiles it alone.
|
||||||
|
#pragma once
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
#include "mods_entry_anim.h" // (the layouts' curves: Key, curve, frames)
|
||||||
|
|
||||||
|
namespace focus_anim {
|
||||||
|
|
||||||
|
using mods_entry::curve;
|
||||||
|
using mods_entry::frames;
|
||||||
|
using mods_entry::Key;
|
||||||
|
using mods_entry::kFrameNs;
|
||||||
|
|
||||||
|
// (PaCommonBtnThin_00's: N_Cursor_00's alpha)
|
||||||
|
constexpr Key kSelect[] = {{0, 0, 0}, {4, 1, 0}};
|
||||||
|
constexpr Key kUnSelect[] = {{0, 1, 0}, {8, 0, 0}};
|
||||||
|
constexpr int64_t kSelectNs = 8 * kFrameNs, kUnSelectNs = 14 * kFrameNs;
|
||||||
|
// (SystemWindow_00's: N_In_00's alpha and its height, its place 160 units up)
|
||||||
|
constexpr Key kInAlpha[] = {{0, 0, 0.02f}, {15, 1, 0.14f}};
|
||||||
|
constexpr Key kInY[] = {{0, 122, 5.6094f}, {5, 142, 2.50235f}, {20, 160, 0}};
|
||||||
|
constexpr Key kOutAlpha[] = {{0, 1, -0.5f}, {2, 0, -0.5f}};
|
||||||
|
constexpr float kInPlace = 160;
|
||||||
|
constexpr int64_t kInNs = 20 * kFrameNs, kOutNs = 2 * kFrameNs;
|
||||||
|
|
||||||
|
/// A button's cursor from one drawing to the next: how much of it shows (N_Cursor_00's alpha, 0 to 1)
|
||||||
|
struct Focus {
|
||||||
|
bool sel = false; // (it had the cursor at the last step)
|
||||||
|
float from = 0; // (shown when the cursor last came or went)
|
||||||
|
int64_t since_ns = 0; // (...and when)
|
||||||
|
float shown = 0;
|
||||||
|
/// One drawing at now: whether the button has the cursor. settle: as it stands once its animation is done (a still,
|
||||||
|
/// a picture that compares, a window just opened)
|
||||||
|
void step(int64_t now, bool selected, bool settle = false) {
|
||||||
|
if (settle || !since_ns) {
|
||||||
|
sel = selected, from = shown = selected ? 1.0f : 0.0f, since_ns = now;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (selected != sel) from = shown, sel = selected, since_ns = now;
|
||||||
|
const float f = frames(now - since_ns);
|
||||||
|
// (a change back half way: from where it is, as mods_entry::Anim, not from the curve's own start)
|
||||||
|
shown = sel ? std::max(from, curve(kSelect, f)) : from * curve(kUnSelect, f);
|
||||||
|
shown = std::clamp(shown, 0.0f, 1.0f);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
/// A dialog's window t ns after it opened (In) or after it closed (Out, out = true): its alpha, and how far under its place
|
||||||
|
/// it is (the layout's units, up positive: 0 in place)
|
||||||
|
struct Window {
|
||||||
|
float alpha = 1, drop = 0;
|
||||||
|
};
|
||||||
|
inline Window window_at(int64_t t, bool out = false) {
|
||||||
|
Window w;
|
||||||
|
if (t < 0) t = 0;
|
||||||
|
const float f = frames(t);
|
||||||
|
if (out) {
|
||||||
|
w.alpha = std::clamp(curve(kOutAlpha, f), 0.0f, 1.0f);
|
||||||
|
return w;
|
||||||
|
}
|
||||||
|
w.alpha = std::clamp(curve(kInAlpha, f), 0.0f, 1.0f);
|
||||||
|
w.drop = std::max(0.0f, kInPlace - curve(kInY, f));
|
||||||
|
return w;
|
||||||
|
}
|
||||||
|
/// (the window takes input once its In has played, as the game's)
|
||||||
|
inline bool window_ready(int64_t t) { return t >= kInNs; }
|
||||||
|
|
||||||
|
} // namespace focus_anim
|
||||||
93
gpu/vk/foliage_shadow.h
Normal file
93
gpu/vk/foliage_shadow.h
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
// (Triforce) The foliage in the sun's far shadow cascades, made cheaper (vk_draw.cpp draw_cull, for dynamic_state; and
|
||||||
|
// foliage_rate, for the mesh draws), apart from Vulkan so that tests/test_foliage_shadow.py can run its rules. After NX2navi's
|
||||||
|
// 754ded9 (epicmeme546's): 18 to 43 fps at Kokiri Forest's entrance on an RX 7900 XT there, the two halves together.
|
||||||
|
//
|
||||||
|
// The game draws its foliage cards (grass, ferns, leaves) double-sided with mesh programs, into the shadow cascades too: the
|
||||||
|
// sun's three cascades are the layers of one depth array, 1600 x 1600 in the owner's logs. Seen from the sun the stacked,
|
||||||
|
// alpha-tested cards cover the same texels many times over: NX2navi counted about 1,000 million fragments in one such draw of
|
||||||
|
// the far cascade. A cascade here is Triforce's own test for one, as for the cascades' depth bias in dynamic_state: depth only,
|
||||||
|
// square, wider than 768 (the characters' own shadow maps, 768 and less, are never touched).
|
||||||
|
//
|
||||||
|
// 1. The cull (RX_VK_FOLIAGE_SHADOW_CULL, on from layer 1 as NX2navi has it). A draw of a mesh program (native, or through the
|
||||||
|
// compute fallback) into a cascade from the given layer on, that culls neither face, is drawn with its back faces culled (its
|
||||||
|
// front ones with an "f"): half the cards' fragments there. Cull mode is dynamic state, so no other pipeline is made. It is not
|
||||||
|
// quite the same picture: a card that faces away from the sun no longer casts its shadow in those cascades. Where cards stand in
|
||||||
|
// crossed pairs or clumps (the game's foliage almost everywhere) the others cover the same texels; a lone card's shadow can go,
|
||||||
|
// in the middle and far cascades only, where a texel is tens of centimetres to metres of ground and such a card is a fraction of
|
||||||
|
// one. NX2navi kept it so (nothing done for a lone card), and so does this; the near cascade (layer 0) is drawn as the game asks.
|
||||||
|
// RX_VK_FOLIAGE_SHADOW_CULL=off: as the game asks everywhere.
|
||||||
|
//
|
||||||
|
// 2. The shading rate (RX_VK_FOLIAGE_SHADOW_VRS, from layer 2 as NX2navi has it). The same draws, of native mesh programs, from
|
||||||
|
// that layer on shade one fragment per 2 x 2 pixels (VK_KHR_fragment_shading_rate, pipelineFragmentShadingRate): the leaves'
|
||||||
|
// alpha test runs a quarter as often. Depth is still tested and written for each texel; what is coarser is the alpha cut-out, in
|
||||||
|
// 2 x 2 texels of the far cascade, metres of ground each. The rate is dynamic state in the pipelines that can draw such a draw and
|
||||||
|
// in no other: native mesh programs with no colour target and a depth one (pipeline_rate), whose key that already names; they
|
||||||
|
// set it at each draw (1 x 1 into any other target), their stage libraries are keyed apart. Where the device has no
|
||||||
|
// pipelineFragmentShadingRate (older Intel, some APUs), without native mesh shaders, or RX_VK_FOLIAGE_SHADOW_VRS=off, the
|
||||||
|
// extension is not enabled and no pipeline changes.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <cstring>
|
||||||
|
|
||||||
|
namespace mkwvk::foliage_shadow {
|
||||||
|
|
||||||
|
constexpr int kNone = 0, kFront = 1, kBack = 2; // (NVN's faces: NVN_FACE_NONE, _FRONT, _BACK)
|
||||||
|
constexpr uint32_t kCascadeMin = 769; // (a cascade is wider than 768: vk_draw.cpp dynamic_state's test)
|
||||||
|
constexpr int kCullFrom = 1, kRateFrom = 2; // (NX2navi's layers: the cull from the middle cascade, the rate in the far one)
|
||||||
|
|
||||||
|
struct Rule {
|
||||||
|
int from = -1; // (the first cascade layer it applies to; -1: off)
|
||||||
|
bool front = false; // (the front faces culled, not the back ones)
|
||||||
|
};
|
||||||
|
|
||||||
|
/// RX_VK_FOLIAGE_SHADOW_CULL=<layer>[f]: from that layer of the cascades on (the game has three, 0 to 2), the back faces culled,
|
||||||
|
/// the front ones with an "f" after the number. Unset or empty: from layer 1, the back faces (NX2navi's). Anything not starting
|
||||||
|
/// with a digit ("off", "-1"): off
|
||||||
|
inline Rule parse(const char* e) {
|
||||||
|
Rule r;
|
||||||
|
if (!e || !*e) return r.from = kCullFrom, r;
|
||||||
|
if (*e < '0' || *e > '9') return r;
|
||||||
|
r.from = std::atoi(e);
|
||||||
|
r.front = std::strchr(e, 'f') != nullptr;
|
||||||
|
return r;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// RX_VK_FOLIAGE_SHADOW_VRS=<layer>: from that layer of the cascades on, one fragment per 2 x 2 pixels. Unset or empty: from
|
||||||
|
/// layer 2 (NX2navi's). Anything not starting with a digit ("off", "-1"): off, and the device's extension is not asked for.
|
||||||
|
/// Returns the first layer, -1 off
|
||||||
|
inline int parse_rate(const char* e) {
|
||||||
|
if (!e || !*e) return kRateFrom;
|
||||||
|
if (*e < '0' || *e > '9') return -1;
|
||||||
|
return std::atoi(e);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A draw into a cascade from layer `from` on (no colour target, a depth target width x height, square and wider than 768, the
|
||||||
|
/// depth view's first layer at from or past it), of a mesh program, that culls neither face as the game draws it
|
||||||
|
inline bool far_foliage(int from, int game, bool mesh_program, int colour_targets, bool depth, uint32_t width, uint32_t height,
|
||||||
|
uint32_t layer) {
|
||||||
|
if (from < 0 || game != kNone || !mesh_program || colour_targets != 0 || !depth) return false;
|
||||||
|
return width == height && width >= kCascadeMin && layer >= (uint32_t)from;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The face a draw culls: the game's (cull, NVN's face), or under the rule the one it names, for far_foliage's draws
|
||||||
|
inline int cull(const Rule& rule, int game, bool mesh_program, int colour_targets, bool depth, uint32_t width, uint32_t height,
|
||||||
|
uint32_t layer) {
|
||||||
|
if (!far_foliage(rule.from, game, mesh_program, colour_targets, depth, width, height, layer)) return game;
|
||||||
|
return rule.front ? kFront : kBack;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a pipeline has the shading rate as dynamic state: the rate is on (on), the program a native mesh one, no colour
|
||||||
|
/// target and a depth one. Only what its key names (vk_draw.cpp pipeline_key): every pipeline of a key is made alike
|
||||||
|
inline bool pipeline_rate(bool on, bool native_mesh, int colour_targets, bool depth_format) {
|
||||||
|
return on && native_mesh && colour_targets == 0 && depth_format;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The fragment size (1 or 2, both ways) a draw through such a pipeline sets: 2 for far_foliage's draws from layer `from` on
|
||||||
|
inline uint32_t rate(int from, int game, bool native_mesh, int colour_targets, bool depth, uint32_t width, uint32_t height,
|
||||||
|
uint32_t layer) {
|
||||||
|
return far_foliage(from, game, native_mesh, colour_targets, depth, width, height, layer) ? 2u : 1u;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::foliage_shadow
|
||||||
23
gpu/vk/fps_cap.h
Normal file
23
gpu/vk/fps_cap.h
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
/* The Game frame rate cap row's "Monitor's rate" (vk_menu.cpp), as plain code: the cap a display's refresh rate gives.
|
||||||
|
host/nn_vi.c fps_max asks the renderer for it at the start (HOS_OPT_DISPLAY_CAP, vk_replay.cpp's option), and the row
|
||||||
|
says the same number beside its name. After NX2navi's fps_limit = display. No Vulkan, no menu:
|
||||||
|
tests/test_fps_cap_monitor.py checks it. */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
namespace mkwvk::fps_cap {
|
||||||
|
|
||||||
|
/// The row's value for Monitor's rate (frame_rate_cap= in the options file): after 60, 90, 120, 144, 165 and 240 (0 to
|
||||||
|
/// 5); 45, 40 and 30 are -1 to -3
|
||||||
|
constexpr int kMonitor = 6;
|
||||||
|
/// The caps the game's pacing and its time step take (host/nn_vi.c): a display under 30 Hz or over 240 gets the nearest
|
||||||
|
constexpr int kLowest = 30, kHighest = 240;
|
||||||
|
|
||||||
|
/// The cap for a display's refresh rate in mHz (59,940 is 60, 143,856 is 144): its rate rounded to whole frames a
|
||||||
|
/// second, kLowest to kHighest. 0 when the rate is not known (mhz 0 or less): the host then keeps the console's 60.
|
||||||
|
inline int of_display(int mhz) {
|
||||||
|
if (mhz <= 0) return 0;
|
||||||
|
const int hz = (int)(((long long)mhz + 500) / 1000);
|
||||||
|
return hz < kLowest ? kLowest : hz > kHighest ? kHighest : hz;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::fps_cap
|
||||||
46
gpu/vk/frame_trace.h
Normal file
46
gpu/vk/frame_trace.h
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
// (Triforce, wp20-onepct) The per-frame trace's split of a frame (RX_VK_FRAME_TRACE=1: triforce_frames.csv beside the game, one
|
||||||
|
// line a frame, vk_replay.cpp status_tick). Its first columns say what the replay did (the draws' parts, sampled and summed over
|
||||||
|
// every thread that records: the parallel replay's helpers too); they could not say what made a slow frame that was not the
|
||||||
|
// draws': the lab's Market frames over the 99th percentile were 7-21% explained by them (r37, r38, r41). These columns, after
|
||||||
|
// the first ones, split every frame the way the stall line splits a frame of 50 ms or more (stall_watch.h): the replay's own
|
||||||
|
// time (its submits and the present phase, on every thread that replays: on the game's threads with the synchronous replay),
|
||||||
|
// the present's own, the dispatches and the queue submits within the replay, the replay thread's waits for its helpers'
|
||||||
|
// segments, and what is left: outside the replay and the present, the game's own update. Of that, the game's threads spend
|
||||||
|
// some in our work: their fence waits and the captures at their submits; and the waits for write-backs still on the GPU (the
|
||||||
|
// captures' late pages, the reads and bindings put off), with the part of them after the writer's recording was done on the
|
||||||
|
// GPU (the submissions' timeline held back by work stopped at an event wait: vk_replay.cpp vk_timeline_wait), and the queue
|
||||||
|
// submits made while a background import ran. Plain functions: tests/test_frame_trace.py
|
||||||
|
#pragma once
|
||||||
|
#include <cstddef>
|
||||||
|
#include <cstdio>
|
||||||
|
|
||||||
|
namespace mkwvk::frame_trace {
|
||||||
|
|
||||||
|
/// One frame's split, in milliseconds
|
||||||
|
struct Split {
|
||||||
|
double interval = 0; // (present to present)
|
||||||
|
double replay = 0; // (the submits and the present phase, every thread that replays: the stall line's "replay")
|
||||||
|
double present = 0; // (the present's own: the swapchain's image, its fence, the present call)
|
||||||
|
double dispatch = 0, qsubmit = 0; // (in replay: the dispatches, the queue submits)
|
||||||
|
double par_wait = 0; // (in replay: the replay thread's waits for the helper threads' segments)
|
||||||
|
double game_wait = 0; // (the game's fence waits, every thread)
|
||||||
|
double captures = 0; // (the submit-time captures on the game's threads)
|
||||||
|
double wb_wait = 0; // (the waits for write-backs still on the GPU, wherever they were: memory sync's capture, a read)
|
||||||
|
double wb_held = 0; // (of them, the part after the writer's recording was done: the timeline held, not the GPU)
|
||||||
|
double qs_bg = 0; // (the queue submits made while a background import ran)
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Outside the replay and the present: the game's own update, its waits for the replay and the GPU among it. As the stall line's
|
||||||
|
inline double outside(const Split& s) { return s.interval - s.replay - s.present; }
|
||||||
|
|
||||||
|
/// The columns, after the trace's first ones (their names; the line's values in this order: line)
|
||||||
|
constexpr const char* kColumns =
|
||||||
|
"replay_ms,present_ms,dispatch_ms,qsubmit_ms,par_wait_ms,outside_ms,game_wait_ms,captures_ms,wb_wait_ms,wb_held_ms,qs_bg_ms";
|
||||||
|
|
||||||
|
/// The values for kColumns, comma-separated, no line end: snprintf's count
|
||||||
|
inline int line(const Split& s, char* out, size_t cap) {
|
||||||
|
return std::snprintf(out, cap, "%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f,%.2f", s.replay, s.present, s.dispatch,
|
||||||
|
s.qsubmit, s.par_wait, outside(s), s.game_wait, s.captures, s.wb_wait, s.wb_held, s.qs_bg);
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::frame_trace
|
||||||
235
gpu/vk/framegen_rows.h
Normal file
235
gpu/vk/framegen_rows.h
Normal file
@@ -0,0 +1,235 @@
|
|||||||
|
/* (Triforce) The Insert menu's Frame generation row (vk_menu.cpp, opts.framegen) with its last choice, AMD FSR 4, and what
|
||||||
|
AMD FSR 4's frame generation (vk_fsr4.cpp, fsr4fg) decides without a GPU: where the choice is offered, which of the
|
||||||
|
versions AMD's library lists is taken, when a frame is a camera cut, when a context still fits and when it is let go,
|
||||||
|
and the order it is taken down in. Plain functions, no Vulkan, no menu: tests/test_framegen_fsr4.py checks them. */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cmath>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdio>
|
||||||
|
|
||||||
|
namespace mkwvk::framegen_rows {
|
||||||
|
|
||||||
|
/// opts.framegen (saved as frame_generation): 0 Off, 1 Auto, 2 to 8 the multiplier, as before (an options file from before
|
||||||
|
/// keeps its meaning), then 9: AMD FSR frame generation, one frame made by AMD's library between each two of the game's
|
||||||
|
/// ((Triforce, FG-1) whichever version AMD's library has for the card: FSR 4's on an RX 9000, FSR 3.1's elsewhere. The
|
||||||
|
/// name FSR4 stays: it is the value's, saved in options files as 9)
|
||||||
|
enum { OFF = 0, AUTO = 1, kMaxMult = 8, FSR4 = 9, kRowMax = FSR4 };
|
||||||
|
|
||||||
|
/// ((Triforce, FG-9) RX_VK_FRAMEGEN_MULT=<0..9>, over the options file for a lab run: the row's whole range, 9 AMD's frame
|
||||||
|
/// generation among it (it ran 2 to 8 alone before, and a 9 became the renderer's own 8x). Not a number: 2, as before
|
||||||
|
inline int env_row(const char* e) {
|
||||||
|
if (!e || !*e) return 2;
|
||||||
|
int v = 0;
|
||||||
|
if (std::sscanf(e, "%d", &v) != 1) return 2;
|
||||||
|
return v < OFF ? (int)OFF : v > kRowMax ? (int)kRowMax : v;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// ((Triforce, FG-1) The row's value 9 by the version AMD's library makes the frames with: "AMD FSR (3.1.6)", "AMD FSR
|
||||||
|
/// (4.0.2)"; "AMD FSR" before it is known. out holds n bytes
|
||||||
|
inline const char* row_label(char* out, size_t n, const char* version) {
|
||||||
|
if (!out || !n) return "";
|
||||||
|
unsigned v[3] = {0, 0, 0};
|
||||||
|
const bool known = version && std::sscanf(version, "%u.%u.%u", &v[0], &v[1], &v[2]) == 3;
|
||||||
|
if (known) std::snprintf(out, n, "AMD FSR (%u.%u.%u)", v[0], v[1], v[2]);
|
||||||
|
else std::snprintf(out, n, "AMD FSR");
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Why AMD FSR 4 cannot make the frames here (the row says it); FG_OK when it can. FG_NOT_YET: not asked, the device is not
|
||||||
|
/// made yet
|
||||||
|
enum Why { FG_OK, FG_NOT_BUILT, FG_ENV_OFF, FG_NOT_YET, FG_NO_LIBRARY, FG_NO_FILE, FG_NO_MODEL, FG_FAILED, FG_WHY_COUNT };
|
||||||
|
|
||||||
|
/// ((Triforce) Why, in words, for the log line a run with the row at AMD FSR gets when AMD's library cannot make its frames)
|
||||||
|
constexpr const char* why_name(Why w) {
|
||||||
|
return w == FG_OK ? "it can" : w == FG_NOT_BUILT ? "not in this build (HOOKSHOT_FSR, Windows)"
|
||||||
|
: w == FG_ENV_OFF ? "switched off (RX_VK_FSR4_FG=0 or RX_VK_FSR4=0)" : w == FG_NOT_YET ? "not asked yet (no device)"
|
||||||
|
: w == FG_NO_LIBRARY ? "AMD's loader is not beside the game" : w == FG_NO_FILE ? "amd_fidelityfx_framegeneration_vk.dll is not beside it"
|
||||||
|
: w == FG_NO_MODEL ? "AMD's library lists no frame generation it can take for this GPU"
|
||||||
|
: w == FG_FAILED ? "it failed earlier in this run" : "?";
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The row's last value on this PC: AMD FSR 4 where it is offered, 8x elsewhere (the row as it was)
|
||||||
|
constexpr int row_hi(bool fsr4_offered) { return fsr4_offered ? (int)FSR4 : (int)kMaxMult; }
|
||||||
|
/// What the row shows, and what runs, for a value: itself, or 2x for a saved AMD FSR 4 on a PC where it is not offered
|
||||||
|
/// (one made frame a frame either way, there by the renderer's own interpolation). The saved value stays for a PC where it is
|
||||||
|
constexpr int shown(int v, bool fsr4_offered) { return v == FSR4 && !fsr4_offered ? 2 : v; }
|
||||||
|
/// The frames shown for each of the game's: 1 at Off, auto_most with Auto (the display's refresh over 30 fps), the
|
||||||
|
/// multiplier, and 2 with AMD FSR 4 (its library makes the one frame between two)
|
||||||
|
constexpr int multiplier(int v, int auto_most) {
|
||||||
|
return v == FSR4 ? 2 : v == AUTO ? auto_most : v < 1 ? 1 : v > kMaxMult ? (int)kMaxMult : v;
|
||||||
|
}
|
||||||
|
/// Whether AMD's library is asked for the made frame: the row at AMD FSR 4, where it can, and not beside NVIDIA DLSS frame
|
||||||
|
/// generation, whose frames go in at the present itself (dlssg: vk_dlssg.cpp says it runs in this session)
|
||||||
|
constexpr bool fsr4_asked(int v, Why why, bool dlssg) { return v == FSR4 && why == FG_OK && !dlssg; }
|
||||||
|
|
||||||
|
/// A version as AMD's library names it ("4.0.2"; one the graphics driver brings carries a star, "4.0.2*"): false when the
|
||||||
|
/// name does not start with three numbers
|
||||||
|
inline bool parse_version(const char* name, unsigned v[3]) {
|
||||||
|
v[0] = v[1] = v[2] = 0;
|
||||||
|
return name && std::sscanf(name, "%u.%u.%u", &v[0], &v[1], &v[2]) == 3;
|
||||||
|
}
|
||||||
|
/// Of the frame generation versions AMD's library lists for this GPU, the one contexts are made with: the newest, the first
|
||||||
|
/// of equal ones, so FSR 4's (4.x) wherever it is listed (RX 9000), else FSR 3.1's (3.1.x: RX 6000 and 7000, whose driver
|
||||||
|
/// lists 3.1.6 alone, NX2navi 5bdf583 on the owner's RX 7900 XT); -1 when it lists none it can read. ((Triforce, FG-1)
|
||||||
|
/// Before, a list of FSR 3.1 alone gave -1 and the row never offered AMD's frame generation on those cards. The row names
|
||||||
|
/// the version it runs, row_label.) only_fsr4 (RX_VK_FSR4_FG=4): 4.x or nothing, as it was
|
||||||
|
inline int pick_version(const char* const* names, int n, bool only_fsr4) {
|
||||||
|
int best = -1;
|
||||||
|
unsigned bv[3] = {0, 0, 0};
|
||||||
|
for (int i = 0; i < n; i++) {
|
||||||
|
unsigned v[3];
|
||||||
|
if (!parse_version(names[i], v) || (only_fsr4 && v[0] != 4)) continue;
|
||||||
|
const bool newer = v[0] != bv[0] ? v[0] > bv[0] : v[1] != bv[1] ? v[1] > bv[1] : v[2] > bv[2];
|
||||||
|
if (best < 0 || newer) best = i, bv[0] = v[0], bv[1] = v[1], bv[2] = v[2];
|
||||||
|
}
|
||||||
|
return best;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The camera in the world, from the view matrix's three rows as the game has them (row major, view = R * world + t: the
|
||||||
|
/// rows are its right, up and back axes, the last column the translation). The position is -R^T t
|
||||||
|
struct Pose {
|
||||||
|
float pos[3]{}, right[3]{}, up[3]{}, fwd[3]{};
|
||||||
|
};
|
||||||
|
inline Pose pose_of(const float v[12]) {
|
||||||
|
Pose p;
|
||||||
|
for (int i = 0; i < 3; i++) {
|
||||||
|
p.right[i] = v[i], p.up[i] = v[4 + i], p.fwd[i] = -v[8 + i];
|
||||||
|
p.pos[i] = -(v[i] * v[3] + v[4 + i] * v[7] + v[8 + i] * v[11]);
|
||||||
|
}
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
/// What counts as a camera cut: the turn (with a move) in degrees for each 60th of a second, and the least the time between
|
||||||
|
/// the two frames is scaled to (in 60ths of a second; four at the most)
|
||||||
|
struct CutRule {
|
||||||
|
float turn_degrees;
|
||||||
|
float min_scale;
|
||||||
|
};
|
||||||
|
/// Frame generation's (vk_fsr4.cpp fsr4fg): NX2navi's first thresholds (6e87c92), 10 degrees, never less than one 60th
|
||||||
|
constexpr CutRule kFrameGenCut{10.0f, 1.0f};
|
||||||
|
/// The upscalers' (FSR 3.1: vk_fsr3_inputs.cpp; DLSS and FSR 4: vk_temporal.cpp's chain): NX2navi's 2495391, 30 degrees.
|
||||||
|
/// A quick swing of the camera around Link turns it 11 to 18 degrees and moves it 0.9 to 1.4 m a frame: at 10 degrees each
|
||||||
|
/// frame of the swing started a new history, the scene at its render size for those frames, blocky at the smaller modes.
|
||||||
|
/// ((Triforce) never less than one 60th, as frame generation's: NX2navi scaled from a quarter of one, but the time here is
|
||||||
|
/// the CPU's between two recordings of the upscale, not the game's step, and the replay records frames back to back
|
||||||
|
/// after a stall (a shader compile, an area load): a few ms apart, which at a quarter took a 1.25 m move, or a 7.5 degree
|
||||||
|
/// turn with 0.125 m, for a cut, an ordinary swing among them. Above 60 fps a cut is the same move as at 60: an area load
|
||||||
|
/// (5 to 180 m in a frame) or a jump of the camera still is one)
|
||||||
|
constexpr CutRule kUpscalerCut{30.0f, 1.0f};
|
||||||
|
|
||||||
|
/// A camera cut between two frames frame_ms apart: the camera moved over 5 units, or turned over the rule's degrees while
|
||||||
|
/// it moved over half a unit, for each 60th of a second (a slower frame may move in proportion, up to four times, a faster
|
||||||
|
/// one down to the rule's least; the game's world is in metres, its near plane at 0.1). A made frame between the two
|
||||||
|
/// pictures of a cut would blend two views that have nothing to do with each other: such a frame starts a new history and
|
||||||
|
/// none is made; an upscaler's history from before a cut is no use after it. The thresholds are NX2navi's, from its camera
|
||||||
|
/// traces of the same game (a median of 0.05 m and 0.08 degrees a frame in play, area loads of 5 to 180 m in one frame)
|
||||||
|
inline bool camera_cut(const Pose& before, const Pose& now, float frame_ms, const CutRule& rule = kFrameGenCut) {
|
||||||
|
const float least = rule.min_scale > 0.0f ? rule.min_scale : 1.0f;
|
||||||
|
const float k = frame_ms / 16.667f;
|
||||||
|
const float scale = !(frame_ms > 0.0f) ? 1.0f : k < least ? least : k > 4.0f ? 4.0f : k;
|
||||||
|
float move2 = 0.0f, dot = 0.0f, la = 0.0f, lb = 0.0f;
|
||||||
|
for (int i = 0; i < 3; i++) {
|
||||||
|
const float d = now.pos[i] - before.pos[i];
|
||||||
|
move2 += d * d, dot += before.fwd[i] * now.fwd[i], la += before.fwd[i] * before.fwd[i], lb += now.fwd[i] * now.fwd[i];
|
||||||
|
}
|
||||||
|
if (!std::isfinite(move2) || !std::isfinite(dot) || !(la > 0.0f) || !(lb > 0.0f)) return true; // (no camera to compare)
|
||||||
|
const float move = std::sqrt(move2);
|
||||||
|
const float cosine = dot / std::sqrt(la * lb);
|
||||||
|
const float turn = std::acos(cosine < -1.0f ? -1.0f : cosine > 1.0f ? 1.0f : cosine); // (radians)
|
||||||
|
const float turn_limit = rule.turn_degrees * 0.017453293f; // (radians)
|
||||||
|
return move > 5.0f * scale || (turn > turn_limit * scale && move > 0.5f * scale);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a present with the row at AMD FSR 4 shows between the game's last frame and this one
|
||||||
|
enum Made {
|
||||||
|
MADE, ///< AMD's library made the frame
|
||||||
|
OWN, ///< it did not take this present (no scene's depth and motion: a menu, a movie; not available; failed): the
|
||||||
|
///< renderer's own interpolation makes the frame, as at 2x
|
||||||
|
SKIP, ///< a new history (the first frame, a gap, a camera cut): AMD's library was given the frame and no frame is made
|
||||||
|
/// ((Triforce, FG-2) the scene's depth and motion are there but AMD's library is not given the frame: no picture
|
||||||
|
/// without the HUD this frame. The game's frame is shown alone: never a made frame of AMD's with the HUD moved in it
|
||||||
|
/// (NX2navi 5bdf583's rule), and not the renderer's own in its place. (HDR Native and an RGBA16F scene, which hold for
|
||||||
|
/// every present, are OWN: held_made)
|
||||||
|
GAME
|
||||||
|
};
|
||||||
|
/// Whether a present's answer shows the game's frame alone, nothing made or blended before it (SKIP, GAME)
|
||||||
|
constexpr bool game_frame_alone(Made m) { return m == SKIP || m == GAME; }
|
||||||
|
|
||||||
|
/// ((Triforce, FG-2) Why a present with the scene's depth and motion was not given to AMD's library, for its log.
|
||||||
|
/// HELD_HDR_NATIVE: the game's own HDR (present_native_pq), whose picture without the HUD only the game could encode.
|
||||||
|
/// HELD_HDR_SCENE: a run whose scene is RenoDX's RGBA16F (vk_temporal.cpp temporal_scene_format), whose upscale output is
|
||||||
|
/// no picture without the HUD this renderer can encode as the presented one is. HELD_NO_HUDLESS: this frame has none
|
||||||
|
enum Held { HELD_NONE, HELD_NO_HUDLESS, HELD_HDR_NATIVE, HELD_HDR_SCENE, HELD_COUNT };
|
||||||
|
/// The reason, from what the present has: the game's own HDR first, then the run's RGBA16F scene (neither can have a
|
||||||
|
/// picture without the HUD), then the picture without the HUD of this frame
|
||||||
|
constexpr Held held_for(bool hdr_native, bool hdr_scene, bool have_hudless) {
|
||||||
|
return hdr_native ? HELD_HDR_NATIVE : hdr_scene ? HELD_HDR_SCENE : !have_hudless ? HELD_NO_HUDLESS : HELD_NONE;
|
||||||
|
}
|
||||||
|
/// ((Triforce) Whether a reason holds for every present while it lasts (HDR Native, the RGBA16F scene), not for one frame
|
||||||
|
/// that is missing its picture without the HUD. Then AMD's library is not asked at all (fsr4fg::frames_wanted: no inputs
|
||||||
|
/// are made for it) and the renderer's own interpolation makes the one frame, as at 2x (OWN), which is what a saved 9 ran
|
||||||
|
/// as on an RX 6000 or 7000 before AMD's FSR 3.1 frame generation was offered there. Only a frame missing its picture
|
||||||
|
/// without the HUD shows the game's frame alone (GAME)
|
||||||
|
constexpr bool held_whole_run(Held h) { return h == HELD_HDR_NATIVE || h == HELD_HDR_SCENE; }
|
||||||
|
/// What a held present shows: the renderer's own frame for a reason that lasts, the game's frame alone for a missing picture
|
||||||
|
constexpr Made held_made(Held h) { return held_whole_run(h) ? OWN : GAME; }
|
||||||
|
|
||||||
|
/// ((Triforce, FG-2) The picture without the HUD is taken at the gameplay upscale's output (its size ow x oh), the present
|
||||||
|
/// is the window texture (sw x sh) cropped to x, y, w, h (vk_present_rect.h present_source_rect). The part of the output that
|
||||||
|
/// is the present's crop, scaled when the two sizes differ (the handheld size's scaled crop, NX2navi 81cdd67), in whole
|
||||||
|
/// pixels inside the output; all zero when there is nothing to take
|
||||||
|
struct SourceRect {
|
||||||
|
int32_t x0 = 0, y0 = 0, x1 = 0, y1 = 0;
|
||||||
|
};
|
||||||
|
inline SourceRect hudless_source(uint32_t ow, uint32_t oh, uint32_t sw, uint32_t sh, uint32_t x, uint32_t y, uint32_t w,
|
||||||
|
uint32_t h) {
|
||||||
|
SourceRect r;
|
||||||
|
if (!ow || !oh || !sw || !sh || !w || !h || x >= sw || y >= sh) return r;
|
||||||
|
const uint64_t rx = x + w > sw ? sw : (uint64_t)x + w, ry = y + h > sh ? sh : (uint64_t)y + h;
|
||||||
|
r.x0 = (int32_t)((uint64_t)x * ow / sw), r.y0 = (int32_t)((uint64_t)y * oh / sh);
|
||||||
|
r.x1 = (int32_t)(rx * ow / sw), r.y1 = (int32_t)(ry * oh / sh);
|
||||||
|
if (r.x1 <= r.x0 || r.y1 <= r.y0) r = SourceRect{};
|
||||||
|
return r;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A frame generation context as it was made: the presented picture's size, the largest scene it takes, and whether the
|
||||||
|
/// scene's far plane is at infinity (a context is made for one kind of depth)
|
||||||
|
struct ContextKey {
|
||||||
|
uint32_t width = 0, height = 0, max_render_width = 0, max_render_height = 0;
|
||||||
|
bool infinite_depth = false;
|
||||||
|
};
|
||||||
|
/// Whether a context made as c takes a frame of w x h with a scene of rw x rh: else it is made again
|
||||||
|
constexpr bool context_fits(const ContextKey& c, uint32_t w, uint32_t h, uint32_t rw, uint32_t rh, bool infinite_depth) {
|
||||||
|
return c.width == w && c.height == h && rw <= c.max_render_width && rh <= c.max_render_height &&
|
||||||
|
c.infinite_depth == infinite_depth;
|
||||||
|
}
|
||||||
|
/// The context for such a frame: the scene is never larger than the picture in play, and a larger one still gets a context
|
||||||
|
constexpr ContextKey context_for(uint32_t w, uint32_t h, uint32_t rw, uint32_t rh, bool infinite_depth) {
|
||||||
|
return ContextKey{w, h, rw > w ? rw : w, rh > h ? rh : h, infinite_depth};
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The presents a context is kept after its last frame: 120 once the row has left AMD FSR 4 (stepping through the row's
|
||||||
|
/// choices passes it: nothing is made and let go again within two seconds), 2 after a failure. Never at the present of
|
||||||
|
/// its last frame: that frame's commands are recorded and not yet on the queue
|
||||||
|
constexpr uint64_t kIdlePresents = 120, kFailedPresents = 2;
|
||||||
|
constexpr bool release_due(bool have, bool failed, uint64_t presents, uint64_t used_at) {
|
||||||
|
return have && presents >= used_at && presents - used_at >= (failed ? kFailedPresents : kIdlePresents);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Taking AMD's frame generation down, in the order that is safe, as one function the real code (vk_fsr4.cpp) and
|
||||||
|
/// tests/test_framegen_fsr4.py both run. ops.stop: no call into AMD's library starts after it (the lock a call holds is
|
||||||
|
/// taken, so one in progress has returned). At the process's exit (HOS_OPT_EXITING, host/hos_relaunch.c) that is all:
|
||||||
|
/// there is no swapchain or thread of AMD's to stop (the renderer presents the made frames itself), and what the context
|
||||||
|
/// holds goes with the process, as the upscaler's does; AMD's library is not called again, so nothing of it runs under
|
||||||
|
/// ExitProcess. In play (the row left AMD FSR 4, another size, a failure): the GPU is past every command buffer that
|
||||||
|
/// used the context (ops.wait_gpu), then AMD's context goes (ops.destroy_context) before the image it wrote into
|
||||||
|
/// (ops.destroy_images), never the other way round
|
||||||
|
template <class Ops> void teardown(Ops& ops, bool exiting) {
|
||||||
|
ops.stop();
|
||||||
|
if (exiting) return;
|
||||||
|
ops.wait_gpu();
|
||||||
|
ops.destroy_context();
|
||||||
|
ops.destroy_images();
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::framegen_rows
|
||||||
@@ -56,4 +56,44 @@ inline float view_distance(const Camera& c, float device_depth) {
|
|||||||
return c.device_to_view_depth[1] / (device_depth - c.device_to_view_depth[0]);
|
return c.device_to_view_depth[1] / (device_depth - c.device_to_view_depth[0]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The jitter sequence's length for FSR 3.1, AMD's own rule (ffxFsr3GetJitterPhaseCount, ffx_upscale.h): 8 times the
|
||||||
|
/// width's scale squared, truncated, with no upper bound: 18 at Quality (1.5x), 23 at Balanced, 32 at Performance, 71 or 72
|
||||||
|
/// at Ultra Performance (3x), 8 for the Native AA. AMD FSR 4 on the frame chain takes it too ((Triforce, FSR4-10)
|
||||||
|
/// vk_temporal.cpp chain); NVIDIA's rule (dlss_math.h dlss_jitter_phases, by area, rounded, 64 at most) stays DLSS's
|
||||||
|
inline unsigned jitter_phases(unsigned render_width, unsigned display_width) {
|
||||||
|
if (!render_width || !display_width) return 8;
|
||||||
|
const float ratio = (float)display_width / (float)render_width;
|
||||||
|
const int n = (int)(8.0f * ratio * ratio);
|
||||||
|
return n < 1 ? 1u : (unsigned)n;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The size FSR 3.1's render-size images are made at (the SDK's maxRenderSize), and whether they are made again, for a frame
|
||||||
|
/// of rw x rh into uw x uh with images of have_w x have_h for an output of have_uw x have_uh (0 x 0: none yet). They are kept
|
||||||
|
/// for any frame that fits them, so the game's dynamic resolution (the Render resolution row's Auto) changes only the
|
||||||
|
/// render size each dispatch is given and the history goes on, as the SDK's FFX_FSR3UPSCALER_ENABLE_DYNAMIC_RESOLUTION
|
||||||
|
/// does: its passes clamp what they read to the frame's render size (ClampUv, ClampLoad). Made again only for another
|
||||||
|
/// output, or for a frame larger than them: at the output's size then (a size that grew is the dynamic resolution's, and
|
||||||
|
/// the images are not made again for its next step), and at once with dynamic (the row's Auto). A fixed share is made at
|
||||||
|
/// its own size. Never smaller than the frame, nor larger than the output unless the frame is
|
||||||
|
struct ImagePlan {
|
||||||
|
unsigned width = 0, height = 0;
|
||||||
|
bool remake = false;
|
||||||
|
};
|
||||||
|
inline ImagePlan plan_images(unsigned have_w, unsigned have_h, unsigned have_uw, unsigned have_uh, unsigned rw, unsigned rh,
|
||||||
|
unsigned uw, unsigned uh, bool dynamic) {
|
||||||
|
ImagePlan p;
|
||||||
|
rw = rw ? rw : 1, rh = rh ? rh : 1, uw = uw ? uw : 1, uh = uh ? uh : 1;
|
||||||
|
const bool same_output = have_w && have_h && have_uw == uw && have_uh == uh;
|
||||||
|
if (same_output && rw <= have_w && rh <= have_h) {
|
||||||
|
p.width = have_w, p.height = have_h;
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
// (a larger frame for the same output: the dynamic resolution's step up)
|
||||||
|
const bool full = dynamic || same_output;
|
||||||
|
p.width = full && uw > rw ? uw : rw;
|
||||||
|
p.height = full && uh > rh ? uh : rh;
|
||||||
|
p.remake = true;
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
|
||||||
} // namespace mkwvk::fsr3_math
|
} // namespace mkwvk::fsr3_math
|
||||||
|
|||||||
148
gpu/vk/game_grid.h
Normal file
148
gpu/vk/game_grid.h
Normal file
@@ -0,0 +1,148 @@
|
|||||||
|
// (Triforce, 11-19) Where the game's 2D layouts are in the window: the 3840 x 2160 grid its menus and lists are laid out on
|
||||||
|
// (the units host/oot_menu.c tells the Mods entry's list in), as the window shows it. The lab's "Title in a 16:9 window (1600x900,
|
||||||
|
// 21:9 picture letterboxed): Mods drawn below the picture, larger, far from the list": the entry was drawn on a 16:9 column of
|
||||||
|
// the window's whole height, which is the grid only when the picture fills the window's height.
|
||||||
|
// - The picture: in the window between black bars, as the present puts it (vk_present_rect.h present_rect, with the
|
||||||
|
// picture's shape: Aspect ratio's 16:10, 21:9 or 32:9, else its pixels').
|
||||||
|
// - The grid in it: the game's layouts keep their 16:9 shape in the picture's middle on a wider picture (host/oot_wide_hud.h:
|
||||||
|
// every layout but the kept backgrounds and fades is squeezed by 16:9 over the picture's ratio, host/hos_main.c
|
||||||
|
// hos_wide_squeeze), its height the picture's; on a narrower one (16:10) the grid is the whole picture, narrower than 16:9
|
||||||
|
// (the layouts squeezed across, not cut). HUD position moves panes of the HUD's own layouts only (oot_hud_out), not
|
||||||
|
// the lists'.
|
||||||
|
// No ImGui and no Vulkan: tests/test_mods_grid.py compiles it alone.
|
||||||
|
#pragma once
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cmath>
|
||||||
|
#include <cstdint>
|
||||||
|
#include "vk_present_rect.h"
|
||||||
|
|
||||||
|
namespace mkwvk::game_grid {
|
||||||
|
|
||||||
|
/// The grid's rectangle in the window's pixels: its left and top, its width and height (2160 units down, 3840 across)
|
||||||
|
struct Place {
|
||||||
|
float x = 0, y = 0, w = 0, h = 0;
|
||||||
|
bool picture = false; // ((11-19) from a picture the present showed; false: the window's column, none known yet)
|
||||||
|
/// ((Triforce, 11-20) the picture itself in the window (the present's rectangle, whole pixels), with picture: the game
|
||||||
|
/// draws nothing outside it)
|
||||||
|
float pic_x = 0, pic_y = 0, pic_w = 0, pic_h = 0;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The game's picture (picture_w x picture_h, 0 not known yet) shown in a window_w x window_h window, aspect the picture's
|
||||||
|
/// shape (0: its pixels'). With no picture known, the window's middle 16:9 column of its whole height, as before
|
||||||
|
inline Place of(uint32_t picture_w, uint32_t picture_h, uint32_t window_w, uint32_t window_h, double aspect) {
|
||||||
|
Place g;
|
||||||
|
if (!window_w || !window_h) return g;
|
||||||
|
if (!picture_w || !picture_h) {
|
||||||
|
g.h = (float)window_h;
|
||||||
|
g.w = std::min((float)window_w, std::floor((float)window_h * 16.0f / 9.0f + 0.5f));
|
||||||
|
g.x = std::floor(((float)window_w - g.w) * 0.5f);
|
||||||
|
return g;
|
||||||
|
}
|
||||||
|
const PresentRect p = present_rect(picture_w, picture_h, nullptr, window_w, window_h, PresentAspect{aspect});
|
||||||
|
const double ratio = (double)p.w / (double)p.h, squeeze = std::min(1.0, (16.0 / 9.0) / ratio);
|
||||||
|
g.h = (float)p.h, g.y = (float)p.y, g.picture = true;
|
||||||
|
g.pic_x = (float)p.x, g.pic_y = (float)p.y, g.pic_w = (float)p.w, g.pic_h = (float)p.h;
|
||||||
|
g.w = (float)((double)p.w * squeeze);
|
||||||
|
g.x = (float)p.x + ((float)p.w - g.w) * 0.5f;
|
||||||
|
return g;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A drawing made on a 16:9 column of the grid's height (its own pixels, from 0: w = h x 16 / 9 wide, unit h / 720) put on
|
||||||
|
/// the grid: x across times kx (1 on a 16:9 grid; under 1 where the layouts are squeezed across), then moved by ox, oy
|
||||||
|
struct Map {
|
||||||
|
float ox = 0, oy = 0, kx = 1;
|
||||||
|
float x(float px) const { return ox + px * kx; }
|
||||||
|
float y(float py) const { return oy + py; }
|
||||||
|
/// (the window's pixel back in the drawing's: the mouse)
|
||||||
|
float back_x(float wx) const { return (wx - ox) / kx; }
|
||||||
|
float back_y(float wy) const { return wy - oy; }
|
||||||
|
};
|
||||||
|
inline float column_w(const Place& g) { return std::floor(g.h * 16.0f / 9.0f + 0.5f); }
|
||||||
|
/// ((Triforce, 11-19) on a 16:9 grid (a wider picture's middle, or a 16:9 one) the column is put on whole pixels, not
|
||||||
|
/// stretched: the text drawn on it is on whole pixels already (ImGui's glyphs), and a part of a pixel across or a scale a
|
||||||
|
/// little off 1 (the grid's width from the picture's, the column's rounded) left it between pixels, soft. Only a grid
|
||||||
|
/// narrower than 16:9 (16:10) is squeezed across, as the game squeezes its layouts there)
|
||||||
|
inline Map map_of(const Place& g) {
|
||||||
|
Map m;
|
||||||
|
const float cw = column_w(g);
|
||||||
|
if (cw > 0 && g.w >= cw - 1.0f) {
|
||||||
|
m.ox = std::round(g.x + (g.w - cw) * 0.5f), m.oy = std::round(g.y), m.kx = 1.0f;
|
||||||
|
return m;
|
||||||
|
}
|
||||||
|
m.ox = std::round(g.x), m.oy = std::round(g.y), m.kx = cw > 0 ? g.w / cw : 1.0f;
|
||||||
|
return m;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A page's 16:9 column (its own pixels from 0: w x h, unit h / 720) and where it goes in the window (map)
|
||||||
|
struct Column {
|
||||||
|
Map map;
|
||||||
|
float w = 0, h = 0;
|
||||||
|
/// ((Triforce, 11-20) over the game's panes: the picture's right edge in the column's own pixels (past w where the
|
||||||
|
/// picture is wider than the grid); 0 otherwise
|
||||||
|
float game_x1 = 0;
|
||||||
|
};
|
||||||
|
/// ((Triforce, 11-19) the column a page of ours is drawn on in a window_w x window_h window. on_game: over the game's own
|
||||||
|
/// panes (its Options screen under our tabs and their header, the Mods entry's list): on the game's grid, as those panes
|
||||||
|
/// are, so that our plates cover them; the lab's "16:9 window, 21:9 picture letterboxed: the game's Default Settings and
|
||||||
|
/// Back and the left edge of its help box show through beside our panel" was our tabs on the window's column of its whole
|
||||||
|
/// height while the game's panes were on the picture's grid. Else (our options over play), and with no picture known yet,
|
||||||
|
/// the window's middle 16:9 column of its whole height, as before (narrower windows: the whole window, not squeezed)
|
||||||
|
inline Column column_of(const Place& grid, float window_w, float window_h, bool on_game) {
|
||||||
|
Column c;
|
||||||
|
if (on_game && grid.picture) {
|
||||||
|
c.map = map_of(grid), c.w = column_w(grid), c.h = grid.h;
|
||||||
|
c.game_x1 = c.map.back_x(grid.pic_x + grid.pic_w);
|
||||||
|
return c;
|
||||||
|
}
|
||||||
|
c.w = std::min(window_w, std::floor(window_h * 16.0f / 9.0f + 0.5f)), c.h = window_h;
|
||||||
|
c.map.ox = std::floor((window_w - c.w) * 0.5f);
|
||||||
|
return c;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// ((Triforce, 11-20) A rectangle of the window, in its pixels
|
||||||
|
struct Rect {
|
||||||
|
float x0 = 0, y0 = 0, x1 = 0, y1 = 0;
|
||||||
|
};
|
||||||
|
/// ((Triforce, 11-20) A dim over a whole window_w x window_h window cut where the game's picture ends (g, from of): out[0]
|
||||||
|
/// the picture, where a dim keeps its colour, then the bars beside, above or below it (up to four, none overlapping, all
|
||||||
|
/// of them and the picture the window exactly), where a dim must be black. The lab's check 3 of 26670c0: our dialogs' dim
|
||||||
|
/// (rgba(8, 5, 2, ...), lighter than black) turned the black bars a dark brown, luminance about 9. Black at the same alpha
|
||||||
|
/// never lightens them; over the picture the dim stays as it was. With no picture known: the whole window, one part.
|
||||||
|
/// Returns how many
|
||||||
|
inline int dim_parts(const Place& g, float window_w, float window_h, Rect out[5]) {
|
||||||
|
out[0] = {0, 0, window_w, window_h};
|
||||||
|
if (!g.picture) return 1;
|
||||||
|
const float x0 = std::clamp(g.pic_x, 0.0f, window_w), x1 = std::clamp(g.pic_x + g.pic_w, x0, window_w);
|
||||||
|
const float y0 = std::clamp(g.pic_y, 0.0f, window_h), y1 = std::clamp(g.pic_y + g.pic_h, y0, window_h);
|
||||||
|
out[0] = {x0, y0, x1, y1};
|
||||||
|
int n = 1;
|
||||||
|
if (y0 > 0) out[n++] = {0, 0, window_w, y0}; // (above)
|
||||||
|
if (y1 < window_h) out[n++] = {0, y1, window_w, window_h}; // (below)
|
||||||
|
if (x0 > 0) out[n++] = {0, y0, x0, y1}; // (left, between those)
|
||||||
|
if (x1 < window_w) out[n++] = {x1, y0, window_w, y1}; // (right)
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// ((Triforce, 11-20) The game's guides band at the foot of its Options screen (SystemOverlay_00's N_Footer_00: W_GuideBg_00
|
||||||
|
/// and W_GuideBg_01, right-aligned 2110 and 2174 units right of the middle, 160 high from 19 under the foot) goes on past the
|
||||||
|
/// 16:9 column's right edge (1920): on a picture wider than the grid its end shows beside the column. The lab's check 3 of
|
||||||
|
/// 26670c0 (the game's own tab: 16x9_10, 21x9_10, fs_10): the band seen to 238 to 240 units past the column's edge (the
|
||||||
|
/// window pane's last texels clear), from 126 units over the foot, and our guides' plate ending at the column's edge left
|
||||||
|
/// 75 px of it in the 1600 x 900 window and 155 px in fullscreen 3440 x 1440. Where the band ends, in units right of the middle
|
||||||
|
constexpr float kGuideBandRight = 2160;
|
||||||
|
/// ((Triforce, 11-20, a review of 1104352) The Options screen chosen from the pause menu (host/oot_menu.c tells it as 2): its
|
||||||
|
/// footer may be the pause menu's own (MenuOverlay_00's band panes end 2110 units right of the middle; in the pause menu's
|
||||||
|
/// fs_30_sys_on_title_screen.png its band fades out by about 2100), not SystemOverlay_00's to 2160. No lab shot of that screen
|
||||||
|
/// yet: our plate ends at the smaller of the two there, so that it never runs about 50 units (33 px in fullscreen 3440 x 1440)
|
||||||
|
/// past the game's band as a dark sliver over the bars' black
|
||||||
|
constexpr float kPauseGuideBandRight = 2110;
|
||||||
|
/// (where our guides' plate ends over the game's panes on column c, in its own pixels from 0: as far as the game's band
|
||||||
|
/// (band_right units right of the middle: kGuideBandRight, the title's Options; kPauseGuideBandRight, the pause menu's), up to
|
||||||
|
/// the picture's right edge (the game draws nothing past it); the column's right edge on a 16:9 or narrower picture (16:10:
|
||||||
|
/// the grid is the whole picture), over play, or with no picture known)
|
||||||
|
inline float guide_band_x1(const Column& c, float band_right = kGuideBandRight) {
|
||||||
|
const float band = c.w * 0.5f + band_right * c.h / 2160.0f;
|
||||||
|
return std::max(c.w, std::min(band, c.game_x1));
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::game_grid
|
||||||
132
gpu/vk/game_language.h
Normal file
132
gpu/vk/game_language.h
Normal file
@@ -0,0 +1,132 @@
|
|||||||
|
/* The Insert menu's Game language row (vk_menu.cpp), as plain code: what language.txt in the game folder chooses and offers,
|
||||||
|
and the file's text with another language chosen. No Vulkan, no menu: tests/test_game_language.py checks it.
|
||||||
|
|
||||||
|
The game's language is the host's to read, at the start (host/nn_applet.c desired_language): HOS_LANGUAGE, else the
|
||||||
|
file's first line that is not blank or a comment and whose first word is one of the game's codes, in any case; English
|
||||||
|
(US) when there is none. Setup's update keeps a folder's choice by the same line, and takes it only when the line is the
|
||||||
|
code alone, spelled as the game spells it (nx2/make_game.py folder_language). The file lists the codes in its comments
|
||||||
|
("# pt-BR Portuguese (Brazil)"), as the host and Setup write it.
|
||||||
|
|
||||||
|
The row offers the languages the file itself names, as its choice or in that list, that the game has; it writes the
|
||||||
|
choice as the code alone on the choice's line, and no code the file does not name. */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstddef>
|
||||||
|
#include <string>
|
||||||
|
#include <string_view>
|
||||||
|
|
||||||
|
namespace mkwvk::game_language {
|
||||||
|
|
||||||
|
struct Language {
|
||||||
|
const char* code;
|
||||||
|
const char* name;
|
||||||
|
};
|
||||||
|
/// The game's languages: host/nn_applet.c's table and nx2/make_game.py's LANGUAGES, in their order (the test holds the
|
||||||
|
/// three together)
|
||||||
|
constexpr Language kLanguages[] = {
|
||||||
|
{"en-US", "English (US)"}, {"en-GB", "English (UK)"}, {"fr", "French (France)"}, {"fr-CA", "French (Canada)"},
|
||||||
|
{"de", "German"}, {"it", "Italian"}, {"es", "Spanish (Spain)"}, {"es-419", "Spanish (Latin America)"},
|
||||||
|
{"pt-BR", "Portuguese (Brazil)"}, {"nl", "Dutch"}, {"pl", "Polish"}, {"ru", "Russian"},
|
||||||
|
{"ja", "Japanese"}, {"ko", "Korean"}, {"zh-Hans", "Chinese (Simplified)"}, {"zh-Hant", "Chinese (Traditional)"},
|
||||||
|
};
|
||||||
|
constexpr int kCount = sizeof kLanguages / sizeof kLanguages[0];
|
||||||
|
constexpr int kDefault = 0; // (English (US): the host's answer when the file chooses nothing)
|
||||||
|
inline unsigned bit(int language) { return 1u << language; }
|
||||||
|
|
||||||
|
/// A code's place in kLanguages, in any case (the host compares so: "pt-br" is pt-BR); -1 when the game has no such code
|
||||||
|
inline int find(std::string_view code) {
|
||||||
|
for (int i = 0; i < kCount; i++) {
|
||||||
|
const std::string_view k = kLanguages[i].code;
|
||||||
|
if (k.size() != code.size()) continue;
|
||||||
|
bool same = true;
|
||||||
|
for (size_t c = 0; c < k.size() && same; c++) {
|
||||||
|
const auto lower = [](char x) { return x >= 'A' && x <= 'Z' ? (char)(x - 'A' + 'a') : x; };
|
||||||
|
same = lower(k[c]) == lower(code[c]);
|
||||||
|
}
|
||||||
|
if (same) return i;
|
||||||
|
}
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a language.txt says
|
||||||
|
struct File {
|
||||||
|
int chosen = -1; ///< the language its choice line names (kLanguages' index); -1: it has no such line
|
||||||
|
unsigned offered = 0; ///< a bit a language the file names: its choice, and each "# <code> ..." line
|
||||||
|
size_t at = 0, len = 0; ///< the choice line's text to replace: after a byte order mark, without its line ending
|
||||||
|
bool crlf = false; ///< its lines end with CR LF
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The first word of s (after spaces and tabs), up to a space, a tab or a line's end
|
||||||
|
inline std::string_view word(std::string_view s) {
|
||||||
|
size_t a = 0;
|
||||||
|
while (a < s.size() && (s[a] == ' ' || s[a] == '\t')) a++;
|
||||||
|
size_t b = a;
|
||||||
|
while (b < s.size() && s[b] != ' ' && s[b] != '\t' && s[b] != '\r' && s[b] != '\n') b++;
|
||||||
|
return s.substr(a, b - a);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The file as the host reads it, line by line: what a line longer than its 79 bytes holds past them is not a line's start,
|
||||||
|
/// a byte order mark may open any line, and the choice is the first line that is neither blank nor a comment and names a
|
||||||
|
/// language of the game (a line that names none is passed over, as there)
|
||||||
|
inline File read(std::string_view text) {
|
||||||
|
File f;
|
||||||
|
f.crlf = text.find("\r\n") != std::string_view::npos;
|
||||||
|
for (size_t start = 0; start < text.size();) {
|
||||||
|
size_t end = text.find('\n', start);
|
||||||
|
if (end == std::string_view::npos) end = text.size();
|
||||||
|
size_t stop = end; // (the line without its ending)
|
||||||
|
if (stop > start && text[stop - 1] == '\r') stop--;
|
||||||
|
std::string_view line = text.substr(start, stop - start);
|
||||||
|
size_t lead = 0; // (bytes of the line before what the host looks at)
|
||||||
|
if (line.size() >= 3 && (unsigned char)line[0] == 0xEF && (unsigned char)line[1] == 0xBB && (unsigned char)line[2] == 0xBF) lead = 3;
|
||||||
|
// (the host's buffer: 79 bytes of the line, cut at a zero byte. It keeps 31 bytes of the word, more than any code has: a
|
||||||
|
// longer word names no language there or here)
|
||||||
|
std::string_view seen = line.substr(0, 79);
|
||||||
|
if (const size_t zero = seen.find('\0'); zero != std::string_view::npos) seen = seen.substr(0, zero);
|
||||||
|
seen = seen.substr(lead < seen.size() ? lead : seen.size());
|
||||||
|
size_t blank = 0;
|
||||||
|
while (blank < seen.size() && (seen[blank] == ' ' || seen[blank] == '\t')) blank++;
|
||||||
|
const std::string_view rest = seen.substr(blank);
|
||||||
|
if (!rest.empty() && rest[0] == '#') { // (a comment: a language it lists, "# <code> <name>")
|
||||||
|
const int listed = find(word(rest.substr(1)));
|
||||||
|
if (listed >= 0) f.offered |= bit(listed);
|
||||||
|
} else if (!rest.empty() && f.chosen < 0) {
|
||||||
|
const int named = find(word(rest));
|
||||||
|
if (named >= 0) f.chosen = named, f.offered |= bit(named), f.at = start + lead, f.len = stop - (start + lead);
|
||||||
|
}
|
||||||
|
start = end + 1;
|
||||||
|
}
|
||||||
|
return f;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The file's text with `language` chosen: its choice line replaced by the code alone (as Setup's update reads it), every
|
||||||
|
/// other byte as it was; with no choice line, the code on a line of its own at the end. The text as it is when the file
|
||||||
|
/// does not name that language: never a code it does not list
|
||||||
|
inline std::string write(std::string_view text, int language) {
|
||||||
|
const File f = read(text);
|
||||||
|
if (language < 0 || language >= kCount || !(f.offered & bit(language))) return std::string(text);
|
||||||
|
const std::string_view code = kLanguages[language].code;
|
||||||
|
std::string out(text);
|
||||||
|
if (f.chosen >= 0) {
|
||||||
|
out.replace(f.at, f.len, code);
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
const char* const eol = f.crlf ? "\r\n" : "\n";
|
||||||
|
if (!out.empty() && out.back() != '\n') out += eol;
|
||||||
|
out += code;
|
||||||
|
out += eol;
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The languages a file offers, in kLanguages' order, into out (kCount of room): how many. None unless the language in
|
||||||
|
/// effect (the file's choice, else the host's English (US)) is among them: the row then has nothing sound to show
|
||||||
|
inline int offered(const File& f, int* out) {
|
||||||
|
const int now = f.chosen >= 0 ? f.chosen : kDefault;
|
||||||
|
if (!(f.offered & bit(now))) return 0;
|
||||||
|
int n = 0;
|
||||||
|
for (int i = 0; i < kCount; i++)
|
||||||
|
if (f.offered & bit(i)) out[n++] = i;
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::game_language
|
||||||
73
gpu/vk/gate_wait.h
Normal file
73
gpu/vk/gate_wait.h
Normal file
@@ -0,0 +1,73 @@
|
|||||||
|
// (Triforce) A submission whose command buffer begins with a wait for an event the game signals from the CPU: the submit waits for
|
||||||
|
// the signal (RX_VK_GATE_WAIT_MS at most) before its capture and its replay, after NX2navi's gate_wait_ms (runtime/vk_exec.c
|
||||||
|
// vk_exec_gate_wait and gate_first_wait).
|
||||||
|
//
|
||||||
|
// OoT's queues order some of their work with events the CPU signals once it has written what the work reads. A submission that
|
||||||
|
// meets such a wait unmet is taken (its capture of the game's memory at the submit) and replayed up to the wait, where it stops; once
|
||||||
|
// the CPU signals, the order thread (vk_replay.cpp sync_watch, a millisecond at a time) or the next submit or present lets it go on,
|
||||||
|
// with a second capture (resume_queue, RX_VK_RESUME_CAPTURE). NX2navi found the signal 1 to 2 ms after the submit call in about
|
||||||
|
// every other frame in Hyrule Field, and made the submit call wait for it, before its copies, as the console's GPU waits at the
|
||||||
|
// wait itself: the capture is then taken after the game has written the frame's data, and the frame is replayed in the submit call
|
||||||
|
// and on the submitting thread (54.5 fps instead of 53 there, the longest frame 33 ms instead of 46). Triforce meets it far less
|
||||||
|
// often (the "captures at a CPU event's wait" of the order line: 100 to 170 in a lab run of 26,000 frames at the Market), and a gate
|
||||||
|
// that runs out holds the submitting thread for the whole bound, so it is off unless asked for, for the lab to measure.
|
||||||
|
// The gate waits only where the replay would stop: the first op of the submission's first command buffer that is not state the
|
||||||
|
// replay only stores (bound programs, buffers, textures, targets, viewports...) is a wait for an event, the wait is not met in the
|
||||||
|
// replay's order now, and the CPU has signalled that event before (vk_replay.cpp cpu_events). It takes no lock while it waits (the
|
||||||
|
// thread that will signal may need the order lock first), and looks at the event's word in the game's memory, which the CPU's signal
|
||||||
|
// writes. It ends at the signal, or at the bound, and the submission then goes as before. After kOuts waits that ran out, once they
|
||||||
|
// are more than a tenth of the waits, it is off for the run (the signal is then one that comes only after the submitting thread goes
|
||||||
|
// on, and every wait would be a stall).
|
||||||
|
// RX_VK_GATE_WAIT_MS unset or 0: off, as before; 1 to 100: the bound in milliseconds (NX2navi's default is 10)
|
||||||
|
// Apart from Vulkan and the stream's op numbers so that tests/test_gate_wait.py can run the rules and the gate over authored streams.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <cstring>
|
||||||
|
#if defined(__x86_64__) || defined(_M_X64) || defined(__i386__) || defined(_M_IX86)
|
||||||
|
#include <immintrin.h>
|
||||||
|
#endif
|
||||||
|
|
||||||
|
namespace mkwvk::gate_wait {
|
||||||
|
|
||||||
|
constexpr int kMaxOps = 256; // (the ops looked at before the wait, at most: NX2navi's)
|
||||||
|
constexpr int64_t kOuts = 4; // (waits that ran out before the gate may go off for the run)
|
||||||
|
|
||||||
|
/// RX_VK_GATE_WAIT_MS: the bound in milliseconds, 0 (off) to 100; unset or not a number: 0
|
||||||
|
inline int ms(const char* env) {
|
||||||
|
if (!env || !*env) return 0;
|
||||||
|
const int v = std::atoi(env);
|
||||||
|
return v < 0 ? 0 : v > 100 ? 100 : v;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The command buffer's first op that is not state (records {uint32 op, uint32 size}, sizes 8-aligned, from p): the wait it begins
|
||||||
|
/// with, or null. kind(op): 1 state the replay only stores (looked past), 2 a wait for an event, 0 anything else (a draw, a copy, a
|
||||||
|
/// signal, a barrier, the end: the gate does not wait)
|
||||||
|
template <class K>
|
||||||
|
const uint8_t* first_wait(const uint8_t* p, K kind) {
|
||||||
|
for (int k = 0; k < kMaxOps; k++) {
|
||||||
|
uint32_t h[2];
|
||||||
|
std::memcpy(h, p, sizeof h);
|
||||||
|
if (h[1] < 8 || (h[1] & 7)) return nullptr;
|
||||||
|
const int what = kind(h[0]);
|
||||||
|
if (what == 2) return p;
|
||||||
|
if (what != 1) return nullptr;
|
||||||
|
p += h[1];
|
||||||
|
}
|
||||||
|
return nullptr;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether the gate goes off for the run, after `gates` waits of which `outs` ran out
|
||||||
|
inline bool off_after(int64_t gates, int64_t outs) {
|
||||||
|
return outs >= kOuts && outs * 10 > gates;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A pause of the host core while it looks (the waiting thread holds no virtual core: the signaller runs)
|
||||||
|
inline void pause() {
|
||||||
|
#if defined(__x86_64__) || defined(_M_X64) || defined(__i386__) || defined(_M_IX86)
|
||||||
|
_mm_pause();
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::gate_wait
|
||||||
311
gpu/vk/gpl_rules.h
Normal file
311
gpu/vk/gpl_rules.h
Normal file
@@ -0,0 +1,311 @@
|
|||||||
|
// (Triforce) The rules of the fast pipeline link (VK_EXT_graphics_pipeline_library; vk_draw.cpp graphics_pipeline,
|
||||||
|
// warm_libraries, prepare_run), apart from Vulkan so that tests/test_gpl_rules.py runs them. Off unless the hidden Debug
|
||||||
|
// tab's Fast pipeline link row or RX_VK_GPL=1 switches it on: nobody has run the game with it.
|
||||||
|
//
|
||||||
|
// What "the driver step" was, and why the link stayed off "until the driver step is safe". A draw can only be linked from
|
||||||
|
// its program's two shader libraries if the driver has them compiled; compiled in the draw they stall it for a compile
|
||||||
|
// (0.2 to 1.1 s measured on NX2navi), so a draw asks for them under VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED
|
||||||
|
// and goes without when they are cold. They are warm only if the start's "driver's part" made them: the step between the
|
||||||
|
// translation and the pipelines (the start-up screen's "Compiling shaders", vk_draw.cpp warm_libraries), where every
|
||||||
|
// kept program variant's two libraries are compiled once, on every processor, into the pipeline cache. That step was not
|
||||||
|
// safe to leave on:
|
||||||
|
// - it hands every shader of the game to the driver's compiler, and a driver's compiler faults on some of them (amdvlk64
|
||||||
|
// on a mesh program, NVIDIA 615.78 on one vertex library at the same point of every preparation, NX2navi's issue 30;
|
||||||
|
// the lab's start of 11-13 that hung with the driver's own lock held after such a fault);
|
||||||
|
// - what it kept of a start that died in it was one empty file, <prepared>.warming: "this step began and did not end".
|
||||||
|
// The next start then left the whole step out and wrote it down as done for that driver, for good. One shader the
|
||||||
|
// driver cannot compile, a start closed from the task manager, a power cut: each left every library cold, and with
|
||||||
|
// cold libraries no draw is linked, while each new pipeline still has two libraries built behind it on the workers
|
||||||
|
// its whole build waits for. The link then costs and gives nothing;
|
||||||
|
// - a fault the guard catches in that step starts the game again (driver_fault_restart), into exactly that.
|
||||||
|
// NX2navi's equivalent step is its library warm (runtime/vk_pipeline.c lw_worker, lb_warm_prepared). What makes it safe
|
||||||
|
// there, and here now: while a worker has a variant in the driver's hands, a journal on disk names it. A start that finds
|
||||||
|
// a journal left over puts the variants it names on a skip list for this driver, which the step leaves out from then on
|
||||||
|
// (each is built at its first draw, as without libraries), and goes on with the step: what the last start compiled comes
|
||||||
|
// back from the driver's own cache in a moment. Only after kTries starts in a row that began the step and did not end it
|
||||||
|
// is the step left out, as it was after the first. NX2navi's journal and skip list are not its owner's lines alone, so
|
||||||
|
// nothing of them is copied: this is written from what they do.
|
||||||
|
// ((Triforce, 11-18) A try is a start that died with the driver compiling for the step: the journal is opened before the
|
||||||
|
// step, and its "began" word says whether that can be so. The stage libraries' pass (warm_libraries) writes it (and
|
||||||
|
// flushes it) just before its first build_graphics, and it stays: their compiles run back to back. The program pass's
|
||||||
|
// compute warms come first and are spread over the translation, which calls no compiler of the driver's: for them the
|
||||||
|
// word is written just before a guarded_compute when none is under way (Journal::Compiling) and written back to 0 when the
|
||||||
|
// last of them returned. A start that died or was stopped with the word at 0 used no try (the next one finds the count it
|
||||||
|
// found itself): before the step, or in the translation between or after the compute warms, the 5-40 minutes after an
|
||||||
|
// update. One with the word written counts as before, a kill in a hung driver call included. A journal of an older build,
|
||||||
|
// without the word, counts as before)
|
||||||
|
//
|
||||||
|
// And the rules of the link itself, which were in graphics_pipeline's lines:
|
||||||
|
// - a draw is linked only from libraries that are all there; a cold or failed library, or a link the driver refuses,
|
||||||
|
// leaves the entry not ready and the whole build to finish it: what a draw does with an entry that is not ready is
|
||||||
|
// what it does without libraries, nothing more is drawn with it;
|
||||||
|
// - the worker's whole (optimised) build always follows a link, and takes its place when it is made; if it fails, the
|
||||||
|
// linked pipeline stays, as before;
|
||||||
|
// - the linked pipeline a whole build replaced is given back to the driver (it was kept for the run: NX2navi measured
|
||||||
|
// 491 MiB of driver memory a session for its replaced links), but only kRetirePresents presents and kRetireMs later:
|
||||||
|
// a draw recorded with it may still be on its way to the GPU. RX_VK_GPL_KEEP_LINKS=1: kept, as before.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <atomic>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdio>
|
||||||
|
#include <deque>
|
||||||
|
#include <mutex>
|
||||||
|
#include <string>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace gpl_rules {
|
||||||
|
|
||||||
|
// ---- an entry: linked first, then the whole build over it
|
||||||
|
|
||||||
|
/// The draw's thread, for a new entry: link is what the libraries linked to, "none" when a library was cold or failed or
|
||||||
|
/// the driver refused the link. True: the entry is ready with it. False: nothing changed, the whole build makes it ready
|
||||||
|
template <class Entry, class Pipeline>
|
||||||
|
bool linked(Entry& e, Pipeline link) {
|
||||||
|
if (link == Pipeline{}) return false;
|
||||||
|
e.fast.store(link, std::memory_order_relaxed);
|
||||||
|
e.pipeline.store(link, std::memory_order_relaxed);
|
||||||
|
e.ready.store(true, std::memory_order_release);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The worker, the whole build over: whole is "none" when it failed. The entry is ready from here on: with the whole
|
||||||
|
/// build, or with the link it had (the build failed), or with nothing (it failed and there was no link: the draws are left
|
||||||
|
/// out, as before). Returns the link the whole build replaced, for retire(); "none" when there is nothing to give back
|
||||||
|
template <class Entry, class Pipeline>
|
||||||
|
Pipeline built(Entry& e, Pipeline whole) {
|
||||||
|
Pipeline replaced{};
|
||||||
|
if (!(whole == Pipeline{})) {
|
||||||
|
e.pipeline.store(whole, std::memory_order_release);
|
||||||
|
replaced = e.fast.exchange(Pipeline{}, std::memory_order_acq_rel);
|
||||||
|
}
|
||||||
|
e.ready.store(true, std::memory_order_release);
|
||||||
|
return replaced;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- the replaced links, given back later
|
||||||
|
|
||||||
|
inline constexpr uint64_t kRetirePresents = 600;
|
||||||
|
inline constexpr uint64_t kRetireMs = 10000;
|
||||||
|
|
||||||
|
/// The links whole builds replaced, until both waits are over (any thread adds, the present's thread takes)
|
||||||
|
template <class Pipeline>
|
||||||
|
class Retired {
|
||||||
|
public:
|
||||||
|
void add(Pipeline p, uint64_t present, uint64_t ms) {
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
q_.push_back({p, present, ms});
|
||||||
|
}
|
||||||
|
/// Those replaced kRetirePresents presents and kRetireMs before now, the oldest first: the caller destroys them
|
||||||
|
std::vector<Pipeline> due(uint64_t present, uint64_t ms) {
|
||||||
|
std::vector<Pipeline> out;
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
while (!q_.empty() && present >= q_.front().present + kRetirePresents && ms >= q_.front().ms + kRetireMs) {
|
||||||
|
out.push_back(q_.front().p);
|
||||||
|
q_.pop_front();
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
size_t waiting() {
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
return q_.size();
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
struct One {
|
||||||
|
Pipeline p;
|
||||||
|
uint64_t present, ms;
|
||||||
|
};
|
||||||
|
std::mutex mu_;
|
||||||
|
std::deque<One> q_;
|
||||||
|
};
|
||||||
|
|
||||||
|
// ---- the driver's part of the preparation: its journal and its skip list
|
||||||
|
|
||||||
|
/// A program variant (Program::key, Program::variant)
|
||||||
|
struct Id {
|
||||||
|
uint64_t key = 0, variant = 0;
|
||||||
|
bool operator==(const Id& o) const { return key == o.key && variant == o.variant; }
|
||||||
|
};
|
||||||
|
|
||||||
|
inline constexpr uint32_t kJournalMagic = 0x4A574654; // ("TFWJ")
|
||||||
|
inline constexpr uint32_t kSkipMagic = 0x53574654; // ("TFWS")
|
||||||
|
inline constexpr int kSlots = 64; // (the workers of the preparation at most: prepare_threads.h)
|
||||||
|
inline constexpr uint32_t kTries = 3; // (starts in a row that may begin the step and not end it)
|
||||||
|
inline constexpr uint32_t kBeganMark = 0x42574654; // ("TFWB": the driver may be compiling for the step)
|
||||||
|
inline constexpr long kBeganAt = 16 + 16 * kSlots; // (the word after the slots; a file without it is an older build's)
|
||||||
|
|
||||||
|
/// What a start found of the last one's journal
|
||||||
|
struct Left {
|
||||||
|
bool found = false; // (a journal was there: the last start began the step and did not end it)
|
||||||
|
bool began = true; // (and the driver may have been compiling for it: false, it used no try)
|
||||||
|
uint32_t tries = 0; // (the starts in a row that did so, the last one counted; 0: none, or another driver's)
|
||||||
|
std::vector<Id> in_hand; // (the variants its workers had in the driver's hands when it ended)
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The journal at path as the last start left it. stamp: what the driver's part is made for now (vk_draw.cpp warmed_stamp).
|
||||||
|
/// A file of another stamp is another driver's or other shaders': found, with nothing to count and no variant to blame. A
|
||||||
|
/// file without the header is an older build's empty marker: one start, no variant named
|
||||||
|
inline Left journal_left(const std::string& path, uint64_t stamp) {
|
||||||
|
Left l;
|
||||||
|
FILE* f = fopen(path.c_str(), "rb");
|
||||||
|
if (!f) return l;
|
||||||
|
l.found = true;
|
||||||
|
uint32_t head[2]{};
|
||||||
|
uint64_t was = 0;
|
||||||
|
if (fread(head, 4, 2, f) != 2 || fread(&was, 8, 1, f) != 1 || head[0] != kJournalMagic) {
|
||||||
|
l.tries = 1;
|
||||||
|
} else if (was == stamp) {
|
||||||
|
l.tries = head[1] ? head[1] : 1;
|
||||||
|
int slots = 0;
|
||||||
|
for (; slots < kSlots; slots++) {
|
||||||
|
uint64_t r[2];
|
||||||
|
if (fread(r, 8, 2, f) != 2) break;
|
||||||
|
if (r[0] || r[1]) l.in_hand.push_back({r[0], r[1]});
|
||||||
|
}
|
||||||
|
// ((Triforce) the began word: 0, the start ended with no compile of the step under way, and it is not counted.
|
||||||
|
// Without the word (an older build's journal, or one cut short) or with anything else in it, it counts, as before)
|
||||||
|
uint32_t began = kBeganMark;
|
||||||
|
if (slots == kSlots && fread(&began, 4, 1, f) == 1 && began == 0 && l.in_hand.empty()) l.began = false, l.tries--;
|
||||||
|
}
|
||||||
|
fclose(f);
|
||||||
|
return l;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a start that found l goes on with the step (true) or leaves it out for this driver, as every start did before
|
||||||
|
inline bool goes_on(const Left& l) { return !l.found || l.tries < kTries; }
|
||||||
|
|
||||||
|
/// The journal of a step under way. Every mark is on its way to the disk before the driver is called
|
||||||
|
class Journal {
|
||||||
|
public:
|
||||||
|
Journal() = default;
|
||||||
|
Journal(const Journal&) = delete;
|
||||||
|
Journal& operator=(const Journal&) = delete;
|
||||||
|
~Journal() { close(); }
|
||||||
|
/// tries: the starts in a row that began the step, this one counted (it counts only while the word is on the disk: began(), Compiling)
|
||||||
|
bool open(const std::string& path, uint64_t stamp, uint32_t tries) {
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
if (f_) fclose(f_);
|
||||||
|
began_.store(false, std::memory_order_relaxed);
|
||||||
|
compiling_ = 0;
|
||||||
|
f_ = fopen(path.c_str(), "wb");
|
||||||
|
if (!f_) return false;
|
||||||
|
const uint32_t head[2] = {kJournalMagic, tries};
|
||||||
|
const uint64_t zero[2 * kSlots]{};
|
||||||
|
const uint32_t not_yet = 0;
|
||||||
|
const bool ok = fwrite(head, 4, 2, f_) == 2 && fwrite(&stamp, 8, 1, f_) == 1 &&
|
||||||
|
fwrite(zero, 8, (size_t)(2 * kSlots), f_) == (size_t)(2 * kSlots) && fwrite(¬_yet, 4, 1, f_) == 1 &&
|
||||||
|
fflush(f_) == 0;
|
||||||
|
if (!ok) fclose(f_), f_ = nullptr;
|
||||||
|
return ok;
|
||||||
|
}
|
||||||
|
/// ((Triforce) Any worker of the stage libraries' pass, just before a compile call: the start counts as a try from
|
||||||
|
/// here on, for good. The first call writes the word; the others wait for it on the lock, then go by the flag. No
|
||||||
|
/// journal: nothing. (A failed write, over bytes the file has already, is tried once: the marks go on as before it)
|
||||||
|
void began() {
|
||||||
|
if (began_.load(std::memory_order_acquire)) return;
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
if (!f_ || began_.load(std::memory_order_relaxed)) return;
|
||||||
|
word(kBeganMark);
|
||||||
|
began_.store(true, std::memory_order_release);
|
||||||
|
}
|
||||||
|
bool has_begun() const { return began_.load(std::memory_order_acquire); }
|
||||||
|
/// ((Triforce) One compile of the step that is not one of a run of them (the program pass's compute warms), held for as
|
||||||
|
/// long as the driver has it: the first of those under way writes the word, the last to return writes 0 over it, so a
|
||||||
|
/// start that dies between them, in the translation, uses no try. Once began() wrote it, the word stays. No journal:
|
||||||
|
/// nothing. (A 0 that cannot be written leaves the word as it was: the start counts, as before)
|
||||||
|
class Compiling {
|
||||||
|
public:
|
||||||
|
explicit Compiling(Journal& j) : j_(j.enter() ? &j : nullptr) {}
|
||||||
|
Compiling(const Compiling&) = delete;
|
||||||
|
Compiling& operator=(const Compiling&) = delete;
|
||||||
|
~Compiling() {
|
||||||
|
if (j_) j_->leave();
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
Journal* j_;
|
||||||
|
};
|
||||||
|
/// Whether the word is written now (by began(), or a Compiling under way): what a start that died here would leave
|
||||||
|
bool counts() {
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
return f_ && (began_.load(std::memory_order_relaxed) || compiling_ > 0);
|
||||||
|
}
|
||||||
|
/// Worker slot's variant, about to go to the driver ({0, 0}: it came back). A slot past the last has no line
|
||||||
|
void mark(int slot, Id id) {
|
||||||
|
if (slot < 0 || slot >= kSlots) return;
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
if (!f_) return;
|
||||||
|
const uint64_t r[2] = {id.key, id.variant};
|
||||||
|
if (fseek(f_, 16 + 16 * slot, SEEK_SET) == 0 && fwrite(r, 8, 2, f_) == 2) fflush(f_);
|
||||||
|
}
|
||||||
|
void close() {
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
if (f_) fclose(f_), f_ = nullptr;
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
void word(uint32_t v) { // (under mu_)
|
||||||
|
if (fseek(f_, kBeganAt, SEEK_SET) == 0 && fwrite(&v, 4, 1, f_) == 1) fflush(f_);
|
||||||
|
}
|
||||||
|
bool enter() {
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
if (!f_ || began_.load(std::memory_order_relaxed)) return false;
|
||||||
|
if (compiling_++ == 0) word(kBeganMark);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
void leave() {
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
if (!f_ || compiling_ == 0) return;
|
||||||
|
if (--compiling_ == 0 && !began_.load(std::memory_order_relaxed)) word(0);
|
||||||
|
}
|
||||||
|
std::mutex mu_;
|
||||||
|
FILE* f_ = nullptr;
|
||||||
|
std::atomic<bool> began_{false};
|
||||||
|
unsigned compiling_ = 0; // (the Compiling under way; mu_)
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The skip list at path for this stamp (another stamp's, or no file: empty)
|
||||||
|
inline std::vector<Id> skip_read(const std::string& path, uint64_t stamp) {
|
||||||
|
std::vector<Id> ids;
|
||||||
|
FILE* f = fopen(path.c_str(), "rb");
|
||||||
|
if (!f) return ids;
|
||||||
|
uint32_t head[2]{};
|
||||||
|
uint64_t was = 0;
|
||||||
|
if (fread(head, 4, 2, f) == 2 && fread(&was, 8, 1, f) == 1 && head[0] == kSkipMagic && was == stamp) {
|
||||||
|
uint64_t r[2];
|
||||||
|
while (ids.size() < 65536 && fread(r, 8, 2, f) == 2) ids.push_back({r[0], r[1]});
|
||||||
|
}
|
||||||
|
fclose(f);
|
||||||
|
return ids;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The skip list at path with more added (each variant once), written whole. Returns the list, as skip_read would give it
|
||||||
|
inline std::vector<Id> skip_add(const std::string& path, uint64_t stamp, const std::vector<Id>& more) {
|
||||||
|
std::vector<Id> ids = skip_read(path, stamp);
|
||||||
|
bool grew = false;
|
||||||
|
for (const Id& id : more) {
|
||||||
|
bool have = false;
|
||||||
|
for (const Id& o : ids) have = have || o == id;
|
||||||
|
if (!have) ids.push_back(id), grew = true;
|
||||||
|
}
|
||||||
|
if (!grew) return ids;
|
||||||
|
const std::string tmp = path + ".tmp";
|
||||||
|
FILE* f = fopen(tmp.c_str(), "wb");
|
||||||
|
if (!f) return ids;
|
||||||
|
const uint32_t head[2] = {kSkipMagic, 0};
|
||||||
|
bool ok = fwrite(head, 4, 2, f) == 2 && fwrite(&stamp, 8, 1, f) == 1;
|
||||||
|
for (const Id& id : ids) {
|
||||||
|
const uint64_t r[2] = {id.key, id.variant};
|
||||||
|
ok = ok && fwrite(r, 8, 2, f) == 2;
|
||||||
|
}
|
||||||
|
ok = fclose(f) == 0 && ok;
|
||||||
|
if (ok) {
|
||||||
|
remove(path.c_str()); // (rename does not replace on Windows)
|
||||||
|
ok = rename(tmp.c_str(), path.c_str()) == 0;
|
||||||
|
}
|
||||||
|
if (!ok) remove(tmp.c_str());
|
||||||
|
return ids;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace gpl_rules
|
||||||
@@ -2,10 +2,48 @@
|
|||||||
there. */
|
there. */
|
||||||
#pragma once
|
#pragma once
|
||||||
|
|
||||||
|
#include <cctype>
|
||||||
|
#include <string>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
namespace mkwvk::gpu_choice {
|
namespace mkwvk::gpu_choice {
|
||||||
|
|
||||||
/// The row's choice as a position in the Vulkan 1.3 devices (0-based), or -1 for Auto. menu: opts.gpu_device (0 Auto,
|
/// The row's choice as a position in the Vulkan 1.3 devices (0-based), or -1 for Auto. menu: opts.gpu_device (0 Auto,
|
||||||
/// n the n-th usable device); usable: how many there are now. A choice past them (a GPU taken out since) is Auto.
|
/// n the n-th usable device); usable: how many there are now. A choice past them (a GPU taken out since) is Auto.
|
||||||
inline int menu_device(int menu, int usable) { return menu > 0 && menu <= usable ? menu - 1 : -1; }
|
inline int menu_device(int menu, int usable) { return menu > 0 && menu <= usable ? menu - 1 : -1; }
|
||||||
|
|
||||||
|
/// ((Triforce) 11-20) A device's name as the choice compares it: lower case, every run of white space one space, none at
|
||||||
|
/// either end. Windows and Vulkan name the same card alike but not always to the space ("AMD Radeon RX 7800 XT" from one
|
||||||
|
/// AMD driver, "AMD Radeon RX 7800 XT" from the other)
|
||||||
|
inline std::string name_key(const std::string& s) {
|
||||||
|
std::string out;
|
||||||
|
bool space = false;
|
||||||
|
for (unsigned char c : s) {
|
||||||
|
if (std::isspace(c) || c < 0x20) { space = !out.empty(); continue; }
|
||||||
|
if (space) out += ' ', space = false;
|
||||||
|
out += (char)std::tolower(c);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// ((Triforce) 11-20, tickets "iGPU used on laptop with dGPU to compile shaders", "Launcher choosing the wrong GPU when
|
||||||
|
/// compiling shaders" and two more) The device a choice means: by its name when one was saved (triforce_options.ini
|
||||||
|
/// gpu_device_name, which the launcher and the menu write with the number), else by the number as before (menu_device).
|
||||||
|
/// The launcher numbered the cards in Windows' order before the game had ever run (no triforce_gpus.txt yet), and Vulkan's
|
||||||
|
/// is often the other way round on a laptop: its "2: RTX 5060" was the game's 2, the 860M, and the build prepared every
|
||||||
|
/// shader there. A name that matches several cards (two of the same) takes the numbered one when its name is that name,
|
||||||
|
/// else the first. No name, or none matching (the card taken out): the number decides. Auto (menu 0) is Auto whatever name
|
||||||
|
/// the file still has. names: the usable devices' names in the game's order. Returns a position in them (0-based) or -1 for Auto
|
||||||
|
inline int pick(const std::string& want, const std::vector<std::string>& names, int menu) {
|
||||||
|
if (menu <= 0) return -1; // (Auto: a name left in the file chooses nothing)
|
||||||
|
const std::string key = name_key(want);
|
||||||
|
if (!key.empty()) {
|
||||||
|
const int numbered = menu_device(menu, (int)names.size());
|
||||||
|
if (numbered >= 0 && name_key(names[(size_t)numbered]) == key) return numbered;
|
||||||
|
for (size_t i = 0; i < names.size(); i++)
|
||||||
|
if (name_key(names[i]) == key) return (int)i;
|
||||||
|
}
|
||||||
|
return menu_device(menu, (int)names.size());
|
||||||
|
}
|
||||||
|
|
||||||
} // namespace mkwvk::gpu_choice
|
} // namespace mkwvk::gpu_choice
|
||||||
|
|||||||
49
gpu/vk/hang_watch.h
Normal file
49
gpu/vk/hang_watch.h
Normal file
@@ -0,0 +1,49 @@
|
|||||||
|
// The hang watch's judgement, apart from its thread, the clock and the stacks (vk_hangdump.cpp). After NX2navi's hang
|
||||||
|
// watch (its runtime/hang_watch.c hw_step): a report when no frame is shown for some seconds, once a stall, a few a run.
|
||||||
|
// tests/test_hang_watch.py
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace mkwvk::hangwatch {
|
||||||
|
|
||||||
|
constexpr int kReports = 3; // (at most this many a run: a game that stalls again and again says so three times)
|
||||||
|
|
||||||
|
// ((Triforce) The stacks asked for from outside: a file of this name beside the report's, "triforce_hang.txt.now" in the game
|
||||||
|
// folder, is looked for every second; the watch takes it away and writes where every thread is, into the log and the
|
||||||
|
// report's file, whether or not a frame is shown. For a game that goes on showing frames and gets nowhere: the owner's
|
||||||
|
// save that did not load on 2026-11-20 showed its loading screen at 240 frames a second for minutes, one game thread
|
||||||
|
// (BaseProcCreateAndDeleteThread) at a whole core, and the stall's own report never came: frames were shown. Asked for
|
||||||
|
// with a file because that needs nothing done to the game's process from outside. At most kAsked a run)
|
||||||
|
constexpr const char* kAskSuffix = ".now";
|
||||||
|
constexpr int kAsked = 16;
|
||||||
|
/// Whether an ask found now is served: the first kAsked of a run (`served`: how many were, counted up here)
|
||||||
|
inline bool ask(int& served) {
|
||||||
|
if (served >= kAsked) return false;
|
||||||
|
served++;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
struct Watch {
|
||||||
|
int64_t presents = -1; // the presents counted at the last look
|
||||||
|
int still = 0; // judged seconds running without one
|
||||||
|
bool told = false; // this stall has its report
|
||||||
|
int reports = 0;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// One second's look. presents: the frames shown so far. judged: whether a second without one counts (the game's window
|
||||||
|
/// in front and not minimized: the game may stop showing frames behind another window, and a watch asked for by name
|
||||||
|
/// judges always). True when a report is due now: `limit` judged seconds running without a present, once per stall (a
|
||||||
|
/// new one needs a present first), at most kReports a run.
|
||||||
|
inline bool step(Watch& w, int64_t presents, bool judged, int limit) {
|
||||||
|
if (presents != w.presents || !judged) {
|
||||||
|
w.presents = presents, w.still = 0, w.told = false;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if (++w.still < limit || w.told || w.reports >= kReports) return false;
|
||||||
|
w.told = true;
|
||||||
|
w.reports++;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::hangwatch
|
||||||
59
gpu/vk/helper_spin.h
Normal file
59
gpu/vk/helper_spin.h
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
// (Triforce, the third performance audit's P6, after NX2navi's replay workers: runtime/vk_exec.c rw_main's idle spin and the
|
||||||
|
// submitting thread's spin for its chunks, epicmeme546, 3010a637, 7534193b, 7eee4817, ffa599ef) A bounded spin before a sleep
|
||||||
|
// on a condition variable, for the parallel replay (vk_replay.cpp): a helper thread with no segment to record spins for the
|
||||||
|
// next before it sleeps (par_worker_main), and the replay thread spins for a segment a helper records before it sleeps for it
|
||||||
|
// (replay_handle). What each waits for is published where the spin can read it without the lock (ParPool::queued, Job::done),
|
||||||
|
// and the condition variable's wait with its predicate follows the spin as before: the spin only makes the wait shorter when
|
||||||
|
// what it waits for comes within it, it never decides anything. The audit's trace: the helpers began a segment 0.262 ms after
|
||||||
|
// they were given it, and the replay thread waited 1.74 ms a frame for them (r22-trace). tests/test_helper_spin.py
|
||||||
|
//
|
||||||
|
// RX_VK_HELPER_SPIN_US=<us>: the spin's bound (100, NX2navi's; at most 1000000); 0: no spin, straight to the wait as before
|
||||||
|
#pragma once
|
||||||
|
#include <chrono>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdlib>
|
||||||
|
#if defined(__x86_64__) || defined(_M_X64) || defined(__i386__) || defined(_M_IX86)
|
||||||
|
#include <immintrin.h>
|
||||||
|
#endif
|
||||||
|
|
||||||
|
namespace helper_spin {
|
||||||
|
|
||||||
|
/// The bound in nanoseconds that RX_VK_HELPER_SPIN_US's text gives (null: unset)
|
||||||
|
inline int64_t bound_ns(const char* text) {
|
||||||
|
if (!text || !*text) return 100000;
|
||||||
|
const long long us = std::atoll(text);
|
||||||
|
return us <= 0 ? 0 : us >= 1000000 ? 1000000000LL : us * 1000;
|
||||||
|
}
|
||||||
|
/// RX_VK_HELPER_SPIN_US, read once
|
||||||
|
inline int64_t spin_ns() {
|
||||||
|
static const int64_t v = bound_ns(std::getenv("RX_VK_HELPER_SPIN_US"));
|
||||||
|
return v;
|
||||||
|
}
|
||||||
|
inline void pause() {
|
||||||
|
#if defined(__x86_64__) || defined(_M_X64) || defined(__i386__) || defined(_M_IX86)
|
||||||
|
_mm_pause();
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
/// Spins until done() holds or ns have passed; whether done() held. ns <= 0: done() once, no spin. *spent (when given): the
|
||||||
|
/// time spun
|
||||||
|
template <class Done>
|
||||||
|
bool until(Done&& done, int64_t ns, int64_t* spent = nullptr) {
|
||||||
|
if (spent) *spent = 0;
|
||||||
|
if (done()) return true;
|
||||||
|
if (ns <= 0) return false;
|
||||||
|
const auto t0 = std::chrono::steady_clock::now();
|
||||||
|
const auto end = t0 + std::chrono::nanoseconds(ns);
|
||||||
|
bool held = false;
|
||||||
|
for (unsigned i = 1;; i++) {
|
||||||
|
pause();
|
||||||
|
if ((held = done())) break;
|
||||||
|
if (!(i & 7) && std::chrono::steady_clock::now() >= end) {
|
||||||
|
held = done();
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (spent) *spent = std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now() - t0).count();
|
||||||
|
return held;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace helper_spin
|
||||||
125
gpu/vk/import_ahead.h
Normal file
125
gpu/vk/import_ahead.h
Normal file
@@ -0,0 +1,125 @@
|
|||||||
|
/* (Triforce) The game's memory imported ahead of its first binding (vk_memory.cpp import_ahead_*). A pool's memory is imported
|
||||||
|
into Vulkan (VK_EXT_external_memory_host) at the first binding that reaches it, by 16 MB chunk; the import is the driver's
|
||||||
|
page locking, 8 to 17 ms a chunk on the owner's RTX 5090, and a binding the GPU writes, or a texture's storage read by the
|
||||||
|
GPU's unswizzle, is imported there and then on the replay's thread: seven in one dispatch made a submit of 79 ms in the
|
||||||
|
owner's test4 log, and the "unswizzled on the GPU" uploads of 22 to 32 ms were imports of their storage. The game's pools sit
|
||||||
|
at the same addresses from run to run (triforce_mirror_keep.txt relies on that too), so the ranges a run imported on its
|
||||||
|
own thread are kept in triforce_imports_ahead.txt beside the game, and in the next runs each is imported by the background
|
||||||
|
import thread as soon as the game's pool table holds memory there, before a binding wants it. A binding that comes while it is
|
||||||
|
under way waits for it, as for any import in the background. Plain functions: tests/test_import_ahead.cpp checks them.
|
||||||
|
RX_VK_IMPORT_AHEAD=1 or 0: on or off on any card; unset: on with NVIDIA's driver only, as the background imports of a
|
||||||
|
binding's reads are (RX_VK_ASYNC_IMPORT: background imports slowed an AMD card's own, vk_memory.cpp gpu_buffer_bind). */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdio>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace mkwvk::import_ahead {
|
||||||
|
|
||||||
|
constexpr size_t kMax = 256; // (ranges kept in the file at most)
|
||||||
|
constexpr size_t kPerLook = 4; // (ranges sent to the background thread at one look, 16 under way at most: a look comes every 8
|
||||||
|
// submissions too, and the test9 session's queue submits stalled beside batches of 16, made
|
||||||
|
// one after the other; each range is sent once a session, vk_memory.cpp)
|
||||||
|
|
||||||
|
/// RX_VK_IMPORT_AHEAD: 1 on, 0 off, unset (or anything else): on for NVIDIA's driver (vendor 0x10de) only, as before test12.
|
||||||
|
/// ((Triforce, 11-15) Off by default since test12: in the lab, at the title, with test12's binaries, the GPU hung after the
|
||||||
|
/// write-back of pool 0x1072000000 (a mirror in pieces) in 6 of 12 runs with the imports ahead on and in 0 of 10 with
|
||||||
|
/// RX_VK_IMPORT_AHEAD=0 (lazy write-back, GPL, BC1, Reflex, the sampler cache and the helper threads cleared, one by one);
|
||||||
|
/// the ahead list held 0x1072000000..0x1077000000, imported on the background thread while that pool's mirror was in use.
|
||||||
|
/// The owner's test10 never hung: its RAM was low all session, which pauses the imports ahead. Back on once the race is
|
||||||
|
/// found and closed. vendor: kept for that, unused)
|
||||||
|
/// ((Triforce, 11-15, later) A race that fits is closed: the host decommitted the game's freed pages under an import the background
|
||||||
|
/// thread had locked and not yet put in the table (vk_memory.cpp bg_held, memory_guest_imported; tests/test_bg_import_held.py), and
|
||||||
|
/// the imports ahead make that window wide where the game frees pages. Still off until a lab A/B with RX_VK_IMPORT_AHEAD=1 at the
|
||||||
|
/// title on a build with that fix hangs no more; the log's "freed under a background import" lines say whether the window was hit)
|
||||||
|
/// ((Triforce, 11-15, the lab A/B) On again: with that fix, 10 of 10 lab runs at the title with the imports ahead on were clean
|
||||||
|
/// and each hit the window 2 to 16 times ("freed under a background import"); with the fix undone (RX_VK_BG_IMPORT_HOLD=0)
|
||||||
|
/// 4 of 10 hung and others showed the title's video garbled. Runs E:\TriforceDev\lab-runs\holdA*, holdB*)
|
||||||
|
inline bool on(const char* env, uint32_t vendor) {
|
||||||
|
if (env && env[0] == '1' && !env[1]) return true;
|
||||||
|
if (env && env[0] == '0' && !env[1]) return false;
|
||||||
|
return vendor == 0x10DE;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// (Triforce, test8) A range imported ahead, [lo, hi), on chunk bounds: what an import on a binding's own thread took, its
|
||||||
|
/// span over the game's pools across chunk ends included (vk_memory.cpp span_game_pools). The first version kept 16 MB chunks
|
||||||
|
/// and imported each on its own: a binding across their ends then made them one import again on its own thread ("not in the
|
||||||
|
/// background: an import beside it grows", 6 of the 13 still made so in the owner's test8 session, and the 64 MB imports that
|
||||||
|
/// took in four of them)
|
||||||
|
struct Range {
|
||||||
|
uint64_t lo = 0, hi = 0;
|
||||||
|
bool operator==(const Range& o) const { return lo == o.lo && hi == o.hi; }
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The file's lines: "lo hi" in hex (a range), or "lo" alone (a chunk, as the first version wrote them); lines that are no such
|
||||||
|
/// numbers, and ranges not on chunk bounds, are left out. Sorted, none inside another, at most kMax
|
||||||
|
inline std::vector<Range> parse(const char* text, uint64_t chunk) {
|
||||||
|
std::vector<Range> out;
|
||||||
|
if (!chunk) return out;
|
||||||
|
for (const char* p = text; p && *p;) {
|
||||||
|
char* end = nullptr;
|
||||||
|
const unsigned long long lo = std::strtoull(p, &end, 16);
|
||||||
|
unsigned long long hi = 0;
|
||||||
|
bool ok = end != p && lo && lo % chunk == 0;
|
||||||
|
if (ok && *end == ' ') {
|
||||||
|
const char* q = end + 1;
|
||||||
|
hi = std::strtoull(q, &end, 16);
|
||||||
|
ok = end != q && hi > lo && hi % chunk == 0 && hi - lo <= (1ull << 30);
|
||||||
|
} else {
|
||||||
|
hi = lo + chunk;
|
||||||
|
}
|
||||||
|
ok = ok && (*end == '\n' || *end == '\r' || !*end);
|
||||||
|
if (ok) out.push_back({lo, hi});
|
||||||
|
while (*p && *p != '\n') p++;
|
||||||
|
if (*p) p++;
|
||||||
|
}
|
||||||
|
std::sort(out.begin(), out.end(), [](const Range& a, const Range& b) { return a.lo != b.lo ? a.lo < b.lo : a.hi > b.hi; });
|
||||||
|
std::vector<Range> kept; // (none inside another)
|
||||||
|
for (const Range& r : out)
|
||||||
|
if (kept.empty() || r.hi > kept.back().hi) kept.push_back(r);
|
||||||
|
if (kept.size() > kMax) kept.resize(kMax);
|
||||||
|
return kept;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// [lo, hi) into the list (sorted, none inside another): true when the list changed (the caller writes the file again). Ranges
|
||||||
|
/// inside the new one go; a new one inside a kept one changes nothing; a full list takes nothing new
|
||||||
|
inline bool add(std::vector<Range>& list, uint64_t lo, uint64_t hi) {
|
||||||
|
if (hi <= lo) return false;
|
||||||
|
for (const Range& r : list)
|
||||||
|
if (r.lo <= lo && hi <= r.hi) return false;
|
||||||
|
const size_t before = list.size();
|
||||||
|
std::erase_if(list, [&](const Range& r) { return lo <= r.lo && r.hi <= hi; });
|
||||||
|
if (list.size() == before && list.size() >= kMax) return false;
|
||||||
|
list.insert(std::lower_bound(list.begin(), list.end(), lo, [](const Range& r, uint64_t v) { return r.lo < v; }), Range{lo, hi});
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The file's text for list
|
||||||
|
inline std::vector<char> text(const std::vector<Range>& list) {
|
||||||
|
std::vector<char> out;
|
||||||
|
char line[48];
|
||||||
|
for (const Range& r : list) {
|
||||||
|
const int n = std::snprintf(line, sizeof line, "%llx %llx\n", (unsigned long long)r.lo, (unsigned long long)r.hi);
|
||||||
|
out.insert(out.end(), line, line + n);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The ranges of list that the game's pools (sorted by lo) reach now and wanted(range) allows (nothing imported over it,
|
||||||
|
/// nothing under way in it), at most n of them
|
||||||
|
template <class Pools, class Wanted>
|
||||||
|
std::vector<Range> due(const std::vector<Range>& list, const Pools& pools, size_t n, Wanted wanted) {
|
||||||
|
std::vector<Range> out;
|
||||||
|
for (const Range& r : list) {
|
||||||
|
if (out.size() >= n) break;
|
||||||
|
auto g = std::lower_bound(pools.begin(), pools.end(), r.hi, [](const auto& p, uint64_t v) { return p.lo < v; });
|
||||||
|
if (g == pools.begin() || std::prev(g)->hi <= r.lo) continue; // (no pool starting below its end reaches into it)
|
||||||
|
if (wanted(r)) out.push_back(r);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::import_ahead
|
||||||
50
gpu/vk/import_touch.h
Normal file
50
gpu/vk/import_touch.h
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
/* (Triforce) The pages of a range read before the driver imports it (vk_memory.cpp import_pool, RX_VK_IMPORT_TOUCH).
|
||||||
|
An import of the game's memory (VK_EXT_external_memory_host) is the driver locking the range's pages inside
|
||||||
|
vkAllocateMemory, and while it does a vkQueueSubmit2 from another thread waits for it. The ranges imported ahead are
|
||||||
|
whole, up to 1 GB, and much of one is pages nobody has touched since they were committed: the driver faults each in under
|
||||||
|
its lock. In the lab's Market stands (RTX 5090, rounds 74 and 77) the allocate of the ranges of 288 to 752 MB took 16 to
|
||||||
|
93 ms and the frame's queue submit stood beside it for as long: the 26 to 64 ms hitches of the owner's walks. With each
|
||||||
|
page read first, on the importing thread and outside the driver, the same allocates took 4 to 13 ms, and the longest
|
||||||
|
queue submit beside an import in play 10 to 12 ms for 71 to 93.
|
||||||
|
Only on the background import thread (the imports another thread's submit stands beside: an import on a binding's own
|
||||||
|
thread gains nothing and the read would add to it), only on Windows, and only where the range was committed whole. There
|
||||||
|
the reads add no memory: the import holds every page of its range in RAM as it is. They are not free in time: a stand's
|
||||||
|
imports of 64 MB or more took 581 and 614 ms with the read (392 and 420 ms reading, 168 and 172 allocating) against 405 ms
|
||||||
|
without, on the background thread; 1,212 ms in the first run after a build.
|
||||||
|
Plain functions: tests/test_import_touch.py checks them.
|
||||||
|
RX_VK_IMPORT_TOUCH=0: off, as before. */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstddef>
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace mkwvk::import_touch {
|
||||||
|
|
||||||
|
constexpr size_t kPage = 4096;
|
||||||
|
constexpr size_t kRun = 4u << 20; // (bytes read in one guarded call: a run that faults is left to the driver, the next goes on)
|
||||||
|
|
||||||
|
constexpr uint64_t kLargeMb = 64; // (imports of this size or more: their own count of log lines, vk_memory.cpp import_pool's report)
|
||||||
|
|
||||||
|
/// RX_VK_IMPORT_TOUCH: on unless it is 0
|
||||||
|
inline bool on(const char* env) { return !(env && env[0] == '0' && !env[1]); }
|
||||||
|
|
||||||
|
/// Whether an import's pages are read first: the switch, on the background import thread, the range committed whole
|
||||||
|
inline bool may(bool on, bool background_thread, bool committed_whole) { return on && background_thread && committed_whole; }
|
||||||
|
|
||||||
|
/// The length of the run at off in a range of bytes: kRun, or what is left
|
||||||
|
inline size_t run(uint64_t bytes, uint64_t off) { return off >= bytes ? 0 : (size_t)(bytes - off < kRun ? bytes - off : kRun); }
|
||||||
|
|
||||||
|
/// One byte of each page of [p, p + n) read, p on a page's start: the pages read. Not inlined: the caller's handler takes a
|
||||||
|
/// fault only across a call (vk_memory.cpp guarded_copy has why)
|
||||||
|
#if defined(__GNUC__) || defined(__clang__)
|
||||||
|
__attribute__((noinline))
|
||||||
|
#endif
|
||||||
|
inline uint64_t read_pages(const volatile uint8_t* p, size_t n) {
|
||||||
|
uint64_t pages = 0;
|
||||||
|
unsigned sink = 0;
|
||||||
|
for (size_t i = 0; i < n; i += kPage) sink += p[i], pages++;
|
||||||
|
(void)sink;
|
||||||
|
return pages;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::import_touch
|
||||||
42
gpu/vk/ini_merge.h
Normal file
42
gpu/vk/ini_merge.h
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
// (Triforce, handoff §9) triforce_options.ini has more than one writer: the game's Insert menu (vk_menu.cpp save, the whole
|
||||||
|
// file at every change) and whoever else changes it while the game runs (the owner in a text editor, a tool, a launcher of
|
||||||
|
// another make). The menu wrote the values it held, so a line changed outside since the game read the file was put back as the
|
||||||
|
// game had it at its next save: that change was lost, however long before the save it came. Now each line the game writes is
|
||||||
|
// merged by its key: the game's value where the game changed it since it last read or wrote the file (its base), else what the
|
||||||
|
// file has now. Two changes made within a second, one in the game and one outside, both stay; one row changed on both sides is
|
||||||
|
// the game's (the later writer). The game itself takes an outside change at its next start, as a file read at the start is.
|
||||||
|
// Plain logic: tests/test_options_writers.py compiles it alone
|
||||||
|
#pragma once
|
||||||
|
#include <map>
|
||||||
|
#include <string>
|
||||||
|
|
||||||
|
namespace ini_merge {
|
||||||
|
using Map = std::map<std::string, std::string>;
|
||||||
|
|
||||||
|
/// A value as lines are compared: without its line end and the spaces around it
|
||||||
|
inline std::string trim(const std::string& v) {
|
||||||
|
size_t a = 0, b = v.size();
|
||||||
|
while (a < b && (v[a] == ' ' || v[a] == '\t')) a++;
|
||||||
|
while (b > a && (v[b - 1] == '\n' || v[b - 1] == '\r' || v[b - 1] == ' ' || v[b - 1] == '\t')) b--;
|
||||||
|
return v.substr(a, b - a);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One line of the file into the map, when it is key=value (a key of something before the '=')
|
||||||
|
inline void add_line(Map& m, const std::string& line) {
|
||||||
|
const size_t eq = line.find('=');
|
||||||
|
if (eq == std::string::npos || eq == 0) return;
|
||||||
|
m[line.substr(0, eq)] = trim(line.substr(eq + 1));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The value the game writes for key: its own (game) where it changed it since base was taken (base: the game's values as of
|
||||||
|
/// its last read or write of the file; a key not in base is the game's), else the file's as it is now (disk), and the game's
|
||||||
|
/// where the file has no such line. *taken: set when the file's was taken over the game's
|
||||||
|
inline std::string pick(const std::string& key, const std::string& game, const Map& base, const Map& disk, bool* taken) {
|
||||||
|
const auto b = base.find(key);
|
||||||
|
if (b == base.end() || b->second != trim(game)) return game; // (changed in the game, or new to the file)
|
||||||
|
const auto d = disk.find(key);
|
||||||
|
if (d == disk.end() || d->second == b->second) return game;
|
||||||
|
if (taken) *taken = true;
|
||||||
|
return d->second;
|
||||||
|
}
|
||||||
|
} // namespace ini_merge
|
||||||
99
gpu/vk/lab_ab.h
Normal file
99
gpu/vk/lab_ab.h
Normal file
@@ -0,0 +1,99 @@
|
|||||||
|
/* (Triforce, lab) In-run A/B. Two runs of one build differ by more than most changes are worth (the other programs on the GPU,
|
||||||
|
which cores the threads landed on, the scene's state), so a change's effect is measured inside ONE run: the frames go in
|
||||||
|
phases of RX_VK_LAB_AB_FRAMES presents (300): none, knob 1, none, knob 2, ... and around again, and the frame trace
|
||||||
|
(RX_VK_FRAME_TRACE, vk_replay.cpp status_tick) carries the phase's knob in its last column (0: none, k: the k-th knob;
|
||||||
|
negative for a phase's first RX_VK_LAB_AB_SKIP frames (30), which a reader leaves out). perf\lowend\lab_ab.py compares each
|
||||||
|
knob's phases with the none-phases on both sides of them.
|
||||||
|
|
||||||
|
RX_VK_LAB_AB=<name>[:<value>],<name>[:<value>],... (at most 16)
|
||||||
|
Knobs joined by '+' are one knob that turns each of them on ("reclast+tlrec+rbsettle").
|
||||||
|
A knob named d_<site> is a delay: a busy wait of <value> microseconds at that site, at most once a frame (delay(), below):
|
||||||
|
what the frame's time answers to a part of it taking longer, which is whether that part is on the frame's critical path
|
||||||
|
(1 ms in for 1 ms of delay) or has slack (nothing). Any other name is a switch the code reads with on("<name>").
|
||||||
|
Not for players: nothing reads these without the variable. */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <atomic>
|
||||||
|
#include <chrono>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <cstring>
|
||||||
|
|
||||||
|
namespace mkwvk::lab_ab {
|
||||||
|
|
||||||
|
struct Knob {
|
||||||
|
char name[64];
|
||||||
|
int64_t value;
|
||||||
|
};
|
||||||
|
inline Knob knobs[16];
|
||||||
|
inline int n_knobs = -1; // (-1: the variable not read yet)
|
||||||
|
inline std::atomic<int> active{0}; // (0: none; k: knobs[k - 1])
|
||||||
|
inline std::atomic<uint64_t> frame_no{0}; // (presents counted here)
|
||||||
|
inline std::atomic<uint64_t> fired[16]; // (the frame each delay last waited in)
|
||||||
|
inline std::atomic<uint32_t> pres_thread{0}; // (the thread that presents: the game's Presentation thread, in the plain mode)
|
||||||
|
inline int64_t phase_frames = 300, skip_frames = 30;
|
||||||
|
|
||||||
|
inline void init() {
|
||||||
|
if (n_knobs >= 0) return;
|
||||||
|
int n = 0;
|
||||||
|
const char* e = getenv("RX_VK_LAB_AB");
|
||||||
|
while (e && *e && n < 16) {
|
||||||
|
const char* end = strchr(e, ',');
|
||||||
|
const size_t len = end ? (size_t)(end - e) : strlen(e);
|
||||||
|
char item[96] = {};
|
||||||
|
memcpy(item, e, len < sizeof item - 1 ? len : sizeof item - 1);
|
||||||
|
char* colon = strchr(item, ':');
|
||||||
|
knobs[n].value = colon ? atoll(colon + 1) : 1;
|
||||||
|
if (colon) *colon = 0;
|
||||||
|
strncpy(knobs[n].name, item, sizeof knobs[n].name - 1);
|
||||||
|
if (knobs[n].name[0]) n++;
|
||||||
|
e = end ? end + 1 : nullptr;
|
||||||
|
}
|
||||||
|
if (const char* f = getenv("RX_VK_LAB_AB_FRAMES")) phase_frames = atoll(f) > 0 ? atoll(f) : 300;
|
||||||
|
if (const char* s = getenv("RX_VK_LAB_AB_SKIP")) skip_frames = atoll(s);
|
||||||
|
n_knobs = n;
|
||||||
|
}
|
||||||
|
/// Once a present, on the presenting thread: the phase this frame belongs to (the trace's column)
|
||||||
|
inline int phase_tick() {
|
||||||
|
init();
|
||||||
|
if (!n_knobs) return 0;
|
||||||
|
const uint64_t f = frame_no.fetch_add(1) + 1;
|
||||||
|
const uint64_t phase = f / (uint64_t)phase_frames; // (0 none, 1 knob 1, 2 none, 3 knob 2, ...)
|
||||||
|
const int k = (phase & 1) ? (int)((phase / 2) % (uint64_t)n_knobs) + 1 : 0;
|
||||||
|
active.store(k, std::memory_order_relaxed);
|
||||||
|
const bool settling = (int64_t)(f % (uint64_t)phase_frames) < skip_frames;
|
||||||
|
return settling ? -(k + 1) : k; // (-1: none, settling; -(k + 1): knob k, settling)
|
||||||
|
}
|
||||||
|
/// Is this switch on in this phase: the phase's knob is it, or names it among several joined by '+' ("reclast+tlrec":
|
||||||
|
/// both on in that knob's phases, so a combination is one knob)
|
||||||
|
inline bool on(const char* name) {
|
||||||
|
if (n_knobs <= 0) return false;
|
||||||
|
const int k = active.load(std::memory_order_relaxed);
|
||||||
|
if (k <= 0) return false;
|
||||||
|
const size_t len = strlen(name);
|
||||||
|
for (const char* p = knobs[k - 1].name; p && *p;) {
|
||||||
|
const char* plus = strchr(p, '+');
|
||||||
|
const size_t n = plus ? (size_t)(plus - p) : strlen(p);
|
||||||
|
if (n == len && !strncmp(p, name, len)) return true;
|
||||||
|
p = plus ? plus + 1 : nullptr;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
inline int64_t value_of(const char* name, int64_t otherwise) {
|
||||||
|
if (n_knobs <= 0) return otherwise;
|
||||||
|
const int k = active.load(std::memory_order_relaxed);
|
||||||
|
return k > 0 && !strcmp(knobs[k - 1].name, name) ? knobs[k - 1].value : otherwise;
|
||||||
|
}
|
||||||
|
/// The delay site <site> (knob "d_<site>"): a busy wait of its microseconds, once a frame
|
||||||
|
inline void delay(const char* site) {
|
||||||
|
if (n_knobs <= 0) return;
|
||||||
|
const int k = active.load(std::memory_order_relaxed);
|
||||||
|
if (k <= 0 || knobs[k - 1].name[0] != 'd' || knobs[k - 1].name[1] != '_' || strcmp(knobs[k - 1].name + 2, site)) return;
|
||||||
|
const uint64_t f = frame_no.load(std::memory_order_relaxed);
|
||||||
|
if (fired[k - 1].exchange(f, std::memory_order_relaxed) == f) return;
|
||||||
|
const auto until = std::chrono::steady_clock::now() + std::chrono::microseconds(knobs[k - 1].value);
|
||||||
|
while (std::chrono::steady_clock::now() < until) {
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::lab_ab
|
||||||
70
gpu/vk/layer_split.h
Normal file
70
gpu/vk/layer_split.h
Normal file
@@ -0,0 +1,70 @@
|
|||||||
|
/* (Triforce) A new layered ASTC texture's first upload spread over a few frames, some of its layers each (vk_texture.cpp
|
||||||
|
texture_touch, split_take). The GPU decodes an ASTC texture (and encodes it again as BC3 or BC1 where the renderer keeps it
|
||||||
|
so) in the upload command buffer of the recording that first draws it. In the owner's test7 session one such texture,
|
||||||
|
1365 x 1365 x 90 layers with 11 levels (57 MB of ASTC, some 890 MB of texels), was the upload of an area change's frame
|
||||||
|
("slow upload 35.0 ms ... ASTC (BC3)"), that frame's upload command buffer took the GPU 114.8 ms, and the next frame's
|
||||||
|
capture waited 117.2 ms for it. The frame's upload budget (new_texture_later) counts bytes of storage, and always lets a
|
||||||
|
frame's first upload through: one texture alone made the backlog. Now such a texture, while the hold of the last picture may
|
||||||
|
leave textures for later, is uploaded a part of its layers a frame: its draws sample the dummy and its frames are held until
|
||||||
|
the last part is in, as for a new texture left whole for the next frame. At the hold's bound, or for a dispatch, what is left
|
||||||
|
goes at once. Plain functions: tests/test_layer_split.cpp checks them. RX_VK_UPLOAD_SPLIT_MB=<MB of texels a frame> (128),
|
||||||
|
0: whole, as before. */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <utility>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace mkwvk::layer_split {
|
||||||
|
|
||||||
|
using Ranges = std::vector<std::pair<uint64_t, uint64_t>>;
|
||||||
|
|
||||||
|
constexpr uint32_t kMaxParts = 6; // (frames at most: upload_owed::kLaterMax is 8, and a part is a frame)
|
||||||
|
|
||||||
|
/// RX_VK_UPLOAD_SPLIT_MB: the texels' MB a part may decode, 0: no split (unset or no number: 128)
|
||||||
|
inline uint64_t budget(const char* env) {
|
||||||
|
if (!env || !*env) return 128ull << 20;
|
||||||
|
char* end = nullptr;
|
||||||
|
const double v = std::strtod(env, &end);
|
||||||
|
if (end == env || *end || v < 0) return 128ull << 20;
|
||||||
|
return (uint64_t)(v * 1048576.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The texels an ASTC texture decodes to (RGBA8, every level and layer)
|
||||||
|
inline uint64_t texel_bytes(uint32_t w, uint32_t h, uint32_t levels, uint32_t layers) {
|
||||||
|
uint64_t n = 0;
|
||||||
|
for (uint32_t l = 0; l < std::max(1u, levels); l++) n += (uint64_t)std::max(1u, w >> l) * std::max(1u, h >> l) * 4;
|
||||||
|
return n * std::max(1u, layers);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The parts a texture of these texels and layers is uploaded in (1: whole)
|
||||||
|
inline uint32_t parts(uint64_t texels, uint32_t layers, uint64_t per_part) {
|
||||||
|
if (!per_part || layers < 2 || texels <= per_part) return 1;
|
||||||
|
const uint64_t p = (texels + per_part - 1) / per_part;
|
||||||
|
return (uint32_t)std::min<uint64_t>({p, (uint64_t)kMaxParts, (uint64_t)layers});
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The layers of `remaining` (ranges of the storage, whole layers of layer_size bytes, in order) split into this frame's part,
|
||||||
|
/// the first `per_frame` layers of them, and the rest
|
||||||
|
inline void take(const Ranges& remaining, uint64_t layer_size, uint32_t per_frame, Ranges& part, Ranges& rest) {
|
||||||
|
part.clear(), rest.clear();
|
||||||
|
if (!layer_size) return;
|
||||||
|
uint64_t left = (uint64_t)std::max(1u, per_frame) * layer_size;
|
||||||
|
for (const auto& [lo, hi] : remaining) {
|
||||||
|
if (hi <= lo) continue;
|
||||||
|
if (!left) { rest.push_back({lo, hi}); continue; }
|
||||||
|
const uint64_t n = std::min(left, hi - lo);
|
||||||
|
part.push_back({lo, lo + n});
|
||||||
|
left -= n;
|
||||||
|
if (lo + n < hi) rest.push_back({lo + n, hi});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The layers a part takes: the texture's layers over its parts, rounded up
|
||||||
|
inline uint32_t per_frame(uint32_t layers, uint32_t n_parts) {
|
||||||
|
return n_parts ? (layers + n_parts - 1) / n_parts : layers;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::layer_split
|
||||||
160
gpu/vk/lib_check.h
Normal file
160
gpu/vk/lib_check.h
Normal file
@@ -0,0 +1,160 @@
|
|||||||
|
// (Triforce, after NX2navi e10ef60's library warm check, runtime/vk_pipeline.c lw_probe and lw_main) Whether the stage libraries
|
||||||
|
// the preparation made are still in the pipeline cache, at a start whose record says they were made (vk_draw.cpp lib_check_cold,
|
||||||
|
// called by prepare_run). Apart from Vulkan so that tests/test_lib_check.py runs its rules alone.
|
||||||
|
//
|
||||||
|
// Why. triforce_prepared.bin's second stamp (vk_draw.cpp warmed_stamp) says the driver's part of the shaders was made for these
|
||||||
|
// programs, this driver and kWarmRecipe, and a start with that stamp goes straight to the precompile. Nothing looked at whether
|
||||||
|
// the libraries were still there. They are kept in our pipeline cache (gpl_libs_cached: asked for as a draw's link asks, the
|
||||||
|
// driver answers from the cache it is handed and not from its own on disk), so they are cold again when mkw_vk_pipelines.bin is
|
||||||
|
// lost or refused (deleted, another device's header, a short read), or when what a library is built with changed without
|
||||||
|
// kWarmRecipe counted up. Cold, no new pipeline is linked: each waits for a whole build, or its draws are left out for the frames
|
||||||
|
// that takes. NX2navi measured it on an RX 7900 XT (26.10.1) with all 42,142 of its prepared libraries cold, one entry each in the
|
||||||
|
// driver's cache after a change of their dynamic states: 93 ms a vertex and 242 ms a fragment library, a walk through a village
|
||||||
|
// skipped 11,180 draws and made 94 draws wait 5.5 s in all, four of them 1.0 to 2.0 s; with them warm again 0.3 and 0.7 ms.
|
||||||
|
//
|
||||||
|
// What. A sample of kSample kept program variants, spread over the program cache (from another place each start), each one's two
|
||||||
|
// shader libraries made as the driver's part makes them, timed, and made once more at once. Where the device can (gpl_ask_only)
|
||||||
|
// they are asked of the caches only, with the flag a draw's link asks with (FAIL_ON_PIPELINE_COMPILE_REQUIRED), and the answer
|
||||||
|
// is the driver's own on every vendor: cold is "a compile is required", the very answer that leaves a draw without its link.
|
||||||
|
// Without the flag (RX_VK_GPL_SYNC) the times say it, against a reference made on the spot: the second making comes from the
|
||||||
|
// cache by construction, the warm time of this driver on this PC, so no fixed time is needed (a library from the cache is 0.3 to
|
||||||
|
// 3 ms on one driver or another, one compiled 30 to 300 ms); cold is a first making over kRatio times the second and kSlackMs
|
||||||
|
// more. The sample is cold when more than half of its variants are, its median; under half a sample (a program cache with fewer
|
||||||
|
// variants) says nothing.
|
||||||
|
//
|
||||||
|
// Cold, the driver's part is made again at once, before the game, as after a driver update: every processor the preparation may
|
||||||
|
// have (prepare_threads.h, which counts the RAM free), the start-up screen's "Compiling shaders", its journal and skip list
|
||||||
|
// (gpl_rules.h), and the cache saved after it. Not on idle threads in play: Triforce's builder behind the game measured 55-64 fps
|
||||||
|
// against 41-47 with it compiling (vk_draw.cpp, before predict_background), and that is what a cold list of tens of thousands of
|
||||||
|
// libraries would do for most of a session. Warm (the start's usual case), nothing more happens than the sample: twelve program
|
||||||
|
// reads and some fifty library lookups.
|
||||||
|
//
|
||||||
|
// It is not made again for ever: a making again holds the start for minutes. <prepared>.check records, for the stamp, that the
|
||||||
|
// check made the part again, and how many starts in a row found the libraries cold again right after (strikes). Past kStrikes the
|
||||||
|
// driver or the PC does not keep them (a cache that does not stay, a driver that answers cold for what it has), and the libraries
|
||||||
|
// are left to the draws that need them, as before this check: with kStrikes 0 the part is made again once, and a start that finds
|
||||||
|
// it cold again right after stops there. NX2navi tries three times, but it warms in play; here each try is a start held. A making
|
||||||
|
// again whose cache could not be written (over the bound, or its save failed) or that warm_begin left out (gpl_rules.h kTries starts that died in it)
|
||||||
|
// is recorded the same way. A start that finds them warm removes the file, and a later loss is made again. RX_VK_LIB_CHECK=0: no
|
||||||
|
// check, as before.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdio>
|
||||||
|
#include <cstring>
|
||||||
|
#include <string>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace lib_check {
|
||||||
|
|
||||||
|
inline constexpr int kSample = 12; // (variants sampled, NX2navi's LW_SAMPLE)
|
||||||
|
inline constexpr int kLook = 2 * kSample; // (programs looked at for them: one may have no variant with libraries)
|
||||||
|
inline constexpr double kRatio = 4.0; // (first making against the second: cold above)
|
||||||
|
inline constexpr double kSlackMs = 3.0; // (and this much more: lookups of a few milliseconds say nothing)
|
||||||
|
inline constexpr int kStrikes = 0; // (made again once, and cold again right after: left to the draws)
|
||||||
|
|
||||||
|
/// RX_VK_LIB_CHECK: on unless "0"
|
||||||
|
inline bool on(const char* env) { return !env || strcmp(env, "0") != 0; }
|
||||||
|
|
||||||
|
/// One variant sampled: its two libraries' first making and the second (ms, both libraries together). asked: made asking the
|
||||||
|
/// caches only; answered_cold: they said a compile is required; made: both were made both times (a library the driver refuses
|
||||||
|
/// says nothing of the cache). Asked, the answer alone counts: a driver that made the library without a compile has it, however
|
||||||
|
/// long its first lookup took. Not asked, the times say it
|
||||||
|
struct Probe {
|
||||||
|
bool asked = false, answered_cold = false, made = false;
|
||||||
|
double first_ms = 0, again_ms = 0;
|
||||||
|
};
|
||||||
|
inline bool counts(const Probe& p) { return p.answered_cold || p.made; }
|
||||||
|
inline bool cold(const Probe& p) {
|
||||||
|
return p.answered_cold || (!p.asked && p.made && p.first_ms > kRatio * p.again_ms + kSlackMs);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a sample comes to: the variants that count, the cold ones, and the median times (for the log)
|
||||||
|
struct Tally {
|
||||||
|
int got = 0, cold = 0, answered = 0;
|
||||||
|
double first_median = 0, again_median = 0;
|
||||||
|
};
|
||||||
|
inline double median(std::vector<double> v) {
|
||||||
|
if (v.empty()) return 0;
|
||||||
|
std::sort(v.begin(), v.end());
|
||||||
|
return v[v.size() / 2];
|
||||||
|
}
|
||||||
|
inline Tally tally(const std::vector<Probe>& probes) {
|
||||||
|
Tally t;
|
||||||
|
std::vector<double> first, again;
|
||||||
|
for (const Probe& p : probes) {
|
||||||
|
if (!counts(p)) continue;
|
||||||
|
t.got++, t.cold += cold(p), t.answered += p.answered_cold;
|
||||||
|
first.push_back(p.first_ms);
|
||||||
|
if (p.made) again.push_back(p.again_ms);
|
||||||
|
}
|
||||||
|
t.first_median = median(first), t.again_median = median(again);
|
||||||
|
return t;
|
||||||
|
}
|
||||||
|
|
||||||
|
enum class Verdict { Unknown, Warm, Cold };
|
||||||
|
inline Verdict verdict(const Tally& t) {
|
||||||
|
if (t.got * 2 < kSample) return Verdict::Unknown;
|
||||||
|
return t.cold * 2 > t.got ? Verdict::Cold : Verdict::Warm;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// k places spread over n, the first at seed's place in the first step: each start samples other programs
|
||||||
|
inline std::vector<size_t> spots(size_t n, size_t k, uint64_t seed) {
|
||||||
|
std::vector<size_t> at;
|
||||||
|
if (!n || !k) return at;
|
||||||
|
const size_t step = std::max<size_t>(1, n / k);
|
||||||
|
for (size_t i = (size_t)(seed % step); i < n && at.size() < k; i += step) at.push_back(i);
|
||||||
|
return at;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- <prepared>.check: "lib-check <stamp> <strikes>"
|
||||||
|
|
||||||
|
struct State {
|
||||||
|
bool found = false;
|
||||||
|
uint64_t stamp = 0;
|
||||||
|
int strikes = 0;
|
||||||
|
};
|
||||||
|
inline State state_read(const std::string& path) {
|
||||||
|
State s;
|
||||||
|
FILE* f = fopen(path.c_str(), "rb");
|
||||||
|
if (!f) return s;
|
||||||
|
char line[96]{};
|
||||||
|
const size_t n = fread(line, 1, sizeof line - 1, f);
|
||||||
|
fclose(f);
|
||||||
|
line[n] = 0;
|
||||||
|
unsigned long long stamp = 0;
|
||||||
|
int strikes = 0;
|
||||||
|
if (sscanf(line, "lib-check %16llx %d", &stamp, &strikes) == 2 && strikes >= 0)
|
||||||
|
s.found = true, s.stamp = stamp, s.strikes = strikes;
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
/// Written whole beside the old one, then put in place (a start that ends in the middle leaves the old file)
|
||||||
|
inline bool state_write(const std::string& path, uint64_t stamp, int strikes) {
|
||||||
|
char line[96];
|
||||||
|
const int n = snprintf(line, sizeof line, "lib-check %016llx %d\n", (unsigned long long)stamp, strikes);
|
||||||
|
const std::string tmp = path + ".tmp";
|
||||||
|
FILE* f = fopen(tmp.c_str(), "wb");
|
||||||
|
if (!f) return false;
|
||||||
|
bool ok = fwrite(line, 1, (size_t)n, f) == (size_t)n;
|
||||||
|
ok = fclose(f) == 0 && ok;
|
||||||
|
if (ok) {
|
||||||
|
remove(path.c_str()); // (rename does not replace on Windows)
|
||||||
|
ok = rename(tmp.c_str(), path.c_str()) == 0;
|
||||||
|
}
|
||||||
|
if (!ok) remove(tmp.c_str());
|
||||||
|
return ok;
|
||||||
|
}
|
||||||
|
|
||||||
|
enum class Do { Nothing, Again, Stop };
|
||||||
|
/// What a start does with a sample's verdict, the record had of the last starts and the stamp now. *next: the strikes to record
|
||||||
|
/// once the part is made again (Again only)
|
||||||
|
inline Do decide(Verdict v, const State& had, uint64_t stamp, int* next) {
|
||||||
|
if (v != Verdict::Cold) return Do::Nothing;
|
||||||
|
const bool same = had.found && had.stamp == stamp; // (made again for this stamp, and cold again since)
|
||||||
|
if (same && had.strikes >= kStrikes) return Do::Stop;
|
||||||
|
*next = same ? had.strikes + 1 : 0;
|
||||||
|
return Do::Again;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace lib_check
|
||||||
214
gpu/vk/mem_parts.h
Normal file
214
gpu/vk/mem_parts.h
Normal file
@@ -0,0 +1,214 @@
|
|||||||
|
// (Triforce) What the process holds in RAM, in parts, for the log's "vk: memory, the parts" line (vk_memory.cpp
|
||||||
|
// memory_parts_tick, every 600 presents) and the rules that react to the PC's RAM running short. Apart from Vulkan and
|
||||||
|
// Windows so that tests/test_mem_parts.py runs it alone.
|
||||||
|
//
|
||||||
|
// Why: a 32 GB PC ran out of RAM in play (v0.4.0-test7: "RAM short (525 to 850 of 32069 MB available)" 40 times, the
|
||||||
|
// working set 20.9 GB, frames of 102 and 245 ms outside the replay and the present while Windows paged), and the log
|
||||||
|
// said only the process's totals. Its private bytes are no guide to the RAM (they grew with the VRAM, 29.7 -> 42.5 GB
|
||||||
|
// while the device's blocks grew 1.4 -> 11.3 GB: the driver's video memory is charged there), the working set is. The
|
||||||
|
// working set is the game's own memory (the guest window: its heap and regions, committed at once, in RAM as the game
|
||||||
|
// touches it) and the rest: the driver's shader modules, pipelines and pipeline cache, the allocator's blocks in system
|
||||||
|
// memory, the heap. The line gives each part that is known, and the game's share of the working set by a sample of
|
||||||
|
// its pages (sample_plan, below): one page in every so many of the committed window asked whether it is in RAM.
|
||||||
|
// (11-16) The rest is split the same way, by what the address space says each run is (Kind): the program's code, the
|
||||||
|
// driver's, other code, mapped files and views, write-combined or uncached memory (what a driver maps for the GPU), and
|
||||||
|
// the heaps and other private memory. A session of v0.4.0-test9 had 9.9 GB of working set outside the game's memory
|
||||||
|
// all through play, with the RAM short, and nothing in the line said what it was.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdio>
|
||||||
|
#include <string>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace mem_parts {
|
||||||
|
|
||||||
|
/// What a run of the address space is (kNone: not counted: free, reserved, or no access)
|
||||||
|
enum Kind : uint8_t { kNone = 0, kGuest, kExe, kDriverCode, kOtherCode, kMapped, kGpuVisible, kPrivate, kKinds };
|
||||||
|
/// A committed run, [lo, hi), and what it is
|
||||||
|
struct Span {
|
||||||
|
uint64_t lo = 0, hi = 0;
|
||||||
|
uint8_t kind = kGuest;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The runs between lo and hi that query(at, &base, &size, &kind) counts, which answers for the region that holds at
|
||||||
|
/// (VirtualQuery's answer: its base, its size from there, and what it is, kNone for what is not counted); false ends the
|
||||||
|
/// walk. [skip_lo, skip_hi) is stepped over without a query (the guest window, which has a walk of its own). At most
|
||||||
|
/// max_queries regions are asked (the walk is a system call a region), and runs of one kind that touch are joined.
|
||||||
|
/// *queries: how many were asked; *whole: false when max_queries stopped the walk before hi
|
||||||
|
template <class Query>
|
||||||
|
std::vector<Span> kind_spans(uint64_t lo, uint64_t hi, Query&& query, size_t max_queries, uint64_t skip_lo = 0,
|
||||||
|
uint64_t skip_hi = 0, size_t* queries = nullptr, bool* whole = nullptr) {
|
||||||
|
std::vector<Span> out;
|
||||||
|
uint64_t at = lo;
|
||||||
|
size_t n = 0;
|
||||||
|
while (at < hi && n < max_queries) {
|
||||||
|
if (at >= skip_lo && at < skip_hi) { // (no query spent on it)
|
||||||
|
at = skip_hi;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
n++;
|
||||||
|
uint64_t base = 0, size = 0;
|
||||||
|
uint8_t kind = kNone;
|
||||||
|
if (!query(at, &base, &size, &kind) || !size) {
|
||||||
|
at = hi; // (the end of the walk: whole as far as anything answers)
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
uint64_t end = base + size > base ? base + size : hi; // (no wrap past the top)
|
||||||
|
if (end <= at) {
|
||||||
|
at = hi;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
if (skip_hi > skip_lo && at < skip_lo && end > skip_lo) end = skip_lo; // (the skipped window is not counted)
|
||||||
|
const uint64_t a = base > at ? base : at, b = end < hi ? end : hi;
|
||||||
|
if (kind != kNone && a < b) {
|
||||||
|
if (!out.empty() && out.back().hi == a && out.back().kind == kind) out.back().hi = b;
|
||||||
|
else out.push_back({a, b, kind});
|
||||||
|
}
|
||||||
|
at = end;
|
||||||
|
}
|
||||||
|
if (queries) *queries = n;
|
||||||
|
if (whole) *whole = at >= hi;
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
/// The committed runs between lo and hi, from query(at, &base, &size, &committed) (the guest window's walk)
|
||||||
|
template <class Query>
|
||||||
|
std::vector<Span> committed_spans(uint64_t lo, uint64_t hi, Query&& query, size_t max_queries, size_t* queries = nullptr,
|
||||||
|
bool* whole = nullptr) {
|
||||||
|
return kind_spans(
|
||||||
|
lo, hi,
|
||||||
|
[&](uint64_t at, uint64_t* base, uint64_t* size, uint8_t* kind) {
|
||||||
|
bool committed = false;
|
||||||
|
if (!query(at, base, size, &committed)) return false;
|
||||||
|
*kind = committed ? kGuest : kNone;
|
||||||
|
return true;
|
||||||
|
},
|
||||||
|
max_queries, 0, 0, queries, whole);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Pages to ask about: one every stride bytes of each committed run (stride a whole number of pages, at least one, made
|
||||||
|
/// larger until the runs' pages are at most max_samples strides), each standing for the bytes up to the next (the run's
|
||||||
|
/// last for what is left). Each run has its first page asked, so there are at most max_samples + spans.size() of them
|
||||||
|
struct Plan {
|
||||||
|
uint64_t committed = 0, stride = 0;
|
||||||
|
std::vector<uint64_t> at, bytes;
|
||||||
|
std::vector<uint8_t> kind; // (each sample's run's)
|
||||||
|
};
|
||||||
|
inline Plan sample_plan(const std::vector<Span>& spans, uint64_t page, size_t max_samples) {
|
||||||
|
Plan p;
|
||||||
|
if (!page) page = 4096;
|
||||||
|
for (const Span& s : spans) p.committed += s.hi - s.lo;
|
||||||
|
if (!max_samples) max_samples = 1;
|
||||||
|
const uint64_t pages = (p.committed + page - 1) / page;
|
||||||
|
const uint64_t every = pages > max_samples ? (pages + max_samples - 1) / max_samples : 1;
|
||||||
|
p.stride = every * page;
|
||||||
|
for (const Span& s : spans) // (a run shorter than the stride counts all of itself by its first page)
|
||||||
|
for (uint64_t a = s.lo; a < s.hi; a += p.stride) {
|
||||||
|
p.at.push_back(a);
|
||||||
|
p.bytes.push_back(s.hi - a < p.stride ? s.hi - a : p.stride);
|
||||||
|
p.kind.push_back(s.kind);
|
||||||
|
if (s.hi - a <= p.stride) break; // (and no step past the top of the address space)
|
||||||
|
}
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
/// What the pages found in RAM stand for, in bytes (resident[i] answers for plan.at[i])
|
||||||
|
inline uint64_t resident_bytes(const Plan& p, const std::vector<bool>& resident) {
|
||||||
|
uint64_t r = 0;
|
||||||
|
for (size_t i = 0; i < p.at.size() && i < resident.size(); i++)
|
||||||
|
if (resident[i]) r += p.bytes[i];
|
||||||
|
return r;
|
||||||
|
}
|
||||||
|
/// The same, by the samples' kinds, into out[kKinds] (added to what is there)
|
||||||
|
inline void resident_by_kind(const Plan& p, const std::vector<bool>& resident, uint64_t out[kKinds]) {
|
||||||
|
for (size_t i = 0; i < p.at.size() && i < resident.size(); i++)
|
||||||
|
if (resident[i] && p.kind[i] < kKinds) out[p.kind[i]] += p.bytes[i];
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The parts the line gives, in bytes (-1: not known on this platform or in this run)
|
||||||
|
struct Parts {
|
||||||
|
int64_t working_set = -1, private_bytes = -1;
|
||||||
|
int64_t guest_committed = -1, guest_resident = -1;
|
||||||
|
int64_t by_kind[kKinds] = {-1, -1, -1, -1, -1, -1, -1, -1}; // (the rest in RAM by what it is; -1: not sampled)
|
||||||
|
int64_t modules = 0, module_bytes = 0; // (program variants' shader modules alive, and their SPIR-V)
|
||||||
|
int64_t packed_stages = 0, packed_bytes = 0; // (stages kept packed instead, and what that takes)
|
||||||
|
int64_t pcache_bytes = -1; // (the pipeline cache as last read or saved; -1: let go)
|
||||||
|
int64_t pipelines = 0, libraries = 0; // (graphics pipelines and stage libraries kept)
|
||||||
|
int64_t drawn = -1; // (of those pipelines, how many a draw has looked up; -1: not known)
|
||||||
|
int64_t released = 0; // (kept pipelines no draw used, given back for the RAM)
|
||||||
|
int64_t system_blocks = -1; // (the allocator's blocks in system memory)
|
||||||
|
int64_t ram_avail = -1, ram_total = -1;
|
||||||
|
};
|
||||||
|
inline std::string mb(int64_t b) {
|
||||||
|
if (b < 0) return "?";
|
||||||
|
char t[32];
|
||||||
|
std::snprintf(t, sizeof t, "%lld", (long long)(b >> 20));
|
||||||
|
return t;
|
||||||
|
}
|
||||||
|
/// The line, without its "vk: " head
|
||||||
|
inline std::string line(const Parts& p) {
|
||||||
|
std::string s = "memory, the parts (MB): working set " + mb(p.working_set) + " (private " + mb(p.private_bytes) + ")";
|
||||||
|
s += "; the game's own memory " + mb(p.guest_committed) + " committed, " + mb(p.guest_resident) + " of it in RAM";
|
||||||
|
if (p.working_set >= 0 && p.guest_resident >= 0)
|
||||||
|
s += "; the rest of the working set " + mb(p.working_set > p.guest_resident ? p.working_set - p.guest_resident : 0);
|
||||||
|
if (p.by_kind[kPrivate] >= 0) {
|
||||||
|
static const char* const names[kKinds] = {"", "", "the program's code", "the driver's code", "other code",
|
||||||
|
"mapped files and views", "write-combined or uncached memory",
|
||||||
|
"the heaps and other private memory"};
|
||||||
|
s += " (sampled:";
|
||||||
|
for (int k = kExe; k < kKinds; k++) s += std::string(k == kExe ? " " : ", ") + names[k] + " " + mb(p.by_kind[k]);
|
||||||
|
s += ")";
|
||||||
|
}
|
||||||
|
char t[160];
|
||||||
|
std::snprintf(t, sizeof t, " (%lld modules", (long long)p.modules);
|
||||||
|
s += ": shader modules " + mb(p.module_bytes) + t;
|
||||||
|
std::snprintf(t, sizeof t, "; %lld stages packed in ", (long long)p.packed_stages);
|
||||||
|
s += t + mb(p.packed_bytes) + ")";
|
||||||
|
s += p.pcache_bytes < 0 ? ", the pipeline cache let go" : ", the pipeline cache " + mb(p.pcache_bytes);
|
||||||
|
std::snprintf(t, sizeof t, ", %lld pipelines and %lld stage libraries kept", (long long)p.pipelines, (long long)p.libraries);
|
||||||
|
s += t;
|
||||||
|
if (p.drawn >= 0) {
|
||||||
|
std::snprintf(t, sizeof t, " (%lld of the pipelines drawn; %lld no draw used given back)", (long long)p.drawn,
|
||||||
|
(long long)p.released);
|
||||||
|
s += t;
|
||||||
|
}
|
||||||
|
s += ", the allocator's blocks in system memory " + mb(p.system_blocks);
|
||||||
|
s += "; RAM " + mb(p.ram_avail) + " of " + mb(p.ram_total) + " available";
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- The PC's RAM running short: what the back end does about it that costs play nothing (vk_memory.cpp ram_watch) ----
|
||||||
|
/// Short: under 1.5 GB available, the imports' rule (vk_memory.cpp imports_give_back), whatever the PC has
|
||||||
|
constexpr uint64_t kShortBytes = 1536ull << 20;
|
||||||
|
inline bool short_now(uint64_t avail) { return avail < kShortBytes; }
|
||||||
|
/// Low: under 3 GB available, or a tenth of the RAM on a PC of more than 30 GB. The pipeline builder behind the game
|
||||||
|
/// (vk_draw.cpp predict_background) rests then: what the driver keeps of each pipeline it builds is RAM
|
||||||
|
inline bool low_now(uint64_t avail, uint64_t total) {
|
||||||
|
const uint64_t tenth = total / 10;
|
||||||
|
return avail < (tenth > (3ull << 30) ? tenth : (3ull << 30));
|
||||||
|
}
|
||||||
|
/// Whether a copy of bytes fits in RAM now with 2 GB to spare: the pipeline cache's save in play fetches the whole cache into
|
||||||
|
/// memory (2 to 3 GB with the stage libraries in it), and with less than that available it is what Windows pages out
|
||||||
|
inline bool copy_fits(uint64_t avail, uint64_t bytes) { return avail > bytes && avail - bytes > (2ull << 30); }
|
||||||
|
/// (11-16) Whether the precompile keeps every pipeline it builds for the session (vk_draw.cpp precompile_keeps_all): only where
|
||||||
|
/// the RAM the game can have, what is available plus what the process holds already (own), is 22 GB or more, on a PC of
|
||||||
|
/// 20 GB or more. The figure is v0.4.0-test9's, a 32 GB PC that started with 18.4 GB free: the game's own memory reached
|
||||||
|
/// 9.7 GB in RAM, the rest of the working set 9.9 GB, of which the precompile's 14,535 graphics and 1,033 compute
|
||||||
|
/// pipelines took about 3.3 GB (the working set before and after it); 9.7 + 6.6 + 3.3 GB, and 2 GB for Windows to page
|
||||||
|
/// with. The PC's total alone says nothing of what its other programs hold
|
||||||
|
constexpr uint64_t kKeepAllBytes = 22ull << 30;
|
||||||
|
inline bool keep_all_pipelines(uint64_t total, uint64_t avail, uint64_t own) {
|
||||||
|
return total >= (20ull << 30) && avail + own >= kKeepAllBytes;
|
||||||
|
}
|
||||||
|
/// (11-18) What the process holds that counts as the game's for keep_all_pipelines: its working set (ws) less what reading the
|
||||||
|
/// pipeline cache added (cache_read; the driver holds that all session, so it is no room for the pipelines). Two 32 GB PCs on
|
||||||
|
/// v0.5.1 with 2.7 and 3.0 GB cache files (4.7 and 4.9 GB of working set once read): 18.2 GB available + 7.0 held and 19.1 +
|
||||||
|
/// 7.7, both over 22 GB, everything kept, and RAM short all session (39 times on one; the other lost the GPU after the save
|
||||||
|
/// loaded). Without the cache: 20.5 and 21.9 GB, kept by none. Not on AMD's driver (amd: the vendor 0x1002), which the rule
|
||||||
|
/// keeps as it was: there a pipeline left to its first draw cost ~100 ms (an RX 6650 XT: 152 of them in one 15.7 s frame),
|
||||||
|
/// worse than the RAM
|
||||||
|
inline uint64_t own_for_keep(uint64_t ws, uint64_t cache_read, bool amd) {
|
||||||
|
if (amd) return ws;
|
||||||
|
return ws > cache_read ? ws - cache_read : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mem_parts
|
||||||
@@ -7,7 +7,9 @@
|
|||||||
|
|
||||||
namespace mkwvk::menu_restart {
|
namespace mkwvk::menu_restart {
|
||||||
|
|
||||||
constexpr int kMax = 16; // (rows that take a restart: the menu has ten; a row past kMax would have no slot)
|
// (rows that take a restart: the menu has about twenty; a row past kMax would have no slot. At most the bits of pending()'s
|
||||||
|
// mask: 32)
|
||||||
|
constexpr int kMax = 32;
|
||||||
|
|
||||||
struct State {
|
struct State {
|
||||||
int n = 0;
|
int n = 0;
|
||||||
|
|||||||
131
gpu/vk/menu_sound_events.h
Normal file
131
gpu/vk/menu_sound_events.h
Normal file
@@ -0,0 +1,131 @@
|
|||||||
|
// (Triforce) The options' own sounds, the game's interface sounds, after NX2navi's menu sounds: which sound a tick of the
|
||||||
|
// menu asks for, and where the sounds wait until the host's sound thread takes them (vk_api.h HOS_OPT_MENU_SOUND;
|
||||||
|
// host/menu_sound.c reads the game's files and plays them).
|
||||||
|
//
|
||||||
|
// Which sound, as NX2navi has it: the options opening and closing; another tab; the cursor on another row, card or button,
|
||||||
|
// and a value changed with Left and Right; a press on something (confirm, or a click); a step back. The menu's input is
|
||||||
|
// one long function (vk_menu.cpp menu_tick) that changes its state in many places, so the sound is told from what the
|
||||||
|
// tick changed: the state is taken before and after it (State), and with the keys the tick saw (Input) it names one
|
||||||
|
// sound at most (sound_of). The mouse resting on a row moves the cursor without a sound, as NX2navi's does.
|
||||||
|
//
|
||||||
|
// Inside the game's own Options screen the options are tabs beside the game's own list (vk_menu.cpp game_options_shown):
|
||||||
|
// going between that list and our tabs with L and R is a tab's move, not an opening; and Back on one of our tabs is the
|
||||||
|
// game's Back too (host/nn_hid.c), which leaves the screen with the game's own sound, so ours is not played over it.
|
||||||
|
// While the game's own list is shown the input and the sounds are the game's.
|
||||||
|
//
|
||||||
|
// The queue: the menu's thread puts, the host's sound thread takes, no lock (one of each). It holds kQueue sounds; one
|
||||||
|
// more is dropped. A sound older than kStaleNs when it is taken is not played: with no sound thread to take them
|
||||||
|
// (the game has not opened its sound yet, or stopped it) the presses of a while ago would otherwise all sound at once.
|
||||||
|
//
|
||||||
|
// No ImGui and no Vulkan: tests/test_menu_sound_events.py compiles it alone.
|
||||||
|
#pragma once
|
||||||
|
#include <atomic>
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace menu_sound {
|
||||||
|
// (vk_api.h HOS_MENU_SOUND_*: vk_menu.cpp holds them against these)
|
||||||
|
constexpr int kNone = 0, kOpen = 1, kClose = 2, kCursor = 3, kDecide = 4, kCancel = 5, kTabLeft = 6, kTabRight = 7;
|
||||||
|
// ((Triforce) and the Mods screen's and entry's own, which sound_of never names: the fanfare of a mod installed, and the
|
||||||
|
// system menu's cursor and "decide" of the game's lists the Mods entry is under. (11-19) The queue took the seven above
|
||||||
|
// only and dropped these without a word: moving onto Mods, A on it, Quit's dialog and the fanfare were silent)
|
||||||
|
constexpr int kFanfare = 8, kSysCursor = 9, kSysDecide = 10, kLast = kSysDecide;
|
||||||
|
|
||||||
|
// (State::page: the tabs' rows, a page over them, and the restart question, whose two buttons L and R choose between)
|
||||||
|
constexpr int kPageRows = 0, kPageAsk = 5;
|
||||||
|
// (State::mode: what the chosen row or card waits for)
|
||||||
|
constexpr int kModeNone = 0,
|
||||||
|
kModeListen = 1, // the Controls page: the new input of a binding
|
||||||
|
kModeEdit = 2, // Left and Right act on it each press: an item's amount, a flag's On and Off
|
||||||
|
kModeConfirm = 3; // Restart or Quit: its second press
|
||||||
|
|
||||||
|
/// What the options show, as far as their sounds go
|
||||||
|
struct State {
|
||||||
|
bool open = false; // the options are open
|
||||||
|
bool embedded = false; // ...as tabs inside the game's own Options screen
|
||||||
|
int page = kPageRows; // what is shown: the tabs' rows, or a page over them (vk_menu.cpp page; the Motion page is one too)
|
||||||
|
int tab = 0; // the tab, of the tabs' rows
|
||||||
|
int group = 0; // a page's section, which L and R turn: the inventory's, the teleport list's, a name page's
|
||||||
|
int cursor = 0; // the row, card, cell or button chosen
|
||||||
|
int mode = kModeNone;
|
||||||
|
int64_t value = 0; // what the chosen row is set to
|
||||||
|
bool operator==(const State&) const = default;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// What a tick's input was, after the mouse's clicks were made the keys they stand for
|
||||||
|
struct Input {
|
||||||
|
bool toggle = false; // the keys that open and close the options over the game (Insert, F1, a gamepad's Home)
|
||||||
|
bool back = false; // Esc, the button B is bound to, a right click
|
||||||
|
bool confirm = false; // Enter, the button A is bound to, a click on what acts by one
|
||||||
|
bool tab_left = false, tab_right = false; // L and R, Q and E, Tab, the tab strip's arrows
|
||||||
|
bool left = false, right = false; // Left and Right, a click on a value's left or right half
|
||||||
|
bool move = false; // Up, Down, Page Up, Page Down, Home, End
|
||||||
|
bool clear = false; // Delete, Backspace (a binding taken away)
|
||||||
|
bool click = false; // the left mouse button went down
|
||||||
|
bool drag = false; // the left mouse button is held on a slider
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The sound of a tick that found the options as `b` and left them as `a`: kNone, or one of the seven
|
||||||
|
inline int sound_of(const State& b, const State& a, const Input& in) {
|
||||||
|
if (!b.open && !a.open) return kNone; // (closed: the keys are the game's)
|
||||||
|
if (!b.open) // (opened: over the game by its keys, or from the game's own list by L or R, which is a tab's move)
|
||||||
|
return in.toggle ? kOpen : a.tab == 0 ? kTabRight : kTabLeft;
|
||||||
|
if (!a.open) {
|
||||||
|
// (closed by a press that acted: a map to travel to, an event, Restart now, an age switched, the question's Later)
|
||||||
|
if (in.confirm || in.left || in.right) return kDecide;
|
||||||
|
if (!b.embedded || in.toggle) return kClose;
|
||||||
|
// (our tabs inside the game's screen: on to the game's own list with L or R, or by a click on its tab, the first)
|
||||||
|
if (in.tab_left || in.tab_right) return in.tab_left ? kTabLeft : kTabRight;
|
||||||
|
// (Back is the game's Back too and has the game's sound; and the game's screen may have gone by itself)
|
||||||
|
return in.click && !in.back ? kTabLeft : kNone;
|
||||||
|
}
|
||||||
|
const bool changed = !(a == b);
|
||||||
|
if (b.mode == kModeListen) { // (a binding waited for its input: a key pressed meanwhile is that input, not the menu's)
|
||||||
|
if (a.mode == kModeListen) return kNone;
|
||||||
|
return a.value != b.value ? kDecide : in.back ? kCancel : kNone; // (bound; given up; or nothing came in time)
|
||||||
|
}
|
||||||
|
if ((in.back || in.toggle) && changed) return kCancel; // (out of a page, a mode, or to the restart question)
|
||||||
|
const bool tabs = in.tab_left || in.tab_right;
|
||||||
|
if (tabs && changed && b.page != kPageAsk) return in.tab_left ? kTabLeft : kTabRight;
|
||||||
|
// (a click on a tab, or on a page's section: by where it is from the one before)
|
||||||
|
if (in.click && a.page == b.page && a.tab != b.tab) return a.tab > b.tab ? kTabRight : kTabLeft;
|
||||||
|
if (in.click && a.page == b.page && a.group != b.group) return a.group > b.group ? kTabRight : kTabLeft;
|
||||||
|
// (a page opened, by a row that may have taken the press for itself; a page left, by a Back its tab took for itself)
|
||||||
|
if (a.page != b.page) return a.page == kPageRows ? kCancel : kDecide;
|
||||||
|
if (in.confirm) return kDecide;
|
||||||
|
if (in.click && (changed || in.left || in.right)) return kDecide;
|
||||||
|
const bool arrows = in.move || in.left || in.right || in.clear || tabs;
|
||||||
|
if (arrows && changed) return kCursor;
|
||||||
|
if ((in.left || in.right) && a.mode == kModeEdit && b.mode == kModeEdit) return kCursor; // (acts each press)
|
||||||
|
if (in.drag && a.value != b.value) return kCursor; // (a slider drawn along)
|
||||||
|
return kNone;
|
||||||
|
}
|
||||||
|
|
||||||
|
constexpr int kQueue = 16;
|
||||||
|
constexpr int64_t kStaleNs = 250000000; // (a quarter second)
|
||||||
|
|
||||||
|
struct Queue {
|
||||||
|
struct Entry { int sound; int64_t ns; };
|
||||||
|
Entry entries[kQueue] = {};
|
||||||
|
std::atomic<unsigned> head{0}, tail{0}; // (taken up to head, put up to tail)
|
||||||
|
|
||||||
|
/// The menu's thread: a sound asked for at now_ns. false when it is none of the sounds (1 to kLast), or the queue is full
|
||||||
|
bool put(int sound, int64_t now_ns) {
|
||||||
|
if (sound <= kNone || sound > kLast) return false;
|
||||||
|
const unsigned t = tail.load(std::memory_order_relaxed);
|
||||||
|
if (t - head.load(std::memory_order_acquire) >= kQueue) return false;
|
||||||
|
entries[t % kQueue] = {sound, now_ns};
|
||||||
|
tail.store(t + 1, std::memory_order_release);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
/// The host's sound thread: the oldest sound waiting that is not stale, 0 when there is none
|
||||||
|
int take(int64_t now_ns) {
|
||||||
|
for (;;) {
|
||||||
|
const unsigned h = head.load(std::memory_order_relaxed);
|
||||||
|
if (h == tail.load(std::memory_order_acquire)) return kNone;
|
||||||
|
const Entry e = entries[h % kQueue];
|
||||||
|
head.store(h + 1, std::memory_order_release);
|
||||||
|
if (now_ns - e.ns <= kStaleNs) return e.sound;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
} // namespace menu_sound
|
||||||
109
gpu/vk/merged_rows.h
Normal file
109
gpu/vk/merged_rows.h
Normal file
@@ -0,0 +1,109 @@
|
|||||||
|
/* (Triforce, options_version 7; 8 on v0.5.1) The Insert menu's rows that each stand for two saved values (vk_menu.cpp), and
|
||||||
|
what an options file from before version 8 becomes (v0.5.1 wrote 7 for its Frame pacing default, with the old rows' keys
|
||||||
|
still in it; the 436f9ac-based builds wrote 7 for this, without them: kVersion is 8 so that both are moved, and the second
|
||||||
|
has nothing to move). The notes "(Triforce, options_version 7)" across the source and the tests name this change by its
|
||||||
|
436f9ac-based number: on this line it is version 8, and version 7 is v0.5.1's Frame pacing default (vk_menu.cpp
|
||||||
|
menu_init, "version < 7"); a file of 7 can still have the old rows' keys. The owner: "there's multiple settings that do
|
||||||
|
multiple things and they contradict each other ... If a setting can be implemented in some other way, then remove the
|
||||||
|
setting entirely". So one row chooses what two did: HDR (and RenoDX HDR), Frame generation (and DLSS frame generation), Frame pacing (and Steady frame rate), Low latency
|
||||||
|
(NVIDIA Reflex or AMD Anti-Lag 2, by the graphics card's maker). The saved values keep their keys and their meanings, for the
|
||||||
|
renderer and the host read them, some before the menu (hdr=, renodx_hdr=, frame_generation=, dlss_frame_generation=,
|
||||||
|
frame_pacing=, steady_frame_rate=, nvidia_reflex=, amd_anti-lag_2=): the rows are views of them.
|
||||||
|
Plain functions, no Vulkan, no menu: tests/test_merged_rows.py checks them, with a copy of the owner's own options. */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
namespace mkwvk::merged_rows {
|
||||||
|
|
||||||
|
// ---- HDR: Off | Expanded | Native | Native + RenoDX (opts.hdr 0 to 2, opts.renodx 0 or 1: RenoDX runs only with Native) ----
|
||||||
|
enum { HDR_OFF = 0, HDR_EXPANDED = 1, HDR_NATIVE = 2, HDR_RENODX = 3 };
|
||||||
|
constexpr int hdr_row(int hdr, int renodx) {
|
||||||
|
return hdr == HDR_NATIVE && renodx ? (int)HDR_RENODX : hdr < 0 ? 0 : hdr > HDR_NATIVE ? (int)HDR_NATIVE : hdr;
|
||||||
|
}
|
||||||
|
inline void hdr_set(int row, int& hdr, int& renodx) {
|
||||||
|
hdr = row >= HDR_RENODX ? (int)HDR_NATIVE : row < 0 ? 0 : row;
|
||||||
|
renodx = row >= HDR_RENODX;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- Frame generation: opts.framegen's own values (framegen_rows.h: 0 Off, 1 Auto, 2 to 8, 9 AMD FSR 4), then NVIDIA DLSS
|
||||||
|
// 2x, 3x and 4x (opts.dlssg 1 to 3, read at the start: a restart). Choosing one sets the other to Off ----
|
||||||
|
enum { FG_DLSS = 10, FG_DLSS_LAST = 12 };
|
||||||
|
constexpr int framegen_row(int framegen, int dlssg) { return dlssg >= 1 && dlssg <= 3 ? FG_DLSS + dlssg - 1 : framegen; }
|
||||||
|
inline void framegen_set(int row, int& framegen, int& dlssg) {
|
||||||
|
if (row >= FG_DLSS) {
|
||||||
|
dlssg = row > FG_DLSS_LAST ? 3 : row - FG_DLSS + 1, framegen = 0;
|
||||||
|
} else {
|
||||||
|
dlssg = 0, framegen = row < 0 ? 0 : row;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/// The row's choices here, in order: Off, Auto, 2x to 8x; AMD FSR 4 where its library makes the frames; NVIDIA DLSS 2x to 4x
|
||||||
|
/// where DLSS frame generation can run. out holds 13 at most; their count
|
||||||
|
inline int framegen_choices(int* out, bool fsr4_offered, bool dlss_offered) {
|
||||||
|
int n = 0;
|
||||||
|
for (int v = 0; v <= 8; v++) out[n++] = v;
|
||||||
|
if (fsr4_offered) out[n++] = 9;
|
||||||
|
for (int v = FG_DLSS; dlss_offered && v <= FG_DLSS_LAST; v++) out[n++] = v;
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- Frame pacing: Off | Even (opts.pace 1) | Steady, any rate | Steady, refresh steps (opts.steady_fps 1 and 2, host/pace_steady.h:
|
||||||
|
// read at the start, a restart). Steady shows frames on an even line too, so it keeps frame_pacing on ----
|
||||||
|
enum { PACE_OFF = 0, PACE_EVEN = 1, PACE_STEADY_ANY = 2, PACE_STEADY_STEPS = 3 };
|
||||||
|
constexpr int pacing_row(int pace, int steady) {
|
||||||
|
return steady == 1 ? (int)PACE_STEADY_ANY : steady == 2 ? (int)PACE_STEADY_STEPS : pace ? (int)PACE_EVEN : (int)PACE_OFF;
|
||||||
|
}
|
||||||
|
inline void pacing_set(int row, int& pace, int& steady) {
|
||||||
|
pace = row != PACE_OFF;
|
||||||
|
steady = row == PACE_STEADY_ANY ? 1 : row == PACE_STEADY_STEPS ? 2 : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- Low latency: Off | On | On + Boost, NVIDIA Reflex on an NVIDIA card (opts.reflex 0 to 2), AMD Anti-Lag 2 on an AMD one
|
||||||
|
// (opts.antilag 0 or 1: no Boost). No row where neither runs ----
|
||||||
|
enum Vendor { LATENCY_NONE = 0, LATENCY_NVIDIA = 1, LATENCY_AMD = 2 };
|
||||||
|
constexpr int latency_hi(Vendor v) { return v == LATENCY_NVIDIA ? 2 : v == LATENCY_AMD ? 1 : 0; }
|
||||||
|
constexpr int latency_row(Vendor v, int reflex, int antilag) {
|
||||||
|
return v == LATENCY_NVIDIA ? (reflex < 0 ? 0 : reflex > 2 ? 2 : reflex) : v == LATENCY_AMD ? (antilag ? 1 : 0) : 0;
|
||||||
|
}
|
||||||
|
inline void latency_set(int row, Vendor v, int& reflex, int& antilag) {
|
||||||
|
if (v == LATENCY_NVIDIA) reflex = row < 0 ? 0 : row > 2 ? 2 : row;
|
||||||
|
else if (v == LATENCY_AMD) antilag = row > 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The next of a row's choices (ascending) from v, dir +1 or -1, wrapping. A value that is not among them (saved where it
|
||||||
|
/// could run) steps to its neighbours by value
|
||||||
|
inline int step_in(const int* choices, int n, int v, int dir) {
|
||||||
|
if (n <= 0) return v;
|
||||||
|
int i = 0;
|
||||||
|
while (i < n && choices[i] < v) i++;
|
||||||
|
if (i < n && choices[i] == v) return choices[(i + (dir > 0 ? 1 : n - 1)) % n];
|
||||||
|
return dir > 0 ? choices[i % n] : choices[(i + n - 1) % n];
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- The options file from before version 8 ----
|
||||||
|
constexpr int kVersion = 8; // (vk_menu.cpp save writes it)
|
||||||
|
constexpr int kHandheldOutput = 7; // (the Output resolution row's 1536x864, the game's handheld profile: opts.output_res 7)
|
||||||
|
/// The keys a file before version 8 can have that the menu no longer writes, each row's job done otherwise: quality_preset (its
|
||||||
|
/// memory limits are the card's own, sized from its video memory), upscaling (the window's filter follows the sizes:
|
||||||
|
/// upscaler_rows.h auto_filter), reflex_frame_limit (off, its old default, until it has been played: RX_VK_REFLEX_LIMIT=1
|
||||||
|
/// for a run, vk_menu.h reflex_limit), handheld_profile (an Output
|
||||||
|
/// resolution), stick_dead_zone (each stick's own row). save drops their lines
|
||||||
|
constexpr const char* kDropped[] = {"quality_preset", "upscaling", "reflex_frame_limit", "handheld_profile", "stick_dead_zone"};
|
||||||
|
constexpr int kDroppedCount = sizeof kDropped / sizeof kDropped[0];
|
||||||
|
/// What a file before version 8 had in those keys that 8 keeps elsewhere (the struct keeps its first name)
|
||||||
|
struct Before7 {
|
||||||
|
int handheld = 0; // (handheld_profile=)
|
||||||
|
int stick_shared = 0; // (stick_dead_zone=: 0 XInput's ~24%, 1 to 5 the shares below)
|
||||||
|
};
|
||||||
|
/// The stick dead zone row's shares, in percent (host/nn_hid.c hos_stick_axis's cuts: a stick's own of the same percent cuts
|
||||||
|
/// at the same place, hos_stick_axis_own); -1 XInput's own, the Left and Right stick dead zones' Default
|
||||||
|
constexpr int kSharedZone[6] = {-1, 5, 10, 15, 20, 30};
|
||||||
|
/// A file from before version 8 as 8 has it, the same in play: the handheld profile as the Output resolution's handheld
|
||||||
|
/// size, the shared stick dead zone in each stick's own row where it had none of its own
|
||||||
|
inline void migrate(int version, const Before7& old, int& output_res, int& lstick_dead_zone, int& rstick_dead_zone) {
|
||||||
|
if (version >= kVersion) return;
|
||||||
|
if (old.handheld) output_res = kHandheldOutput;
|
||||||
|
const int shared = old.stick_shared > 0 && old.stick_shared < 6 ? kSharedZone[old.stick_shared] : -1;
|
||||||
|
if (lstick_dead_zone < 0) lstick_dead_zone = shared;
|
||||||
|
if (rstick_dead_zone < 0) rstick_dead_zone = shared;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::merged_rows
|
||||||
135
gpu/vk/mesh_ahead.h
Normal file
135
gpu/vk/mesh_ahead.h
Normal file
@@ -0,0 +1,135 @@
|
|||||||
|
// (Triforce) The indirect mesh draws of a pass with their records prepared at the first of them, after NX2navi's mesh_ahead
|
||||||
|
// (runtime/vk_exec_draw.c mesh_ring_entry, runtime/vk_exec.c op_linear and vk_exec_mesh_run).
|
||||||
|
//
|
||||||
|
// NVN's indirect mesh draw reads {count, first} and Vulkan's {x, y, z}: each one's records are copied into a ring as {count, 1, 1}
|
||||||
|
// before it (vk_draw.cpp mesh_indirect_convert). A copy cannot be recorded inside a pass, so every indirect mesh draw ended its pass
|
||||||
|
// and the next draw began it again behind a barrier, three barriers and a fill and a copy between (OoT's foliage, grass and leaves:
|
||||||
|
// about 16 a frame at the Market, 45 in NX2navi's village). Now the first indirect mesh draw of a run copies the records of the ones
|
||||||
|
// after it as well, in the same barriers; those find their entry and stay in the pass. A run is the indirect mesh draws after it with
|
||||||
|
// only ops between that store state, draw, or reach the CPU alone (the counter reports, which the replay writes as it records, and the
|
||||||
|
// render conditions, which it reads): nothing between runs outside a pass or writes memory where the GPU could, so no work of the
|
||||||
|
// game's between can write the records after they were copied (the culling compute that writes them is a dispatch, which ends the run;
|
||||||
|
// a draw's shader writes a later draw's records only with the game's barrier between, which ends it too, as on the console, where the
|
||||||
|
// GPU's front end reads them early). An entry is taken only by the draw it was copied for: its op, the same record address, count,
|
||||||
|
// stride and word; the replay clears the table at any other op (and at each command buffer's start), and a draw the replay did not
|
||||||
|
// draw (its render condition) leaves its entry unused. At most kMax draws and kMaxBytes of ring a run, so the ring (4 MB) never comes
|
||||||
|
// round inside one. RX_VK_MESH_AHEAD=0: each draw copies its own, as before.
|
||||||
|
//
|
||||||
|
// Apart from Vulkan and the stream's op numbers so that tests/test_mesh_ahead.py can run the scan and the table over authored streams.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstring>
|
||||||
|
|
||||||
|
namespace mkwvk::mesh_ahead {
|
||||||
|
|
||||||
|
constexpr int kMax = 32; // (the draws after the first whose records it copies, at most)
|
||||||
|
constexpr int kMaxOps = 2048; // (the ops looked at after it, at most: NX2navi's MESH_RUN_OPS)
|
||||||
|
constexpr uint64_t kMaxBytes = 1ull << 20; // (a run's ring bytes, at most: a quarter of vk_draw.cpp's kMeshRing)
|
||||||
|
|
||||||
|
/// RX_VK_MESH_AHEAD: on unless "0"
|
||||||
|
inline bool on(const char* env) {
|
||||||
|
return !(env && env[0] == '0' && !env[1]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What an indirect mesh draw's conversion copies: its op in the stream (where the replay meets it), its records' address, how many,
|
||||||
|
/// their stride and the word that holds the count
|
||||||
|
struct Draw {
|
||||||
|
const uint8_t* op = nullptr;
|
||||||
|
uint64_t indirect = 0;
|
||||||
|
uint32_t n = 0, stride = 0, word = 0;
|
||||||
|
};
|
||||||
|
inline bool same(const Draw& a, const Draw& b) {
|
||||||
|
return a.op == b.op && a.indirect == b.indirect && a.n == b.n && a.stride == b.stride && a.word == b.word;
|
||||||
|
}
|
||||||
|
/// What the conversion copies for an indirect mesh draw of the stream's max and stride (the game's ints): its records, 1 to 4096 of
|
||||||
|
/// them (as many as the draw may draw), 8 bytes apart when the stride is not given
|
||||||
|
inline Draw draw_of(const uint8_t* op, uint64_t indirect, int32_t max, int32_t stride, uint32_t word) {
|
||||||
|
Draw d;
|
||||||
|
d.op = op, d.indirect = indirect, d.word = word;
|
||||||
|
d.n = (uint32_t)(max < 1 ? 1 : max > 4096 ? 4096 : max);
|
||||||
|
d.stride = stride > 0 ? (uint32_t)stride : 8u;
|
||||||
|
return d;
|
||||||
|
}
|
||||||
|
/// A draw's ring bytes (vk_draw.cpp's: {count, 1, 1} a record, 16-aligned)
|
||||||
|
inline uint64_t ring_bytes(uint32_t n) {
|
||||||
|
return ((uint64_t)n * 12 + 15) & ~15ull;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The ops after the one at `at` (records {uint32 op, uint32 size}, sizes 8-aligned), up to `end` (null: none): the indirect mesh draws
|
||||||
|
/// among them while every op between belongs in a run, into out, at most max; at most kMaxOps ops looked at. kind(op, record): 0 the
|
||||||
|
/// run ends there (the stream's end too), 1 it may be in a run, 2 an indirect mesh draw (in the run, and collected)
|
||||||
|
template <class K>
|
||||||
|
int scan(const uint8_t* at, const uint8_t* end, const uint8_t** out, int max, K kind) {
|
||||||
|
uint32_t h[2];
|
||||||
|
std::memcpy(h, at, sizeof h);
|
||||||
|
if (h[1] < 8 || (h[1] & 7)) return 0;
|
||||||
|
const uint8_t* p = at + h[1];
|
||||||
|
int n = 0;
|
||||||
|
for (int ops = 0; ops < kMaxOps && n < max && p != end; ops++) {
|
||||||
|
std::memcpy(h, p, sizeof h);
|
||||||
|
if (h[1] < 8 || (h[1] & 7)) break;
|
||||||
|
const int k = kind(h[0], p);
|
||||||
|
if (!k) break;
|
||||||
|
if (k == 2) out[n++] = p;
|
||||||
|
p += h[1];
|
||||||
|
}
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// How many of the run's draws after the first get entries: in order, while the ring bytes of the first and theirs stay within kMaxBytes
|
||||||
|
/// (and no more than the table holds)
|
||||||
|
inline int fit(const Draw* next, int n, uint32_t first_n) {
|
||||||
|
if (n > kMax) n = kMax;
|
||||||
|
uint64_t bytes = ring_bytes(first_n);
|
||||||
|
int k = 0;
|
||||||
|
while (k < n && bytes + ring_bytes(next[k].n) <= kMaxBytes) bytes += ring_bytes(next[k].n), k++;
|
||||||
|
return k;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the replay knows of the indirect mesh draw it is about to draw (set before draw(), vk_replay.cpp; taken by the conversion):
|
||||||
|
/// its op, and the draws after it in its run (none when they are in the table already)
|
||||||
|
struct Pending {
|
||||||
|
struct Next { // (as the stream has it: draw_of makes the conversion's numbers)
|
||||||
|
const uint8_t* op;
|
||||||
|
uint64_t indirect;
|
||||||
|
int32_t max, stride;
|
||||||
|
};
|
||||||
|
const uint8_t* op = nullptr;
|
||||||
|
Next next[kMax];
|
||||||
|
int n = 0;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The records copied ahead: the run's draws after its first, in order, each with its ring entry
|
||||||
|
struct Table {
|
||||||
|
struct Entry {
|
||||||
|
Draw d;
|
||||||
|
uint64_t at = 0;
|
||||||
|
};
|
||||||
|
Entry e[kMax];
|
||||||
|
int n = 0, i = 0; // (entries, and the first not taken)
|
||||||
|
void clear() { n = i = 0; }
|
||||||
|
bool holds(const uint8_t* op) const {
|
||||||
|
for (int k = i; k < n; k++)
|
||||||
|
if (e[k].d.op == op) return true;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
/// The ring entry copied for d (its op, and the same records): true, and the entries before it are let go (draws the replay did not
|
||||||
|
/// draw); false when there is none or it does not match, and the table is cleared
|
||||||
|
bool take(const Draw& d, uint64_t* at) {
|
||||||
|
for (int k = i; k < n; k++)
|
||||||
|
if (e[k].d.op == d.op) {
|
||||||
|
if (!same(e[k].d, d)) break;
|
||||||
|
*at = e[k].at;
|
||||||
|
i = k + 1;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
clear();
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
inline thread_local Pending t_pending;
|
||||||
|
inline thread_local Table t_table;
|
||||||
|
|
||||||
|
} // namespace mkwvk::mesh_ahead
|
||||||
112
gpu/vk/mod_cheat_rows.h
Normal file
112
gpu/vk/mod_cheat_rows.h
Normal file
@@ -0,0 +1,112 @@
|
|||||||
|
// (Triforce) The System tab's rows for the cheats of the mods' cheat files: what the host names (host/mod_cheats.c), what
|
||||||
|
// the rows show, and what goes back to the host when the player switches one (vk_api.h HOS_OPT_MODCHEAT_*).
|
||||||
|
//
|
||||||
|
// The cheats are the host's: it reads the files at the start and runs the ones that are on, once a game frame. The menu
|
||||||
|
// has a row for each, with the file's own name: the row table is fixed, so it holds kSlots rows for them that are not
|
||||||
|
// shown until the host has named its cheats, and a file's cheats past kSlots have no row (the host's log says so). A
|
||||||
|
// name arrives a byte a call, as the save rows' lines do, with the line the row shows under its help after a line feed.
|
||||||
|
// A row is On | Off like the menu's toggles; a master code and a cheat that cannot run here have a row that cannot be
|
||||||
|
// changed. Which cheats are on is kept by the host (triforce_cheats.txt), not in the options file.
|
||||||
|
//
|
||||||
|
// The host's calls (text, add, done, take) come on the game's thread, the rows are drawn and pressed on the renderer's:
|
||||||
|
// names and lines are written before count says the rows are there and never after; the switches are atomics.
|
||||||
|
//
|
||||||
|
// No ImGui and no Vulkan: tests/test_mod_cheat_rows.py compiles it alone.
|
||||||
|
#pragma once
|
||||||
|
#include <atomic>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstring>
|
||||||
|
|
||||||
|
namespace mod_cheat_rows {
|
||||||
|
constexpr int kSlots = 96;
|
||||||
|
constexpr int kName = 80; // (a cheat's name is at most 63 bytes; " (cheat)" after one that is a row's of the menu's own)
|
||||||
|
constexpr int kLine = 200; // (the line under the row's help)
|
||||||
|
constexpr int kOn = 1, kAlways = 2, kRefused = 4; // (vk_api.h HOS_MODCHEAT_*)
|
||||||
|
|
||||||
|
struct List {
|
||||||
|
char names[kSlots][kName] = {};
|
||||||
|
char lines[kSlots][kLine] = {};
|
||||||
|
unsigned char bits[kSlots] = {};
|
||||||
|
std::atomic<int> on[kSlots] = {};
|
||||||
|
std::atomic<int> count{0}; // (the rows shown: 0 until the host's list is whole)
|
||||||
|
// the host's side
|
||||||
|
char in[kName + kLine] = {};
|
||||||
|
int in_n = 0, added = 0;
|
||||||
|
int told[kSlots] = {}; // (what the host knows of each switch)
|
||||||
|
|
||||||
|
/// HOS_OPT_MODCHEAT_TEXT: the next byte of the row being named
|
||||||
|
void text(int byte) {
|
||||||
|
if (byte > 0 && byte < 256 && in_n + 1 < (int)sizeof in) in[in_n++] = (char)byte;
|
||||||
|
}
|
||||||
|
/// HOS_OPT_MODCHEAT_ADD: the bytes so far are the next row's name and, after a line feed, its line. taken: names a
|
||||||
|
/// row may not have (rows of the menu's own that act by their name): such a cheat is shown with " (cheat)" after it
|
||||||
|
void add(int row_bits, const char* const* taken = nullptr, int n_taken = 0) {
|
||||||
|
in[in_n] = 0;
|
||||||
|
in_n = 0;
|
||||||
|
if (count.load(std::memory_order_acquire) || added >= kSlots) return; // (shown already, or no row left)
|
||||||
|
const int slot = added++;
|
||||||
|
char* feed = strchr(in, '\n');
|
||||||
|
if (feed) *feed = 0;
|
||||||
|
int n = (int)strlen(in);
|
||||||
|
if (n > 63) { // (not in the middle of a letter)
|
||||||
|
n = 63;
|
||||||
|
while (n && ((unsigned char)in[n] & 0xC0) == 0x80) n--;
|
||||||
|
}
|
||||||
|
memcpy(names[slot], in, (size_t)n);
|
||||||
|
names[slot][n] = 0;
|
||||||
|
if (!n) strcpy(names[slot], "(no name)");
|
||||||
|
for (int i = 0; i < n_taken; i++)
|
||||||
|
if (!strcmp(names[slot], taken[i])) strcat(names[slot], " (cheat)");
|
||||||
|
if (feed) {
|
||||||
|
size_t ln = strlen(feed + 1);
|
||||||
|
if (ln > kLine - 1) ln = kLine - 1;
|
||||||
|
memcpy(lines[slot], feed + 1, ln);
|
||||||
|
lines[slot][ln] = 0;
|
||||||
|
for (char* p = lines[slot]; *p; p++)
|
||||||
|
if (*p == '\n') *p = ' ';
|
||||||
|
}
|
||||||
|
bits[slot] = (unsigned char)(row_bits & (kOn | kAlways | kRefused));
|
||||||
|
const int is_on = (row_bits & kAlways) || ((row_bits & kOn) && !(row_bits & kRefused));
|
||||||
|
on[slot].store(is_on, std::memory_order_relaxed);
|
||||||
|
told[slot] = is_on;
|
||||||
|
}
|
||||||
|
/// HOS_OPT_MODCHEAT_COUNT: the list is whole: the rows are shown from now on
|
||||||
|
void done(int n) {
|
||||||
|
in_n = 0;
|
||||||
|
if (count.load(std::memory_order_acquire)) return;
|
||||||
|
count.store(n < 0 ? 0 : n < added ? n : added, std::memory_order_release);
|
||||||
|
}
|
||||||
|
int shown() const { return count.load(std::memory_order_acquire); }
|
||||||
|
/// A row that cannot be changed: a master code, or a cheat that cannot run here
|
||||||
|
bool fixed(int slot) const { return slot < 0 || slot >= kSlots || (bits[slot] & (kAlways | kRefused)); }
|
||||||
|
bool is_on(int slot) const { return slot >= 0 && slot < kSlots && on[slot].load(std::memory_order_relaxed); }
|
||||||
|
/// What a row that cannot be changed shows in its value's place
|
||||||
|
const char* fixed_text(int slot) const { return slot >= 0 && slot < kSlots && (bits[slot] & kRefused) ? "Not supported" : "Always on"; }
|
||||||
|
const char* line(int slot) const { return slot >= 0 && slot < kSlots ? lines[slot] : ""; }
|
||||||
|
/// The slot of a row by its name's address (the rows' names are this list's own), -1 for any other row
|
||||||
|
int slot_of(const char* name) const {
|
||||||
|
const uintptr_t first = (uintptr_t)names[0], at = (uintptr_t)name;
|
||||||
|
return at >= first && at - first < sizeof names ? (int)((at - first) / kName) : -1;
|
||||||
|
}
|
||||||
|
/// A press on a row: Left is On and Right is Off, as they are drawn; confirm flips. true when the switch changed
|
||||||
|
bool press(int slot, bool left, bool right, bool confirm) {
|
||||||
|
if (slot < 0 || slot >= shown() || fixed(slot)) return false;
|
||||||
|
const int was = on[slot].load(std::memory_order_relaxed);
|
||||||
|
const int now = confirm ? !was : left ? 1 : right ? 0 : was;
|
||||||
|
if (now == was) return false;
|
||||||
|
on[slot].store(now, std::memory_order_relaxed);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
/// HOS_OPT_MODCHEAT_ASK: the next switch that is not as the host knows it, (slot + 1) * 2 + its state; 0 none
|
||||||
|
int take() {
|
||||||
|
const int n = shown();
|
||||||
|
for (int slot = 0; slot < n; slot++) {
|
||||||
|
const int now = on[slot].load(std::memory_order_relaxed);
|
||||||
|
if (fixed(slot) || now == told[slot]) continue;
|
||||||
|
told[slot] = now;
|
||||||
|
return (slot + 1) * 2 + (now ? 1 : 0);
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
} // namespace mod_cheat_rows
|
||||||
109
gpu/vk/mods_entry_anim.h
Normal file
109
gpu/vk/mods_entry_anim.h
Normal file
@@ -0,0 +1,109 @@
|
|||||||
|
// (Triforce, 11-19) The Mods entry under the title's list, animated as the game animates its own entries. Each entry of the
|
||||||
|
// title's list (Continue, New Game, Options, ...) is the part PaCommonBtnTitle_00 of the Title archive's layout, and the
|
||||||
|
// part plays its own animations as its button takes and lets go of the cursor: Select, UnSelect, LoopRandom while it has
|
||||||
|
// it, Decide when it is pressed. The Mods entry is drawn by the renderer one pitch under the list's last entry
|
||||||
|
// (vk_menu.cpp draw_title_mods); here are its times and amounts, read from the part's curves (the layout's keys: a frame,
|
||||||
|
// a value and a slope, Hermite between them, 60 frames a second) and from the title layout's (Title_00):
|
||||||
|
// Select every pane of the chosen look from 0 to 1, the unchosen text from 1 to 0, over 1 frame
|
||||||
|
// UnSelect back, over 8 frames
|
||||||
|
// LoopRandom while chosen: 0.6, 0.5 at frame 20, 0.6 at 40 (the filigree's strength: 0.47 to 0.6 in the lab's shot)
|
||||||
|
// Decide 16 frames: the filigree from 1 to 0.85 of its size from frame 8 (its glow to 0.95), the band's sparkle
|
||||||
|
// (W_Bg_01) from 0 at frame 2 to 1 at 16; the game's next screen comes after it
|
||||||
|
// the list Title_00's 15-frame rise from 0 to 1 (its keys' slopes 0.02 and 0.14), as the entry comes with the list;
|
||||||
|
// and gone, as UnSelect, once the list has not been seen for 150 ms (vk_menu.cpp mods_list_fresh's: a game
|
||||||
|
// frame late, at the 30 fps cap or in a hitch, is not the list going); back after that, its In goes on from
|
||||||
|
// where the entry was, not from nothing
|
||||||
|
// No ImGui and no Vulkan: tests/test_mods_entry_anim.py compiles it alone.
|
||||||
|
#pragma once
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace mods_entry {
|
||||||
|
|
||||||
|
/// A key of the layout's curves: its frame, its value, its slope (a value a frame)
|
||||||
|
struct Key {
|
||||||
|
float frame, value, slope;
|
||||||
|
};
|
||||||
|
/// A curve at frame f: the first key's value before it, the last's after; two keys at one frame are a step
|
||||||
|
inline float curve(const Key* k, int n, float f) {
|
||||||
|
if (f <= k[0].frame) return k[0].value;
|
||||||
|
for (int i = 0; i + 1 < n; i++) {
|
||||||
|
const Key &a = k[i], &b = k[i + 1];
|
||||||
|
if (f > b.frame) continue;
|
||||||
|
const float len = b.frame - a.frame;
|
||||||
|
if (len <= 0) return b.value;
|
||||||
|
const float t = (f - a.frame) / len, t2 = t * t, t3 = t2 * t;
|
||||||
|
return (2 * t3 - 3 * t2 + 1) * a.value + (t3 - 2 * t2 + t) * a.slope * len + (-2 * t3 + 3 * t2) * b.value +
|
||||||
|
(t3 - t2) * b.slope * len;
|
||||||
|
}
|
||||||
|
return k[n - 1].value;
|
||||||
|
}
|
||||||
|
|
||||||
|
// (PaCommonBtnTitle_00's curves, as the layout keeps them)
|
||||||
|
constexpr Key kSelect[] = {{0, 0, 0}, {1, 1, 0}};
|
||||||
|
constexpr Key kUnSelect[] = {{0, 1, 0}, {8, 0, 0}};
|
||||||
|
constexpr Key kLoop[] = {{0, 0.6f, 0}, {20, 0.5f, 0}, {40, 0.6f, 0}};
|
||||||
|
constexpr Key kDecideDeco[] = {{8, 1, 0}, {8, 1, -0.04341f}, {16, 0.85f, 0}};
|
||||||
|
constexpr Key kDecideGlow[] = {{8, 1, 0}, {8, 1, -0.01246f}, {16, 0.95f, 0}};
|
||||||
|
constexpr Key kDecideFlash[] = {{0, 0, 0}, {2, 0, 0}, {16, 1, 0}};
|
||||||
|
// (Title_00's: the list coming in)
|
||||||
|
constexpr Key kListIn[] = {{0, 0, 0.02f}, {15, 1, 0.14f}};
|
||||||
|
template <int N>
|
||||||
|
inline float curve(const Key (&k)[N], float f) { return curve(k, N, f); }
|
||||||
|
|
||||||
|
constexpr int64_t kFrameNs = 16666667; // (a frame of the layout's: 1/60 s)
|
||||||
|
constexpr int64_t kDecideNs = 16 * kFrameNs; // (Decide's length: the Mods screen opens after it)
|
||||||
|
constexpr int64_t kLoopNs = 40 * kFrameNs;
|
||||||
|
constexpr int64_t kListGoneNs = 150000000; // (the list not seen for this long: it went. As mods_list_fresh)
|
||||||
|
constexpr int64_t kListShownNs = kListGoneNs + 8 * kFrameNs; // (...and the entry has gone with it after this)
|
||||||
|
inline float frames(int64_t ns) { return (float)((double)ns / (double)kFrameNs); }
|
||||||
|
|
||||||
|
/// How far into the list's In (ns) it shows `amount` (0 to 1): its curve rises all the way, so a search of its 15 frames
|
||||||
|
inline int64_t in_at(float amount) {
|
||||||
|
if (amount <= 0) return 0;
|
||||||
|
float lo = 0, hi = 15;
|
||||||
|
for (int i = 0; i < 24; i++) {
|
||||||
|
const float mid = (lo + hi) * 0.5f;
|
||||||
|
(curve(kListIn, mid) < amount ? lo : hi) = mid;
|
||||||
|
}
|
||||||
|
return (int64_t)((double)hi * (double)kFrameNs);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The entry's state from one drawing to the next: how chosen it looks (focus) and how much of it shows with the list
|
||||||
|
struct Anim {
|
||||||
|
bool sel = false; // (it had the cursor at the last step)
|
||||||
|
float from = 0; // (focus when the cursor last came or went)
|
||||||
|
int64_t since_ns = 0; // (...and when)
|
||||||
|
bool there = false; // (the list was seen at the last step)
|
||||||
|
int64_t list_ns = 0; // (when the list came)
|
||||||
|
float gone_from = 0; // (shown when the list went)
|
||||||
|
float focus = 0, shown = 0;
|
||||||
|
/// One drawing at now: whether the entry has the cursor, how long since the host last told of the list
|
||||||
|
void step(int64_t now, bool selected, int64_t list_age_ns) {
|
||||||
|
if (!since_ns || selected != sel) from = since_ns ? focus : (selected ? 1.0f : 0.0f), sel = selected, since_ns = now;
|
||||||
|
const float f = frames(now - since_ns);
|
||||||
|
focus = sel ? std::max(from, curve(kSelect, f)) : from * curve(kUnSelect, f);
|
||||||
|
const bool seen = list_age_ns < kListGoneNs;
|
||||||
|
if (seen && !there) list_ns = now - in_at(shown); // ((Triforce, 11-19) from where it is: no blink to nothing)
|
||||||
|
if (!seen && there) gone_from = shown;
|
||||||
|
there = seen;
|
||||||
|
shown = seen ? curve(kListIn, frames(now - list_ns)) : gone_from * curve(kUnSelect, frames(list_age_ns - kListGoneNs));
|
||||||
|
shown = std::clamp(shown, 0.0f, 1.0f), focus = std::clamp(focus, 0.0f, 1.0f);
|
||||||
|
}
|
||||||
|
/// LoopRandom's value now (it runs from the cursor's coming)
|
||||||
|
float loop(int64_t now) const { return curve(kLoop, frames((now - since_ns) % kLoopNs)); }
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Decide at t ns after the press: the filigree's size, its glow's, the band's sparkle (0 to 1)
|
||||||
|
struct Decide {
|
||||||
|
float deco = 1, glow = 1, flash = 0;
|
||||||
|
};
|
||||||
|
inline Decide decide_at(int64_t t) {
|
||||||
|
Decide d;
|
||||||
|
if (t < 0) return d;
|
||||||
|
const float f = std::min(frames(t), 16.0f);
|
||||||
|
d.deco = curve(kDecideDeco, f), d.glow = curve(kDecideGlow, f), d.flash = curve(kDecideFlash, f);
|
||||||
|
return d;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mods_entry
|
||||||
541
gpu/vk/occ_answer.h
Normal file
541
gpu/vk/occ_answer.h
Normal file
@@ -0,0 +1,541 @@
|
|||||||
|
// (Triforce, 11-17) Samples-passed reports answered from occlusion queries (RX_VK_OCC_QUERY=1, vk_replay.cpp "occlusion
|
||||||
|
// queries"): the bookkeeping, header only so that tests/test_occ_answer.py checks it without a device.
|
||||||
|
//
|
||||||
|
// The game resets NVN's samples-passed counter, draws an object's proxy and reports the count (its render job f_80084de0,
|
||||||
|
// the report from near lr 0x800876dc); its CPU reads the count back once it has waited for the frame, and does not draw an
|
||||||
|
// object whose count is 0. The replay writes every report as it passes it, in stream order, long before the GPU has run
|
||||||
|
// the draws it counts, and has always written 4096 there (everything visible). It still does: that placeholder is what
|
||||||
|
// anything reading the record early sees, as before. With RX_VK_OCC_QUERY=1 the draws between a reset and its report are
|
||||||
|
// also counted by a Vulkan occlusion query, and once the recording that holds them is done on the GPU the count is
|
||||||
|
// written into the same record, over the placeholder, as NX2navi writes it at completion. The game's report records
|
||||||
|
// rotate (a ring of them: an address stands for another object at its next report), so a count is written only into the
|
||||||
|
// record of the report it counted, and only while no later report has been written there (Writers), and only over the
|
||||||
|
// value the replay left in the word (a compare-and-write: the game's CPU re-arms its records between frames). A wrong 0
|
||||||
|
// hides an object the player should see, so a count goes in only when it is whole; otherwise the record keeps its 4096.
|
||||||
|
// The game also predicates draws on these words (render conditions, NOP_RENDER_COND): as NX2navi decides them
|
||||||
|
// (runtime/vk_exec_xfer.c cond_enabled), a word whose newest report is a samples-passed one with its count still to come is
|
||||||
|
// unknown, and the condition renders; a word whose count was written is read from memory, where a real 0 can skip the draw.
|
||||||
|
//
|
||||||
|
// Counter one counting, from a reset to its report, over the queries its draws began: the replay's (one per NVN
|
||||||
|
// queue's state, under replay_mu) or a parallel replay's segment's (the helper thread's: a segment starts with
|
||||||
|
// none, what came before it is another thread's). Matched to its recording by the recording's serial (each
|
||||||
|
// begin of a slot takes a new one), never by the command buffer's handle, which a slot reuses
|
||||||
|
// Writers per report address, the newest report written there (its place in the stream, the value it left, whether its
|
||||||
|
// count is pending, written or not to come): what a count read back may be written over, what a condition reads
|
||||||
|
// Cond a render condition's answer from its two words' sources (cond_decide)
|
||||||
|
// complete a done recording's counts into their records (through the caller's compare-and-write), or why not
|
||||||
|
// Stats what happened, for the line every 600 presents
|
||||||
|
// Sites the first 64 report addresses with their counts, for the lab to find the one RX_VK_OCC_QUERY_SKIP keeps at 4096
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdio>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <cstring>
|
||||||
|
#include <unordered_map>
|
||||||
|
#include <unordered_set>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace mkwvk::occ {
|
||||||
|
|
||||||
|
constexpr uint64_t kVisible = 4096; // (the placeholder every report gets at replay; it stays unless its whole count comes)
|
||||||
|
constexpr int64_t kLateAge = 8; // (presents: a count read back this long after its report is not written)
|
||||||
|
constexpr int64_t kKeepAge = 16; // (presents: a report address not reported again for longer leaves Writers)
|
||||||
|
|
||||||
|
/// RX_VK_OCC_QUERY_SKIP=<addr>[,<addr>...]: report addresses (hex with 0x, or decimal) that keep the fixed answer.
|
||||||
|
/// Separated by ',', ';' or spaces; anything else ends the list
|
||||||
|
inline std::vector<uint64_t> parse_addrs(const char* e) {
|
||||||
|
std::vector<uint64_t> v;
|
||||||
|
while (e && *e) {
|
||||||
|
while (*e == ',' || *e == ';' || *e == ' ') e++;
|
||||||
|
if (!*e) break;
|
||||||
|
char* end = nullptr;
|
||||||
|
const unsigned long long a = strtoull(e, &end, 0);
|
||||||
|
if (end == e) break;
|
||||||
|
v.push_back((uint64_t)a);
|
||||||
|
e = end;
|
||||||
|
}
|
||||||
|
std::sort(v.begin(), v.end());
|
||||||
|
v.erase(std::unique(v.begin(), v.end()), v.end());
|
||||||
|
return v;
|
||||||
|
}
|
||||||
|
inline bool listed(const std::vector<uint64_t>& sorted, uint64_t addr) {
|
||||||
|
return std::binary_search(sorted.begin(), sorted.end(), addr);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a knob list (RX_VK_LAB_AB's knobs, each maybe several names joined by '+') names this switch anywhere
|
||||||
|
inline bool knob_names(const char* knob, const char* name) {
|
||||||
|
const size_t len = strlen(name);
|
||||||
|
for (const char* p = knob; p && *p;) {
|
||||||
|
const char* plus = strchr(p, '+');
|
||||||
|
const size_t n = plus ? (size_t)(plus - p) : strlen(p);
|
||||||
|
if (n == len && !strncmp(p, name, len)) return true;
|
||||||
|
p = plus ? plus + 1 : nullptr;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The samples of the queries [first, first + n) of a block read with VK_QUERY_RESULT_64_BIT |
|
||||||
|
/// VK_QUERY_RESULT_WITH_AVAILABILITY_BIT ({samples, available} per query). False when one is not available
|
||||||
|
inline bool sum_range(const uint64_t* pairs, uint32_t first, uint32_t n, uint64_t* out) {
|
||||||
|
uint64_t sum = 0;
|
||||||
|
for (uint32_t i = first; i < first + n; i++) {
|
||||||
|
if (!pairs[2 * i + 1]) return false;
|
||||||
|
sum += pairs[2 * i];
|
||||||
|
}
|
||||||
|
*out = sum;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A measured report: its address, its queries in its recording's block, its place in the stream (the same in the dry pass
|
||||||
|
/// that wrote its placeholder and on the thread that recorded its draws) and the present it was recorded at
|
||||||
|
struct Rep {
|
||||||
|
uint64_t addr = 0;
|
||||||
|
uint32_t first = 0, n = 0;
|
||||||
|
uint64_t pos = 0;
|
||||||
|
int64_t present = 0;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// One samples-passed counting. The replay calls reset() at a SAMPLES_PASSED reset, draw_enter() as a draw goes into draw(),
|
||||||
|
/// begin() where a query may begin (the draw's pass open; may_begin() first, cheaply), draw_issued() for each draw recorded,
|
||||||
|
/// report() at the report. A query is begun inside a pass and ended at the pass's end or the report, so the queries of one
|
||||||
|
/// count follow one another in the block of the recording that holds them (first.. the block's next). The recording is
|
||||||
|
/// known by its serial (a new one each time a slot is begun): a draw or report in another recording, or in the same slot
|
||||||
|
/// begun again, loses the count
|
||||||
|
struct Counter {
|
||||||
|
enum State { kUnknown, kCounting, kReported };
|
||||||
|
enum Outcome { kMeasured, kNoReset, kIncomplete, kDouble };
|
||||||
|
State state = kUnknown;
|
||||||
|
int slot = -1; // (the recording slot whose block the queries are in; -1: none, the count is lost)
|
||||||
|
uint64_t serial = 0; // (that slot's recording: occ_slot_begin's serial; 0: none)
|
||||||
|
uint32_t first = 0; // (the count's first query in the block)
|
||||||
|
int open = -1; // (the query begun and not ended, its index in the block)
|
||||||
|
bool lost = false; // (a draw in another recording, or the block full)
|
||||||
|
uint32_t ops = 0, issued = 0; // (draws that went into draw() while counting, and those it recorded)
|
||||||
|
enum Why { kNone, kSkipped, kOtherCmd, kFull };
|
||||||
|
Why why = kNone;
|
||||||
|
|
||||||
|
/// A parallel replay's segment starts (or ends), or the replay's counting is given up: nothing is counting
|
||||||
|
void segment() { *this = Counter{}; }
|
||||||
|
bool counting(bool keep) const { return state == kCounting || (state == kReported && keep); }
|
||||||
|
/// A reset: counting from the block's next query of slot's recording serial_ (slot -1 or serial 0: the command buffer
|
||||||
|
/// is no recording's, the count is lost)
|
||||||
|
void reset(int slot_, uint64_t serial_, uint32_t next) {
|
||||||
|
const bool none = slot_ < 0 || !serial_;
|
||||||
|
state = kCounting, slot = none ? -1 : slot_, serial = none ? 0 : serial_, first = next, open = -1, lost = none;
|
||||||
|
ops = issued = 0;
|
||||||
|
why = none ? kOtherCmd : kNone;
|
||||||
|
}
|
||||||
|
void draw_enter(bool keep) {
|
||||||
|
if (counting(keep)) ops++;
|
||||||
|
}
|
||||||
|
void draw_issued(bool keep) {
|
||||||
|
if (counting(keep)) issued++;
|
||||||
|
}
|
||||||
|
/// Whether a query might begin at a draw (before the recording's serial is looked up)
|
||||||
|
bool may_begin(bool keep) const { return counting(keep) && open < 0 && !lost; }
|
||||||
|
/// A draw in an open pass of the recording serial_: true when a query is to begin (the caller takes the block's next
|
||||||
|
/// index and sets open); a draw in another recording loses the count
|
||||||
|
bool begin(bool keep, uint64_t serial_) {
|
||||||
|
if (!may_begin(keep)) return false;
|
||||||
|
if (serial_ != serial) {
|
||||||
|
lost = true, why = kOtherCmd;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
void full() { lost = true, why = kFull; }
|
||||||
|
/// The report, in the recording serial_ (the open query already ended by the caller): what it measured. *keep_now: the
|
||||||
|
/// game reported twice without a reset, which NVN counts on from the reset (cumulative): from then on the counting goes
|
||||||
|
/// on past a report. A report in another recording than the reset's is incomplete
|
||||||
|
Outcome report(bool keep, uint64_t serial_, uint32_t next, Rep* rep, bool* keep_now) {
|
||||||
|
*keep_now = false;
|
||||||
|
if (state == kUnknown) return kNoReset;
|
||||||
|
if (state == kReported && !keep) { // (the draws since the first report went uncounted: none of this count is whole)
|
||||||
|
*keep_now = true;
|
||||||
|
lost = true, why = kSkipped;
|
||||||
|
return kDouble;
|
||||||
|
}
|
||||||
|
if (!lost && serial_ != serial) lost = true, why = kOtherCmd;
|
||||||
|
const bool complete = !lost && ops == issued;
|
||||||
|
if (!complete && !lost) why = kSkipped;
|
||||||
|
state = kReported;
|
||||||
|
if (!complete) return kIncomplete;
|
||||||
|
rep->first = first, rep->n = next - first;
|
||||||
|
return kMeasured;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
/// A map from 64-bit keys, open addressing with linear probes and backward-shift deletion (no tombstones): no allocation per
|
||||||
|
/// entry. The replay placed one entry for every samples-passed report (~2,600 a frame at the Market) in a std::unordered_map,
|
||||||
|
/// a node allocated and freed each time: 0.76 ms a frame of the Presentation Thread (all-thread sample, 2026-11-20)
|
||||||
|
template <class V>
|
||||||
|
class FlatU64 {
|
||||||
|
public:
|
||||||
|
struct Slot {
|
||||||
|
uint64_t key = 0;
|
||||||
|
V val{};
|
||||||
|
bool used = false;
|
||||||
|
};
|
||||||
|
V* find(uint64_t k) {
|
||||||
|
if (!n_) return nullptr;
|
||||||
|
for (size_t i = idx(k);; i = (i + 1) & mask_) {
|
||||||
|
Slot& s = s_[i];
|
||||||
|
if (!s.used) return nullptr;
|
||||||
|
if (s.key == k) return &s.val;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const V* find(uint64_t k) const { return const_cast<FlatU64*>(this)->find(k); }
|
||||||
|
V& operator[](uint64_t k) {
|
||||||
|
if (V* v = find(k)) return *v;
|
||||||
|
if ((n_ + 1) * 10 > s_.size() * 7) grow();
|
||||||
|
size_t i = idx(k);
|
||||||
|
while (s_[i].used) i = (i + 1) & mask_;
|
||||||
|
s_[i].key = k, s_[i].val = V{}, s_[i].used = true;
|
||||||
|
n_++;
|
||||||
|
return s_[i].val;
|
||||||
|
}
|
||||||
|
size_t count(uint64_t k) const { return find(k) ? 1 : 0; }
|
||||||
|
size_t erase(uint64_t k) {
|
||||||
|
if (!n_) return 0;
|
||||||
|
size_t i = idx(k);
|
||||||
|
for (;; i = (i + 1) & mask_) {
|
||||||
|
if (!s_[i].used) return 0;
|
||||||
|
if (s_[i].key == k) break;
|
||||||
|
}
|
||||||
|
// backward shift: the entries after the hole that would be found from a slot at or before it move into it
|
||||||
|
for (size_t j = (i + 1) & mask_;; j = (j + 1) & mask_) {
|
||||||
|
if (!s_[j].used) break;
|
||||||
|
const size_t home = idx(s_[j].key);
|
||||||
|
const bool moves = i <= j ? (home <= i || home > j) : (home <= i && home > j);
|
||||||
|
if (moves) s_[i] = s_[j], i = j;
|
||||||
|
}
|
||||||
|
s_[i] = Slot{};
|
||||||
|
n_--;
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
template <class F>
|
||||||
|
void erase_if(F pred) { // (the keys to go first, then each erased: a shift during the walk would skip or revisit entries)
|
||||||
|
std::vector<uint64_t> gone;
|
||||||
|
for (const Slot& s : s_)
|
||||||
|
if (s.used && pred(s.key, s.val)) gone.push_back(s.key);
|
||||||
|
for (uint64_t k : gone) erase(k);
|
||||||
|
}
|
||||||
|
bool empty() const { return n_ == 0; }
|
||||||
|
size_t size() const { return n_; }
|
||||||
|
|
||||||
|
private:
|
||||||
|
size_t idx(uint64_t k) const { return (size_t)((k * 0x9E3779B97F4A7C15ull) >> shift_) & mask_; }
|
||||||
|
void grow() {
|
||||||
|
std::vector<Slot> old;
|
||||||
|
old.swap(s_);
|
||||||
|
const size_t cap = old.empty() ? 1024 : old.size() * 2;
|
||||||
|
s_.assign(cap, Slot{});
|
||||||
|
mask_ = cap - 1;
|
||||||
|
shift_ = 64;
|
||||||
|
for (size_t c = cap; c > 1; c >>= 1) shift_--;
|
||||||
|
n_ = 0;
|
||||||
|
for (const Slot& s : old)
|
||||||
|
if (s.used) (*this)[s.key] = s.val;
|
||||||
|
}
|
||||||
|
std::vector<Slot> s_;
|
||||||
|
size_t n_ = 0, mask_ = 0;
|
||||||
|
int shift_ = 64;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Per report address, the newest samples-passed report the replay wrote there (in stream order: placed) and where its
|
||||||
|
/// count stands: NX2navi's Pending table (runtime/vk_exec_xfer.c: seq, written, unknown, "the last report to addr
|
||||||
|
/// decides"). A count read back goes into its record only while its report is the newest there (the game's records rotate,
|
||||||
|
/// and a later report at the same address is another object's, whose placeholder stays), and a render condition reads a
|
||||||
|
/// word whose newest report's count is still to come as unknown (Src kPendingSrc: renders, NX2navi's cond_enabled "rx ==
|
||||||
|
/// 2"). A record rewritten in the stream after its report (a small buffer clear the replay applies in stream order, or a
|
||||||
|
/// report of another counter type) leaves the table: its count is not written there, and a condition reads memory, as before
|
||||||
|
class Writers {
|
||||||
|
public:
|
||||||
|
enum Verdict { kWrite, kSuperseded, kUnknown, kFixed };
|
||||||
|
enum State : uint8_t {
|
||||||
|
kPending, // (its count is to be written and has not been yet)
|
||||||
|
kWritten, // (its count is in the record: completion wrote it)
|
||||||
|
kFinal // (no count comes: skip list, knob off, a clear flag, not counted whole, lost, late, the CPU wrote it)
|
||||||
|
};
|
||||||
|
enum Src { kNone, kPendingSrc, kWrittenSrc, kFinalSrc }; // (what a render condition reading the word finds)
|
||||||
|
struct Placed {
|
||||||
|
uint64_t pos = 0;
|
||||||
|
int64_t present = 0;
|
||||||
|
uint64_t value = 0; // (what the replay, or a count written since, put in the word: a count is written only over it)
|
||||||
|
State state = kFinal;
|
||||||
|
};
|
||||||
|
/// A report's placeholder `value` written at the replay, at pos in the stream (the newest there from now on). write:
|
||||||
|
/// its count is to be written once read back (not on RX_VK_OCC_QUERY_SKIP, the lab knob on, not a clear flag)
|
||||||
|
void placed(uint64_t addr, uint64_t pos, int64_t present, bool write, uint64_t value) {
|
||||||
|
m_[addr] = Placed{pos, present, value, write ? kPending : kFinal};
|
||||||
|
page_[page(addr)] = 1, page_[page(addr + 7)] = 1;
|
||||||
|
}
|
||||||
|
bool is_report(uint64_t addr) const { return m_.count(addr) != 0; }
|
||||||
|
/// What a count read back for the report at (addr, pos) may do. (Two reports at one address with no draw between share
|
||||||
|
/// a place; in stream order the last one's count is written last, over the first one's)
|
||||||
|
Verdict verdict(uint64_t addr, uint64_t pos) const {
|
||||||
|
const auto* it = m_.find(addr);
|
||||||
|
if (!it) return kUnknown; // (not reported for kKeepAge presents, or rewritten in the stream)
|
||||||
|
if (it->pos > pos) return kSuperseded; // (reported again since: another object's record now)
|
||||||
|
if (it->pos != pos) return kUnknown;
|
||||||
|
return it->state == kFinal ? kFixed : kWrite;
|
||||||
|
}
|
||||||
|
/// The value the report at addr left in its word (the placeholder, or the count written since)
|
||||||
|
uint64_t value(uint64_t addr) const {
|
||||||
|
const auto* it = m_.find(addr);
|
||||||
|
return !it ? 0 : it->value;
|
||||||
|
}
|
||||||
|
/// The count of the report at (addr, pos) is in its record now
|
||||||
|
void wrote(uint64_t addr, uint64_t pos, uint64_t count) {
|
||||||
|
auto* it = m_.find(addr);
|
||||||
|
if (it && it->pos == pos) it->state = kWritten, it->value = count;
|
||||||
|
}
|
||||||
|
/// No count comes for the report at (addr, pos) (not counted whole, lost, late, never sent, the CPU wrote the record):
|
||||||
|
/// a condition reads its word from memory from now on, as before
|
||||||
|
void settle(uint64_t addr, uint64_t pos) {
|
||||||
|
auto* it = m_.find(addr);
|
||||||
|
if (it && it->pos == pos && it->state == kPending) it->state = kFinal;
|
||||||
|
}
|
||||||
|
/// A report of another counter type at addr (a timestamp: base behaviour) is the newest there now
|
||||||
|
bool erase(uint64_t addr) { return m_.erase(addr) != 0; }
|
||||||
|
/// A small buffer clear of [lo, hi) applied in stream order (NOP_CLEAR_BUF): the records whose count word it reaches leave
|
||||||
|
/// the table (NX2navi's clear_small and clr_mask, by whole records: memory holds the clear's value from now on, and the
|
||||||
|
/// count of a report before it is not written over it). Report addresses are 4-byte aligned at least. Returns how many
|
||||||
|
int cleared(uint64_t lo, uint64_t hi) {
|
||||||
|
if (hi <= lo || m_.empty() || (!page_[page(lo)] && !page_[page(hi - 1)])) return 0;
|
||||||
|
int n = 0;
|
||||||
|
for (uint64_t k = ((lo >= 7 ? lo - 7 : 0) + 3) & ~3ull; k < hi; k += 4) n += (int)m_.erase(k);
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
/// What a render condition finds in the word at addr: no samples-passed report (memory, as before), its count pending
|
||||||
|
/// (unknown: renders), written (memory holds it), or none to come (memory). A count pending longer than kLateAge
|
||||||
|
/// presents is not written any more (complete: late): memory
|
||||||
|
Src word(uint64_t addr, int64_t now) const {
|
||||||
|
const auto* it = m_.find(addr);
|
||||||
|
if (!it) return kNone;
|
||||||
|
switch (it->state) {
|
||||||
|
case kPending: return now - it->present > kLateAge ? kFinalSrc : kPendingSrc;
|
||||||
|
case kWritten: return kWrittenSrc;
|
||||||
|
case kFinal: break;
|
||||||
|
}
|
||||||
|
return kFinalSrc;
|
||||||
|
}
|
||||||
|
/// The addresses not reported for more than `age` presents go (a count still to come for them is not written)
|
||||||
|
void prune(int64_t now, int64_t age) {
|
||||||
|
m_.erase_if([now, age](uint64_t, const Placed& p) { return now - p.present > age; });
|
||||||
|
}
|
||||||
|
size_t size() const { return m_.size(); }
|
||||||
|
|
||||||
|
private:
|
||||||
|
// (the pages a report went into, by a hash of the page, as NX2navi's rep_pages: a clear elsewhere looks nothing up)
|
||||||
|
static uint32_t page(uint64_t a) { return (uint32_t)(((a >> 12) * 0x9E3779B97F4A7C15ull) >> 48); }
|
||||||
|
FlatU64<Placed> m_; // (by report address: FlatU64, no allocation per report)
|
||||||
|
std::vector<uint8_t> page_ = std::vector<uint8_t>(1u << 16);
|
||||||
|
};
|
||||||
|
|
||||||
|
/// A render condition (NOP_RENDER_COND: mode 0 renders when the counter words at addr and addr + 16 are equal, mode 1 when
|
||||||
|
/// they differ) over the words' sources (Writers::word), after NX2navi's cond_enabled: a samples-passed count not done yet
|
||||||
|
/// renders; a written one, as any other word, is read from memory (base_enabled: the replay's answer from memory). active:
|
||||||
|
/// RX_VK_OCC_QUERY=1 or the lab knob's phase (else the answer is the base one, whatever the table says); cond_on: false
|
||||||
|
/// with RX_VK_OCC_COND=0, every condition on a samples-passed word renders
|
||||||
|
struct Cond {
|
||||||
|
enum Kind { kNotSamples, kBase, kUnknown, kForced, kWrittenOn, kWrittenOff, kMemory };
|
||||||
|
bool enabled;
|
||||||
|
Kind kind;
|
||||||
|
};
|
||||||
|
inline Cond cond_decide(bool base_enabled, Writers::Src sx, Writers::Src sy, bool active, bool cond_on) {
|
||||||
|
if (sx == Writers::kNone && sy == Writers::kNone) return {base_enabled, Cond::kNotSamples};
|
||||||
|
if (!active) return {base_enabled, Cond::kBase};
|
||||||
|
if (!cond_on) return {true, Cond::kForced};
|
||||||
|
if (sx == Writers::kPendingSrc || sy == Writers::kPendingSrc) return {true, Cond::kUnknown};
|
||||||
|
if (sx == Writers::kWrittenSrc || sy == Writers::kWrittenSrc)
|
||||||
|
return {base_enabled, base_enabled ? Cond::kWrittenOn : Cond::kWrittenOff};
|
||||||
|
return {base_enabled, Cond::kMemory};
|
||||||
|
}
|
||||||
|
inline const char* src_name(Writers::Src s) {
|
||||||
|
return s == Writers::kPendingSrc ? "pending" : s == Writers::kWrittenSrc ? "written" : s == Writers::kFinalSrc ? "memory, no count to come" : "memory";
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What happened since the last line (under the caller's lock)
|
||||||
|
struct Stats {
|
||||||
|
enum At { kAtWait, kAtBegin, kAtPresent, kAtReuse };
|
||||||
|
// at the replay
|
||||||
|
int64_t reports = 0; // samples-passed reports written (the placeholder, 4096 as before)
|
||||||
|
int64_t in_segments = 0; // of them by the dry pass of a parallel replay
|
||||||
|
int64_t skipped = 0; // on RX_VK_OCC_QUERY_SKIP: no count to be written
|
||||||
|
int64_t fixed = 0; // the lab knob occq off in this phase: no count to be written
|
||||||
|
int64_t flagged = 0; // the count word a clear flag (NVN_COUNTER_WRITTEN): left alone
|
||||||
|
int64_t rewritten = 0; // records rewritten in the stream after their report (a small clear, another counter type)
|
||||||
|
int64_t measured = 0, no_reset = 0, incomplete = 0, doubles = 0; // counted whole by queries, and those not
|
||||||
|
int64_t inc_skipped = 0, inc_other_cmd = 0, inc_full = 0; // why not whole
|
||||||
|
// render conditions that read a samples-passed report's word (Cond::Kind), and the work they skipped
|
||||||
|
int64_t conds = 0, cond_base = 0, cond_unknown = 0, cond_forced = 0, cond_written_on = 0, cond_written_off = 0, cond_memory = 0;
|
||||||
|
int64_t draws_skipped = 0, clears_skipped = 0;
|
||||||
|
// at completion
|
||||||
|
int64_t written = 0, zeros = 0; // counts written into their records (of them 0: the object hidden)
|
||||||
|
int64_t lost = 0; // a query not available once its recording was done: 4096 stays
|
||||||
|
int64_t dropped = 0; // their recording never went to the GPU
|
||||||
|
int64_t late = 0; // read back more than kLateAge presents after the report: not written
|
||||||
|
int64_t superseded = 0; // the record reported again first (the ring came round): not written
|
||||||
|
int64_t unknown = 0; // the record no longer known (pruned, or rewritten in the stream): not written
|
||||||
|
int64_t kept_fixed = 0; // its report placed with no count to be written (skip list, knob off)
|
||||||
|
int64_t rearmed = 0; // the word no longer held the replay's value (the CPU wrote the record since): not written
|
||||||
|
int64_t lat_sum = 0, lat_ge1 = 0, lat_ge2 = 0; // presents from the report to the write, over written
|
||||||
|
int64_t at[4] = {}; // recordings read: at a game's fence wait, a recording's begin, a present, the slot's reuse
|
||||||
|
int64_t addrs = 0; // distinct report addresses since the last line
|
||||||
|
void cond(Cond::Kind k) {
|
||||||
|
if (k == Cond::kNotSamples) return;
|
||||||
|
conds++;
|
||||||
|
(k == Cond::kBase ? cond_base : k == Cond::kUnknown ? cond_unknown : k == Cond::kForced ? cond_forced
|
||||||
|
: k == Cond::kWrittenOn ? cond_written_on : k == Cond::kWrittenOff ? cond_written_off : cond_memory)++;
|
||||||
|
}
|
||||||
|
/// The line, into out
|
||||||
|
int line(char* out, size_t size, int64_t presents, size_t tracked) const {
|
||||||
|
return snprintf(out, size,
|
||||||
|
"vk: occlusion queries (RX_VK_OCC_QUERY), %lld presents: %lld samples-passed reports, 4096 written at "
|
||||||
|
"the replay as before (%lld by a parallel replay's dry pass; %lld skipped (RX_VK_OCC_QUERY_SKIP), %lld "
|
||||||
|
"lab knob occq off, %lld a clear flag; %lld records rewritten in the stream after their report); %lld "
|
||||||
|
"counted whole, %lld without their reset on the recording thread, %lld incomplete (%lld draws left out, "
|
||||||
|
"%lld over two recordings, %lld block full), %lld reported twice without a reset. At completion: %lld "
|
||||||
|
"counts written (%lld of them 0), %lld lost (not available), %lld never sent, %lld late, %lld superseded "
|
||||||
|
"(the record reported again first), %lld no longer known, %lld kept 4096 (skip list or knob off), %lld "
|
||||||
|
"not written: the record no longer held the replay's value; mean latency %.2f presents, %lld written "
|
||||||
|
"after the next present, %lld after two; recordings read at: %lld a game's fence wait, %lld a "
|
||||||
|
"recording's begin, %lld a present, %lld the slot's reuse. Render conditions on samples-passed words: "
|
||||||
|
"%lld (%lld as before: knob occq off; %lld rendered: a count pending (unknown); %lld rendered: "
|
||||||
|
"RX_VK_OCC_COND=0; %lld rendered from a written count; %lld skipped from a written count; %lld from "
|
||||||
|
"memory, no count to come); skipped by them: %lld draws and dispatches, %lld clears; %lld addresses "
|
||||||
|
"this period, %zu tracked",
|
||||||
|
(long long)presents, (long long)reports, (long long)in_segments, (long long)skipped, (long long)fixed,
|
||||||
|
(long long)flagged, (long long)rewritten, (long long)measured, (long long)no_reset,
|
||||||
|
(long long)incomplete, (long long)inc_skipped, (long long)inc_other_cmd, (long long)inc_full,
|
||||||
|
(long long)doubles, (long long)written, (long long)zeros, (long long)lost, (long long)dropped,
|
||||||
|
(long long)late, (long long)superseded, (long long)unknown, (long long)kept_fixed, (long long)rearmed,
|
||||||
|
written ? (double)lat_sum / (double)written : 0.0, (long long)lat_ge1, (long long)lat_ge2,
|
||||||
|
(long long)at[kAtWait], (long long)at[kAtBegin], (long long)at[kAtPresent], (long long)at[kAtReuse],
|
||||||
|
(long long)conds, (long long)cond_base, (long long)cond_unknown, (long long)cond_forced,
|
||||||
|
(long long)cond_written_on, (long long)cond_written_off, (long long)cond_memory,
|
||||||
|
(long long)draws_skipped, (long long)clears_skipped, (long long)addrs, tracked);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
/// A done recording's reports (in stream order) into their records. pairs: its block read with VK_QUERY_RESULT_64_BIT |
|
||||||
|
/// VK_QUERY_RESULT_WITH_AVAILABILITY_BIT over the `used` queries it took (readable false: the read failed). A count is
|
||||||
|
/// written only when every query of it is available, it is read back at most kLateAge presents after its report, and its
|
||||||
|
/// report is still the newest at its address with a count to be written there (Writers::verdict); then write(addr, expect,
|
||||||
|
/// count) puts it in the record only while the word still holds expect, what the replay (or an earlier count of the same
|
||||||
|
/// place) left there, and says whether it did: the game's CPU re-arms its records between frames, and a count read back
|
||||||
|
/// late must not land over that (NX2navi treats a report as pending until its value is written; here the write itself is
|
||||||
|
/// a compare-and-write). Anything else leaves the record as it is, and the report's count is settled (no count comes: a
|
||||||
|
/// condition reads memory). measured(addr, count): each whole count read back (the sites)
|
||||||
|
template <class Write, class Measured>
|
||||||
|
void complete(const std::vector<Rep>& reps, const uint64_t* pairs, uint32_t used, bool readable, Writers& w, int64_t now,
|
||||||
|
Stats& st, Write&& write, Measured&& measured) {
|
||||||
|
for (const Rep& p : reps) {
|
||||||
|
uint64_t sum = 0;
|
||||||
|
const bool inside = p.n <= used && p.first <= used - p.n; // (its queries in the block the recording took)
|
||||||
|
if (!readable || !inside || !sum_range(pairs, p.first, p.n, &sum)) {
|
||||||
|
st.lost++;
|
||||||
|
w.settle(p.addr, p.pos);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
measured(p.addr, sum);
|
||||||
|
if (now - p.present > kLateAge) {
|
||||||
|
st.late++;
|
||||||
|
w.settle(p.addr, p.pos);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
switch (w.verdict(p.addr, p.pos)) {
|
||||||
|
case Writers::kSuperseded: st.superseded++; continue;
|
||||||
|
case Writers::kUnknown: st.unknown++; continue;
|
||||||
|
case Writers::kFixed: st.kept_fixed++; continue;
|
||||||
|
case Writers::kWrite: break;
|
||||||
|
}
|
||||||
|
if (!write(p.addr, w.value(p.addr), sum)) {
|
||||||
|
st.rearmed++;
|
||||||
|
w.settle(p.addr, p.pos);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
w.wrote(p.addr, p.pos, sum);
|
||||||
|
const int64_t lat = std::max<int64_t>(0, now - p.present);
|
||||||
|
st.written++, st.zeros += sum ? 0 : 1, st.lat_sum += lat, st.lat_ge1 += lat >= 1, st.lat_ge2 += lat >= 2;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/// A recording's reports that never went to the GPU: no count comes for them
|
||||||
|
inline void dropped(const std::vector<Rep>& reps, Writers& w, Stats& st) {
|
||||||
|
for (const Rep& p : reps) w.settle(p.addr, p.pos);
|
||||||
|
st.dropped += (int64_t)reps.size();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The first 64 report addresses the replay saw, with how often each was reported, what its queries counted and how many
|
||||||
|
/// counts went into it
|
||||||
|
struct Sites {
|
||||||
|
struct Site {
|
||||||
|
uint64_t addr = 0;
|
||||||
|
int64_t reports = 0, measured = 0, written = 0;
|
||||||
|
uint64_t last = 0, lo = UINT64_MAX, hi = 0;
|
||||||
|
};
|
||||||
|
static constexpr int kMost = 64;
|
||||||
|
Site s[kMost];
|
||||||
|
int n = 0;
|
||||||
|
Site* at(uint64_t addr, bool add) {
|
||||||
|
for (int i = 0; i < n; i++)
|
||||||
|
if (s[i].addr == addr) return &s[i];
|
||||||
|
if (!add || n == kMost) return nullptr;
|
||||||
|
s[n].addr = addr;
|
||||||
|
return &s[n++];
|
||||||
|
}
|
||||||
|
void reported(uint64_t addr) {
|
||||||
|
if (Site* x = at(addr, true)) x->reports++;
|
||||||
|
}
|
||||||
|
void measured(uint64_t addr, uint64_t count) {
|
||||||
|
if (Site* x = at(addr, false)) x->measured++, x->last = count, x->lo = std::min(x->lo, count), x->hi = std::max(x->hi, count);
|
||||||
|
}
|
||||||
|
void wrote(uint64_t addr) {
|
||||||
|
if (Site* x = at(addr, false)) x->written++;
|
||||||
|
}
|
||||||
|
/// Lines of eight sites each: "addr reports/measured last [lo..hi] written"
|
||||||
|
template <class Log>
|
||||||
|
void log(Log&& out) const {
|
||||||
|
for (int b = 0; b < n; b += 8) {
|
||||||
|
char line[1024];
|
||||||
|
int k = snprintf(line, sizeof line, "vk: occlusion query sites %d-%d of %d (address: reports/measured, last count "
|
||||||
|
"[lowest..highest], counts written):", b + 1, std::min(n, b + 8), n);
|
||||||
|
for (int i = b; i < std::min(n, b + 8) && k < (int)sizeof line - 96; i++) {
|
||||||
|
const Site& x = s[i];
|
||||||
|
k += snprintf(line + k, sizeof line - k, " %#llx: %lld/%lld %llu [%llu..%llu] %lld;", (unsigned long long)x.addr,
|
||||||
|
(long long)x.reports, (long long)x.measured, (unsigned long long)x.last,
|
||||||
|
(unsigned long long)(x.measured ? x.lo : 0), (unsigned long long)x.hi, (long long)x.written);
|
||||||
|
}
|
||||||
|
out(line);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The distinct addresses reported since the last line (bounded)
|
||||||
|
/// (a fixed open-addressing table of 131072 keys, 0 meaning empty (a report address is never 0), cleared at each line: it
|
||||||
|
/// was a std::unordered_set, a node allocated for each report's address only to count them for the log, 2,600 a frame)
|
||||||
|
struct Distinct {
|
||||||
|
static constexpr size_t kSlots = 1u << 17, kMost = 65536;
|
||||||
|
std::vector<uint64_t> slot = std::vector<uint64_t>(kSlots, 0);
|
||||||
|
int64_t n = 0;
|
||||||
|
void note(uint64_t addr) {
|
||||||
|
if (!addr || n >= (int64_t)kMost) return;
|
||||||
|
for (size_t i = (size_t)((addr * 0x9E3779B97F4A7C15ull) >> 47) & (kSlots - 1);; i = (i + 1) & (kSlots - 1)) {
|
||||||
|
if (slot[i] == addr) return;
|
||||||
|
if (!slot[i]) {
|
||||||
|
slot[i] = addr, n++;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
int64_t take() {
|
||||||
|
const int64_t r = n;
|
||||||
|
if (n) std::fill(slot.begin(), slot.end(), 0), n = 0;
|
||||||
|
return r;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
} // namespace mkwvk::occ
|
||||||
138
gpu/vk/oot_debug_link.h
Normal file
138
gpu/vk/oot_debug_link.h
Normal file
@@ -0,0 +1,138 @@
|
|||||||
|
/* (Triforce, OoT debug tools) The Debug tab's Game flags and Cutscenes pages (gpu/vk/vk_menu.cpp, gpu/vk/debug_names.h) and
|
||||||
|
the game's side (host/hos_main.c): the lines that go between them, a byte a call (HOS_OPT_DEBUG_ASK one way,
|
||||||
|
HOS_OPT_DEBUG_TEXT and HOS_OPT_DEBUG_SAID the other: gpu/vk/vk_api.h). C and C++; tests/test_debug_link.py runs it.
|
||||||
|
|
||||||
|
The game knows a flag (a Bool of its GameData) by the MurmurHash3 of its name alone, and an event by the name of its
|
||||||
|
file. Neither list is in this repository: the pages read the names from the player's own game files, in memory.
|
||||||
|
|
||||||
|
The pages ask:
|
||||||
|
W <hash> <hash> ... the flags the page shows now, at most OOT_WATCH_MAX, eight hex digits each: their states are
|
||||||
|
sent twice a second from then on (a W with none: no more)
|
||||||
|
F <hash> <0|1> the flag cleared or set, through the game's own setter
|
||||||
|
E <name> the event of that name started, as one of the game's actors starts it
|
||||||
|
The game's side says:
|
||||||
|
OOT_SAID_FLAGS "<sum> <states>": the watch list's oot_watch_sum, then a character a flag in the list's order:
|
||||||
|
1 set, 0 not, - the save data has no such flag, ? no save is loaded
|
||||||
|
OOT_SAID_STATUS what became of an F or an E, in words, for the page's foot */
|
||||||
|
#ifndef OOT_DEBUG_LINK_H
|
||||||
|
#define OOT_DEBUG_LINK_H
|
||||||
|
#include <stddef.h>
|
||||||
|
#include <stdint.h>
|
||||||
|
#include <stdio.h>
|
||||||
|
#include <string.h>
|
||||||
|
|
||||||
|
enum { OOT_WATCH_MAX = 64, OOT_NAME_MAX = 95, OOT_LINE_MAX = 4 + 9 * OOT_WATCH_MAX };
|
||||||
|
enum { OOT_SAID_FLAGS = 1, OOT_SAID_STATUS = 2 };
|
||||||
|
|
||||||
|
/* MurmurHash3 (32-bit, seed 0) of a name's bytes, as the game's GameData getters and setters take it */
|
||||||
|
static inline uint32_t oot_name_hash(const char *text, size_t n) {
|
||||||
|
uint32_t h = 0;
|
||||||
|
size_t i = 0;
|
||||||
|
for (; i + 4 <= n; i += 4) {
|
||||||
|
uint32_t k = (uint32_t)(uint8_t)text[i] | (uint32_t)(uint8_t)text[i + 1] << 8 | (uint32_t)(uint8_t)text[i + 2] << 16 |
|
||||||
|
(uint32_t)(uint8_t)text[i + 3] << 24;
|
||||||
|
k *= 0xcc9e2d51u, k = k << 15 | k >> 17, k *= 0x1b873593u;
|
||||||
|
h ^= k, h = h << 13 | h >> 19, h = h * 5 + 0xe6546b64u;
|
||||||
|
}
|
||||||
|
uint32_t k = 0;
|
||||||
|
if ((n & 3) == 3) k ^= (uint32_t)(uint8_t)text[i + 2] << 16;
|
||||||
|
if ((n & 3) >= 2) k ^= (uint32_t)(uint8_t)text[i + 1] << 8;
|
||||||
|
if (n & 3) k ^= (uint8_t)text[i], k *= 0xcc9e2d51u, k = k << 15 | k >> 17, k *= 0x1b873593u, h ^= k;
|
||||||
|
h ^= (uint32_t)n;
|
||||||
|
h ^= h >> 16, h *= 0x85ebca6bu, h ^= h >> 13, h *= 0xc2b2ae35u, h ^= h >> 16;
|
||||||
|
return h;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* a character of a flag's or an event's name: the game's names are letters, digits, _ . and - */
|
||||||
|
static inline int oot_name_char(int ch) {
|
||||||
|
return (ch >= 'a' && ch <= 'z') || (ch >= 'A' && ch <= 'Z') || (ch >= '0' && ch <= '9') || ch == '_' || ch == '.' || ch == '-';
|
||||||
|
}
|
||||||
|
/* a name of 1 to OOT_NAME_MAX such characters */
|
||||||
|
static inline int oot_name_ok(const char *name) {
|
||||||
|
size_t n = 0;
|
||||||
|
while (name[n] && n <= OOT_NAME_MAX && oot_name_char((unsigned char)name[n])) n++;
|
||||||
|
return n > 0 && n <= OOT_NAME_MAX && !name[n];
|
||||||
|
}
|
||||||
|
|
||||||
|
/* what pairs a line of states with the list it answers: the flags' hashes in their order */
|
||||||
|
static inline uint32_t oot_watch_sum(const uint32_t *hashes, int n) {
|
||||||
|
uint32_t sum = 0x9e3779b9u + (uint32_t)n;
|
||||||
|
for (int i = 0; i < n; i++) sum = (sum << 5 | sum >> 27) ^ hashes[i];
|
||||||
|
return sum;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* eight hex digits at *at, which moves past them: 1 when there were eight */
|
||||||
|
static inline int oot_hex8(const char **at, uint32_t *value) {
|
||||||
|
uint32_t v = 0;
|
||||||
|
for (int i = 0; i < 8; i++) {
|
||||||
|
const int ch = (unsigned char)(*at)[i];
|
||||||
|
const int digit = ch >= '0' && ch <= '9' ? ch - '0' : ch >= 'a' && ch <= 'f' ? ch - 'a' + 10 : -1;
|
||||||
|
if (digit < 0) return 0;
|
||||||
|
v = v << 4 | (uint32_t)digit;
|
||||||
|
}
|
||||||
|
*at += 8, *value = v;
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
typedef struct OotAsk {
|
||||||
|
int kind; /* 'W', 'F' or 'E' */
|
||||||
|
uint32_t hash; /* F: the flag */
|
||||||
|
int value; /* F: 0 or 1 */
|
||||||
|
int count; /* W: how many */
|
||||||
|
uint32_t watch[OOT_WATCH_MAX]; /* W: the flags */
|
||||||
|
char name[OOT_NAME_MAX + 1]; /* E: the event */
|
||||||
|
} OotAsk;
|
||||||
|
|
||||||
|
/* A line of the pages taken apart: 1 when it is one of the three, whole and well formed (anything else is dropped) */
|
||||||
|
static inline int oot_ask_parse(const char *line, OotAsk *ask) {
|
||||||
|
memset(ask, 0, sizeof *ask);
|
||||||
|
const char *at = line + 1;
|
||||||
|
if (line[0] == 'W') {
|
||||||
|
while (*at) {
|
||||||
|
if (*at++ != ' ' || ask->count == OOT_WATCH_MAX || !oot_hex8(&at, &ask->watch[ask->count])) return 0;
|
||||||
|
ask->count++;
|
||||||
|
}
|
||||||
|
ask->kind = 'W';
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if (line[0] == 'F') {
|
||||||
|
if (*at++ != ' ' || !oot_hex8(&at, &ask->hash) || !ask->hash || *at++ != ' ' || (*at != '0' && *at != '1') || at[1]) return 0;
|
||||||
|
ask->value = *at == '1';
|
||||||
|
ask->kind = 'F';
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
if (line[0] == 'E') {
|
||||||
|
if (*at++ != ' ' || !oot_name_ok(at)) return 0;
|
||||||
|
memcpy(ask->name, at, strlen(at) + 1);
|
||||||
|
ask->kind = 'E';
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* The pages' lines written (out of OOT_LINE_MAX at least): their length, 0 when the line cannot be made */
|
||||||
|
static inline int oot_ask_watch(char *out, size_t cap, const uint32_t *hashes, int n) {
|
||||||
|
if (n < 0 || n > OOT_WATCH_MAX || cap < 2 + 9 * (size_t)n) return 0;
|
||||||
|
int used = snprintf(out, cap, "W");
|
||||||
|
for (int i = 0; i < n; i++) used += snprintf(out + used, cap - (size_t)used, " %08x", (unsigned)hashes[i]);
|
||||||
|
return used;
|
||||||
|
}
|
||||||
|
static inline int oot_ask_flag(char *out, size_t cap, uint32_t hash, int value) {
|
||||||
|
return hash && cap >= 13 ? snprintf(out, cap, "F %08x %d", (unsigned)hash, value ? 1 : 0) : 0;
|
||||||
|
}
|
||||||
|
static inline int oot_ask_event(char *out, size_t cap, const char *name) {
|
||||||
|
return oot_name_ok(name) && cap >= strlen(name) + 3 ? snprintf(out, cap, "E %s", name) : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* A line of states read: 1 when it answers the list of that sum and has n states of the four characters; they are in
|
||||||
|
states then */
|
||||||
|
static inline int oot_states_parse(const char *line, uint32_t sum, int n, char *states) {
|
||||||
|
uint32_t said;
|
||||||
|
const char *at = line;
|
||||||
|
if (!oot_hex8(&at, &said) || said != sum || *at++ != ' ' || (int)strlen(at) != n) return 0;
|
||||||
|
for (int i = 0; i < n; i++)
|
||||||
|
if (at[i] != '0' && at[i] != '1' && at[i] != '-' && at[i] != '?') return 0;
|
||||||
|
memcpy(states, at, (size_t)n);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
#endif
|
||||||
60
gpu/vk/pace_rule.h
Normal file
60
gpu/vk/pace_rule.h
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
// (Triforce) Who holds a frame of the game to the Game frame rate cap: RivaTuner Statistics Server, NVIDIA Reflex or
|
||||||
|
// the game's own wait, and what RivaTuner is asked to hold (gpu/vk/vk_pace.cpp, where the game's frame starts). Plain
|
||||||
|
// C with no system call, so that tests/test_rtss_limit.py runs every combination of rows through it.
|
||||||
|
//
|
||||||
|
// The owner's rule: RivaTuner is the game's limiter whenever it can hold the game. It holds presents, and it counts
|
||||||
|
// every one, the frames frame generation makes too. So what it can hold depends on where the presents come from:
|
||||||
|
//
|
||||||
|
// No frame generation: one present for each of the game's frames, on the replay's thread. RivaTuner's limit is the
|
||||||
|
// cap, and holding the presents holds the game through the present queue.
|
||||||
|
//
|
||||||
|
// The Frame generation row's fixed multipliers (2x to 8x, AMD FSR 3.1's 2x and AMD FSR 4's frame): the display thread
|
||||||
|
// (vk_framegen.cpp display_main) presents every frame of every set, the multiplier's number for each of the game's.
|
||||||
|
// RivaTuner's limit is the cap times the multiplier, and holding those presents holds the game: the display thread
|
||||||
|
// then takes the game's frames at the cap, and the replay's thread waits for it (framegen_record) as the game waits
|
||||||
|
// for the replay. The display thread's own pacing of the made frames has to yield meanwhile (pace_display_yields):
|
||||||
|
// it spaces the presents by the game's measured frame time, which under RivaTuner's hold is RivaTuner's own spacing
|
||||||
|
// read back; it can only ever fall behind that, never catch up, and each hitch would leave the game slower for good.
|
||||||
|
//
|
||||||
|
// The Frame generation row's Auto: a frame for each refresh of the screen, at that refresh's time, however many the
|
||||||
|
// game makes (a game frame is given none, one or several). The presents follow the screen, not the game, so no limit
|
||||||
|
// on them holds the game to its cap: RivaTuner is kept out (RTSS_OUT_AUTO) and the game's own wait holds the frames.
|
||||||
|
//
|
||||||
|
// NVIDIA DLSS frame generation: the game's one present goes into NVIDIA's Streamline, which shows the made frames
|
||||||
|
// and the game's own from inside its closed library, on a pace of its own that cannot be told to yield, and shows
|
||||||
|
// one frame for each of the game's while it holds generation back (a menu, a load, a window not in front). What a
|
||||||
|
// hold on each of its presents does to that pacing cannot be worked out from this code, so RivaTuner is kept out
|
||||||
|
// (RTSS_OUT_DLSSG) and the game's own wait, or Reflex with its row, holds the frames. HOS_RTSS_DLSSG=1 lets RivaTuner
|
||||||
|
// try there all the same, with the cap times the multiplier in effect and every safeguard of rtss_decide.h.
|
||||||
|
#pragma once
|
||||||
|
#include <stdint.h>
|
||||||
|
|
||||||
|
#include "rtss_decide.h"
|
||||||
|
|
||||||
|
enum { PACE_GAME = 0, PACE_REFLEX = 1, PACE_RTSS = 2 };
|
||||||
|
enum {
|
||||||
|
PACE_FG_OFF, // no frame generation: a present for each of the game's frames
|
||||||
|
PACE_FG_FIXED, // the display thread shows mult frames for each of the game's
|
||||||
|
PACE_FG_AUTO, // the display thread shows a frame for each refresh of the screen
|
||||||
|
PACE_FG_DLSSG, // NVIDIA's Streamline shows mult frames for each of the game's (1 while it holds generation back)
|
||||||
|
};
|
||||||
|
|
||||||
|
// What RivaTuner is asked to hold where the game's frame starts: the limit on the frames it counts, every frame
|
||||||
|
// presented, in millihertz; or why it is kept out (RTSS_OUT_*); RTSS_UNSET while the host has told no period yet.
|
||||||
|
// period_ns: the period the host paces the game's frames by (the cap in effect). fg, mult: where the presents come
|
||||||
|
// from and how many for each of the game's frames now. dlssg_try: HOS_RTSS_DLSSG=1.
|
||||||
|
static inline int64_t pace_rtss_want(uint64_t period_ns, int fg, int mult, int dlssg_try) {
|
||||||
|
if (!period_ns) return RTSS_UNSET;
|
||||||
|
if (fg == PACE_FG_AUTO) return RTSS_OUT_AUTO;
|
||||||
|
if (fg == PACE_FG_DLSSG && !dlssg_try) return RTSS_OUT_DLSSG;
|
||||||
|
return rtss_cap_mhz(period_ns) * (fg == PACE_FG_OFF || mult < 1 ? 1 : mult);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Who holds this frame. rtss: RivaTuner holds it (rtss_frame_holds of the worker's word, with no movie playing and the
|
||||||
|
// frames' own check passed). reflex: Reflex would hold it (vk_dlssg.cpp pace), which is asked only when RivaTuner does
|
||||||
|
// not: with RivaTuner as the limiter Reflex is given no frame limit, whatever the Reflex frame limit row says. Exactly
|
||||||
|
// one of the three for every frame: the host waits when, and only when, the answer is PACE_GAME.
|
||||||
|
static inline int pace_owner(int rtss, int reflex) { return rtss ? PACE_RTSS : reflex ? PACE_REFLEX : PACE_GAME; }
|
||||||
|
|
||||||
|
// The display thread's own pacing of made frames stands aside: RivaTuner holds the frames it presents.
|
||||||
|
static inline int pace_display_yields(int owner, int fg) { return owner == PACE_RTSS && fg == PACE_FG_FIXED; }
|
||||||
54
gpu/vk/pass_barrier.h
Normal file
54
gpu/vk/pass_barrier.h
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
// (Triforce) The barrier a draw pass starts with (vk_texture.cpp pass_start_barrier), apart from Vulkan so that
|
||||||
|
// tests/test_pass_barrier.py can run its rule.
|
||||||
|
//
|
||||||
|
// Before every draw pass the replay records a barrier: what the work before it wrote is made visible to it. A full one (all
|
||||||
|
// commands, all memory) makes the GPU drain: the new pass's vertex, task and mesh work waits until the pass before has drawn its
|
||||||
|
// last fragment. The game mode (RX_VK_PASS_BARRIER=game) keeps that wait for the new pass's fragment stages only (its fragment
|
||||||
|
// shaders, depth and stencil tests and colour output), where a draw pass comes right after another with nothing recorded
|
||||||
|
// between into its command buffer: no barrier of the game's (nvnCommandBufferBarrier is a full barrier and ends the pass), no copy,
|
||||||
|
// clear, uniform update or dispatch. ((Triforce, P4) by the command buffer's own count, cmd_work.h: what other command buffers get
|
||||||
|
// meanwhile is not between the two passes on the queue; a recording's uploads go before its commands, behind full barriers of their
|
||||||
|
// own.) The new pass's vertex, task and mesh work may then overlap the earlier pass's fragments, as on the
|
||||||
|
// console's GPU, where NVN asks for a barrier before rendering results are read. What changes order is only a vertex, task or
|
||||||
|
// mesh stage that reads or writes what the pass before touched with no barrier from the game between; every fragment-side access
|
||||||
|
// stays ordered after all earlier work, the writes and the reads (so a clear or a draw that writes a target an earlier pass
|
||||||
|
// sampled waits for that pass).
|
||||||
|
//
|
||||||
|
// Not where the replay itself ended the pass to keep a draw or a clear apart from its own pass's work (vk_draw.cpp draw_hazards,
|
||||||
|
// clear_after_draws, note_pass_split): those passes start with a full barrier, as before, since the draw that follows may read the
|
||||||
|
// pass's targets in its vertex or mesh stage. Not with RX_VK_BARRIER_EVERY_DRAW (an experiment that ends the pass at each draw for a
|
||||||
|
// full barrier there). ((Triforce, P4) With the parallel replay too: a segment is a command buffer of its own, begun with a full
|
||||||
|
// barrier, so its first pass start is a full one; RX_VK_BARRIER_CONTEXT=0, the one count of every thread's commands: not while the
|
||||||
|
// helper threads record, as before.)
|
||||||
|
//
|
||||||
|
// On by default with NVIDIA's driver only, where NX2navi turned its own game-only barriers on (its barrier_game, "about a tenth off
|
||||||
|
// the graphics card's time for a frame", 34 screenshots the same on an RTX 5090); nothing in Triforce's notes records a fault of
|
||||||
|
// this one, and none of the owner's runs had it on. NX2navi's rule is broader: between two passes it orders the attachments alone
|
||||||
|
// (no shader reads), and between two dispatches nothing. An attachment-only barrier leaves a target's write unordered with the
|
||||||
|
// shader reads of it before: across two passes, the fault Triforce's notes record within one (vk_draw.cpp clear_after_draws: a
|
||||||
|
// clear while draws before it still sampled its target, a band of the picture for a frame). So it is not taken; the dispatches'
|
||||||
|
// barriers are vk_draw.cpp dispatch's, not this rule's.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstring>
|
||||||
|
|
||||||
|
namespace mkwvk::pass_barrier {
|
||||||
|
|
||||||
|
/// RX_VK_PASS_BARRIER: "game" the game mode on any card; "full" (or any other value) a full barrier at every pass start, as
|
||||||
|
/// before; unset or empty: the game mode with NVIDIA's driver (vendor 0x10de), full on every other card. every_draw:
|
||||||
|
/// RX_VK_BARRIER_EVERY_DRAW is set, which keeps every pass start full
|
||||||
|
inline bool game(const char* env, uint32_t vendor, bool every_draw) {
|
||||||
|
if (every_draw) return false;
|
||||||
|
if (env && *env) return std::strcmp(env, "game") == 0;
|
||||||
|
return vendor == 0x10DE;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a pass start gets the fragment-stage barrier: the game mode, and a draw pass the last thing on the command buffer (the
|
||||||
|
/// pass ended on this command buffer, its count as it was then, and not a pass the replay split). ((Triforce, P4) no parallel
|
||||||
|
/// replay's argument: vk_texture.cpp folds RX_VK_BARRIER_CONTEXT=0's standing aside into game)
|
||||||
|
inline bool relaxed(bool game, bool after_pass) {
|
||||||
|
return game && after_pass;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::pass_barrier
|
||||||
144
gpu/vk/pcache_bound.h
Normal file
144
gpu/vk/pcache_bound.h
Normal file
@@ -0,0 +1,144 @@
|
|||||||
|
// (Triforce) The bound on the pipeline cache's file (mkw_vk_pipelines.bin; vk_draw.cpp pcache_init, pipeline_cache_save),
|
||||||
|
// apart from Vulkan so that tests/test_pcache_bound.py can run it. As NX2navi's (runtime/vk_cache_file.h and its cache_cap).
|
||||||
|
//
|
||||||
|
// The file is read into memory whole at every start and handed to the driver, which keeps its own copy for the session, and
|
||||||
|
// every save fetches all of it again. Nothing bounded it: a player's file reached 4.1 GB on a 32 GB PC whose log said "RAM
|
||||||
|
// short" all session, the lab's 3.9 GB. The rule of 11-14 (the file is kept for one set of translated shaders) stops the
|
||||||
|
// growth from update to update; what one set of shaders can reach on some driver, nothing stops. So:
|
||||||
|
// - a file over the bound is not read at all: its size is asked before a byte of it is, so it never takes its size in
|
||||||
|
// memory. The cache starts empty, the driver's own cache serves the pipelines, and the first save writes a new file
|
||||||
|
// over the old one;
|
||||||
|
// - data over the bound is not written: the file keeps its last save, which is under the bound and is read at the next
|
||||||
|
// start; what was built after that save is compiled again then (from the driver's own cache, where it has one). A file
|
||||||
|
// the next start would refuse is never made;
|
||||||
|
// - a file at or under the bound is read and written as before, byte for byte (in pieces now: no single read call has
|
||||||
|
// to take gigabytes), and one whose read ends early is no cache at all, never part of one.
|
||||||
|
// The bound: RX_VK_PCACHE_MAX_MB megabytes when set (0: none, as before); unset, an eighth of the PC's RAM, 2048 MB at
|
||||||
|
// least and 8192 MB at most (16 GB: 2048, 32 GB: 4096, 64 GB and more: 8192). Files measured: 647 MB fresh and 1146 MB
|
||||||
|
// with the precompile's 13,000 pipelines (vk_draw.cpp's notes), 1631 MB in an install on the lab's PC: all under the
|
||||||
|
// least of them. Never below 2048: a bound under what one run builds would have the data refused at every save, and every
|
||||||
|
// start compile again (NX2navi's 512 MB did that on an RTX 2080 Ti). The RAM, not what is free of it at the start: the
|
||||||
|
// same file is not read at one start and refused at the next.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdio>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <new>
|
||||||
|
#include <stdexcept>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace pcache_bound {
|
||||||
|
|
||||||
|
inline constexpr uint64_t kLeastMb = 2048, kMostMb = 8192;
|
||||||
|
/// A read asks the C library for this much at a time
|
||||||
|
inline constexpr size_t kPiece = (size_t)64 << 20;
|
||||||
|
|
||||||
|
/// The bound in bytes, 0: none. env: RX_VK_PCACHE_MAX_MB (null: unset); ram_bytes: the PC's RAM, 0 when it is not known
|
||||||
|
/// (the least then). A value that is no number counts as unset, never as "no bound"
|
||||||
|
inline uint64_t bytes(const char* env, uint64_t ram_bytes) {
|
||||||
|
if (env && *env) {
|
||||||
|
char* end = nullptr;
|
||||||
|
const unsigned long long mb = strtoull(env, &end, 10);
|
||||||
|
if (end != env && !*end && env[0] != '-') return mb > (UINT64_MAX >> 20) ? 0 : (uint64_t)mb << 20;
|
||||||
|
}
|
||||||
|
return std::clamp<uint64_t>((ram_bytes >> 20) / 8, kLeastMb, kMostMb) << 20;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a file or data of that size is within the bound (0: none)
|
||||||
|
inline bool within(uint64_t size, uint64_t bound) { return !bound || size <= bound; }
|
||||||
|
|
||||||
|
// ((Triforce, 11-18) The carry over an update. The rule of 11-14 left a file made for other shaders (an update that changed the
|
||||||
|
// translation) unread: right for the growth, but on AMD's driver, which keeps no cache of its own for these, it meant every
|
||||||
|
// pipeline and stage library compiled again after each such update: 87 minutes before the title on an RX 7900 XT (32 of them
|
||||||
|
// the 14,467 noted pipelines, 53 the 88,742 stage libraries). What the update did not change still matches the old file: of
|
||||||
|
// two caches a translator apart on the lab's PC, 39.6 % of the SPIR-V modules were the same (12.3 % of their bytes). So a file
|
||||||
|
// for other shaders is read all the same when it is no larger than the carry limit: the driver finds again what it compiled for
|
||||||
|
// the shaders that did not change, and the save writes old and new together for the new shaders. The limit stops the growth:
|
||||||
|
// a fresh file is 650-860 MB, one carried and saved again is over 1536 MB, and the next update's start leaves it unread, as
|
||||||
|
// before. RX_VK_PCACHE_CARRY_MB=<MB> sets the limit, 0: never carried (the rule of 11-14 alone))
|
||||||
|
inline constexpr uint64_t kCarryMb = 1536;
|
||||||
|
/// The carry limit in bytes (0: none carried). env: RX_VK_PCACHE_CARRY_MB (null or no number: the default)
|
||||||
|
inline uint64_t carry_limit(const char* env) {
|
||||||
|
if (env && *env) {
|
||||||
|
char* end = nullptr;
|
||||||
|
const unsigned long long mb = strtoull(env, &end, 10);
|
||||||
|
if (end != env && !*end && env[0] != '-') return mb > (UINT64_MAX >> 20) ? kCarryMb << 20 : (uint64_t)mb << 20;
|
||||||
|
}
|
||||||
|
return kCarryMb << 20;
|
||||||
|
}
|
||||||
|
/// Whether a file for other shaders (not same) of size bytes is read all the same: carried over the update. And only when it
|
||||||
|
/// leaves room under the bound (bound, 0: none) for as much again: the update's own pipelines go into the same file, and a save
|
||||||
|
/// over the bound is not written, so a carried file that left no room would have them compiled again at every start
|
||||||
|
/// And only on a PC with the RAM for it (ram_bytes, 0: not known, then never): the driver holds the carried data for the whole
|
||||||
|
/// session, up to 1-1.5 GB more than a fresh file, and on 16 GB PCs that already run short of RAM (logs that say so dozens of
|
||||||
|
/// times a session) that is paging and hitches in play. 31 GB and more: a 32 GB PC reports a little under 32
|
||||||
|
inline constexpr uint64_t kCarryRamGb = 31;
|
||||||
|
inline bool carried(bool same, uint64_t size, uint64_t limit, uint64_t bound, uint64_t ram_bytes) {
|
||||||
|
return !same && limit && size > 32 && size <= limit && (!bound || size <= bound / 2) && ram_bytes >= (kCarryRamGb << 30);
|
||||||
|
}
|
||||||
|
|
||||||
|
enum class Read {
|
||||||
|
Ok,
|
||||||
|
None, // (no such file: a first start)
|
||||||
|
Unsized, // (its length could not be asked)
|
||||||
|
Short, // (not longer than a pipeline cache's header)
|
||||||
|
OverBound, // (longer than the bound: not a byte of it read)
|
||||||
|
NoMemory, // (no block of its size)
|
||||||
|
Cut, // (the read ended early: nothing of it is used)
|
||||||
|
};
|
||||||
|
|
||||||
|
inline const char* why(Read r) {
|
||||||
|
switch (r) {
|
||||||
|
case Read::Unsized: return "its size cannot be asked";
|
||||||
|
case Read::Short: return "it is shorter than a pipeline cache's header";
|
||||||
|
case Read::OverBound: return "it is over the bound";
|
||||||
|
case Read::NoMemory: return "there is no memory for a block of its size";
|
||||||
|
case Read::Cut: return "reading it ended early";
|
||||||
|
default: return "";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The whole file into out, or nothing: out is empty unless the answer is Ok. *len: the file's length whenever it could be
|
||||||
|
/// asked, read or not. least: a file of at most this many bytes is refused; bound: one longer than this too (0: none)
|
||||||
|
inline Read read(const char* path, uint64_t least, uint64_t bound, std::vector<uint8_t>& out, uint64_t* len) {
|
||||||
|
out.clear();
|
||||||
|
*len = 0;
|
||||||
|
FILE* f = fopen(path, "rb");
|
||||||
|
if (!f) return Read::None;
|
||||||
|
#ifdef _WIN32
|
||||||
|
int64_t n = _fseeki64(f, 0, SEEK_END) ? -1 : _ftelli64(f);
|
||||||
|
if (n >= 0 && _fseeki64(f, 0, SEEK_SET)) n = -1;
|
||||||
|
#else
|
||||||
|
int64_t n = fseeko(f, 0, SEEK_END) ? -1 : (int64_t)ftello(f);
|
||||||
|
if (n >= 0 && fseeko(f, 0, SEEK_SET)) n = -1;
|
||||||
|
#endif
|
||||||
|
Read r = Read::Ok;
|
||||||
|
if (n >= 0) *len = (uint64_t)n;
|
||||||
|
if (n < 0) r = Read::Unsized;
|
||||||
|
else if ((uint64_t)n <= least) r = Read::Short;
|
||||||
|
else if (!within((uint64_t)n, bound)) r = Read::OverBound;
|
||||||
|
else if ((uint64_t)n > SIZE_MAX) r = Read::NoMemory;
|
||||||
|
else {
|
||||||
|
try {
|
||||||
|
out.resize((size_t)n);
|
||||||
|
} catch (const std::bad_alloc&) {
|
||||||
|
r = Read::NoMemory;
|
||||||
|
} catch (const std::length_error&) {
|
||||||
|
r = Read::NoMemory;
|
||||||
|
}
|
||||||
|
size_t got = 0;
|
||||||
|
while (r == Read::Ok && got < out.size()) {
|
||||||
|
const size_t piece = std::min(out.size() - got, kPiece);
|
||||||
|
const size_t had = fread(out.data() + got, 1, piece, f);
|
||||||
|
got += had;
|
||||||
|
if (had != piece) r = Read::Cut;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fclose(f);
|
||||||
|
if (r != Read::Ok) out.clear(), out.shrink_to_fit();
|
||||||
|
return r;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace pcache_bound
|
||||||
237
gpu/vk/pcache_carry.h
Normal file
237
gpu/vk/pcache_carry.h
Normal file
@@ -0,0 +1,237 @@
|
|||||||
|
// (Triforce) The pipeline cache's file carried over a stamp change at an update's preparation, capped (vk_draw.cpp pcache_init,
|
||||||
|
// pipeline_cache_save, pcache_carry_rebuild; vk_main.cpp's HOOKSHOT_PREPARE_ONLY start), apart from Vulkan so that
|
||||||
|
// tests/test_pcache_carry.py runs its rules.
|
||||||
|
//
|
||||||
|
// Why. mkw_vk_pipelines.bin is kept for one stamp (program_cache_stamp, written beside it as <file>.for: the rule of 11-14).
|
||||||
|
// Any change of the stamp threw the whole file away, and the stamp changes with every translator update, a Robust buffers
|
||||||
|
// toggle, a vk_bank.cpp edit no translation calls (436f9ac -> int2: sampler_bias_changed). The update's preparation then built
|
||||||
|
// the ~91,600 stage libraries and the precompiled pipelines into an empty cache: 5608 s for the warm on AMD 26.10.1 with its
|
||||||
|
// own cache wiped, 1868 s for the precompile, where a cache that has them takes 16.4 s and 4.4 s. The driver finds an entry by
|
||||||
|
// its own hash of the whole creation, so an entry of the old file whose shader did not change is a hit, and one whose shader
|
||||||
|
// did is never used, never wrong.
|
||||||
|
//
|
||||||
|
// Why the rule of 11-14 is there all the same: growth. With the old files always read, each translator update added its
|
||||||
|
// ~650 MB to pipelines of shaders that no longer exist, and a player's file reached 4.1 GB, "RAM short" in play. So:
|
||||||
|
// - only at an update's preparation (HOOKSHOT_PREPARE_ONLY, Setup's "Preparing shaders"), only a file of this device and
|
||||||
|
// driver (the header's vendor, device and pipelineCacheUUID, checked as before) within the file's bound (pcache_bound.h),
|
||||||
|
// and only while the file is at most kCapNum/kCapDen (1.25) times its base;
|
||||||
|
// - the base: the size of the last set built fresh, kept beside the file in <file>.base (Side) with the number of carries
|
||||||
|
// since. A file with no side file of its own (one written before this rule, or under RX_VK_PCACHE_KEEP_OLD) is taken as
|
||||||
|
// one fresh set: the rule of 11-14 kept it for one stamp. While a file has never been carried into, what play adds to it
|
||||||
|
// is that set's too, and raises the base; once carried into, the base stays what it was, so the cap is the whole growth
|
||||||
|
// over every carry since the set was built (cumulative, not a quarter more at each update);
|
||||||
|
// - a carried session whose cache grows past what it may hold (limit: the cap, or the file's bound where that is lower, so
|
||||||
|
// a carried cache never ends up over the bound unwritten) stops building into it as soon as that is seen (looked at every
|
||||||
|
// kCheckMs while the preparation's workers build, and at each save; while the stage libraries are built, as soon as a
|
||||||
|
// sample of them shows that the rate it grows at clearly takes it there: the early drop, below), does not write it, and
|
||||||
|
// names this stamp in <file>.nocarry, so that a preparation begun again after a kill starts fresh rather than carrying the
|
||||||
|
// same file again.
|
||||||
|
// vk_main.cpp's preparation then builds the shaders' driver part again into a fresh cache (the compute pipelines the
|
||||||
|
// carried run built and the stage libraries, from the program cache: nothing translated again), which is saved with a
|
||||||
|
// new base. What the carried run compiled is in the driver's own cache by then, where the driver keeps one;
|
||||||
|
// - a carried save writes the new stamp beside the file (unlike RX_VK_PCACHE_KEEP_OLD): it is these shaders' file now. So
|
||||||
|
// the side file says too that the carried preparation is not done yet (Side::open): one that was stopped (Setup's Cancel,
|
||||||
|
// the builder's stop after a long silence, a closed window) saved its merged cache under the new stamp, and the
|
||||||
|
// preparation begun again would read it as these shaders' own file, uncapped, and build the rest of the warm into it with
|
||||||
|
// only the bound to stop it. It is read as carried instead (resumed: capped, refused by .nocarry, built again into a fresh
|
||||||
|
// cache past its limit), until a carried preparation goes through to its end (vk_main.cpp, pcache_carry_done). Its save
|
||||||
|
// writes the side file before the data and the .for (begun), so that one cut between them is read so too;
|
||||||
|
// RX_VK_PCACHE_CARRY=0: no carry, the rule of 11-14 as it was. The play file (pcache_play.h) keeps its own rule: it is
|
||||||
|
// for one stamp, as before. A cache changes how long a pipeline takes to make, never the pipeline.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <cstring>
|
||||||
|
|
||||||
|
namespace pcache_carry {
|
||||||
|
|
||||||
|
inline constexpr uint64_t kCapNum = 5, kCapDen = 4; // (1.25 times the base)
|
||||||
|
inline constexpr char kMagic[8] = {'T', 'F', 'P', 'C', 'B', 'A', 'S', 'E'};
|
||||||
|
inline constexpr size_t kSideBytes = 40; // (magic 8, the stamp's 16 hex digits, base 8, carries 4, open 4)
|
||||||
|
|
||||||
|
/// RX_VK_PCACHE_CARRY: on unless set to 0
|
||||||
|
inline bool on(const char* env) { return !env || std::atoi(env) != 0; }
|
||||||
|
|
||||||
|
/// The most a file of that base may hold and still be carried into, or saved by a carried session
|
||||||
|
inline uint64_t cap(uint64_t base) {
|
||||||
|
return base > UINT64_MAX / kCapNum ? UINT64_MAX : base * kCapNum / kCapDen;
|
||||||
|
}
|
||||||
|
inline bool fits(uint64_t size, uint64_t base) { return size <= cap(base); }
|
||||||
|
/// What a carried session's cache may hold: its cap, or the file's bound (pcache_bound.h, 0: none) where that is lower. Over
|
||||||
|
/// the bound it could not be written at all (pipeline_cache_save), and the update's preparation would end with the old file
|
||||||
|
inline uint64_t limit(uint64_t base, uint64_t bound) {
|
||||||
|
const uint64_t c = cap(base);
|
||||||
|
return bound && bound < c ? bound : c;
|
||||||
|
}
|
||||||
|
/// How often the preparation looks at a carried cache's size while its workers build (vk_draw.cpp pcache_carry_check)
|
||||||
|
inline constexpr unsigned kCheckMs = 10000;
|
||||||
|
|
||||||
|
// (Triforce, 11-18) The early drop (vk_draw.cpp warm_libraries, pcache_carry_early). Past its limit alone was seen only once it
|
||||||
|
// got there: on an RX 7900 XT an update carried a 1973 MB file built by an older translator, nearly every stage library missed
|
||||||
|
// (pre-raster 1.3-2.0 s each, fragment 0.18 s, as cold), and the cache reached 2469 MB of its 2467 after 2040 s and 36,906 of
|
||||||
|
// the ~91,600 libraries (16,018 of 42,532 programs): 34 minutes built into a cache that was then let go, before the fresh build
|
||||||
|
// began. So a carried warm builds a sample of the whole order first, and the cache's growth over it, per program of the sample,
|
||||||
|
// times the programs of the whole order, on top of what it held when they began, is where it ends at this rate (projected);
|
||||||
|
// clearly past its limit (drop_early), it is dropped then, as past it (pcache_carry_over, .nocarry, the rebuild into a fresh
|
||||||
|
// cache), a few minutes in rather than half an hour.
|
||||||
|
// - the sample: every kSampleStride-th program of the warm's order (sample_order), built before the others, which wait for its
|
||||||
|
// projection to begin (so that the growth looked at is the sample's alone), and looked at once. Not the order's head: the
|
||||||
|
// warm takes the programs a state list loaded and those the notes name first (9,420 of 42,532 programs, some 20,000
|
||||||
|
// libraries), the others in the program cache's file order, so that hits and misses come in stretches of it. That run had
|
||||||
|
// one: 12,391 libraries between 960 and 1080 s at 0.17 s a pre-raster, as hits, then 5.2 s each. A head that misses
|
||||||
|
// while the rest hits projected 4.5 times its growth (42,532 / 9,420), past any margin: dropped, and the ~70,000 libraries
|
||||||
|
// that would have been hits compiled cold into the fresh cache. Every 32nd of the whole order follows such stretches as
|
||||||
|
// they lie. The sample costs nothing built (the same programs, only sooner), but the notes' programs wait for it: some
|
||||||
|
// 1,330 programs, ~2,900 libraries, a few minutes cold, much less where they hit; and the workers idle while its last
|
||||||
|
// programs are built;
|
||||||
|
// - the margin: the projected growth over the warm at least kProjectNum/kProjectDen (2) times the room the cache had at its
|
||||||
|
// start (its limit less its size then), for what the sample's libraries differ from the others in size. A carry that would
|
||||||
|
// not fit and is not dropped early is dropped later all the same, once it is past (over); at least kProjectLibs stage
|
||||||
|
// libraries in the sample, or none is made (a warm of under ~22,000 programs, whose cold build takes less time);
|
||||||
|
// - the growth is counted from the cache's size when the libraries began (what prepare_run's compute pipelines added before
|
||||||
|
// is in that size, and so out of the room). The compute pass is not projected (its programs done are translations, the
|
||||||
|
// compute ones first, and nothing after them builds into the cache) and keeps the look past the limit alone, as do
|
||||||
|
// precompile_run and the saves;
|
||||||
|
// RX_VK_PCACHE_CARRY_PROJECT=0: only once past the limit, and the warm in its own order, as before
|
||||||
|
inline constexpr uint64_t kSampleStride = 32; // (the sample: every 32nd program of the warm's order)
|
||||||
|
inline constexpr uint64_t kProjectLibs = 1500; // (and at least so many stage libraries built for it)
|
||||||
|
inline constexpr uint64_t kProjectNum = 2, kProjectDen = 1; // (the margin: the projected growth over the room at the start)
|
||||||
|
|
||||||
|
/// RX_VK_PCACHE_CARRY_PROJECT: on unless set to 0
|
||||||
|
inline bool project_on(const char* env) { return !env || std::atoi(env) != 0; }
|
||||||
|
|
||||||
|
/// How many of the warm's total programs the sample takes: every kSampleStride-th, from the first
|
||||||
|
inline uint64_t sample_count(uint64_t total) { return total / kSampleStride + (total % kSampleStride != 0); }
|
||||||
|
/// The program of the warm's order (0 .. total-1) built at its claim c (c < total): the sample's first (c < sample_count, every
|
||||||
|
/// kSampleStride-th, in order), then the others in their order. Every program once
|
||||||
|
inline uint64_t sample_order(uint64_t c, uint64_t total) {
|
||||||
|
const uint64_t m = sample_count(total);
|
||||||
|
if (c < m) return c * kSampleStride;
|
||||||
|
c -= m;
|
||||||
|
return c / (kSampleStride - 1) * kSampleStride + c % (kSampleStride - 1) + 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Where a cache of now bytes, from bytes when the warm began, ends at the rate it grew with done of total programs done (the
|
||||||
|
/// growth per program done, times the programs left, on top of now). Nothing grown, nothing done or nothing left: now
|
||||||
|
inline uint64_t projected(uint64_t now, uint64_t from, uint64_t done, uint64_t total) {
|
||||||
|
if (!done || now <= from || done >= total) return now;
|
||||||
|
const double end = (double)now + (double)(now - from) * (double)(total - done) / (double)done;
|
||||||
|
return end >= 18446744073709549568.0 ? UINT64_MAX : (uint64_t)end;
|
||||||
|
}
|
||||||
|
/// Whether a carried cache is dropped now on its projection, its sample built (done of the total programs, the sample's
|
||||||
|
/// libs stage libraries, now bytes): the sample large enough, it grew, programs are left, it had room under its limit at the
|
||||||
|
/// warm's start (limit: the cap, or the bound where lower), and the projected growth is at least the margin times that room.
|
||||||
|
/// Past the limit already, the growth may be under the margin: over says it
|
||||||
|
inline bool drop_early(uint64_t now, uint64_t from, uint64_t done, uint64_t total, uint64_t libs, uint64_t limit) {
|
||||||
|
if (libs < kProjectLibs || from >= limit || now <= from || done >= total) return false;
|
||||||
|
const uint64_t grown = projected(now, from, done, total) - from;
|
||||||
|
return (double)grown * (double)kProjectDen >= (double)(limit - from) * (double)kProjectNum;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What <file>.base says: the stamp of the file it was written with (the .for's 16 hex digits), the base, the carries since,
|
||||||
|
/// and whether the file was saved by a carried preparation that has not gone through to its end (open: 1; a side file of
|
||||||
|
/// before this, whose 4 last bytes were spare and 0, is not)
|
||||||
|
struct Side {
|
||||||
|
char stamp[16]{};
|
||||||
|
uint64_t base = 0;
|
||||||
|
uint32_t carried = 0;
|
||||||
|
uint32_t open = 0;
|
||||||
|
};
|
||||||
|
|
||||||
|
inline void put(const Side& s, uint8_t out[kSideBytes]) {
|
||||||
|
std::memset(out, 0, kSideBytes);
|
||||||
|
std::memcpy(out, kMagic, 8);
|
||||||
|
std::memcpy(out + 8, s.stamp, 16);
|
||||||
|
std::memcpy(out + 24, &s.base, 8);
|
||||||
|
std::memcpy(out + 32, &s.carried, 4);
|
||||||
|
std::memcpy(out + 36, &s.open, 4);
|
||||||
|
}
|
||||||
|
/// False: not a side file (too short, another magic)
|
||||||
|
inline bool get(const uint8_t* in, size_t n, Side* s) {
|
||||||
|
if (n < kSideBytes || std::memcmp(in, kMagic, 8)) return false;
|
||||||
|
std::memcpy(s->stamp, in + 8, 16);
|
||||||
|
std::memcpy(&s->base, in + 24, 8);
|
||||||
|
std::memcpy(&s->carried, in + 32, 4);
|
||||||
|
std::memcpy(&s->open, in + 36, 4);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The file's history at a start: its side file when that was written with the file's own stamp (for: the .for's 16 digits,
|
||||||
|
/// null when there is none), else one fresh set of the file's size
|
||||||
|
inline Side history(bool side_read, const Side& side, const char* file_for, uint64_t file_size) {
|
||||||
|
if (side_read && file_for && !std::memcmp(side.stamp, file_for, 16) && side.base) return side;
|
||||||
|
Side s;
|
||||||
|
if (file_for) std::memcpy(s.stamp, file_for, 16);
|
||||||
|
s.base = file_size;
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a file for another stamp is read at this start (carried): an update's preparation, the switch on, no carry refused
|
||||||
|
/// for these shaders already (refused: <file>.nocarry names this stamp), the file within its bound and at most cap(base) of
|
||||||
|
/// its history. The device's header is checked after, on its bytes
|
||||||
|
inline bool may_carry(bool on, bool preparing, bool refused, bool within_bound, uint64_t file_size, const Side& had) {
|
||||||
|
return on && preparing && !refused && within_bound && file_size > 32 && had.base && fits(file_size, had.base);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a file for these shaders (same: its .for names this stamp) is read as carried all the same at this start
|
||||||
|
/// (resumed): an update's preparation, the switch on, and its history open, saved by a carried preparation that did not go
|
||||||
|
/// through to its end. Read with the history it has (Start::Same: no carry counted again), capped as the carried run was;
|
||||||
|
/// refused (<file>.nocarry names this stamp: that run went past its limit), it is not read at all, and the cache starts fresh
|
||||||
|
inline bool resumed(bool on, bool preparing, bool same, const Side& had) {
|
||||||
|
return on && preparing && same && had.open != 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether the side file says a carried session of these shaders (stamp: this stamp's 16 digits) began a save that did not
|
||||||
|
/// get as far as the .for (file_for: its 16 digits, null when there is none): open, named for these shaders, while the .for
|
||||||
|
/// names another stamp or none. A carried save writes the side file first, then the data, then the .for (vk_draw.cpp
|
||||||
|
/// pipeline_cache_save), so that a save cut anywhere (a crash, a power cut, a kill in a checkpoint of warm_checkpoint.h)
|
||||||
|
/// leaves the session's own history: the file there is the old one or the merged one, both within that session's limit.
|
||||||
|
/// Read with that history as a file of these shaders whose carried preparation is not done (resumed), never as one fresh set
|
||||||
|
/// of its size, uncapped
|
||||||
|
inline bool begun(bool side_read, const Side& side, const char* stamp, const char* file_for) {
|
||||||
|
return side_read && side.open && side.base && !std::memcmp(side.stamp, stamp, 16) &&
|
||||||
|
(!file_for || std::memcmp(file_for, stamp, 16));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// ((Triforce, 11-19) Whether v0.5.2's carry outside Setup's preparation (pcache_bound.h carried: a file for other shaders
|
||||||
|
/// read whole by its size and the PC's RAM) is asked at all: not at the preparation (may_carry's), not for a file read as
|
||||||
|
/// carried or resumed already, not when a carry was refused for these shaders (refused: <file>.nocarry names them. So
|
||||||
|
/// warm_libraries leaves a file it dropped for what it held that nothing asks for, its .for gone: read again, its cache would
|
||||||
|
/// be over the bound again and not written, and the stage libraries compiled at their first draw every session), and not for
|
||||||
|
/// a carried preparation's save cut short (begun: its own capped history, read as resumed)
|
||||||
|
inline bool plain_carry(bool preparing, bool carried, bool resumed, bool refused, bool begun) {
|
||||||
|
return !preparing && !carried && !resumed && !refused && !begun;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The session's history from the start: fresh (nothing read), the file's own (read for these shaders), or one carry more
|
||||||
|
/// (read across a stamp change: open until its preparation goes through to its end)
|
||||||
|
enum class Start { Fresh, Same, Carried };
|
||||||
|
inline Side session(Start how, const Side& had) {
|
||||||
|
Side s;
|
||||||
|
if (how == Start::Fresh) return s;
|
||||||
|
s = had;
|
||||||
|
if (how == Start::Carried) s.carried = had.carried + 1, s.open = 1;
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a session's cache of size bytes is past what it may hold: a carried session's over its limit (the cap, or the bound
|
||||||
|
/// where lower). The others never (the bound alone, as before: pipeline_cache_save)
|
||||||
|
inline bool over(bool carried_session, uint64_t size, const Side& s, uint64_t bound) {
|
||||||
|
return carried_session && size > limit(s.base, bound);
|
||||||
|
}
|
||||||
|
/// Whether a save of size bytes is written as far as the carry goes: not when it is over
|
||||||
|
inline bool save_ok(bool carried_session, uint64_t size, const Side& s, uint64_t bound) {
|
||||||
|
return !over(carried_session, size, s, bound);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The history once size bytes are saved under stamp: a file never carried into is one set, so its base follows it (the
|
||||||
|
/// largest saved); one carried into keeps its base. Open stays as the session has it
|
||||||
|
inline Side saved(const Side& s, uint64_t size, const char* stamp) {
|
||||||
|
Side n = s;
|
||||||
|
std::memcpy(n.stamp, stamp, 16);
|
||||||
|
if (!n.carried && size > n.base) n.base = size;
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace pcache_carry
|
||||||
67
gpu/vk/pcache_gate.h
Normal file
67
gpu/vk/pcache_gate.h
Normal file
@@ -0,0 +1,67 @@
|
|||||||
|
// (Triforce) The pipeline cache let go in play when the PC's RAM runs short (vk_draw.cpp pcache_let_go): the gate every
|
||||||
|
// pipeline creation with the cache passes, so that the cache is destroyed only once no creation is using it. Apart from
|
||||||
|
// Vulkan so that tests/test_pcache_gate.py runs it alone, with threads. (11-16: the letting go is off by default,
|
||||||
|
// RX_VK_PCACHE_RAM=1: in v0.4.0-test9 it gave back nothing the working set showed; the gate stays, passed as ever)
|
||||||
|
//
|
||||||
|
// Why: the driver holds the whole cache in RAM for the run, as it was read from mkw_vk_pipelines.bin (2.6 to 2.7 GB on a
|
||||||
|
// 32 GB PC with the stage libraries in it, v0.4.0-test5 to test7; the lab measured as much more RAM in use at the title
|
||||||
|
// with a 3.9 GB file than with a fresh one). In play it serves the pipelines built there, and the driver's own cache on
|
||||||
|
// disk serves most of those too (NVIDIA's: ~460 of ~12,900 precompiled pipelines found in ours). When the PC has under
|
||||||
|
// 1.5 GB of RAM available, Windows pages, and a frame waits 100-250 ms for it; the cache is then worth less than the RAM.
|
||||||
|
// Vulkan's rule: a cache may be destroyed once no command that takes it runs (pipelines made with it do not depend on it).
|
||||||
|
// Every creation that may run in play takes the cache through Gate::Use; Gate::close turns new ones away (they create
|
||||||
|
// without a cache) and waits for those inside. Sequentially consistent atomics: a Use that entered before close saw no
|
||||||
|
// creation inside has either left or sees the gate closed.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <atomic>
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace pcache_gate {
|
||||||
|
|
||||||
|
class Gate {
|
||||||
|
public:
|
||||||
|
/// In: true, the caller may use the cache until leave(); false, the gate is closed (the caller goes without a cache)
|
||||||
|
bool enter() {
|
||||||
|
users_.fetch_add(1);
|
||||||
|
if (closed_.load()) {
|
||||||
|
users_.fetch_sub(1);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
void leave() { users_.fetch_sub(1); }
|
||||||
|
/// Closed from now on, and returns once nobody is inside; wait() is called while somebody is (a sleep). Once only:
|
||||||
|
/// false when it was closed already
|
||||||
|
template <class Wait>
|
||||||
|
bool close(Wait&& wait) {
|
||||||
|
if (closed_.exchange(true)) return false;
|
||||||
|
while (users_.load()) wait();
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
bool closed() const { return closed_.load(); }
|
||||||
|
int64_t inside() const { return users_.load(); }
|
||||||
|
|
||||||
|
private:
|
||||||
|
std::atomic<int64_t> users_{0};
|
||||||
|
std::atomic<bool> closed_{false};
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The cache for one creation: cache itself while the gate is open, none once it is closed (or when none was asked for).
|
||||||
|
/// Held for as long as the creation runs: a temporary in the call's full expression does (Use(gate, pcache).cache)
|
||||||
|
template <class Cache>
|
||||||
|
struct Use {
|
||||||
|
Gate& gate;
|
||||||
|
Cache cache;
|
||||||
|
bool in;
|
||||||
|
Use(Gate& g, Cache want) : gate(g), cache(want), in(want != Cache{} && g.enter()) {
|
||||||
|
if (!in) cache = Cache{};
|
||||||
|
}
|
||||||
|
~Use() {
|
||||||
|
if (in) gate.leave();
|
||||||
|
}
|
||||||
|
Use(const Use&) = delete;
|
||||||
|
Use& operator=(const Use&) = delete;
|
||||||
|
};
|
||||||
|
|
||||||
|
} // namespace pcache_gate
|
||||||
69
gpu/vk/pcache_play.h
Normal file
69
gpu/vk/pcache_play.h
Normal file
@@ -0,0 +1,69 @@
|
|||||||
|
// (Triforce) The pipeline cache's play file (vk_draw.cpp pcache_init, cached_graphics, cached_compute, pipeline_cache_save),
|
||||||
|
// apart from Vulkan so that tests/test_pcache_play.py runs its rules. NX2navi's play cache (NX2NAVI_PIPELINE_CACHE_PLAY,
|
||||||
|
// runtime/vk_pipeline.c pipe_create_in, cache_load, cache_save; chrg6, at 62cc3b13), written for Triforce's cache.
|
||||||
|
//
|
||||||
|
// Why. mkw_vk_pipelines.bin holds what the preparation built and what earlier runs made: 2.6 GB on the owner's PC with the
|
||||||
|
// stage libraries. A file can only be written whole, so a save after a few new pipelines fetched all of it from the driver
|
||||||
|
// into memory and wrote it again: the owner's test18 start wrote 2,683,916 KB right after its precompile, for 116 KB new,
|
||||||
|
// on the start-up's own thread. In play the 64 MB growth rule (pipeline_cache_save, 11-14) already waits for more; the start's
|
||||||
|
// precompile and the game's close did not.
|
||||||
|
//
|
||||||
|
// What. Once the preparation is done, what a run compiles goes into a second cache with a file of its own,
|
||||||
|
// mkw_vk_pipelines_play.bin beside the first (with its own .for, the same shaders' stamp): a creation with the main cache asks
|
||||||
|
// it first with VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED_BIT (the main cache or the driver's own gives the
|
||||||
|
// pipeline at once), and only when that answers VK_PIPELINE_COMPILE_REQUIRED goes to the play cache with the creation's own
|
||||||
|
// flags: one there from an earlier run comes back, one asked of the caches only is answered by it too, the rest is compiled
|
||||||
|
// into it (create). A save writes the play file when it changed, and the main file only when something was compiled into the
|
||||||
|
// main cache since its last save (the preparation, which builds into it as before) (save_what). The play file is read back at
|
||||||
|
// the next start; once it has reached kFoldMb (RX_VK_PCACHE_FOLD_MB) it is merged into the main cache at that start, the
|
||||||
|
// session builds into the main cache as before, its first save writes the main file and the play file is removed (fold_due).
|
||||||
|
// Without pipelineCreationCacheControl, or RX_VK_PCACHE_PLAY_FILE=0, there is no play cache: everything as before.
|
||||||
|
// A cache changes how long a pipeline takes to make, never the pipeline: nothing of this can change a picture.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace pcache_play {
|
||||||
|
|
||||||
|
inline constexpr uint64_t kFoldMb = 64; // (NX2navi's NX2NAVI_PIPELINE_CACHE_PLAY_MB default)
|
||||||
|
|
||||||
|
/// Whether a creation goes through the play cache: with the main cache (not none, not the gate's none), a play cache there,
|
||||||
|
/// and no preparation under way (it builds into the main cache)
|
||||||
|
inline bool routes(bool main_cache, bool play_cache, bool preparing) {
|
||||||
|
return main_cache && play_cache && !preparing;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// At the start, the play file read: whether it is folded into the main cache now (fold_bytes 0: never)
|
||||||
|
inline bool fold_due(uint64_t play_bytes, uint64_t fold_bytes) {
|
||||||
|
return fold_bytes && play_bytes >= fold_bytes;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a save writes: the play file (there is a play cache, its data's size changed since its last write) and the main
|
||||||
|
/// file (there is no play cache, or pipelines were compiled into the main cache since its last save)
|
||||||
|
struct SaveWhat {
|
||||||
|
bool play = false, main = false;
|
||||||
|
};
|
||||||
|
inline SaveWhat save_what(bool play_cache, uint64_t play_now, uint64_t play_saved, int64_t main_built) {
|
||||||
|
SaveWhat w;
|
||||||
|
w.play = play_cache && play_now && play_now != play_saved;
|
||||||
|
w.main = !play_cache || main_built > 0;
|
||||||
|
return w;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One creation (create(cache, info, out) makes it, as vkCreate*Pipelines with one info). route: routes(). fail_bit:
|
||||||
|
/// VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED_BIT, compile_required: VK_PIPELINE_COMPILE_REQUIRED. *into_play is set
|
||||||
|
/// when the play cache was asked (it made or answered the creation)
|
||||||
|
template <class Info, class Cache, class Pipeline, class Result, class Create>
|
||||||
|
Result create(bool route, Cache main, Cache play, const Info* ci, Pipeline* out, uint32_t fail_bit, Result compile_required,
|
||||||
|
Create make, bool* into_play) {
|
||||||
|
if (!route) return make(main, ci, out);
|
||||||
|
Info asked = *ci;
|
||||||
|
asked.flags |= fail_bit;
|
||||||
|
Result r = make(main, &asked, out);
|
||||||
|
if (r != compile_required) return r;
|
||||||
|
*out = Pipeline{};
|
||||||
|
if (into_play) *into_play = true;
|
||||||
|
return make(play, ci, out);
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace pcache_play
|
||||||
542
gpu/vk/perf_machine.cpp
Normal file
542
gpu/vk/perf_machine.cpp
Normal file
@@ -0,0 +1,542 @@
|
|||||||
|
// (Triforce) The performance overlay's machine numbers (perf_machine.h): a thread that samples them twice a second while
|
||||||
|
// the FPS counter row is on Full, and stops when it leaves Full. The OS's counters for the CPU, the memory and the GPU's
|
||||||
|
// load (Windows: GetSystemTimes, GetProcessTimes, GlobalMemoryStatusEx, K32GetProcessMemoryInfo, and PDH for the thermal
|
||||||
|
// zones and the GPU's engines and memory, as Task Manager reads them; Linux: /proc), and the GPU vendor's driver library
|
||||||
|
// for its temperature: NVML on NVIDIA, ADL's PM log on AMD, both loaded at run time from the system's folders
|
||||||
|
// and kept for the process while they have the GPU. Nothing of it is linked: on a PC without them (an Intel GPU, an old driver) their numbers stay
|
||||||
|
// -1 and the overlay shows "-"; a vendor library with no device of the rendering GPU is shut down and let go at once.
|
||||||
|
// Nothing here runs on the presenting thread but start, stop and latest, which only take the lock (start makes the
|
||||||
|
// thread); the thread runs at the lowest priority.
|
||||||
|
//
|
||||||
|
// (Triforce: from NX2navi runtime/perfmon.c (nvml_open, nvml_sample, adl_open, adl_sample, sampler, perfmon_mode),
|
||||||
|
// runtime/platform/win32.c (pdh_load, pdh_query, pdh_items, plat_load_open, plat_cpu_load, plat_gpu_load, plat_mem_info),
|
||||||
|
// runtime/platform/linux.c (plat_cpu_load, plat_mem_info) and runtime/sysinfo.c (sysinfo_cpu_name), at 62cc3b13.) Contains
|
||||||
|
// code Copyright (c) 2026 Revivedx and the Savage Wind Recomp contributors, Copyright (c) 2026 Peiikai (MIT), and AMD
|
||||||
|
// Display Library declarations (AdapterInfo, ADLSingleSensorData, ADLPMLogDataOutput, the ADL_PMLOG_* sensor numbers and
|
||||||
|
// the ADL2_* signatures) Copyright (c) 2016 - 2022 Advanced Micro Devices, Inc. (MIT): the notice is in perf_machine.h.
|
||||||
|
#include "perf_machine.h"
|
||||||
|
|
||||||
|
#include <chrono>
|
||||||
|
#include <condition_variable>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <mutex>
|
||||||
|
#include <string>
|
||||||
|
#include <system_error>
|
||||||
|
#include <thread>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
#ifdef _WIN32
|
||||||
|
#ifndef WIN32_LEAN_AND_MEAN
|
||||||
|
#define WIN32_LEAN_AND_MEAN
|
||||||
|
#endif
|
||||||
|
#ifndef NOMINMAX
|
||||||
|
#define NOMINMAX
|
||||||
|
#endif
|
||||||
|
#include <windows.h>
|
||||||
|
#include <pdh.h>
|
||||||
|
#include <pdhmsg.h>
|
||||||
|
#include <psapi.h>
|
||||||
|
#if defined(_M_X64) || defined(_M_IX86) || defined(__x86_64__) || defined(__i386__)
|
||||||
|
#include <intrin.h>
|
||||||
|
#define PERF_CPUID_MS 1
|
||||||
|
#endif
|
||||||
|
#else
|
||||||
|
#include <dlfcn.h>
|
||||||
|
#include <sys/resource.h>
|
||||||
|
#include <time.h>
|
||||||
|
#include <unistd.h>
|
||||||
|
#ifdef __linux__
|
||||||
|
#include <sys/syscall.h>
|
||||||
|
#endif
|
||||||
|
#if defined(__x86_64__) || defined(__i386__)
|
||||||
|
#include <cpuid.h>
|
||||||
|
#define PERF_CPUID_GNU 1
|
||||||
|
#endif
|
||||||
|
#endif
|
||||||
|
|
||||||
|
namespace mkwvk::perf_machine {
|
||||||
|
namespace {
|
||||||
|
|
||||||
|
void (*g_log)(const char*) = nullptr;
|
||||||
|
void note(const char* line) {
|
||||||
|
if (g_log) g_log(line);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- a library some PCs lack (a GPU vendor's): a bare name from the system's folders only
|
||||||
|
void* lib_open(const char* name) {
|
||||||
|
#ifdef _WIN32
|
||||||
|
const bool bare = !std::strchr(name, '\\') && !std::strchr(name, '/');
|
||||||
|
return (void*)LoadLibraryExA(name, nullptr, bare ? LOAD_LIBRARY_SEARCH_SYSTEM32 : 0);
|
||||||
|
#else
|
||||||
|
return dlopen(name, RTLD_NOW | RTLD_LOCAL);
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
void* lib_sym(void* lib, const char* name) {
|
||||||
|
#ifdef _WIN32
|
||||||
|
return lib ? (void*)GetProcAddress((HMODULE)lib, name) : nullptr;
|
||||||
|
#else
|
||||||
|
return lib ? dlsym(lib, name) : nullptr;
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
/// A library that has nothing for the rendering GPU, let go (NVML or ADL on a PC whose game renders on another card)
|
||||||
|
void lib_close(void* lib) {
|
||||||
|
#ifdef _WIN32
|
||||||
|
if (lib) FreeLibrary((HMODULE)lib);
|
||||||
|
#else
|
||||||
|
if (lib) dlclose(lib);
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- the CPU's name: CPUID's brand string, without the spaces around it
|
||||||
|
void cpu_brand(char* out, size_t n) {
|
||||||
|
out[0] = 0;
|
||||||
|
unsigned r[12] = {};
|
||||||
|
#if defined(PERF_CPUID_MS)
|
||||||
|
int top[4];
|
||||||
|
__cpuid(top, (int)0x80000000);
|
||||||
|
if ((unsigned)top[0] < 0x80000004u) return;
|
||||||
|
for (int i = 0; i < 3; i++) __cpuid((int*)&r[i * 4], (int)(0x80000002u + (unsigned)i));
|
||||||
|
#elif defined(PERF_CPUID_GNU)
|
||||||
|
if (__get_cpuid_max(0x80000000, nullptr) < 0x80000004u) return;
|
||||||
|
for (unsigned i = 0; i < 3; i++) __get_cpuid(0x80000002u + i, &r[i * 4], &r[i * 4 + 1], &r[i * 4 + 2], &r[i * 4 + 3]);
|
||||||
|
#else
|
||||||
|
(void)n;
|
||||||
|
return;
|
||||||
|
#endif
|
||||||
|
char b[49];
|
||||||
|
std::memcpy(b, r, 48);
|
||||||
|
b[48] = 0;
|
||||||
|
const char* p = b;
|
||||||
|
while (*p == ' ') p++;
|
||||||
|
snprintf(out, n, "%s", p);
|
||||||
|
for (char* e = out + std::strlen(out); e > out && e[-1] == ' '; e--) e[-1] = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- NVML (NVIDIA): its structs as the library lays them out, no NVIDIA header
|
||||||
|
struct NvmlUtil { unsigned gpu, memory; };
|
||||||
|
struct NvmlMem { unsigned long long total, free, used; };
|
||||||
|
struct Nvml {
|
||||||
|
bool tried = false, ok = false;
|
||||||
|
void* dev = nullptr;
|
||||||
|
int (*util)(void*, NvmlUtil*) = nullptr;
|
||||||
|
int (*temp)(void*, int, unsigned*) = nullptr;
|
||||||
|
int (*mem)(void*, NvmlMem*) = nullptr;
|
||||||
|
int (*clock)(void*, int, unsigned*) = nullptr;
|
||||||
|
} nv;
|
||||||
|
|
||||||
|
template <class F> F sym(void* lib, const char* name) { return (F)lib_sym(lib, name); }
|
||||||
|
|
||||||
|
/// Once, on the sampler's thread: the device whose name is the rendering GPU's, else none (a PC that renders on another
|
||||||
|
/// vendor's GPU: NVML's numbers would be another card's). With none NVML is shut down and let go: it is kept only for the
|
||||||
|
/// device it samples
|
||||||
|
void nvml_open(const char* gpu) {
|
||||||
|
nv.tried = true;
|
||||||
|
#ifdef _WIN32
|
||||||
|
static const char* const names[] = {"nvml.dll", "C:\\Program Files\\NVIDIA Corporation\\NVSMI\\nvml.dll"};
|
||||||
|
#else
|
||||||
|
static const char* const names[] = {"libnvidia-ml.so.1"};
|
||||||
|
#endif
|
||||||
|
void* lib = nullptr;
|
||||||
|
for (const char* n : names)
|
||||||
|
if (!lib) lib = lib_open(n);
|
||||||
|
if (!lib) return;
|
||||||
|
const auto init = sym<int (*)()>(lib, "nvmlInit_v2");
|
||||||
|
const auto count = sym<int (*)(unsigned*)>(lib, "nvmlDeviceGetCount_v2");
|
||||||
|
const auto handle = sym<int (*)(unsigned, void**)>(lib, "nvmlDeviceGetHandleByIndex_v2");
|
||||||
|
const auto name = sym<int (*)(void*, char*, unsigned)>(lib, "nvmlDeviceGetName");
|
||||||
|
nv.util = sym<int (*)(void*, NvmlUtil*)>(lib, "nvmlDeviceGetUtilizationRates");
|
||||||
|
nv.temp = sym<int (*)(void*, int, unsigned*)>(lib, "nvmlDeviceGetTemperature");
|
||||||
|
nv.mem = sym<int (*)(void*, NvmlMem*)>(lib, "nvmlDeviceGetMemoryInfo");
|
||||||
|
// ((Triforce, 11-17) no power reading (nvmlDeviceGetPowerUsage): the owner's call, power polling is reported to cost
|
||||||
|
// frames and stutter on some cards; the overlay's GPU row has no watts)
|
||||||
|
nv.clock = sym<int (*)(void*, int, unsigned*)>(lib, "nvmlDeviceGetClockInfo");
|
||||||
|
const auto shutdown = sym<int (*)()>(lib, "nvmlShutdown");
|
||||||
|
const bool up = init && count && handle && name && init() == 0;
|
||||||
|
unsigned n = 0;
|
||||||
|
if (up && count(&n) == 0)
|
||||||
|
for (unsigned i = 0; i < n && !nv.dev; i++) {
|
||||||
|
void* d = nullptr;
|
||||||
|
char nm[96] = "";
|
||||||
|
if (handle(i, &d) != 0 || name(d, nm, sizeof nm) != 0) continue;
|
||||||
|
if (name_match(gpu, nm)) nv.dev = d;
|
||||||
|
}
|
||||||
|
nv.ok = nv.dev != nullptr;
|
||||||
|
if (up) note(nv.ok ? "NVML has the rendering GPU: its temperature" : "NVML has no device of the rendering GPU's name");
|
||||||
|
if (nv.ok) return;
|
||||||
|
if (up && shutdown) shutdown(); // (nvmlInit's own count of users given back, then the library)
|
||||||
|
lib_close(lib);
|
||||||
|
nv.util = nullptr, nv.temp = nullptr, nv.mem = nullptr, nv.clock = nullptr;
|
||||||
|
}
|
||||||
|
void nvml_sample(Machine& m) {
|
||||||
|
if (!nv.ok) return;
|
||||||
|
NvmlUtil u;
|
||||||
|
NvmlMem mem;
|
||||||
|
unsigned v;
|
||||||
|
if (m.gpu_pct < 0 && nv.util && nv.util(nv.dev, &u) == 0) m.gpu_pct = (float)u.gpu; // (the system's counters first)
|
||||||
|
if (nv.mem && nv.mem(nv.dev, &mem) == 0) {
|
||||||
|
if (m.vram_used_mb < 0) m.vram_used_mb = (float)(mem.used >> 20);
|
||||||
|
m.vram_total_mb = (float)(mem.total >> 20);
|
||||||
|
}
|
||||||
|
if (nv.temp && nv.temp(nv.dev, 0 /* NVML_TEMPERATURE_GPU */, &v) == 0) m.gpu_temp_c = (float)v;
|
||||||
|
if (nv.clock && nv.clock(nv.dev, 0 /* NVML_CLOCK_GRAPHICS */, &v) == 0) m.gpu_clock_mhz = (float)v;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- ADL (AMD): the PM log the driver keeps for Radeon RX 5000 and later (the numbers Adrenalin's overlay shows); an
|
||||||
|
// older GPU answers with an error and its numbers stay -1. Windows only
|
||||||
|
#ifdef _WIN32
|
||||||
|
constexpr int kAdlMaxPath = 256, kAdlSensors = 256;
|
||||||
|
enum {
|
||||||
|
ADL_PMLOG_CLK_GFXCLK = 1,
|
||||||
|
ADL_PMLOG_TEMPERATURE_EDGE = 8,
|
||||||
|
ADL_PMLOG_INFO_ACTIVITY_GFX = 19,
|
||||||
|
ADL_PMLOG_TEMPERATURE_HOTSPOT = 27,
|
||||||
|
};
|
||||||
|
struct AdlAdapterInfo { // (AdapterInfo, the Windows layout)
|
||||||
|
int iSize, iAdapterIndex;
|
||||||
|
char strUDID[kAdlMaxPath];
|
||||||
|
int iBusNumber, iDeviceNumber, iFunctionNumber, iVendorID;
|
||||||
|
char strAdapterName[kAdlMaxPath], strDisplayName[kAdlMaxPath];
|
||||||
|
int iPresent, iExist;
|
||||||
|
char strDriverPath[kAdlMaxPath], strDriverPathExt[kAdlMaxPath], strPNPString[kAdlMaxPath];
|
||||||
|
int iOSDisplayIndex;
|
||||||
|
};
|
||||||
|
struct AdlSensor { int supported, value; };
|
||||||
|
struct AdlPmLog { int size; AdlSensor sensors[kAdlSensors]; };
|
||||||
|
typedef void*(__stdcall* AdlAlloc)(int);
|
||||||
|
struct Adl {
|
||||||
|
bool tried = false, ok = false;
|
||||||
|
int index = 0;
|
||||||
|
void* ctx = nullptr;
|
||||||
|
int (*pmlog)(void*, int, AdlPmLog*) = nullptr;
|
||||||
|
AdlPmLog log; // (the sampler's buffer)
|
||||||
|
} adl;
|
||||||
|
void* __stdcall adl_alloc(int n) { return std::malloc((size_t)n); }
|
||||||
|
|
||||||
|
/// Once, on the sampler's thread, when NVML has not the GPU: the adapter of the rendering GPU's name whose PM log answers
|
||||||
|
/// (ADL lists one index per display output of a GPU). With none ADL's context is destroyed and the library let go
|
||||||
|
void adl_open(const char* gpu) {
|
||||||
|
adl.tried = true;
|
||||||
|
void* lib = lib_open("atiadlxx.dll");
|
||||||
|
if (!lib) return;
|
||||||
|
const auto create = sym<int (*)(AdlAlloc, int, void**)>(lib, "ADL2_Main_Control_Create");
|
||||||
|
const auto destroy = sym<int (*)(void*)>(lib, "ADL2_Main_Control_Destroy");
|
||||||
|
const auto count = sym<int (*)(void*, int*)>(lib, "ADL2_Adapter_NumberOfAdapters_Get");
|
||||||
|
const auto info = sym<int (*)(void*, AdlAdapterInfo*, int)>(lib, "ADL2_Adapter_AdapterInfo_Get");
|
||||||
|
adl.pmlog = sym<int (*)(void*, int, AdlPmLog*)>(lib, "ADL2_New_QueryPMLogData_Get");
|
||||||
|
int n = 0;
|
||||||
|
const bool up = create && count && info && adl.pmlog && create(adl_alloc, 1, &adl.ctx) == 0;
|
||||||
|
AdlAdapterInfo* a = up && count(adl.ctx, &n) == 0 && n > 0 && n <= 256 ? (AdlAdapterInfo*)std::calloc((size_t)n, sizeof *a) : nullptr;
|
||||||
|
if (a) {
|
||||||
|
for (int i = 0; i < n; i++) a[i].iSize = sizeof *a;
|
||||||
|
if (info(adl.ctx, a, n * (int)sizeof *a) == 0) {
|
||||||
|
for (int i = 0; i < n && !adl.ok; i++) {
|
||||||
|
if (a[i].iVendorID != 1002 || !name_match(gpu, a[i].strAdapterName)) continue; // (ADL writes 0x1002 as decimal 1002)
|
||||||
|
std::memset(&adl.log, 0, sizeof adl.log);
|
||||||
|
adl.log.size = sizeof adl.log;
|
||||||
|
if (adl.pmlog(adl.ctx, a[i].iAdapterIndex, &adl.log) != 0) continue;
|
||||||
|
adl.index = a[i].iAdapterIndex;
|
||||||
|
adl.ok = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
std::free(a);
|
||||||
|
}
|
||||||
|
if (up) note(adl.ok ? "ADL has the rendering GPU's PM log: its temperature" : "ADL has no PM log for the rendering GPU");
|
||||||
|
if (adl.ok) return;
|
||||||
|
if (up && destroy) destroy(adl.ctx); // (the context ADL2_Main_Control_Create made, then the library)
|
||||||
|
adl.ctx = nullptr, adl.pmlog = nullptr;
|
||||||
|
lib_close(lib);
|
||||||
|
}
|
||||||
|
void adl_sample(Machine& m) {
|
||||||
|
if (!adl.ok) return;
|
||||||
|
std::memset(&adl.log, 0, sizeof adl.log);
|
||||||
|
adl.log.size = sizeof adl.log;
|
||||||
|
if (adl.pmlog(adl.ctx, adl.index, &adl.log) != 0) return;
|
||||||
|
const AdlSensor* v = adl.log.sensors;
|
||||||
|
if (m.gpu_pct < 0 && v[ADL_PMLOG_INFO_ACTIVITY_GFX].supported) m.gpu_pct = (float)v[ADL_PMLOG_INFO_ACTIVITY_GFX].value;
|
||||||
|
if (v[ADL_PMLOG_TEMPERATURE_EDGE].supported) m.gpu_temp_c = (float)v[ADL_PMLOG_TEMPERATURE_EDGE].value;
|
||||||
|
else if (v[ADL_PMLOG_TEMPERATURE_HOTSPOT].supported) m.gpu_temp_c = (float)v[ADL_PMLOG_TEMPERATURE_HOTSPOT].value;
|
||||||
|
// ((Triforce, 11-17) the card's power draw is not read: see nvml_open)
|
||||||
|
if (v[ADL_PMLOG_CLK_GFXCLK].supported) m.gpu_clock_mhz = (float)v[ADL_PMLOG_CLK_GFXCLK].value;
|
||||||
|
}
|
||||||
|
#else
|
||||||
|
struct Adl { bool tried = false, ok = false; } adl;
|
||||||
|
void adl_open(const char*) { adl.tried = true; }
|
||||||
|
void adl_sample(Machine&) {}
|
||||||
|
#endif
|
||||||
|
|
||||||
|
// ---- the OS's counters
|
||||||
|
#ifdef _WIN32
|
||||||
|
typedef PDH_STATUS(WINAPI* PdhOpenQueryW_t)(LPCWSTR, DWORD_PTR, PDH_HQUERY*);
|
||||||
|
typedef PDH_STATUS(WINAPI* PdhAddEnglishCounterW_t)(PDH_HQUERY, LPCWSTR, DWORD_PTR, PDH_HCOUNTER*);
|
||||||
|
typedef PDH_STATUS(WINAPI* PdhCollectQueryData_t)(PDH_HQUERY);
|
||||||
|
typedef PDH_STATUS(WINAPI* PdhGetFormattedCounterArrayW_t)(PDH_HCOUNTER, DWORD, LPDWORD, LPDWORD, PPDH_FMT_COUNTERVALUE_ITEM_W);
|
||||||
|
typedef PDH_STATUS(WINAPI* PdhCloseQuery_t)(PDH_HQUERY);
|
||||||
|
struct Pdh {
|
||||||
|
PdhOpenQueryW_t open = nullptr; // (set only when every one of them is there)
|
||||||
|
PdhAddEnglishCounterW_t add = nullptr;
|
||||||
|
PdhCollectQueryData_t collect = nullptr;
|
||||||
|
PdhGetFormattedCounterArrayW_t get = nullptr;
|
||||||
|
PdhCloseQuery_t close = nullptr;
|
||||||
|
};
|
||||||
|
const Pdh& pdh() { // (pdh.dll, loaded at the first use, kept)
|
||||||
|
static const Pdh p = [] {
|
||||||
|
Pdh r;
|
||||||
|
void* l = lib_open("pdh.dll");
|
||||||
|
if (!l) return r;
|
||||||
|
r.add = sym<PdhAddEnglishCounterW_t>(l, "PdhAddEnglishCounterW");
|
||||||
|
r.collect = sym<PdhCollectQueryData_t>(l, "PdhCollectQueryData");
|
||||||
|
r.get = sym<PdhGetFormattedCounterArrayW_t>(l, "PdhGetFormattedCounterArrayW");
|
||||||
|
r.close = sym<PdhCloseQuery_t>(l, "PdhCloseQuery");
|
||||||
|
const auto open = sym<PdhOpenQueryW_t>(l, "PdhOpenQueryW");
|
||||||
|
if (r.add && r.collect && r.get && r.close) r.open = open;
|
||||||
|
return r;
|
||||||
|
}();
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
/// A PDH query with its wildcard counters ("\Object(*)\Counter"); q null when it could not be opened
|
||||||
|
struct PdhQuery {
|
||||||
|
PDH_HQUERY q = nullptr;
|
||||||
|
PDH_HCOUNTER c[2] = {nullptr, nullptr};
|
||||||
|
void open(const wchar_t* c0, const wchar_t* c1) {
|
||||||
|
const Pdh& p = pdh();
|
||||||
|
if (!p.open || p.open(nullptr, 0, &q) != ERROR_SUCCESS) {
|
||||||
|
q = nullptr;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (p.add(q, c0, 0, &c[0]) != ERROR_SUCCESS) c[0] = nullptr;
|
||||||
|
if (!c1 || p.add(q, c1, 0, &c[1]) != ERROR_SUCCESS) c[1] = nullptr;
|
||||||
|
if (!c[0] && !c[1]) close();
|
||||||
|
}
|
||||||
|
void close() {
|
||||||
|
if (q) pdh().close(q);
|
||||||
|
q = nullptr;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
/// A counter's valid instances: their names (narrowed: the instance names are ASCII) and values. False when there is no
|
||||||
|
/// valid data yet (a rate counter needs two collections)
|
||||||
|
struct Items {
|
||||||
|
std::vector<std::string> names;
|
||||||
|
std::vector<const char*> ptrs;
|
||||||
|
std::vector<double> values;
|
||||||
|
};
|
||||||
|
bool pdh_items(PDH_HCOUNTER c, Items& out) {
|
||||||
|
out.names.clear(), out.ptrs.clear(), out.values.clear();
|
||||||
|
DWORD size = 0, count = 0;
|
||||||
|
const Pdh& p = pdh();
|
||||||
|
if (!c || p.get(c, PDH_FMT_DOUBLE | PDH_FMT_NOCAP100, &size, &count, nullptr) != (PDH_STATUS)PDH_MORE_DATA || !size) return false;
|
||||||
|
std::vector<unsigned char> buf(size);
|
||||||
|
auto* items = (PDH_FMT_COUNTERVALUE_ITEM_W*)buf.data();
|
||||||
|
if (p.get(c, PDH_FMT_DOUBLE | PDH_FMT_NOCAP100, &size, &count, items) != ERROR_SUCCESS) return false;
|
||||||
|
for (DWORD i = 0; i < count; i++) {
|
||||||
|
if (items[i].FmtValue.CStatus != PDH_CSTATUS_VALID_DATA) continue;
|
||||||
|
std::string n;
|
||||||
|
for (const wchar_t* w = items[i].szName; w && *w; w++) n += (char)(*w < 128 ? *w : '?');
|
||||||
|
out.names.push_back(std::move(n));
|
||||||
|
out.values.push_back(items[i].FmtValue.doubleValue);
|
||||||
|
}
|
||||||
|
for (const std::string& n : out.names) out.ptrs.push_back(n.c_str());
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
uint64_t ft64(FILETIME f) { return (uint64_t)f.dwHighDateTime << 32 | f.dwLowDateTime; }
|
||||||
|
|
||||||
|
struct Sources {
|
||||||
|
CpuTimes cpu; // (the sample before's)
|
||||||
|
PdhQuery thermal, gpu; // (gpu: the engines' utilisation, the adapters' dedicated memory)
|
||||||
|
char luid[32] = ""; // (the rendering GPU's, luid_name; empty: any)
|
||||||
|
Items items;
|
||||||
|
};
|
||||||
|
void sources_open(Sources& s, const Gpu& g) {
|
||||||
|
if (g.has_luid) luid_name(g.luid, s.luid, sizeof s.luid);
|
||||||
|
s.thermal.open(L"\\Thermal Zone Information(*)\\Temperature", nullptr);
|
||||||
|
s.gpu.open(L"\\GPU Engine(*)\\Utilization Percentage", L"\\GPU Adapter Memory(*)\\Dedicated Usage");
|
||||||
|
char line[160];
|
||||||
|
snprintf(line, sizeof line, "Windows' counters: GPU engines %s (%s), thermal zones %s", s.gpu.q ? "open" : "not there",
|
||||||
|
s.luid[0] ? s.luid : "any adapter", s.thermal.q ? "open" : "not there");
|
||||||
|
note(line);
|
||||||
|
}
|
||||||
|
void sources_close(Sources& s) { s.thermal.close(), s.gpu.close(); }
|
||||||
|
void sample_os(Sources& s, Machine& m) {
|
||||||
|
FILETIME idle, kern, user, cr, ex, pk, pu;
|
||||||
|
if (GetSystemTimes(&idle, &kern, &user) && GetProcessTimes(GetCurrentProcess(), &cr, &ex, &pk, &pu)) {
|
||||||
|
CpuTimes now;
|
||||||
|
now.idle = ft64(idle), now.kernel = ft64(kern), now.user = ft64(user), now.proc = ft64(pk) + ft64(pu);
|
||||||
|
cpu_load(s.cpu, now, &m.cpu_pct, &m.cpu_game_pct);
|
||||||
|
s.cpu = now;
|
||||||
|
}
|
||||||
|
const Pdh& p = pdh();
|
||||||
|
// (the firmware's ACPI zones, near the CPU on most boards: the hottest under 150 degrees, from kelvin)
|
||||||
|
if (s.thermal.q && p.collect(s.thermal.q) == ERROR_SUCCESS && pdh_items(s.thermal.c[0], s.items))
|
||||||
|
for (double k : s.items.values) {
|
||||||
|
const double c = k - 273.15;
|
||||||
|
if (c > m.cpu_temp_c && c < 150) m.cpu_temp_c = (float)c;
|
||||||
|
}
|
||||||
|
if (s.gpu.q && p.collect(s.gpu.q) == ERROR_SUCCESS) {
|
||||||
|
if (pdh_items(s.gpu.c[0], s.items))
|
||||||
|
m.gpu_pct = engine_load(s.items.ptrs.data(), s.items.values.data(), (int)s.items.values.size(), s.luid);
|
||||||
|
if (pdh_items(s.gpu.c[1], s.items))
|
||||||
|
m.vram_used_mb = adapter_memory_mb(s.items.ptrs.data(), s.items.values.data(), (int)s.items.values.size(), s.luid);
|
||||||
|
}
|
||||||
|
MEMORYSTATUSEX ms{};
|
||||||
|
ms.dwLength = sizeof ms;
|
||||||
|
if (GlobalMemoryStatusEx(&ms) && ms.ullTotalPhys) {
|
||||||
|
m.ram_total_mb = (float)(ms.ullTotalPhys >> 20);
|
||||||
|
m.ram_used_mb = (float)((ms.ullTotalPhys - ms.ullAvailPhys) >> 20);
|
||||||
|
}
|
||||||
|
PROCESS_MEMORY_COUNTERS pmc{};
|
||||||
|
pmc.cb = sizeof pmc;
|
||||||
|
if (K32GetProcessMemoryInfo(GetCurrentProcess(), &pmc, sizeof pmc)) m.ram_game_mb = (float)(pmc.WorkingSetSize >> 20);
|
||||||
|
}
|
||||||
|
#else
|
||||||
|
struct Sources {
|
||||||
|
CpuTimes cpu; // (/proc/stat's busy jiffies in kernel's place with idle 0, its total in user's; the process's in jiffies)
|
||||||
|
};
|
||||||
|
void sources_open(Sources&, const Gpu&) {}
|
||||||
|
void sources_close(Sources&) {}
|
||||||
|
void sample_os(Sources& s, Machine& m) {
|
||||||
|
unsigned long long v[10] = {};
|
||||||
|
int n = 0;
|
||||||
|
if (FILE* f = std::fopen("/proc/stat", "r")) {
|
||||||
|
n = std::fscanf(f, "cpu %llu %llu %llu %llu %llu %llu %llu %llu %llu %llu", &v[0], &v[1], &v[2], &v[3], &v[4], &v[5],
|
||||||
|
&v[6], &v[7], &v[8], &v[9]);
|
||||||
|
std::fclose(f);
|
||||||
|
}
|
||||||
|
timespec ts{};
|
||||||
|
clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &ts);
|
||||||
|
const long tick = sysconf(_SC_CLK_TCK);
|
||||||
|
if (n >= 8 && tick > 0) {
|
||||||
|
uint64_t total = 0;
|
||||||
|
for (int i = 0; i < 8; i++) total += v[i]; // (guest time is already inside user time)
|
||||||
|
const uint64_t idle = v[3] + v[4];
|
||||||
|
CpuTimes now; // (cpu_load's terms: kernel + user the total, idle the idle)
|
||||||
|
now.kernel = total, now.user = 0, now.idle = idle;
|
||||||
|
now.proc = (uint64_t)(((double)ts.tv_sec + ts.tv_nsec / 1e9) * (double)tick);
|
||||||
|
cpu_load(s.cpu, now, &m.cpu_pct, &m.cpu_game_pct);
|
||||||
|
s.cpu = now;
|
||||||
|
}
|
||||||
|
if (FILE* f = std::fopen("/proc/meminfo", "r")) {
|
||||||
|
char line[128];
|
||||||
|
unsigned long long total = 0, avail = 0, k;
|
||||||
|
while (std::fgets(line, sizeof line, f)) {
|
||||||
|
if (std::sscanf(line, "MemTotal: %llu kB", &k) == 1) total = k;
|
||||||
|
else if (std::sscanf(line, "MemAvailable: %llu kB", &k) == 1) avail = k;
|
||||||
|
}
|
||||||
|
std::fclose(f);
|
||||||
|
if (total) m.ram_total_mb = (float)(total >> 10), m.ram_used_mb = (float)((total - std::min(total, avail)) >> 10);
|
||||||
|
}
|
||||||
|
if (FILE* f = std::fopen("/proc/self/statm", "r")) {
|
||||||
|
unsigned long long size, resident;
|
||||||
|
if (std::fscanf(f, "%llu %llu", &size, &resident) == 2)
|
||||||
|
m.ram_game_mb = (float)((resident * (uint64_t)sysconf(_SC_PAGESIZE)) >> 20);
|
||||||
|
std::fclose(f);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#endif
|
||||||
|
|
||||||
|
// ---- the sampler: runs while want is set, twice a second
|
||||||
|
struct State {
|
||||||
|
std::mutex mu; // (everything here)
|
||||||
|
std::condition_variable cv;
|
||||||
|
bool want = false, alive = false;
|
||||||
|
int live = 0; // (the sampler threads made and not yet ended: live_samplers)
|
||||||
|
Gpu gpu;
|
||||||
|
Machine latest;
|
||||||
|
};
|
||||||
|
State& st() { // (never destroyed: a sampler still asleep at the process's end holds its lock)
|
||||||
|
static State* s = new State;
|
||||||
|
return *s;
|
||||||
|
}
|
||||||
|
void clear(Machine& m) { // (the numbers, not the names)
|
||||||
|
Machine e;
|
||||||
|
std::memcpy(e.gpu_name, m.gpu_name, sizeof e.gpu_name);
|
||||||
|
std::memcpy(e.cpu_name, m.cpu_name, sizeof e.cpu_name);
|
||||||
|
e.samples = m.samples;
|
||||||
|
m = e;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The sampler's thread below the game's own (the replay's and the presenting threads): its counters can wait in the driver
|
||||||
|
void lowest_priority() {
|
||||||
|
#ifdef _WIN32
|
||||||
|
SetThreadPriority(GetCurrentThread(), THREAD_PRIORITY_LOWEST);
|
||||||
|
#elif defined(__linux__)
|
||||||
|
setpriority(PRIO_PROCESS, (id_t)syscall(SYS_gettid), 19); // (Linux: a nice of a thread's own, by its id)
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
|
void sampler() {
|
||||||
|
lowest_priority();
|
||||||
|
State& s = st();
|
||||||
|
Gpu gpu;
|
||||||
|
{
|
||||||
|
std::lock_guard lk(s.mu);
|
||||||
|
gpu = s.gpu;
|
||||||
|
}
|
||||||
|
Machine m;
|
||||||
|
snprintf(m.gpu_name, sizeof m.gpu_name, "%s", gpu.name);
|
||||||
|
cpu_brand(m.cpu_name, sizeof m.cpu_name);
|
||||||
|
if (!nv.tried) nvml_open(gpu.name);
|
||||||
|
if (!nv.ok && !adl.tried) adl_open(gpu.name);
|
||||||
|
Sources src;
|
||||||
|
sources_open(src, gpu);
|
||||||
|
std::unique_lock lk(s.mu);
|
||||||
|
while (s.want) {
|
||||||
|
lk.unlock();
|
||||||
|
clear(m);
|
||||||
|
sample_os(src, m);
|
||||||
|
if (gpu.vram_total_mb > 0) m.vram_total_mb = gpu.vram_total_mb;
|
||||||
|
nvml_sample(m);
|
||||||
|
adl_sample(m);
|
||||||
|
m.samples++;
|
||||||
|
lk.lock();
|
||||||
|
s.latest = m;
|
||||||
|
if (s.want) s.cv.wait_for(lk, std::chrono::milliseconds(500), [&] { return !s.want; });
|
||||||
|
}
|
||||||
|
s.alive = false; // (in the same locked part as the last look at want: a start after it makes a new thread)
|
||||||
|
lk.unlock();
|
||||||
|
sources_close(src);
|
||||||
|
lk.lock();
|
||||||
|
s.live--; // (this thread's last step: live_samplers)
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
void start(const Gpu& gpu, void (*log)(const char* line)) {
|
||||||
|
State& s = st();
|
||||||
|
std::lock_guard lk(s.mu);
|
||||||
|
s.want = true;
|
||||||
|
if (s.alive) return;
|
||||||
|
g_log = log;
|
||||||
|
s.gpu = gpu;
|
||||||
|
s.latest = Machine{}; // ("-" until the first sample, not the numbers of the time before)
|
||||||
|
s.alive = true;
|
||||||
|
s.live++;
|
||||||
|
// (a thread the system will not make: no sampler, and the panel's machine numbers stay "-" (latest, emptied above). Never
|
||||||
|
// the exception, which would reach menu_tick on the game's thread; the next start tries again)
|
||||||
|
try {
|
||||||
|
std::thread(sampler).detach();
|
||||||
|
} catch (const std::system_error&) {
|
||||||
|
s.alive = false;
|
||||||
|
s.live--;
|
||||||
|
note("its sampler's thread could not be made: the machine's numbers stay \"-\"");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void stop() {
|
||||||
|
State& s = st();
|
||||||
|
std::lock_guard lk(s.mu);
|
||||||
|
s.want = false;
|
||||||
|
s.cv.notify_all();
|
||||||
|
}
|
||||||
|
|
||||||
|
Machine latest() {
|
||||||
|
State& s = st();
|
||||||
|
std::lock_guard lk(s.mu);
|
||||||
|
return s.latest;
|
||||||
|
}
|
||||||
|
|
||||||
|
int live_samplers() {
|
||||||
|
State& s = st();
|
||||||
|
std::lock_guard lk(s.mu);
|
||||||
|
return s.live;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::perf_machine
|
||||||
177
gpu/vk/perf_machine.h
Normal file
177
gpu/vk/perf_machine.h
Normal file
@@ -0,0 +1,177 @@
|
|||||||
|
// (Triforce) The performance overlay's machine numbers (vk_menu.cpp draw_perf, the FPS counter row on Full): the GPU's
|
||||||
|
// load and temperature (no power: removed 11-17, the owner's call), the video memory the card has in use and in all, the CPU's load, temperature and this
|
||||||
|
// game's share of it, the system memory in use, in all and the game's, and the GPU's and the CPU's names. Sampled twice a
|
||||||
|
// second on a thread of their own (perf_machine.cpp) while the row is on Full, since some of the calls take milliseconds:
|
||||||
|
// the presenting thread only reads the latest sample. A number the machine does not give is -1, and the overlay shows "-"
|
||||||
|
// (an AMD or Intel card has no NVIDIA library, an old Radeon no PM log, a board no thermal zone, Linux no temperatures).
|
||||||
|
//
|
||||||
|
// (Triforce: from NX2navi runtime/perfmon.c, runtime/perfmon.h, runtime/platform/win32.c plat_cpu_load, plat_gpu_load and
|
||||||
|
// plat_mem_info, runtime/platform/linux.c and runtime/sysinfo.c sysinfo_cpu_name, at 62cc3b13.) That code contains code
|
||||||
|
// Copyright (c) 2026 Revivedx and the Savage Wind Recomp contributors, Copyright (c) 2026 Peiikai (MIT: the performance
|
||||||
|
// overlay of github.com/Revivedx/SavageWindRecomp, its NVML calls and sampling loop), and its AMD Display Library
|
||||||
|
// declarations Copyright (c) 2016 - 2022 Advanced Micro Devices, Inc. (MIT: github.com/GPUOpen-LibrariesAndSDKs/display-library).
|
||||||
|
// Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
|
||||||
|
// documentation files (the "Software"), to deal in the Software without restriction, including without limitation the
|
||||||
|
// rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit
|
||||||
|
// persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and
|
||||||
|
// this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS
|
||||||
|
// PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
||||||
|
// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||||
|
// HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
|
||||||
|
// ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||||
|
//
|
||||||
|
// The parts below with no clock and no system call of their own (cpu_load, luid_name, engine_load, adapter_memory_mb,
|
||||||
|
// cell_text, local_heap_mb, name_match) are what tests/test_perf_overlay.cpp runs; it runs the sampler too, for a GPU of no
|
||||||
|
// vendor's name.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstddef>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdio>
|
||||||
|
#include <cstring>
|
||||||
|
|
||||||
|
namespace mkwvk::perf_machine {
|
||||||
|
|
||||||
|
struct Machine {
|
||||||
|
char gpu_name[96] = ""; // (the GPU the game renders on, as Vulkan names it)
|
||||||
|
char cpu_name[64] = ""; // (the CPU's brand string, CPUID)
|
||||||
|
float gpu_pct = -1; // (the busiest engine type of the rendering GPU, as Task Manager shows it; else NVML's or ADL's)
|
||||||
|
float gpu_temp_c = -1; // (NVML; ADL's edge temperature, else its hotspot)
|
||||||
|
float vram_used_mb = -1; // (the card's dedicated memory in use by every program: Windows' adapter counter, else NVML)
|
||||||
|
float vram_total_mb = -1; // (the card's: NVML, else the device-local heaps Vulkan reports, added up)
|
||||||
|
float gpu_clock_mhz = -1; // (the graphics clock: NVML, else ADL's; the VRAM row's third cell when the game's figure is unknown)
|
||||||
|
float cpu_pct = -1; // (the whole machine, percent of all logical processors, since the sample before)
|
||||||
|
float cpu_game_pct = -1; // (this process, the same percent)
|
||||||
|
float cpu_temp_c = -1; // (the hottest ACPI thermal zone: a zone near the CPU on most boards, not the package itself)
|
||||||
|
float ram_used_mb = -1, ram_total_mb = -1, ram_game_mb = -1; // (in use, installed, this process's working set)
|
||||||
|
int samples = 0; // (made since the sampler started: 0 until the first)
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The GPU the game renders on, given when the sampler starts
|
||||||
|
struct Gpu {
|
||||||
|
char name[96] = "";
|
||||||
|
uint8_t luid[8] = {}; // (Vulkan's deviceLUID: Windows' counters name an adapter by it)
|
||||||
|
bool has_luid = false;
|
||||||
|
float vram_total_mb = -1;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Starts the sampler (the row went to Full) unless it runs; gpu is read then. log, or null: a line on what was found
|
||||||
|
/// (NVML, ADL, the counters), once a run
|
||||||
|
void start(const Gpu& gpu, void (*log)(const char* line));
|
||||||
|
/// Stops it (the row left Full): its thread ends at its next wake, within a moment
|
||||||
|
void stop();
|
||||||
|
/// The latest sample (all -1 before the first): a copy, under the sampler's lock for as long as the copy takes
|
||||||
|
Machine latest();
|
||||||
|
/// (for tests/test_perf_overlay.cpp) The sampler threads alive now: 1 while it runs, 0 once it has ended (a second start
|
||||||
|
/// while one runs makes no second)
|
||||||
|
int live_samplers();
|
||||||
|
|
||||||
|
// ---- the parts with no clock and no system call ----
|
||||||
|
|
||||||
|
/// The CPU's times as GetSystemTimes and GetProcessTimes give them (100 ns units; kernel includes idle), or /proc/stat's
|
||||||
|
/// (busy and total jiffies in kernel and idle's place) with the process's in the same unit
|
||||||
|
struct CpuTimes {
|
||||||
|
uint64_t idle = 0, kernel = 0, user = 0, proc = 0;
|
||||||
|
};
|
||||||
|
/// The machine's and this process's share of all logical processors between two readings, percent; -1 when there is no
|
||||||
|
/// reading before (before.kernel 0) or no time went by
|
||||||
|
inline void cpu_load(const CpuTimes& before, const CpuTimes& now, float* total_pct, float* game_pct) {
|
||||||
|
*total_pct = *game_pct = -1;
|
||||||
|
const uint64_t total = (now.kernel - before.kernel) + (now.user - before.user); // (kernel time includes idle time)
|
||||||
|
if (!before.kernel || !total || now.kernel < before.kernel || now.user < before.user) return;
|
||||||
|
const uint64_t idle = now.idle >= before.idle ? now.idle - before.idle : 0;
|
||||||
|
*total_pct = 100.0f * (float)(total - std::min(total, idle)) / (float)total;
|
||||||
|
*game_pct = 100.0f * (float)(now.proc >= before.proc ? now.proc - before.proc : 0) / (float)total;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The adapter's part of a counter's instance names, from Vulkan's deviceLUID (the LUID struct's memory: LowPart, then
|
||||||
|
/// HighPart): "luid_0x<high>_0x<low>", lower case
|
||||||
|
inline void luid_name(const uint8_t luid[8], char* out, size_t n) {
|
||||||
|
uint32_t lo, hi;
|
||||||
|
std::memcpy(&lo, luid, 4);
|
||||||
|
std::memcpy(&hi, luid + 4, 4);
|
||||||
|
snprintf(out, n, "luid_0x%08x_0x%08x", hi, lo);
|
||||||
|
}
|
||||||
|
/// The "luid_0x..._0x..." part of an instance name, lower case, into out (27 bytes at least); empty when it has none
|
||||||
|
inline void instance_luid(const char* name, char* out) {
|
||||||
|
const char* p = std::strstr(name, "luid_");
|
||||||
|
size_t n = 0;
|
||||||
|
while (p && p[n] && n < 26) out[n] = (char)(p[n] >= 'A' && p[n] <= 'Z' ? p[n] + 32 : p[n]), n++; // (luid_0x%08x_0x%08x: 26)
|
||||||
|
out[n] = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The GPU's load from "\GPU Engine(*)\Utilization Percentage": instances "pid_<n>_luid_0x.._0x.._phys_<n>_eng_<n>_engtype_<type>"
|
||||||
|
/// with each process's share of an engine. Per adapter and engine type the shares add up; the load is the busiest type,
|
||||||
|
/// 100 at most. luid: the adapter's (luid_name), or empty for any. -1 when no instance counts
|
||||||
|
inline float engine_load(const char* const* names, const double* values, int n, const char* luid) {
|
||||||
|
enum { kTypes = 32 };
|
||||||
|
struct { char luid[32], type[32]; double sum; } t[kTypes];
|
||||||
|
int nt = 0;
|
||||||
|
for (int i = 0; i < n; i++) {
|
||||||
|
char lu[32];
|
||||||
|
instance_luid(names[i], lu);
|
||||||
|
if (luid[0] && std::strcmp(lu, luid)) continue;
|
||||||
|
const char* ty = std::strstr(names[i], "engtype_");
|
||||||
|
ty = ty ? ty + 8 : "";
|
||||||
|
int k = 0;
|
||||||
|
while (k < nt && (std::strcmp(t[k].luid, lu) || std::strncmp(t[k].type, ty, 31))) k++;
|
||||||
|
if (k == nt) {
|
||||||
|
if (nt == kTypes) continue;
|
||||||
|
std::memcpy(t[k].luid, lu, sizeof lu);
|
||||||
|
std::strncpy(t[k].type, ty, 31);
|
||||||
|
t[k].type[31] = 0;
|
||||||
|
t[k].sum = 0;
|
||||||
|
nt++;
|
||||||
|
}
|
||||||
|
t[k].sum += values[i];
|
||||||
|
}
|
||||||
|
float load = -1;
|
||||||
|
for (int k = 0; k < nt; k++)
|
||||||
|
if (t[k].sum > load) load = (float)(t[k].sum > 100 ? 100 : t[k].sum);
|
||||||
|
return load;
|
||||||
|
}
|
||||||
|
/// The adapter's dedicated memory in use from "\GPU Adapter Memory(*)\Dedicated Usage" (instances
|
||||||
|
/// "luid_0x.._0x.._phys_<n>", bytes), MB: the instances of luid added up (-1 when they come to nothing), or with luid
|
||||||
|
/// empty the adapter that uses the most (-1 when there is none)
|
||||||
|
inline float adapter_memory_mb(const char* const* names, const double* values, int n, const char* luid) {
|
||||||
|
double sum = 0, best = -1;
|
||||||
|
for (int i = 0; i < n; i++) {
|
||||||
|
char lu[32];
|
||||||
|
instance_luid(names[i], lu);
|
||||||
|
if (luid[0] && !std::strcmp(lu, luid)) sum += values[i];
|
||||||
|
if (values[i] > best) best = values[i];
|
||||||
|
}
|
||||||
|
if (luid[0] ? sum <= 0 : best < 0) return -1;
|
||||||
|
return (float)((luid[0] ? sum : best) / 1048576.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A number of the overlay's as its panel writes it: fmt with x, or "-" when x is under 0 (the machine, its driver or the game
|
||||||
|
/// gives none: no NVIDIA or AMD library for this GPU, no thermal zone, no sample yet); true for "-", which the panel dims
|
||||||
|
/// (NX2navi's perf_panel.cpp cell)
|
||||||
|
inline bool cell_text(char* out, size_t n, const char* fmt, double x) {
|
||||||
|
if (x < 0) {
|
||||||
|
snprintf(out, n, "-");
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
snprintf(out, n, fmt, x);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The card's memory as Vulkan reports it, MB: the heaps with local_bit (VK_MEMORY_HEAP_DEVICE_LOCAL_BIT) added up, the
|
||||||
|
/// system memory a card can also reach left out; -1 when there is none (NX2navi's perf_gpu_once). Heap: Vulkan's
|
||||||
|
/// VkMemoryHeap, or any struct with its size and flags
|
||||||
|
template <class Heap> float local_heap_mb(const Heap* heaps, uint32_t n, uint32_t local_bit) {
|
||||||
|
uint64_t bytes = 0;
|
||||||
|
for (uint32_t i = 0; i < n; i++)
|
||||||
|
if (heaps[i].flags & local_bit) bytes += heaps[i].size;
|
||||||
|
return bytes ? (float)(bytes >> 20) : -1.0f;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a vendor library's device name is the rendering GPU's: either within the other (NVIDIA's names match NVIDIA's
|
||||||
|
/// GPUs only), both not empty
|
||||||
|
inline bool name_match(const char* vulkan, const char* vendor) {
|
||||||
|
return vulkan[0] && vendor[0] && (std::strstr(vulkan, vendor) || std::strstr(vendor, vulkan));
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::perf_machine
|
||||||
140
gpu/vk/perf_stats.h
Normal file
140
gpu/vk/perf_stats.h
Normal file
@@ -0,0 +1,140 @@
|
|||||||
|
// (Triforce) The performance overlay's frame times (vk_menu.cpp, the FPS counter row on Full): a ring of the
|
||||||
|
// game's frame times, and what the overlay says of them: the mean, the longest, the median, the 99th percentile and
|
||||||
|
// the 1% and 0.1% lows over the last ten seconds, and the newest of them for the graph. After what NX2navi's overlay shows
|
||||||
|
// (its runtime/perfmon.c; none of its code is here: its 0.1% low and its rule of 100 frames for both lows are, computed
|
||||||
|
// here by the same nearest rank as the 1% low). Plain functions with no clock and no lock of their own, so that
|
||||||
|
// tests/test_perf_overlay.py runs them (tests/test_perf_overlay.cpp); the caller gives the times and keeps one thread in
|
||||||
|
// at a time.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace mkwvk::perf_stats {
|
||||||
|
|
||||||
|
constexpr int kFrames = 2048; // the frame times kept (a power of two)
|
||||||
|
constexpr int64_t kWindowNs = 10000000000ll; // the summary counts the frames of the last ten seconds
|
||||||
|
constexpr int64_t kGapNs = 1000000000ll; // held longer than this, a frame is a load or a pause: no frame time
|
||||||
|
constexpr int kLowFrames = 100; // under this many frames there is no 1% low yet, nor a 0.1% low
|
||||||
|
|
||||||
|
struct Ring {
|
||||||
|
uint32_t us[kFrames] = {}; // frame times in microseconds
|
||||||
|
uint32_t count = 0; // frame times written since the start: the newest is us[(count - 1) % kFrames]
|
||||||
|
int64_t last_ns = 0; // when the newest frame came; 0 before the first
|
||||||
|
};
|
||||||
|
|
||||||
|
/// A frame of the game came at now_ns: the time since the one before goes into the ring. The first frame, a frame
|
||||||
|
/// after a gap (kGapNs) and a clock that did not move on only start the next interval.
|
||||||
|
inline void note(Ring& r, int64_t now_ns) {
|
||||||
|
const int64_t d = r.last_ns ? now_ns - r.last_ns : 0;
|
||||||
|
r.last_ns = now_ns;
|
||||||
|
if (d <= 0 || d > kGapNs) return;
|
||||||
|
r.us[r.count++ % kFrames] = (uint32_t)((d + 500) / 1000);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// How many frame times the ring holds
|
||||||
|
inline int held(const Ring& r) { return (int)std::min<uint32_t>(r.count, (uint32_t)kFrames); }
|
||||||
|
|
||||||
|
/// The newest n frame times (as many as there are), the oldest of them first, in milliseconds: the graph. Returns how
|
||||||
|
/// many were written.
|
||||||
|
inline int newest(const Ring& r, float* out, int n) {
|
||||||
|
const int m = std::min(std::max(n, 0), held(r));
|
||||||
|
for (int i = 0; i < m; i++) out[i] = (float)r.us[(r.count - (uint32_t)m + (uint32_t)i) % kFrames] / 1000.0f;
|
||||||
|
return m;
|
||||||
|
}
|
||||||
|
|
||||||
|
struct Summary {
|
||||||
|
int frames = 0; // the frame times counted
|
||||||
|
float avg_ms = 0; // their mean
|
||||||
|
float max_ms = 0; // the longest
|
||||||
|
float p50_ms = 0; // the median
|
||||||
|
float p99_ms = 0; // the 99th percentile: 99 of 100 frames took no longer
|
||||||
|
float fps = 0; // the frame rate of the mean
|
||||||
|
float low1_fps = -1; // the 1% low: the frame rate of the 99th percentile; -1 with fewer than kLowFrames frames
|
||||||
|
float low01_fps = -1; // (Triforce: from NX2navi's perfmon.c lows_summary) the 0.1% low: the frame rate of the 99.9th
|
||||||
|
// percentile (under 1000 frames, the longest); -1 with fewer than kLowFrames frames
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Of n values sorted upwards, the one at percentile q (0 to 1), by the nearest rank: the smallest value that at least
|
||||||
|
/// that share of them does not exceed.
|
||||||
|
inline uint32_t percentile(const uint32_t* sorted, int n, double q) {
|
||||||
|
if (n <= 0) return 0;
|
||||||
|
int rank = (int)(q * n);
|
||||||
|
if ((double)rank < q * n) rank++; // (the ceiling)
|
||||||
|
return sorted[std::min(std::max(rank, 1), n) - 1];
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the overlay says of the newest frames: those whose times add up to window_ns, at least one.
|
||||||
|
inline Summary summarize(const Ring& r, int64_t window_ns = kWindowNs) {
|
||||||
|
Summary s;
|
||||||
|
static thread_local uint32_t v[kFrames];
|
||||||
|
const int have = held(r);
|
||||||
|
int n = 0;
|
||||||
|
uint64_t sum = 0;
|
||||||
|
for (int i = 0; i < have; i++) { // (the newest first)
|
||||||
|
const uint32_t us = r.us[(r.count - 1u - (uint32_t)i) % kFrames];
|
||||||
|
if (n && (int64_t)(sum + us) * 1000 > window_ns) break;
|
||||||
|
v[n++] = us, sum += us;
|
||||||
|
}
|
||||||
|
if (!n) return s;
|
||||||
|
std::sort(v, v + n);
|
||||||
|
s.frames = n;
|
||||||
|
s.avg_ms = (float)((double)sum / n / 1000.0);
|
||||||
|
s.max_ms = (float)v[n - 1] / 1000.0f;
|
||||||
|
s.p50_ms = (float)percentile(v, n, 0.50) / 1000.0f;
|
||||||
|
s.p99_ms = (float)percentile(v, n, 0.99) / 1000.0f;
|
||||||
|
s.fps = sum ? (float)(1e6 * n / (double)sum) : 0;
|
||||||
|
if (n >= kLowFrames && s.p99_ms > 0) s.low1_fps = 1000.0f / s.p99_ms;
|
||||||
|
if (n >= kLowFrames) {
|
||||||
|
const uint32_t p999 = percentile(v, n, 0.999);
|
||||||
|
if (p999) s.low01_fps = 1000000.0f / (float)p999;
|
||||||
|
}
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- (Triforce) the overlay's figures per frame (NX2navi's waits block: GPU frame prep, GPU busy per frame, Pipeline waits,
|
||||||
|
// Shader waits, Texture uploads), from running counts read twice a second (vk_menu.cpp perf_sample) ----
|
||||||
|
|
||||||
|
/// The running counts the figures come from, as of one sample: the game's frames, the replay's time to record them and the
|
||||||
|
/// GPU's in the recordings (ns; the GPU's stays 0 without timestamps), the textures uploaded, and the replay's waits for
|
||||||
|
/// pipelines and for translations (us, vk_draw.cpp draw_wait_split)
|
||||||
|
struct Counts {
|
||||||
|
int64_t frames = 0, prep_ns = 0, gpu_ns = 0, uploads = 0, pipe_us = 0, shader_us = 0;
|
||||||
|
};
|
||||||
|
/// The counts of the sample before (have: false until there is one)
|
||||||
|
struct Base {
|
||||||
|
bool have = false;
|
||||||
|
Counts at;
|
||||||
|
};
|
||||||
|
/// The figures per game frame since the sample before, ms or uploads a frame; -1 where there is none (the panel's "-")
|
||||||
|
struct PerFrame {
|
||||||
|
double prep_ms = -1, gpu_ms = -1, pipe_ms = -1, shader_ms = -1, uploads = -1;
|
||||||
|
};
|
||||||
|
enum Step {
|
||||||
|
STEP_FIRST, ///< no counts to go from (the first sample, or fresh): every figure "-"
|
||||||
|
STEP_NEW, ///< out has the figures since the sample before
|
||||||
|
STEP_KEEP, ///< no game frame since: the panel keeps what it shows (out untouched)
|
||||||
|
};
|
||||||
|
/// One sample's step. fresh: the panel was turned on now, so the counts of the sample before are of another time. Each figure
|
||||||
|
/// is its count's change over the frames' change; a count that went back is 0, and the GPU's time "-" when it did not move
|
||||||
|
/// on (no timestamps on this device). now becomes the next sample's base in every case
|
||||||
|
inline Step per_frame(Base& b, const Counts& now, bool fresh, PerFrame* out) {
|
||||||
|
const Counts was = b.at;
|
||||||
|
const bool had = b.have && !fresh;
|
||||||
|
b.have = true, b.at = now;
|
||||||
|
if (!had) {
|
||||||
|
*out = PerFrame{};
|
||||||
|
return STEP_FIRST;
|
||||||
|
}
|
||||||
|
const int64_t n = now.frames - was.frames;
|
||||||
|
if (n <= 0) return STEP_KEEP;
|
||||||
|
const auto per = [n](int64_t d, double unit) { return (double)(d > 0 ? d : 0) / unit / (double)n; };
|
||||||
|
out->prep_ms = per(now.prep_ns - was.prep_ns, 1e6);
|
||||||
|
out->gpu_ms = now.gpu_ns > was.gpu_ns ? per(now.gpu_ns - was.gpu_ns, 1e6) : -1;
|
||||||
|
out->pipe_ms = per(now.pipe_us - was.pipe_us, 1e3);
|
||||||
|
out->shader_ms = per(now.shader_us - was.shader_us, 1e3);
|
||||||
|
out->uploads = per(now.uploads - was.uploads, 1);
|
||||||
|
return STEP_NEW;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::perf_stats
|
||||||
237
gpu/vk/pipe_order.h
Normal file
237
gpu/vk/pipe_order.h
Normal file
@@ -0,0 +1,237 @@
|
|||||||
|
/* (Triforce, wp18) The pipelines made ahead in the order the last sessions' draws first needed them (vk_draw.cpp
|
||||||
|
graphics_pipeline, order_met, warm_index_set, pipelines_warm, warm_variant, warm_hold, warm_link_take; the file
|
||||||
|
triforce_pipeline_order.bin beside the pipeline notes).
|
||||||
|
|
||||||
|
With the programs read ahead in the sessions' order (program_order.h) the save's load-in frame reads none of its programs
|
||||||
|
any more (lab round 37: "0 programs read from the cache by the draws, 298 taken read ahead"), but it still makes its
|
||||||
|
pipelines: "pipelines 86.5" ms of its 249 to 270 (86 to 92 in every run), and 178 ms summed over the helpers in the Market's
|
||||||
|
arrival frame. Where the precompile keeps no pipelines (the RAM spared), each pipeline of a load is new at its first draw,
|
||||||
|
which links it on its own thread from its stage libraries, those made from the driver's caches there too. The pipeline warm
|
||||||
|
(pipe_warm.h) makes them ahead for a program the read-ahead has read, from the pipeline notes: up to 32 a program variant,
|
||||||
|
every pipeline any session ever built with it. In round 37 it made 1964 links and the draws took 54: its bounds (512 links,
|
||||||
|
the oldest let go beyond; 1024 libraries no draw has had, none made beyond) were full well before the load, of pipelines the
|
||||||
|
load did not draw, and of the program order's read-ahead the newest, the farthest from the draws, were what it kept.
|
||||||
|
|
||||||
|
So each session keeps the pipelines its draws first needed (a new entry, or one the precompile made that no draw had), in
|
||||||
|
that order (Record: the first kKeysMax), and writes them beside the notes now and then: this session's first, then those of
|
||||||
|
the sessions before it has not met (program_order.h merge; the same form of file, its own magic). The next session finds
|
||||||
|
each one's note (Index) and from the title on, when the read-ahead has read a program variant, the warm makes the pipelines
|
||||||
|
the order names for it, and with an order only those (kPerVariant at most): their stage libraries, asked of the caches only,
|
||||||
|
and their links, held for the draws (Held). The draw that needs one takes its link as it takes the warm's now.
|
||||||
|
- At most kLinksMax links held: beyond, the one farthest ahead in the order goes, not the oldest (the nearest are the next
|
||||||
|
frames'). At most kLibsMax libraries the warm made that no draw has had: none made beyond.
|
||||||
|
- Where the draws are: the middle place of the last kRecent pipelines they needed first (Front, program_order.h's rule: one
|
||||||
|
met out of the order's turn moves nothing). A link held kBehind places behind it goes, and none is made there: that
|
||||||
|
session went elsewhere.
|
||||||
|
- A link that goes takes with it the libraries the warm made for it that no draw has had and no other link held uses
|
||||||
|
(let_go_libs), so that the libraries' bound does not fill with what the draws did not come to.
|
||||||
|
Nothing changes what a draw gets: the same pipeline from the same libraries, linked a moment earlier on another thread.
|
||||||
|
|
||||||
|
Plain functions: tests/test_pipe_order.cpp checks them. RX_VK_PIPELINE_ORDER=0: neither read nor written, the warm from the
|
||||||
|
notes as before. */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <deque>
|
||||||
|
#include <iterator>
|
||||||
|
#include <map>
|
||||||
|
#include <unordered_map>
|
||||||
|
#include <unordered_set>
|
||||||
|
#include <utility>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
#include "program_order.h"
|
||||||
|
|
||||||
|
namespace mkwvk::pipe_order {
|
||||||
|
|
||||||
|
constexpr size_t kKeysMax = program_order::kKeysMax; // (pipelines a session's order keeps, and the file)
|
||||||
|
constexpr size_t kPerVariant = 32; // (the order's pipelines made for a program variant, at most: as the notes')
|
||||||
|
constexpr size_t kLinksMax = 1024; // (links held for draws that have not come: a load's, beside the title's taken)
|
||||||
|
constexpr size_t kLibsMax = 2048; // (stage libraries the warm made that no draw has had: two a program variant of 1024)
|
||||||
|
constexpr int64_t kBehind = 512; // (a link held this many places behind where the draws are goes, and none is made there)
|
||||||
|
constexpr size_t kRecent = 16; // (where the draws are: the middle place of the last this many they needed first)
|
||||||
|
constexpr uint32_t kMagic = 0x50505446; // ("FTPP")
|
||||||
|
|
||||||
|
/// RX_VK_PIPELINE_ORDER: on unless "0"
|
||||||
|
inline bool on(const char* env) { return !(env && env[0] == '0' && !env[1]); }
|
||||||
|
|
||||||
|
/// The file: program_order.h's form, its own magic
|
||||||
|
inline std::vector<uint8_t> write(const std::vector<uint64_t>& keys) { return program_order::write(keys, kMagic); }
|
||||||
|
inline std::vector<uint64_t> read(const uint8_t* data, size_t n) { return program_order::read(data, n, kMagic); }
|
||||||
|
using program_order::merge;
|
||||||
|
using Record = program_order::Record;
|
||||||
|
|
||||||
|
/// One pipeline of the order: its place, its note's record in the notes file
|
||||||
|
struct Entry {
|
||||||
|
int64_t place = 0;
|
||||||
|
uint32_t rec = 0;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The order as the warm takes it: the pipelines with a note by program variant (the caller's key for it), each list in the
|
||||||
|
/// order, at most per a variant; and every key's place, for where the draws are
|
||||||
|
struct Index {
|
||||||
|
std::unordered_map<uint64_t, std::vector<Entry>> by_variant;
|
||||||
|
std::unordered_map<uint64_t, int64_t> place;
|
||||||
|
size_t named = 0, variants = 0; // (the pipelines with a note, and their variants, as built)
|
||||||
|
/// note(key, &variant, &rec): the note of key, false when the notes have none
|
||||||
|
template <class Note>
|
||||||
|
void build(const std::vector<uint64_t>& order, Note note, size_t per = kPerVariant) {
|
||||||
|
by_variant.clear(), place.clear(), named = 0;
|
||||||
|
for (size_t i = 0; i < order.size(); i++) {
|
||||||
|
const uint64_t k = order[i];
|
||||||
|
if (!k || !place.emplace(k, (int64_t)i).second) continue;
|
||||||
|
uint64_t variant = 0;
|
||||||
|
uint32_t rec = 0;
|
||||||
|
if (!note(k, &variant, &rec)) continue;
|
||||||
|
std::vector<Entry>& list = by_variant[variant];
|
||||||
|
if (list.size() >= per) continue;
|
||||||
|
list.push_back({(int64_t)i, rec});
|
||||||
|
named++;
|
||||||
|
}
|
||||||
|
variants = by_variant.size();
|
||||||
|
}
|
||||||
|
/// The variant's pipelines, taken out (each variant once a session); none when the order names none for it
|
||||||
|
std::vector<Entry> take(uint64_t variant) {
|
||||||
|
const auto it = by_variant.find(variant);
|
||||||
|
if (it == by_variant.end()) return {};
|
||||||
|
std::vector<Entry> out = std::move(it->second);
|
||||||
|
by_variant.erase(it);
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
/// key's place in the order; -1: it names none
|
||||||
|
int64_t at(uint64_t key) const {
|
||||||
|
const auto it = place.find(key);
|
||||||
|
return it == place.end() ? -1 : it->second;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Where the draws are in the order
|
||||||
|
struct Front {
|
||||||
|
std::deque<int64_t> recent; // (the places of the last kRecent pipelines the draws needed first)
|
||||||
|
int64_t furthest = -1; // (the furthest the middle of recent has been)
|
||||||
|
/// A draw needed the pipeline at place first (-1: one the order does not name, nothing moves): true when where the draws
|
||||||
|
/// are moved on
|
||||||
|
bool met(int64_t place) {
|
||||||
|
if (place < 0) return false;
|
||||||
|
recent.push_back(place);
|
||||||
|
if (recent.size() > kRecent) recent.pop_front();
|
||||||
|
std::vector<int64_t> s(recent.begin(), recent.end());
|
||||||
|
std::nth_element(s.begin(), s.begin() + s.size() / 2, s.end());
|
||||||
|
if (s[s.size() / 2] <= furthest) return false;
|
||||||
|
furthest = s[s.size() / 2];
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
/// Whether the draws went kBehind places past place (nothing made there, and what is held there goes)
|
||||||
|
bool passed(int64_t place) const { return furthest >= 0 && place < furthest - kBehind; }
|
||||||
|
/// The first place not passed
|
||||||
|
int64_t from() const { return furthest - kBehind; }
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The links held for draws that have not come, by pipeline key, each with its place and the keys of the libraries it was
|
||||||
|
/// linked from. At most max: beyond, the one farthest ahead goes. Not thread-safe: the caller's lock
|
||||||
|
template <class Link>
|
||||||
|
class Held {
|
||||||
|
public:
|
||||||
|
struct One {
|
||||||
|
uint64_t key = 0;
|
||||||
|
int64_t place = 0;
|
||||||
|
Link link{};
|
||||||
|
uint64_t libs[4]{};
|
||||||
|
uint32_t n = 0;
|
||||||
|
};
|
||||||
|
explicit Held(size_t max = kLinksMax) : max_(max) {}
|
||||||
|
/// link held for key at place, linked from n libraries; into gone: the new one when key is held already, the farthest
|
||||||
|
/// ahead beyond max (the new one, when it is that). The caller destroys what goes (no draw has had it)
|
||||||
|
void put(uint64_t key, int64_t place, Link link, const uint64_t* libs, uint32_t n, std::vector<One>& gone) {
|
||||||
|
One o;
|
||||||
|
o.key = key, o.place = place, o.link = link, o.n = n > 4 ? 4 : n;
|
||||||
|
for (uint32_t i = 0; i < o.n; i++) o.libs[i] = libs[i];
|
||||||
|
if (place_of_.count(key)) {
|
||||||
|
gone.push_back(o);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
by_place_.emplace(std::make_pair(place, key), o);
|
||||||
|
place_of_.emplace(key, place);
|
||||||
|
while (by_place_.size() > max_) {
|
||||||
|
const auto last = std::prev(by_place_.end());
|
||||||
|
gone.push_back(last->second);
|
||||||
|
place_of_.erase(last->first.second);
|
||||||
|
by_place_.erase(last);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/// The link held for key, taken out (a draw's from now on): false when none is
|
||||||
|
bool take(uint64_t key, One& out) {
|
||||||
|
const auto it = place_of_.find(key);
|
||||||
|
if (it == place_of_.end()) return false;
|
||||||
|
const auto at = by_place_.find({it->second, key});
|
||||||
|
out = at->second;
|
||||||
|
by_place_.erase(at);
|
||||||
|
place_of_.erase(it);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
/// Those placed before at, taken out into gone
|
||||||
|
void behind(int64_t at, std::vector<One>& gone) {
|
||||||
|
const auto end = by_place_.lower_bound({at, 0});
|
||||||
|
for (auto it = by_place_.begin(); it != end;) {
|
||||||
|
gone.push_back(it->second);
|
||||||
|
place_of_.erase(it->first.second);
|
||||||
|
it = by_place_.erase(it);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/// Those linked from library lib, taken out into gone (it is let go)
|
||||||
|
void drop_using(uint64_t lib, std::vector<One>& gone) {
|
||||||
|
for (auto it = by_place_.begin(); it != by_place_.end();) {
|
||||||
|
if (!linked_from(it->second, lib)) {
|
||||||
|
++it;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
gone.push_back(it->second);
|
||||||
|
place_of_.erase(it->first.second);
|
||||||
|
it = by_place_.erase(it);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/// Whether a link at place would stay held: room left, or nearer than the farthest held (made, not let go at once)
|
||||||
|
bool room(int64_t place) const {
|
||||||
|
return by_place_.size() < max_ || (!by_place_.empty() && place < std::prev(by_place_.end())->first.first);
|
||||||
|
}
|
||||||
|
/// Whether a link held was linked from library lib
|
||||||
|
bool uses(uint64_t lib) const {
|
||||||
|
for (const auto& [at, o] : by_place_)
|
||||||
|
if (linked_from(o, lib)) return true;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
/// Every link held, taken out into out, the nearest first
|
||||||
|
void all(std::vector<One>& out) {
|
||||||
|
for (const auto& [at, o] : by_place_) out.push_back(o);
|
||||||
|
by_place_.clear();
|
||||||
|
place_of_.clear();
|
||||||
|
}
|
||||||
|
size_t size() const { return by_place_.size(); }
|
||||||
|
|
||||||
|
private:
|
||||||
|
static bool linked_from(const One& o, uint64_t lib) {
|
||||||
|
for (uint32_t i = 0; i < o.n; i++)
|
||||||
|
if (o.libs[i] == lib) return true;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
size_t max_;
|
||||||
|
std::map<std::pair<int64_t, uint64_t>, One> by_place_; // ((place, key): the nearest first)
|
||||||
|
std::unordered_map<uint64_t, int64_t> place_of_;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Of the libraries of the links that went (gone), those to let go with them, each once: the warm's own that no draw has had
|
||||||
|
/// (mine(lib)) and that no link still held uses (used(lib))
|
||||||
|
template <class One, class Mine, class Used>
|
||||||
|
std::vector<uint64_t> let_go_libs(const std::vector<One>& gone, Mine mine, Used used) {
|
||||||
|
std::vector<uint64_t> out;
|
||||||
|
std::unordered_set<uint64_t> seen;
|
||||||
|
for (const One& o : gone)
|
||||||
|
for (uint32_t i = 0; i < o.n; i++) {
|
||||||
|
const uint64_t lib = o.libs[i];
|
||||||
|
if (seen.insert(lib).second && mine(lib) && !used(lib)) out.push_back(lib);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace mkwvk::pipe_order
|
||||||
52
gpu/vk/pipe_release.h
Normal file
52
gpu/vk/pipe_release.h
Normal file
@@ -0,0 +1,52 @@
|
|||||||
|
// (Triforce) The kept pipelines no draw has had, given back to the driver while the PC's RAM is short (vk_draw.cpp
|
||||||
|
// pipelines_release_tick). Apart from Vulkan so that tests/test_pipe_release.py runs it alone.
|
||||||
|
//
|
||||||
|
// Why: on a PC of 20 GB or more the precompile builds every pipeline the notes name before the title and keeps it for the
|
||||||
|
// session (vk_draw.cpp precompile_run). On v0.4.0-test9's 32 GB PC that was 14,535 graphics and 1,033 compute pipelines,
|
||||||
|
// and the working set grew by 3.6 GB over the precompile (5517 -> 9094 MB): about 235 KB a pipeline at most, its
|
||||||
|
// programs and notes in that too. The RAM was short all session after (900 to 1,500 MB available, 43 "RAM short" lines),
|
||||||
|
// with 9.9 GB of the working set outside the game's own memory. How many of those pipelines a session draws depends on
|
||||||
|
// where it goes; the ones no draw has had are the cheapest to give back: nothing recorded them, no thread's table holds
|
||||||
|
// them (only graphics_pipeline hands an entry out, and it marks the entry first), and the driver's own cache and ours
|
||||||
|
// build them again in well under a millisecond when a draw first needs one (the precompile measured 0.2-0.4 ms each
|
||||||
|
// with the caches warm), on a worker, the draw left out or waiting meanwhile, as for any pipeline new to a session.
|
||||||
|
// (The idea is NX2navi's pipe_release, which gives back the pipelines no draw has used while video memory is short; the
|
||||||
|
// code is this back end's own.)
|
||||||
|
//
|
||||||
|
// The rules: a pass at most every so many presents while the RAM is short (due), a batch at most a pass (take), and
|
||||||
|
// only entries the caller says may go: no draw has had them, they are built (ready, with a pipeline) and have no fast
|
||||||
|
// link. The caller holds the table's locks and destroys what take moved out.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <cstddef>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <memory>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace pipe_release {
|
||||||
|
|
||||||
|
constexpr size_t kBatch = 512;
|
||||||
|
|
||||||
|
/// Whether a pass is due: the RAM short now, and at least every presents since the last pass (last 0: none yet)
|
||||||
|
inline bool due(bool ram_short, uint64_t presents, uint64_t last, uint64_t every) {
|
||||||
|
return ram_short && (last == 0 || presents >= last + every);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Up to batch entries of map (key -> std::unique_ptr<Entry>) for which may_go(entry) says yes, taken out of it into out.
|
||||||
|
/// The others stay where they are. Returns how many were taken
|
||||||
|
template <class Map, class Entry, class MayGo>
|
||||||
|
size_t take(Map& map, size_t batch, std::vector<std::unique_ptr<Entry>>& out, MayGo&& may_go) {
|
||||||
|
size_t n = 0;
|
||||||
|
for (auto it = map.begin(); it != map.end() && n < batch;) {
|
||||||
|
if (it->second && may_go(*it->second)) {
|
||||||
|
out.push_back(std::move(it->second));
|
||||||
|
it = map.erase(it);
|
||||||
|
n++;
|
||||||
|
} else {
|
||||||
|
++it;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace pipe_release
|
||||||
199
gpu/vk/pipe_wait.h
Normal file
199
gpu/vk/pipe_wait.h
Normal file
@@ -0,0 +1,199 @@
|
|||||||
|
// (Triforce) What a draw does whose pipeline is not ready yet (vk_draw.cpp draw, graphics_pipeline), apart from Vulkan so that
|
||||||
|
// tests/test_pipe_wait.py runs it. NX2navi's pipeline-miss rules (runtime/vk_exec_draw.c prepare_draw_ex, frame_wait,
|
||||||
|
// last_wait, link_budget; runtime/vk_target_frames.h tfr_pass_recent; runtime/vk_pipeline.c lib_worker; by chrg6 and
|
||||||
|
// epicmeme546, at 62cc3b13) brought to Triforce's own design: the rules and their numbers are NX2navi's, the code is
|
||||||
|
// written for this renderer.
|
||||||
|
//
|
||||||
|
// What a draw did before, and still does by default: one whose pipeline a worker still builds was left out when its targets
|
||||||
|
// held a picture from an earlier frame (vk_draw.cpp targets_persistent) and its shaders write no memory, so that the next
|
||||||
|
// frame draws it; any other waited for its pipeline however long that took (RX_VK_PIPE_SKIP=0: every one waited for).
|
||||||
|
// Triforce links a pipeline from its program's stage libraries on the draw's own thread at its first miss (gpl_rules.h), so
|
||||||
|
// a draw whose libraries the driver has is never left out; what is not ready after that is a compile on a worker.
|
||||||
|
//
|
||||||
|
// The rules added, each with its switch:
|
||||||
|
// - kept(): a pass with a colour target drawn for the first time counts as one drawn once, whatever its depth target is
|
||||||
|
// (NX2navi's NX2NAVI_PIPE_SKIP=any). The game's item icon capture draws into a new colour target with the per-frame depth
|
||||||
|
// buffer, once: a draw left out there stayed missing from the icon until the item was captured again. Its draws wait, as
|
||||||
|
// for any target drawn once. RX_VK_PIPE_SKIP_NEW=0: the depth target alone keeps the pass, as before.
|
||||||
|
// - choose(), a full-screen draw (at most 6 vertices over half the pass or more: a lighting, fog or tone mapping pass) into
|
||||||
|
// kept targets: left out, its frame lacks the pass, a black or unlit picture with the interface intact. Where the hold of
|
||||||
|
// the last picture can take that frame (present_hold.h; the draws left out for a pipeline count as owed, vk_replay.cpp
|
||||||
|
// hold_step) it is left out and the frame is not shown, as before the hold existed for pipelines. Where it cannot (the hold
|
||||||
|
// at its bound, resting after one, or off) the draw waits for its pipeline from the frame's wait pool (NX2navi's
|
||||||
|
// NX2NAVI_PIPE_FRAME_WAIT_MS, kFrameWaitMs) and then from kLastWaitMs (NX2navi's FULLSCREEN_LAST_MS), and is left out only
|
||||||
|
// after both. NX2navi waits from the pool first and holds after; here the order is the other way: Triforce's link was tried
|
||||||
|
// at the miss already, so what the draw would wait for is a compile, and a held frame costs the game's thread nothing.
|
||||||
|
// RX_VK_PIPE_FRAME_WAIT_MS=<ms>: the pool; 0: left out, as before.
|
||||||
|
// - choose(), a draw into targets drawn once or a draw whose shaders write memory: waited for as before, unless
|
||||||
|
// RX_VK_PIPE_ONESHOT_BUDGET=1 (NX2navi's NX2NAVI_ONESHOT_BUDGET, on there): from the frame's wait pool, and left out once it
|
||||||
|
// is spent. Off here: a draw left out of a target the game draws once (a volume, a lookup table) may leave it wrong until
|
||||||
|
// the game draws it again, and a slow frame is better than a wrong one.
|
||||||
|
// - LinkBudget: the synchronous links of a frame (graphics_pipeline) take at most RX_VK_PIPE_LINK_MS (NX2navi's
|
||||||
|
// NX2NAVI_PIPE_LINK_MS, 6 there); past it a new pipeline is linked by a worker instead, its draws left out or waited for
|
||||||
|
// meanwhile. 0 (the default): every link on the draw's thread, as before. NX2navi's other half of that rule, the 3 ms a
|
||||||
|
// draw waits for one pipeline's link (NX2NAVI_PIPE_LINK_DRAW_MS), has no counterpart: its links are made by worker threads
|
||||||
|
// and the draw waits for them, while here the link is made in the draw.
|
||||||
|
// - libs_first(), a pipeline whose stage libraries were cold (or whose link the budget left to a worker): the worker builds
|
||||||
|
// those libraries first and links the pipeline from them, the entry ready with that link, and the whole optimised build
|
||||||
|
// follows as a job of its own (NX2navi's NX2NAVI_PIPE_FAST_WORKERS). Before, the whole build came first and the cold
|
||||||
|
// libraries after it, so the draws of such a pipeline waited for the whole build. The same objects are made either way.
|
||||||
|
// A draw that waits runs the first job itself (vk_jobs.cpp jobs_wait): the libraries and the link, not the whole build.
|
||||||
|
// RX_VK_PIPE_FAST_WORKERS=0: the whole build first, as before.
|
||||||
|
// A wait from a budget is vk_jobs.cpp jobs_wait_for: the pipeline's job goes to the front of the queue, and is not run on the
|
||||||
|
// draw's thread (a compile there would outlast the bound).
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <atomic>
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace pipe_wait {
|
||||||
|
|
||||||
|
inline constexpr int64_t kMs = 1000000;
|
||||||
|
inline constexpr int64_t kFrameWaitMs = 50; // (NX2navi's NX2NAVI_PIPE_FRAME_WAIT_MS default)
|
||||||
|
inline constexpr int64_t kLastWaitMs = 40; // (NX2navi's FULLSCREEN_LAST_MS)
|
||||||
|
inline constexpr int64_t kLinkMs = 6; // (NX2navi's NX2NAVI_PIPE_LINK_MS default; here 0 unless asked)
|
||||||
|
inline constexpr int64_t kMinWaitNs = kMs; // (a wait shorter than this is none: NX2navi counts its waits in whole ms)
|
||||||
|
inline constexpr int kFullVertices = 6; // (a full-screen triangle or quad, or a tile: NX2navi's count <= 6)
|
||||||
|
|
||||||
|
// ---- the pass: kept from an earlier frame, or drawn once
|
||||||
|
|
||||||
|
/// A target of the pass as kept() sees it: there, drawn in an earlier frame, or not drawn before this frame (drawn first in
|
||||||
|
/// this frame, or never)
|
||||||
|
struct Target {
|
||||||
|
bool there = false;
|
||||||
|
bool earlier = false;
|
||||||
|
};
|
||||||
|
inline Target target(bool there, uint64_t first_render_frame, uint64_t now) {
|
||||||
|
return {there, there && first_render_frame && first_render_frame < now};
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a draw into this pass may be left out while its pipeline builds: the game draws its targets frame after frame.
|
||||||
|
/// n colour targets c, the depth target d. new_colour_rule (RX_VK_PIPE_SKIP_NEW): a colour target not drawn before this
|
||||||
|
/// frame makes a pass whose depth target alone was drawn before one drawn once. Without it: any target drawn before keeps
|
||||||
|
/// the pass (targets_persistent, as before)
|
||||||
|
inline bool kept(const Target* c, int n, Target d, bool new_colour_rule) {
|
||||||
|
bool colour_earlier = false, colour_new = false;
|
||||||
|
for (int i = 0; i < n; i++) {
|
||||||
|
if (!c[i].there) continue;
|
||||||
|
colour_earlier |= c[i].earlier;
|
||||||
|
colour_new |= !c[i].earlier;
|
||||||
|
}
|
||||||
|
if (colour_earlier) return true;
|
||||||
|
return d.earlier && !(new_colour_rule && colour_new);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A full-screen draw: direct, at most kFullVertices vertices in one instance, over half the pass or more (the scissor's
|
||||||
|
/// area as the draw has it, the whole target when it has none)
|
||||||
|
inline bool full_screen(bool mesh, bool indirect, int count, int instances, uint64_t area, uint64_t pass_area) {
|
||||||
|
return !mesh && !indirect && count > 0 && count <= kFullVertices && instances <= 1 && pass_area &&
|
||||||
|
area * 2 >= pass_area;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- a presented frame's budgets
|
||||||
|
|
||||||
|
/// What the draws of one presented frame may still spend (any thread; a new frame number fills it again). The draws of the
|
||||||
|
/// parallel replay's helper threads share it: a frame's total is approximate where two of them charge at once
|
||||||
|
class Pool {
|
||||||
|
public:
|
||||||
|
explicit Pool(int64_t per_frame_ns = 0) : per_frame_(per_frame_ns) {}
|
||||||
|
void per_frame(int64_t ns) { per_frame_ = ns; }
|
||||||
|
int64_t per_frame() const { return per_frame_; }
|
||||||
|
int64_t left(uint64_t frame) {
|
||||||
|
fill(frame);
|
||||||
|
return left_.load(std::memory_order_relaxed);
|
||||||
|
}
|
||||||
|
void charge(uint64_t frame, int64_t ns) {
|
||||||
|
fill(frame);
|
||||||
|
int64_t was = left_.load(std::memory_order_relaxed);
|
||||||
|
while (!left_.compare_exchange_weak(was, std::max<int64_t>(0, was - std::max<int64_t>(0, ns)), std::memory_order_relaxed)) {
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
void fill(uint64_t frame) {
|
||||||
|
uint64_t was = frame_.load(std::memory_order_relaxed);
|
||||||
|
if (was != frame && frame_.compare_exchange_strong(was, frame, std::memory_order_relaxed))
|
||||||
|
left_.store(per_frame_, std::memory_order_relaxed);
|
||||||
|
}
|
||||||
|
int64_t per_frame_ = 0;
|
||||||
|
std::atomic<uint64_t> frame_{~0ull};
|
||||||
|
std::atomic<int64_t> left_{0};
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The synchronous links of a frame (RX_VK_PIPE_LINK_MS): links() true while the frame's links have time left (always, with
|
||||||
|
/// no budget)
|
||||||
|
class LinkBudget {
|
||||||
|
public:
|
||||||
|
explicit LinkBudget(int64_t per_frame_ns = 0) : pool_(per_frame_ns) {}
|
||||||
|
bool on() const { return pool_.per_frame() > 0; }
|
||||||
|
bool links(uint64_t frame) { return !on() || pool_.left(frame) > 0; }
|
||||||
|
void spent(uint64_t frame, int64_t ns) {
|
||||||
|
if (on()) pool_.charge(frame, ns);
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
Pool pool_;
|
||||||
|
};
|
||||||
|
|
||||||
|
// ---- the draw's choice
|
||||||
|
|
||||||
|
enum class Act { Skip, Wait, WaitAll };
|
||||||
|
enum Kind { K_NONE, K_ONESHOT, K_FULL, K_LAST };
|
||||||
|
|
||||||
|
struct Rules {
|
||||||
|
bool always_wait = false; // (RX_VK_PIPE_SKIP=0: every pipeline waited for)
|
||||||
|
bool oneshot_budget = false; // (RX_VK_PIPE_ONESHOT_BUDGET=1)
|
||||||
|
bool full_wait = true; // (RX_VK_PIPE_FRAME_WAIT_MS not 0)
|
||||||
|
};
|
||||||
|
|
||||||
|
struct Choice {
|
||||||
|
Act act = Act::Skip;
|
||||||
|
int64_t ns = 0; // (Wait: for at most this long, then left out)
|
||||||
|
Kind kind = K_NONE; // (Wait: which budget the time is charged to)
|
||||||
|
};
|
||||||
|
|
||||||
|
/// A draw whose pipeline is not ready. kept: its pass (kept()); writes: its shaders write memory; full: full_screen(); held:
|
||||||
|
/// the hold of the last picture would take this frame if it lacks a draw (present_hold.h DEFERS, the pipelines counted);
|
||||||
|
/// pool_ns, last_ns: what the frame's wait pool and its last wait have left
|
||||||
|
inline Choice choose(const Rules& r, bool kept, bool writes, bool full, bool held, int64_t pool_ns, int64_t last_ns) {
|
||||||
|
if (r.always_wait) return {Act::WaitAll, 0, K_NONE};
|
||||||
|
if (!kept || writes) {
|
||||||
|
if (!r.oneshot_budget) return {Act::WaitAll, 0, K_NONE};
|
||||||
|
if (pool_ns >= kMinWaitNs) return {Act::Wait, pool_ns, K_ONESHOT};
|
||||||
|
return {Act::Skip, 0, K_NONE};
|
||||||
|
}
|
||||||
|
if (!full || held || !r.full_wait) return {Act::Skip, 0, K_NONE};
|
||||||
|
if (pool_ns >= kMinWaitNs) return {Act::Wait, pool_ns, K_FULL};
|
||||||
|
if (last_ns >= kMinWaitNs) return {Act::Wait, last_ns, K_LAST};
|
||||||
|
return {Act::Skip, 0, K_NONE};
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- a pipeline whose libraries were cold: the worker's order
|
||||||
|
|
||||||
|
/// The job a draw may wait for (its entry's ready flag), for a pipeline the draw could not link: cold names the libraries the
|
||||||
|
/// draw found cold (0: not), link_later says the draw did not try (LinkBudget). link(cold) links from the libraries, naming
|
||||||
|
/// those still cold in cold (gpl_link); build_cold(i, key) builds one (gpl_build_cold); linked(pipeline) makes the entry
|
||||||
|
/// ready with a link (gpl_rules::linked); whole_later() queues the whole build over it as a job of its own, whole_now() builds
|
||||||
|
/// it here and makes the entry ready with it (gpl_rules::built). Returns true when the entry is ready with a link
|
||||||
|
template <class Pipeline, class Link, class BuildCold, class Linked, class WholeLater, class WholeNow>
|
||||||
|
bool libs_first(uint64_t cold0, uint64_t cold1, bool link_later, Link link, BuildCold build_cold, Linked linked,
|
||||||
|
WholeLater whole_later, WholeNow whole_now) {
|
||||||
|
uint64_t cold[2] = {cold0, cold1};
|
||||||
|
Pipeline l{};
|
||||||
|
if (link_later) l = link(cold);
|
||||||
|
if (l == Pipeline{} && (cold[0] || cold[1])) {
|
||||||
|
for (int i = 0; i < 2; i++)
|
||||||
|
if (cold[i]) build_cold(i, cold[i]);
|
||||||
|
uint64_t still[2] = {0, 0};
|
||||||
|
l = link(still);
|
||||||
|
}
|
||||||
|
if (!(l == Pipeline{}) && linked(l)) {
|
||||||
|
whole_later();
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
whole_now();
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace pipe_wait
|
||||||
301
gpu/vk/pipe_warm.h
Normal file
301
gpu/vk/pipe_warm.h
Normal file
@@ -0,0 +1,301 @@
|
|||||||
|
// (Triforce) The pipelines of a load made behind its draws (vk_draw.cpp pipelines_warm, called by the program read-ahead's
|
||||||
|
// workers: vk_shader.cpp readahead_main). Apart from Vulkan so that tests/test_pipe_warm.py runs it alone.
|
||||||
|
//
|
||||||
|
// Why: v0.4.0-test16's load at Castle Town spent 59.6, 79.5 and 37.3 ms a frame in "pipelines" on the thread replaying
|
||||||
|
// (frames 8769, 8789 and 8793), with 267, 203 and 99 programs new to the draws and no waits for a pipeline. That is the fast
|
||||||
|
// link at each new pipeline's first draw: its two stage libraries made from the caches (the driver had all 1654 the session
|
||||||
|
// asked for, none cold) and their link (1.35 ms for 6.01 links a frame over that session, some 0.2 ms a link). That PC's
|
||||||
|
// precompile keeps no pipelines (the RAM spared: precompile_keeps_all), so every pipeline of a load is new at its first
|
||||||
|
// draw. The program read-ahead reads a load's programs on four workers before their draws, in the notes' order
|
||||||
|
// (program_readahead.h), and the notes hold each pipeline's whole state. So the worker that has just read a program, before
|
||||||
|
// any draw can have it, makes the stage libraries of the pipelines the notes name for it, asking the caches only, and links
|
||||||
|
// them. The draw finds the link made and goes on; the worker's whole build follows it as before.
|
||||||
|
//
|
||||||
|
// The rules, here and in vk_draw.cpp:
|
||||||
|
// - level (RX_VK_PIPELINE_WARM): 0 none, as before; 1 the stage libraries only; 2 (the default) the libraries and the links
|
||||||
|
// - only where the precompile keeps no pipelines, with the fast link on and asking the caches only (gpl_ask_only). A
|
||||||
|
// library the driver would have to compile is left to its draw, which hands it to a worker as before. Nothing is made
|
||||||
|
// while the RAM is low or short (may_warm)
|
||||||
|
// - at most kNotesPerVariant notes a program variant (the first in the notes' order), each variant once a session
|
||||||
|
// - what it holds is bounded: at most kLinksMax links made for draws that have not come (the oldest let go beyond: Held),
|
||||||
|
// at most kLibsMax stage libraries made that no draw has had (none made beyond). A library or link a draw has had is the
|
||||||
|
// draw's from then on, as if the draw had made it
|
||||||
|
// - while the RAM is short, what it holds goes back to the driver (vk_draw.cpp warm_let_go, from pipelines_release_tick):
|
||||||
|
// none of it was ever in a recording, and no warm has one of its libraries in hand then
|
||||||
|
// - ((Triforce, wp18) with the last sessions' order of pipelines (pipe_order.h, RX_VK_PIPELINE_ORDER), the pipelines that
|
||||||
|
// order names for a variant read and no other, their links held by their place in it with its own bounds; the notes'
|
||||||
|
// others are not made then, and the warm does not follow the draws)
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstddef>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdlib>
|
||||||
|
#include <deque>
|
||||||
|
#include <iterator>
|
||||||
|
#include <list>
|
||||||
|
#include <mutex>
|
||||||
|
#include <unordered_map>
|
||||||
|
#include <unordered_set>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace pipe_warm {
|
||||||
|
|
||||||
|
constexpr size_t kNotesPerVariant = 32; // (notes warmed a program variant, at most)
|
||||||
|
constexpr size_t kLinksMax = 512; // (links held for draws that have not come: about one load's, as the shelf's 512 programs)
|
||||||
|
constexpr size_t kLibsMax = 1024; // (stage libraries made that no draw has had: two a program for the shelf's 512)
|
||||||
|
|
||||||
|
/// RX_VK_PIPELINE_WARM: unset 2; 0, 1 or 2 as given; below 0 is 0, above 2 is 2
|
||||||
|
inline int level(const char* env) {
|
||||||
|
if (!env || !*env) return 2;
|
||||||
|
const int v = atoi(env);
|
||||||
|
return v < 0 ? 0 : v > 2 ? 2 : v;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a worker makes anything now: a level, the fast link on, asking the caches only, the RAM neither low nor short
|
||||||
|
inline bool may_warm(int level, bool fast_link, bool ask_only, bool ram_low, bool ram_short) {
|
||||||
|
return level > 0 && fast_link && ask_only && !ram_low && !ram_short;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The notes' records by program variant (the caller's key for it), at most per a variant, in the order they are added
|
||||||
|
inline void index_add(std::unordered_map<uint64_t, std::vector<uint32_t>>& index, uint64_t variant, uint32_t rec,
|
||||||
|
size_t per = kNotesPerVariant) {
|
||||||
|
std::vector<uint32_t>& recs = index[variant];
|
||||||
|
if (recs.size() < per) recs.push_back(rec);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether the warm may make one more stage library, with made of its own that no draw has had
|
||||||
|
inline bool lib_room(size_t made, size_t max = kLibsMax) { return made < max; }
|
||||||
|
|
||||||
|
/// The links held for draws that have not come, by pipeline key, each with the keys of the libraries it was linked from.
|
||||||
|
/// At most max: beyond, the oldest goes. Not thread-safe: the caller's lock
|
||||||
|
template <class Link>
|
||||||
|
class Held {
|
||||||
|
public:
|
||||||
|
struct One {
|
||||||
|
uint64_t key = 0;
|
||||||
|
Link link{};
|
||||||
|
uint64_t libs[4]{};
|
||||||
|
uint32_t n = 0;
|
||||||
|
};
|
||||||
|
explicit Held(size_t max = kLinksMax) : max_(max) {}
|
||||||
|
/// link held for key, linked from n libraries; what goes into gone: the new one when key is held already, the oldest
|
||||||
|
/// beyond max. The caller destroys what goes (no draw has had it)
|
||||||
|
void put(uint64_t key, Link link, const uint64_t* libs, uint32_t n, std::vector<Link>& gone) {
|
||||||
|
if (by_key_.count(key)) {
|
||||||
|
gone.push_back(link);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
One o;
|
||||||
|
o.key = key, o.link = link, o.n = n > 4 ? 4 : n;
|
||||||
|
for (uint32_t i = 0; i < o.n; i++) o.libs[i] = libs[i];
|
||||||
|
order_.push_back(o);
|
||||||
|
by_key_[key] = std::prev(order_.end());
|
||||||
|
while (order_.size() > max_) {
|
||||||
|
gone.push_back(order_.front().link);
|
||||||
|
by_key_.erase(order_.front().key);
|
||||||
|
order_.pop_front();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/// The link held for key, taken out (a draw's from now on): false when none is
|
||||||
|
bool take(uint64_t key, One& out) {
|
||||||
|
const auto it = by_key_.find(key);
|
||||||
|
if (it == by_key_.end()) return false;
|
||||||
|
out = *it->second;
|
||||||
|
order_.erase(it->second);
|
||||||
|
by_key_.erase(it);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
/// ((Triforce, test18) the links held that were linked from library lib, taken out into gone: it is let go)
|
||||||
|
void drop_using(uint64_t lib, std::vector<Link>& gone) {
|
||||||
|
for (auto it = order_.begin(); it != order_.end();) {
|
||||||
|
bool uses = false;
|
||||||
|
for (uint32_t i = 0; i < it->n; i++) uses |= it->libs[i] == lib;
|
||||||
|
if (!uses) {
|
||||||
|
++it;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
gone.push_back(it->link);
|
||||||
|
by_key_.erase(it->key);
|
||||||
|
it = order_.erase(it);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
/// ((Triforce, wp18) whether a link held was linked from library lib: pipe_order.h let_go_libs)
|
||||||
|
bool uses(uint64_t lib) const {
|
||||||
|
for (const One& o : order_)
|
||||||
|
for (uint32_t i = 0; i < o.n; i++)
|
||||||
|
if (o.libs[i] == lib) return true;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
/// Every link held, taken out into out, the oldest first
|
||||||
|
void all(std::vector<Link>& out) {
|
||||||
|
for (const One& o : order_) out.push_back(o.link);
|
||||||
|
order_.clear();
|
||||||
|
by_key_.clear();
|
||||||
|
}
|
||||||
|
size_t size() const { return order_.size(); }
|
||||||
|
|
||||||
|
private:
|
||||||
|
size_t max_;
|
||||||
|
std::list<One> order_; // (the oldest first)
|
||||||
|
std::unordered_map<uint64_t, typename std::list<One>::iterator> by_key_;
|
||||||
|
};
|
||||||
|
|
||||||
|
// ---- (Triforce, test18) Following the draws. In the owner's test18 session the warm made 2266 links and 2462 libraries and
|
||||||
|
// its draws took 276 of the links, of the 1659 pipelines the session drew (all but 15 of them noted). The read-ahead reads 32
|
||||||
|
// programs after each one the draws take, far more than a load takes, and the warm made everything it read: by the end of the
|
||||||
|
// load (the line at frame ~3540) the libraries no draw had had were at their bound of 1024, so nothing read after that had a
|
||||||
|
// library made or a link (the warm makes no link without all its libraries), and 1207 links had been let go as the oldest
|
||||||
|
// beyond 512, among them ones the load came to later. With RX_VK_PIPELINE_WARM_FOLLOW=1 (off by default since test19, below):
|
||||||
|
// - where the draws are (Front): the last kPoints places in the read-ahead's order (the notes', then the compute programs')
|
||||||
|
// the draws consumed (took a program read ahead, or read one themselves), each with its time;
|
||||||
|
// - a program read within kLinkAhead places after one of those consumed in the last kRecentMs has its libraries made and
|
||||||
|
// linked; one within kLibAhead its libraries only (cheap, and shared by its other pipelines: its draw links them itself, a
|
||||||
|
// fraction of a millisecond); one farther, or with no recent place before it (the draws stopped taking), nothing (want);
|
||||||
|
// - the libraries no draw has had are kept up to kLibsKeep, the oldest let go beyond (Made; the held links made from one
|
||||||
|
// go with it, Held::drop_using), so the bound never stops the warm as kLibsMax did;
|
||||||
|
// Off (the default): every program read is warmed whole and the libraries stop at kLibsMax, as before.
|
||||||
|
// tests/pipe_warm_sim.cpp runs both ways through a synthetic load.
|
||||||
|
//
|
||||||
|
// (Triforce, test18) The compute pipelines too (vk_draw.cpp warm_compute). test18's load spent 67.6 ms of its first frame in
|
||||||
|
// "dispatch": that PC's precompile builds no compute pipelines either (built once for the caches, then left to play), so the
|
||||||
|
// first dispatch of each compute program made its pipeline on the thread replaying, asked of the caches (a hit, but a
|
||||||
|
// creation all the same), or queued it and waited up to 50 ms. The read-ahead reads the compute programs play dispatched
|
||||||
|
// (triforce_computes.bin, after the notes' programs); the worker that has read one makes its pipeline the same way, of the
|
||||||
|
// caches only, and the dispatch that takes the program has it. One a program on the read-ahead's shelf (512 at most, and the
|
||||||
|
// game lists 286), destroyed when the shelf lets the program go, given back from the shelf while the RAM is short.
|
||||||
|
// RX_VK_COMPUTE_WARM=1 to have it; off by default since test19 (below): none, as before.
|
||||||
|
constexpr size_t kPoints = 16; // (the draws' latest places kept)
|
||||||
|
constexpr int64_t kLinkAhead = 64; // (places after a recent one: libraries and links)
|
||||||
|
constexpr int64_t kLibAhead = 192; // (places after a recent one: libraries only)
|
||||||
|
constexpr uint64_t kRecentMs = 2000; // (a place older than this is not where the draws are)
|
||||||
|
constexpr size_t kLibsKeep = 2048; // (following: the libraries no draw has had, kept; the oldest let go beyond)
|
||||||
|
|
||||||
|
/// On unless "0"
|
||||||
|
inline bool on(const char* env) { return !(env && env[0] == '0' && !env[1]); }
|
||||||
|
/// ((Triforce, test19) RX_VK_PIPELINE_WARM_FOLLOW and RX_VK_COMPUTE_WARM: off unless "1" (opt_in))
|
||||||
|
inline bool opt_in(const char* env) { return env && env[0] == '1' && !env[1]; }
|
||||||
|
|
||||||
|
// ---- (Triforce, test19) What the warm may cost. The lab's test19 tour (w19-new-1, the one run of four whose precompile kept
|
||||||
|
// no pipelines, so the one where the warm ran at all; the other three kept every pipeline, and there the warm and its switches
|
||||||
|
// do nothing) had the warm spend 2538 ms on the read-ahead's four workers, 6862 libraries and 3662 links made for 556 links
|
||||||
|
// taken, 3143 libraries let go as the oldest, and 2833 compute pipelines made for the 378 compute programs the game lists: the
|
||||||
|
// read-ahead's shelf let 7647 programs go and read many again, and the warm made them again. A worker warming is a worker not
|
||||||
|
// reading: that run's draws read 500 programs themselves, and waited 28 times (69 ms) for one being read. So:
|
||||||
|
// - at most kBudgetMsPerSecond of the workers' time a second goes to the warm, kBudgetBurstMs saved up at most (Budget): past
|
||||||
|
// it, a program read is parked as it is, its draws link it as before;
|
||||||
|
// - nothing is made twice in a session: a library the warm let go as the oldest is not made again by the warm (a draw that
|
||||||
|
// needs it makes it), a compute pipeline is made once a program variant (Once)
|
||||||
|
constexpr double kBudgetMsPerSecond = 250; // (a quarter of one worker)
|
||||||
|
constexpr double kBudgetBurstMs = 500;
|
||||||
|
|
||||||
|
/// The workers' time the warm may spend: per_second ms a second, burst saved up at most. Any thread
|
||||||
|
class Budget {
|
||||||
|
public:
|
||||||
|
explicit Budget(double per_second = kBudgetMsPerSecond, double burst = kBudgetBurstMs)
|
||||||
|
: per_second_(per_second), burst_(burst), tokens_(burst) {}
|
||||||
|
/// Whether the warm may begin something now (anything left of the budget)
|
||||||
|
bool allows(uint64_t now_ms) {
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
refill(now_ms);
|
||||||
|
return tokens_ > 0;
|
||||||
|
}
|
||||||
|
/// ms of the workers' time spent (the budget may go below nothing: what is spent is spent)
|
||||||
|
void spent(double ms, uint64_t now_ms) {
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
refill(now_ms);
|
||||||
|
tokens_ -= ms;
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
void refill(uint64_t now_ms) {
|
||||||
|
if (last_ && now_ms > last_) tokens_ = std::min(burst_, tokens_ + (double)(now_ms - last_) * per_second_ / 1000.0);
|
||||||
|
last_ = now_ms;
|
||||||
|
}
|
||||||
|
std::mutex mu_;
|
||||||
|
double per_second_, burst_, tokens_;
|
||||||
|
uint64_t last_ = 0;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Keys done once a session: first(key) is true the first time only. Not thread-safe: the caller's lock
|
||||||
|
class Once {
|
||||||
|
public:
|
||||||
|
bool first(uint64_t key) { return done_.insert(key).second; }
|
||||||
|
bool done(uint64_t key) const { return done_.count(key) != 0; }
|
||||||
|
size_t size() const { return done_.size(); }
|
||||||
|
|
||||||
|
private:
|
||||||
|
std::unordered_set<uint64_t> done_;
|
||||||
|
};
|
||||||
|
|
||||||
|
enum class Want { none, libraries, links };
|
||||||
|
|
||||||
|
/// What the warm makes of a program that far (Front::ahead) after the nearest place the draws consumed lately
|
||||||
|
inline Want want(int64_t ahead, int64_t link_ahead = kLinkAhead, int64_t lib_ahead = kLibAhead) {
|
||||||
|
if (ahead < 0 || ahead > lib_ahead) return Want::none;
|
||||||
|
return ahead <= link_ahead ? Want::links : Want::libraries;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Where the draws are: their latest places in the read-ahead's order, with when. Any thread
|
||||||
|
class Front {
|
||||||
|
public:
|
||||||
|
void consumed(int64_t at, uint64_t ms) {
|
||||||
|
if (at < 0) return;
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
points_[next_++ % kPoints] = {at, ms, true};
|
||||||
|
}
|
||||||
|
/// How far at lies after the nearest place at or before it consumed in the last recent_ms (0: that place); -1: none
|
||||||
|
int64_t ahead(int64_t at, uint64_t now_ms, uint64_t recent_ms = kRecentMs) const {
|
||||||
|
if (at < 0) return -1;
|
||||||
|
std::lock_guard lk(mu_);
|
||||||
|
int64_t best = -1;
|
||||||
|
for (const Point& p : points_) {
|
||||||
|
if (!p.set || p.at > at || (now_ms > p.ms && now_ms - p.ms > recent_ms)) continue;
|
||||||
|
if (best < 0 || at - p.at < best) best = at - p.at;
|
||||||
|
}
|
||||||
|
return best;
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
struct Point {
|
||||||
|
int64_t at = 0;
|
||||||
|
uint64_t ms = 0;
|
||||||
|
bool set = false;
|
||||||
|
};
|
||||||
|
mutable std::mutex mu_;
|
||||||
|
Point points_[kPoints];
|
||||||
|
size_t next_ = 0;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// The libraries the warm made, in the order it made them: the oldest still the warm's (still(key)) is the one let go.
|
||||||
|
/// Not thread-safe: the caller's lock
|
||||||
|
class Made {
|
||||||
|
public:
|
||||||
|
void add(uint64_t key) { q_.push_back(key); }
|
||||||
|
/// The oldest key still(key) says is the warm's, taken out into key; false when there is none. Keys a draw had since
|
||||||
|
/// are passed over and dropped
|
||||||
|
template <class Still>
|
||||||
|
bool oldest(uint64_t& key, Still still) {
|
||||||
|
while (!q_.empty()) {
|
||||||
|
key = q_.front();
|
||||||
|
q_.pop_front();
|
||||||
|
if (still(key)) return true;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
/// The keys no longer the warm's dropped once the list is four times max (a draw had them)
|
||||||
|
template <class Still>
|
||||||
|
void trim(size_t max, Still still) {
|
||||||
|
if (q_.size() <= 4 * max) return;
|
||||||
|
std::deque<uint64_t> keep;
|
||||||
|
for (const uint64_t k : q_)
|
||||||
|
if (still(k)) keep.push_back(k);
|
||||||
|
q_.swap(keep);
|
||||||
|
}
|
||||||
|
size_t size() const { return q_.size(); }
|
||||||
|
|
||||||
|
private:
|
||||||
|
std::deque<uint64_t> q_;
|
||||||
|
};
|
||||||
|
|
||||||
|
} // namespace pipe_warm
|
||||||
61
gpu/vk/prepare_threads.h
Normal file
61
gpu/vk/prepare_threads.h
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
// (Triforce) How many workers the shader preparation before the title runs on (vk_draw.cpp: prepare_run, warm_libraries,
|
||||||
|
// state_list_prepare), apart from Vulkan so that tests/test_prepare_threads.py can run it.
|
||||||
|
//
|
||||||
|
// It was half the processors, 32 at most: "below normal, and half the CPUs: on all of them at High the driver's compiles
|
||||||
|
// froze the whole PC". That was a process started at High priority (scripts/run-vk.ps1 starts its test runs so): a thread
|
||||||
|
// below normal in such a process is still above every program of normal priority, and with one on each processor nothing
|
||||||
|
// else got a turn. A player's game is a process of normal priority, where a worker below normal gives way to everything
|
||||||
|
// else on the PC, and nothing of the game runs during the preparation: there the other half of the processors stood idle
|
||||||
|
// through a wait of minutes (165 s with 12 workers on a 24-thread PC). NX2navi prepares on every processor it is given.
|
||||||
|
//
|
||||||
|
// So: every processor, 64 at most, unless the process was started above normal priority (half, as before). Nothing
|
||||||
|
// bounded the memory a worker takes: each holds one program's translation at a time and the driver's compiler for one
|
||||||
|
// pipeline, and how much the driver's part is has not been measured here (the compute programs go first, so every worker
|
||||||
|
// is in the driver's compiler at once when the preparation begins). So the workers past half are given only what RAM is
|
||||||
|
// free when the preparation starts, 1 GB each; with the RAM not known, half. Never fewer than half: what it was.
|
||||||
|
// RX_VK_PREPARE_THREADS=<n> sets the number outright, 1 to 64.
|
||||||
|
//
|
||||||
|
// (Triforce, 11-18) And measured: on AMD's Windows driver for RDNA 1 and 2 (vk_vendor.h amd_older_compiler) the compiles do
|
||||||
|
// not add up. With an empty driver cache the preparation begins with the compute programs, up to 64 texture variants each,
|
||||||
|
// every worker in the driver's compiler at once (warm_compute): on an RX 6800 XT (24 processors) each of those compiles
|
||||||
|
// ran 6-8 times slower with 22 workers than with 6 (the 8x4x4 programs 17-37 s against 2.4-3.8 s, the 128x1x1 ones
|
||||||
|
// 120-145 s against 22 s), and the translation pass ended at minute 24 against minute 15; 12 workers ended its compute part
|
||||||
|
// a minute behind 6, 16 between 12 and 22. Neither the shared pipeline cache nor the shader modules were it. But the
|
||||||
|
// steps after it, the stage libraries and the precompile, mostly graphics pipelines, went slower on 6 (the libraries 2222 s
|
||||||
|
// against 2074 s on 23; the precompile about 35 min against 27): so the cap is the translation pass's alone (translate,
|
||||||
|
// vk_draw.cpp prepare_run), kAmdOlderMost at most on that driver; the switch still decides outright. 8 workers on every
|
||||||
|
// step (11-18) ended the translation at minute 15 as 6 did, the libraries in about 2010 s and the precompile in about
|
||||||
|
// 27 min as 23 did, title at 4640 s: 8 it is. The libraries kept only about 4 of the 8 busy, the precompile's last
|
||||||
|
// pipelines all 8: both left on count's
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace prepare_threads {
|
||||||
|
|
||||||
|
/// The free RAM a worker past half the processors must find when the preparation starts
|
||||||
|
inline constexpr uint64_t kRamAWorker = 1ull << 30;
|
||||||
|
|
||||||
|
/// What it was, and what the preparation behind the game is counted from: half the processors, 1 to 32
|
||||||
|
inline unsigned half(unsigned cpus) { return std::clamp(cpus / 2, 1u, 32u); }
|
||||||
|
|
||||||
|
/// The most workers of the translation pass on AMD's Windows driver for RDNA 1 and 2 (above)
|
||||||
|
inline constexpr unsigned kAmdOlderMost = 8;
|
||||||
|
|
||||||
|
/// cpus: the logical processors; ram_free: the physical memory free now, 0 when it is not known; above_normal: the process
|
||||||
|
/// was started above normal priority; asked: RX_VK_PREPARE_THREADS, 0 or less when it is not set
|
||||||
|
inline unsigned count(unsigned cpus, uint64_t ram_free, bool above_normal, int asked) {
|
||||||
|
if (asked > 0) return (unsigned)std::min(asked, 64);
|
||||||
|
const unsigned least = half(cpus), most = std::clamp(cpus, 1u, 64u);
|
||||||
|
if (above_normal || !ram_free) return least;
|
||||||
|
return (unsigned)std::clamp<uint64_t>(ram_free / kRamAWorker, least, most);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The translation pass's workers (prepare_run): all, count's; on AMD's older driver kAmdOlderMost at most, unless the
|
||||||
|
/// switch asked for a number
|
||||||
|
inline unsigned translate(unsigned all, bool amd_older, bool asked) {
|
||||||
|
return amd_older && !asked ? std::min(all, kAmdOlderMost) : all;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace prepare_threads
|
||||||
90
gpu/vk/present_hold.h
Normal file
90
gpu/vk/present_hold.h
Normal file
@@ -0,0 +1,90 @@
|
|||||||
|
// (Triforce) Hold the last picture while new textures come in (vk_replay.cpp do_present), apart from Vulkan so that
|
||||||
|
// tests/test_present_hold.py can run it.
|
||||||
|
//
|
||||||
|
// At an area change one frame took every new texture's upload at once: 523 ms for 119 uploads of 363 MB in the owner's test5
|
||||||
|
// session, 107 and 164 ms for some 400 in others. New textures now have a budget a frame too (vk_texture.cpp
|
||||||
|
// new_texture_later): once a frame's uploads took it, a new texture a draw samples is left for the next frame, and the draw
|
||||||
|
// samples the dummy in its place. Asked what should show meanwhile, the owner chose "hold the last picture": a frame drawn
|
||||||
|
// with any texture still owed its first bytes is not shown, the window gets the last complete picture again, and the game
|
||||||
|
// goes on stepping underneath. This is the rule of when a frame is held and when the hold ends:
|
||||||
|
//
|
||||||
|
// - a frame drawn with every texture it sampled complete is shown, and ends a hold;
|
||||||
|
// - a frame drawn with a texture owed its first bytes is held, while the hold is younger than its bound (RX_VK_HOLD_MS,
|
||||||
|
// 250 ms by default);
|
||||||
|
// - once a hold reaches its bound no texture is left for later any more (the flush) and the frame after that is shown
|
||||||
|
// whatever it holds, which ends the hold: a picture is never held for longer than the bound and the frame after it;
|
||||||
|
// - after a hold that reached its bound no hold begins, and nothing is left for later, for a rest of 120 presents (2 s at
|
||||||
|
// 60), twice as long after each such hold that follows (at most 1920), back to 120 once a hold ends by itself or the
|
||||||
|
// presents since the rest's end are more than 600: a card that cannot keep its textures (one at its video memory
|
||||||
|
// allowance, whose render targets are made again from memory frame after frame) gets its frames as before, not a frozen
|
||||||
|
// picture every few seconds.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cstdint>
|
||||||
|
|
||||||
|
namespace present_hold {
|
||||||
|
|
||||||
|
/// What the draws of the next frame may do with a new texture once the frame's uploads took their budget (vk_texture.cpp)
|
||||||
|
enum Mode {
|
||||||
|
AS_BEFORE = 0, // no hold: as before the hold (a new ASTC texture's own rule, upload_defer, draws the placeholder)
|
||||||
|
DEFERS = 1, // left for the next frame (the frame is then held)
|
||||||
|
FLUSH = 2, // the hold's bound: none left, every upload now
|
||||||
|
};
|
||||||
|
|
||||||
|
inline constexpr uint64_t kRest = 120, kRestMax = 1920, kQuiet = 600;
|
||||||
|
|
||||||
|
struct State {
|
||||||
|
bool holding = false;
|
||||||
|
int64_t since_ns = 0; // (the first held present's time)
|
||||||
|
int64_t frames = 0; // (this hold's frames so far)
|
||||||
|
bool flush = false; // (this hold reached its bound)
|
||||||
|
uint64_t rest_until = 0; // (no hold before this present)
|
||||||
|
uint64_t rest = kRest;
|
||||||
|
// (counts, for the log: all of the session)
|
||||||
|
int64_t holds = 0, held_frames = 0, held_ns = 0, longest_ns = 0, bounded = 0;
|
||||||
|
// (the hold that ended at the last decide, for its log line: frames, time, and whether at the bound; 0 frames: none)
|
||||||
|
int64_t ended_frames = 0, ended_ns = 0;
|
||||||
|
bool ended_bounded = false;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// What the draws of the frame after present `present` may do (bound_ns 0: the hold is off)
|
||||||
|
inline Mode mode(const State& s, int64_t bound_ns, uint64_t present) {
|
||||||
|
if (bound_ns <= 0) return AS_BEFORE;
|
||||||
|
if (s.flush) return FLUSH;
|
||||||
|
return present >= s.rest_until ? DEFERS : AS_BEFORE;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The frame at present `present` (counted from 1), at `now_ns`: drawn with a texture owed its first bytes (owed) or with
|
||||||
|
/// every one complete. true: it is held (the window keeps the picture before it), false: it is shown
|
||||||
|
inline bool decide(State& s, int64_t bound_ns, bool owed, int64_t now_ns, uint64_t present) {
|
||||||
|
s.ended_frames = 0, s.ended_ns = 0, s.ended_bounded = false;
|
||||||
|
if (bound_ns <= 0 && !s.holding) return false;
|
||||||
|
if (s.holding) {
|
||||||
|
const int64_t held = std::max<int64_t>(0, now_ns - s.since_ns);
|
||||||
|
if (!owed || s.flush || bound_ns <= 0) { // (complete, or the frame after the bound: shown, the hold over)
|
||||||
|
s.holding = false;
|
||||||
|
s.held_ns += held, s.longest_ns = std::max(s.longest_ns, held);
|
||||||
|
s.ended_frames = s.frames, s.ended_ns = held, s.ended_bounded = s.flush;
|
||||||
|
if (s.flush) {
|
||||||
|
s.bounded++;
|
||||||
|
s.rest_until = present + s.rest;
|
||||||
|
s.rest = std::min(s.rest * 2, kRestMax);
|
||||||
|
} else {
|
||||||
|
s.rest = kRest;
|
||||||
|
}
|
||||||
|
s.flush = false;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if (held >= bound_ns) s.flush = true; // (this one still held; the next drawn with every upload made)
|
||||||
|
s.frames++, s.held_frames++;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (!owed || present < s.rest_until) return false; // (complete, or resting: shown as it is)
|
||||||
|
if (present >= s.rest_until + kQuiet) s.rest = kRest;
|
||||||
|
s.holding = true, s.since_ns = now_ns, s.frames = 1, s.flush = false;
|
||||||
|
s.holds++, s.held_frames++;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace present_hold
|
||||||
32
gpu/vk/present_mode.h
Normal file
32
gpu/vk/present_mode.h
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
/* The swapchain's present mode (vk_main.cpp create_swapchain), as plain code: which of the modes a surface lists is asked
|
||||||
|
for, by the Insert menu's VSync row and the two switches. No Vulkan: tests/test_vsync_row.py checks it.
|
||||||
|
|
||||||
|
VSync on, the default, is the choice from before the row. Mailbox where the driver has it: the newest finished frame at
|
||||||
|
each refresh of the display, no tearing, and nothing held back (the game paces itself, host/nn_vi.c). Without mailbox,
|
||||||
|
immediate on Windows (AMD's driver has no mailbox, and FIFO held anything under 60 fps at 30) and FIFO elsewhere
|
||||||
|
(immediate on NVIDIA's X11 path turns vblank off for every output, so other monitors and programs tear).
|
||||||
|
|
||||||
|
VSync off asks for immediate wherever the surface lists it: a frame is on screen the moment it is presented, up to a
|
||||||
|
refresh sooner, and torn where that falls inside a refresh. It lifts no limit: the frame rate is the game's own cap
|
||||||
|
either way. Where On is immediate already (Windows without mailbox) Off changes nothing, and where the surface lists
|
||||||
|
no immediate Off is On.
|
||||||
|
|
||||||
|
RX_VK_FIFO (set) forces FIFO, and RX_VK_IMMEDIATE (set) asks for immediate, over the row, as before it. */
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
namespace mkwvk::present_mode {
|
||||||
|
|
||||||
|
enum Mode { kFifo = 0, kMailbox = 1, kImmediate = 2 };
|
||||||
|
|
||||||
|
/// The mode to ask for. mailbox, immediate: the surface lists them (FIFO always is there). vsync: the VSync row.
|
||||||
|
/// windows: a Windows build. force_fifo, want_immediate: RX_VK_FIFO and RX_VK_IMMEDIATE are set
|
||||||
|
inline Mode choose(bool mailbox, bool immediate, bool vsync, bool windows, bool force_fifo, bool want_immediate) {
|
||||||
|
if (force_fifo) return kFifo;
|
||||||
|
const Mode waits = mailbox ? kMailbox : kFifo;
|
||||||
|
const bool at_once = want_immediate || !vsync || (windows && waits == kFifo);
|
||||||
|
return at_once && immediate ? kImmediate : waits;
|
||||||
|
}
|
||||||
|
|
||||||
|
inline const char* name(int mode) { return mode == kMailbox ? "mailbox" : mode == kImmediate ? "immediate" : "fifo"; }
|
||||||
|
|
||||||
|
} // namespace mkwvk::present_mode
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user