Add a switch that compares textures over 64 KiB by content before uploading them again, and hash eight chains at once.

While the player runs through a town the game moves about 32 textures to new storage every four or five frames, back to back in one block, and a frame after their first upload every page of the block shows a write again. A texture up to 64 KiB keeps a hash of its storage and is uploaded again only when its bytes changed: of those, 0 to 4 in a hundred are (9 of 440 in one recording of 600 frames). A larger one has no hash and 75 to 94 in a hundred are uploaded again, whatever the size and format: 3,034 second uploads in those 600 frames beside 4,233 first ones, the ASTC ones decoded and encoded on the GPU a second time. The frames of the second uploads take 0.9 to 2.5 ms longer than frames without uploads, and the frames after them wait 0.3 to 1.3 ms longer for the GPU (six recordings of one player).

tex_same_big = 1 has textures up to 8 MiB keep the hash too. It is off by default: the bytes are read once more at the first upload and once at the comparison, 15 us each for 300 KiB, which together is about what the second upload takes on the submitting thread (27 us), so what it gains is the GPU's work and whatever else a second upload costs its frame, and that is not measured in play. A texture whose comparisons found other bytes twice in a row takes no hash any more, and an area load takes none.

The hash runs eight chains side by side and folds them, 20 GB/s against 7 for the one chain on a Ryzen 7 9850X3D, which the small textures' comparisons get as well. Each step is still a bijection of its chain and of the word it takes.

The pool log's buckets line counts the hashed uploads and the comparisons when there are any. --probe-vkmem has a case that passes with the switch off and on: the same bytes written again, other bytes, and the same bytes after two rewrites in a row.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
chrg6
2026-11-12 10:27:46 -05:00
parent 65dacfa7c7
commit a389f2450f
4 changed files with 105 additions and 8 deletions

View File

@@ -973,6 +973,39 @@ static int test_textures(pool_t *tpool, pool_t *phys) {
vk_texture_stats(NULL, NULL, &re1);
#ifdef _WIN32 // elsewhere an import's pinning can mark pages written again without a CPU write: the count is not exact
GCHECK(re1 == re0 + 1, "storage rewritten: %llu re-uploads (want 1)", (unsigned long long)(re1 - re0));
// The same bytes written again, so that every page of the storage shows a write. The texture is over 64 KiB: with
// NX2NAVI_TEX_SAME_BIG it is compared by content and left as it is, without the switch it is uploaded again. Other
// bytes upload either way, and the image holds them. After two comparisons in a row that found other bytes its
// uploads take no hash any more, so the same bytes written again then upload with the switch too.
{
int big = rt_knob("TEX_SAME_BIG", "0") == 1;
volatile uint8_t *sm = (volatile uint8_t *)t.mem;
uint64_t re2, re3, re4, re5;
GCHECK(t.L.size > (64u << 10), "the RGBA8 texture's storage is %llu bytes, not over 64 KiB", (unsigned long long)t.L.size);
for (uint64_t o = 0; o < t.L.size; o += 0x1000) sm[o] = sm[o];
if (read_back(&t, 0, lin, 1, "RGBA8 written again with its own bytes")) return 1;
vk_texture_stats(NULL, NULL, &re2);
GCHECK(re2 == re1 + (big ? 0 : 1), "the same bytes written again: %llu re-uploads (want %d)",
(unsigned long long)(re2 - re1), big ? 0 : 1);
if (fill_storage(&t, lin)) return 1;
for (unsigned lv = 0; lv < 3; lv++)
if (read_back(&t, lv, lin, 1, "RGBA8 rewritten after the comparison")) return 1;
vk_texture_stats(NULL, NULL, &re3);
GCHECK(re3 == re2 + 1, "other bytes after the comparison: %llu re-uploads (want 1)", (unsigned long long)(re3 - re2));
if (fill_storage(&t, lin)) return 1;
if (read_back(&t, 0, lin, 1, "RGBA8 rewritten a second time in a row")) return 1;
vk_texture_stats(NULL, NULL, &re4);
GCHECK(re4 == re3 + 1, "other bytes a second time: %llu re-uploads (want 1)", (unsigned long long)(re4 - re3));
for (uint64_t o = 0; o < t.L.size; o += 0x1000) sm[o] = sm[o];
if (read_back(&t, 2, lin, 1, "RGBA8 written again with its own bytes after two rewrites")) return 1;
vk_texture_stats(NULL, NULL, &re5);
GCHECK(re5 == re4 + 1, "the same bytes after two rewrites in a row: %llu re-uploads (want 1: no hash is taken any "
"more)", (unsigned long long)(re5 - re4));
printf("probe-vkmem: a texture over 64 KiB written again with its own bytes: %s (NX2NAVI_TEX_SAME_BIG=%d); other "
"bytes upload, and after two rewrites in a row the same bytes do too\n",
big ? "left as it is" : "uploaded again", big);
g_gpu_checks += 5;
}
#endif
if (read_back(&t, 1, lin, 0, "RGBA8 ReadTexels from storage")) return 1;