Make the ASTC scratch trim only try the upload lock, so it can no longer deadlock with an upload.

vk_astc_scratch_trim runs at every present and held the decoder's lock while vk_defer_buffer waited for cmd_lock.
The upload path takes the two locks the other way round: vk_upload_begin takes cmd_lock, then vk_astc_record and
scratch_get take the decoder's lock. A present that trimmed while another thread was between vk_upload_begin and
scratch_get therefore hung both threads. Without the hang, the present still waited for as long as an upload flush
held cmd_lock through its fence waits. vk_memory.c gets vk_defer_buffer_try, which defers only when it gets cmd_lock
at once. When the lock is busy the trim leaves the scratch as it is, and the next present tries again.

tests/astc_trim_check builds the real vk_astc.c with platform/common.c's recursive lock. It checks that the trim gives
way while another thread holds cmd_lock, that it goes through once the lock is free, and that the forced interleaving
ends. Before the change the check reported the deadlock (exit 2). Checks that passed: astc_trim_check, pass_cc_check
on every runtime file (vk.h changed), no new -Wall -Wextra warnings in vk_astc.c and vk_memory.c, and the pass's
test matrix (35 ok; the 6 failures are the baseline's).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
chrg6
2026-11-13 17:05:51 -05:00
parent a473e763b2
commit b755cdc536
5 changed files with 269 additions and 3 deletions

View File

@@ -6016,6 +6016,16 @@ void vk_defer_free(void *p) { if (p) defer(DEAD_HOST, (uint64_t)(uintptr_t)p, NU
void vk_defer_buffer(VkBuffer buf, VmaAllocation alloc) {
if (buf) defer(DEAD_BUFFER_VMA, (uint64_t)(uintptr_t)buf, alloc);
}
// For a caller that holds a lock the upload path takes inside cmd_lock: vk_astc.c vk_astc_scratch_trim holds the
// decoder's lock, which vk_astc.c scratch_get takes under vk_upload_begin's cmd_lock, so waiting for cmd_lock there
// would reverse that order. defer takes cmd_lock again, which the recursive lock allows.
int vk_defer_buffer_try(VkBuffer buf, VmaAllocation alloc) {
plat_once(&cmd_once, cmd_init, NULL);
if (!plat_rmutex_trylock(&cmd_lock)) return 0;
if (buf) defer(DEAD_BUFFER_VMA, (uint64_t)(uintptr_t)buf, alloc);
plat_rmutex_unlock(&cmd_lock);
return 1;
}
void vk_mem_note_submit(uint64_t timeline_value) {
plat_once(&cmd_once, cmd_init, NULL);