Let the submitting thread go on past the replay workers' groups, with up to four groups in flight.

With NX2NAVI_REPLAY_PIPE=1 (off by default) the submitting thread hands a group's chunks to the workers, takes chunks
of it itself until none is left to take, and goes on with the ops after the group in a new command buffer after the
chunks' places in the part (vk_sync_append_later), also into the next group's front pass.
Up to four groups are handed out and not in their part yet; their chunks wait in one queue in the order of the stream.
The submitting thread waits for them only where the order needs it: for room, before a part goes to the GPU, before
copies and texture or buffer clears, where DLAA's run or the contact shadows' hook acts, and where a lookup would make
a texture entry or take back a retired one (the op goes again afterwards, as a chunk's does in turn).

What the chunks read stays as the stream had it while the submitting thread runs ahead: uniform snapshot drops count
from their place in the stream, a chunk reads only the inline uniform updates before its group, the HDR present's
source goes by stream position, the texture pool set syncs under the chunks' lock, each group keeps its own render
conditions, occlusion segments and closed query slots, and the queries after a group's last report go into the next
segment made after it. A group's first chunk runs its pass hooks after the group before began its pass; turns wait
for every earlier chunk, of earlier groups too. Locks the chunks take, the submitting thread takes while it is behind.

Draws keep the sampled-texture descriptors they resolved per draw context and reuse them without the texture lock
while their slot is unchanged; the workers waited for that lock at nearly every draw. Texture lookups that change
nothing no longer wait for their turn. The perf line says where the submitting thread waited, what the chunks took,
and which texture lookups waited; NX2NAVI_GPU_TIMELINE also shows the chunks and the waits.

Off by default because it does not pay yet: in Hyrule Field with 3 workers it shortens the replay (11.5 against 12.1
ms a frame) but not the frame (49.2 against 49.6 fps in the same batch). The GPU gets a group's commands only once its
last chunk is recorded, so the main thread's wait for the GPU after the frame's first submission grew by 0.8 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
epicmeme546
2026-11-12 03:06:33 -05:00
committed by chrg6
parent 2ac56ecc6f
commit ffa599ef33
14 changed files with 623 additions and 165 deletions

View File

@@ -542,8 +542,22 @@ const VkTexImage *vk_texture_by_id_locked(int id, view_t *view, int *has_view);
// be done yet; vk_exec_turn waits until they are (no-op elsewhere).
int vk_texture_deferred(void);
int vk_texture_deferred_take(void);
void vk_texture_deferred_stats(uint64_t out[3]); // since the last call, by what the lookup found: none, retired, released
int vk_exec_turn_due(void);
int vk_exec_in_chunk(void); // the calling thread records a chunk of a parallel group
// vk_exec_shared: other threads may record now (a chunk, or the submitting thread past a group, NX2NAVI_REPLAY_PIPE),
// so the shared tables lock; vk_exec_behind: the calling thread is that submitting thread; vk_exec_hooks_wait: then
// waits until the group's first chunk ran the pass hooks.
int vk_exec_shared(void);
int vk_exec_behind(void);
void vk_exec_hooks_wait(void);
// A pass hook (vk_contact_pass_begin, vk_dlaa_pass) about to change what draws read or to record: the submitting
// thread past a group waits for the group first; a group's hooks run in its first chunk, in turn already.
void vk_exec_hook_turn(void);
// Where the calling thread records in the order of the stream as the replay workers cut it (later records larger).
uint64_t vk_exec_stream_pos(void);
// The epoch of a uniform snapshot drop the calling thread makes (write) or of the drops it sees (vk_usnap.c).
uint64_t vk_exec_epoch(int write);
void vk_exec_turn(void);
VkSampler vk_sampler_by_id_locked(int id);
// The slot's sampler with nearest filtering and no anisotropy: a depth texture read without a compare lands on one texel
@@ -719,13 +733,15 @@ int vk_exec_record(VkCommandBuffer cb, const uint64_t *handles, int n, uint64_t
// batch) without a signal, and returns a new begun one for the rest of the submission. The submission's timeline value
// is signalled by its last part, whose signal covers every earlier part in submission order.
VkCommandBuffer vk_sync_split(void);
// Inside vk_exec_record, for the replay workers (vk_exec.c): vk_sync_append ends the command buffer recorded so far,
// puts it and then cbs (recorded by the workers) into the part, in that order, and returns a new begun one for the rest
// of the part; vk_sync_append_room says whether the part has room for n more. A worker claims and begins a command
// buffer of its own pool for the submission being recorded with vk_sync_worker_begin(w) and ends it with
// vk_sync_worker_end, on its own thread, while the submitting thread waits for it.
// Inside vk_exec_record, for the replay workers (vk_exec.c): vk_sync_append_later ends the command buffer recorded so
// far, puts it into the part followed by n places for command buffers the workers record (*at the first), and returns
// a new begun one for the rest of the part; vk_sync_append_fill puts the workers' command buffers into their places,
// before the part ends; vk_sync_append_room says whether the part has room for n more. A worker claims and begins a
// command buffer of its own pool for the submission being recorded with vk_sync_worker_begin(w) and ends it with
// vk_sync_worker_end, on its own thread.
int vk_sync_append_room(int n);
VkCommandBuffer vk_sync_append(const VkCommandBuffer *cbs, int n);
VkCommandBuffer vk_sync_append_later(int n, int *at);
void vk_sync_append_fill(int at, const VkCommandBuffer *cbs, int n);
VkCommandBuffer vk_sync_worker_begin(int w);
void vk_sync_worker_end(VkCommandBuffer cb);
extern int vk_sub_in_frame; // replay: the submission being recorded is the frame's n-th (0 the first; vk_backend.c)