Apply texture and sampler registrations between submissions, in the order the game made them.

A registration the game made while a submission was being replayed changed the slot table under that submission,
whose texture pool set is written as the replay goes and read when it runs. Such a registration now waits and takes
effect once the submission has gone to the queue; with the replay on its own thread it goes into the replay queue
ahead of the next item. NX2NAVI_REG_ORDER=0 restores the old behaviour. In a town session the white kerb, stair and
wall-top edges of single frames were gone with it; the log counts the registrations that waited.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

docs/maxwell.md

@@ -90,7 +90,7 @@ Source shorthands:
 | 4.7 | **Virtual memory pools** map physical pool pages; the same bytes reachable at two GPU addresses. | Two `VkBuffer`s over one allocation need alias-aware barriers. | — | — |
 | 4.8 | **Events gate queues**: a command buffer's `WaitEvent` holds its queue until a `SignalEvent` of another queue, or `nvnEventSignal` on the CPU, gives the event the value; the GPU reads what the commands behind the wait use only from then on. Ocarina's worker queues wait for the main queue's signals and the other way round, and the main queue waits each frame for a signal the CPU gives once the game has written the frame's data, before, after or during the main queue's submit call. | One queue in submission order, and the replay copies what a submission reads (VRAM mirror sync, uniform snapshot) when its recording starts. | A wait reached before its signal stops its submission, which goes on when the signal has been replayed (`vk_exec.c`, "event order across the game's queues"). A CPU signal counts for a submission only when it came before those copies; one that comes during the replay stops the submission, and it goes on at once in a submission of its own (`vk_exec_xfer.c`, "events the CPU signals"). Passing the wait on it drew single frames from copies taken before the game had finished writing: the scene's shadowing missing or the scene black, the interface as it was. | — |
 | 4.8 | **Command list calls**: a `CallCommands` / `CallList` is not a GPU subroutine. A call is stored as a control entry; at submit the CPU walks the entries and recurses into the called list, emitting gpfifo entries that point at the command words, which the GPU fetches later (prefetch can be switched off per entry). So list structure is fixed at submit, command words are read when fetched. | n/a | Ours: `OP_CALL` is walked at replay, `CallCommands` inlines the recording at record time (`vk_exec.c` `walk`, `nvn_api_cmd.c` `inline_recording`). Equivalent for control flow; deferred bind handles are the data read late (2.4). | — |
-| 4.9 | **The texture pool is read as the GPU goes** (bindless `TEX.B` handles index it), and the GPU runs a submission right after it is submitted, so it sees the pool as of about the submit; Ocarina registers the next submission's textures right after submitting. | A descriptor a pending command buffer may read must not be updated (update-after-bind allows only unused ones), and on RADV a view of memory allocated after a submit is not in that submit's page tables. | A replay part handed to the queue early (`NX2NAVI_SPLIT_DRAWS`) kept reading the set later parts wrote: GPU page fault (`gfxhub`, TCP read) on the 128×128 probe pass of Kokiri Forest's first world frame, about 40 s in, nearly every run on RX 9070 XT. Ours: one pool set per submission from a ring, sealed when a part goes; the next sync that has entries to write takes another set, and no part is split off while every other set belongs to the same submission (`vk_exec_draw.c` `vk_exec_pool_split`, `pool_take_set`). | — |
+| 4.9 | **The texture pool is read as the GPU goes** (bindless `TEX.B` handles index it), and the GPU runs a submission right after it is submitted, so it sees the pool as of about the submit; Ocarina registers the next submission's textures right after submitting. | A descriptor a pending command buffer may read must not be updated (update-after-bind allows only unused ones), and on RADV a view of memory allocated after a submit is not in that submit's page tables. | A replay part handed to the queue early (`NX2NAVI_SPLIT_DRAWS`) kept reading the set later parts wrote: GPU page fault (`gfxhub`, TCP read) on the 128×128 probe pass of Kokiri Forest's first world frame, about 40 s in, nearly every run on RX 9070 XT. Ours: one pool set per submission from a ring, sealed when a part goes; the next sync that has entries to write takes another set, and no part is split off while every other set belongs to the same submission (`vk_exec_draw.c` `vk_exec_pool_split`, `pool_take_set`). A registration the game makes while a submission is being replayed takes effect after that submission has gone to the queue, and with the replay on its own thread where the game made it among its submits (`vk_backend.c` "registrations in submit order", `NX2NAVI_REG_ORDER`). | — |
 
 ## 5. Fixed function
 

runtime/vk.h

@@ -274,6 +274,10 @@ void vk_pool_fini(pool_t *p);              // drops images cut from the pool; wa
 int  vk_pool_map_virtual(pool_t *v, int n, const void *requests);   // nvn_mapping_t[n], checked by the front end
 void vk_texture_register(int id, const tex_t *t, const view_t *v);  // texture descriptor slot (copied)
 void vk_sampler_register(int id, const samp_par *s);                // sampler descriptor slot (copied)
+// vk_backend.c "registrations in submit order": vk_sync.c submit calls these around the replay of a submission of the
+// game's commands. A registration the game makes in between waits and takes effect at the end.
+void vk_reg_replay_begin(void);
+void vk_reg_replay_end(uint64_t fence);
 void vk_texture_release(const tex_t *t);   // TextureFinalize: the image stays for slots that still name its storage
 void vk_read_texels(const tex_t *t, const view_t *v, const NVNcopyRegion *r, uint8_t *dst, uint64_t row, uint64_t img);
 void vk_write_texels(const tex_t *t, const view_t *v, const NVNcopyRegion *r, const uint8_t *src, uint64_t row, uint64_t img);

runtime/vk_backend.c

@@ -262,13 +262,104 @@ static void present_image(const tex_t *t, const int crop[4], uint64_t at_ns, uin
     if (rdoc_active()) { vk_present_drain(); rdoc_after_present(frame); }
 }
 
+// ---- Registrations in submit order
+// The console's GPU reads the texture and sampler pools as it runs a submission, right after the submit, so a draw sees
+// the slots as they were at about its submit (docs/maxwell.md 4.9). The replay takes longer than that, and writes the
+// submission's bindless set as it goes; the set is read when the submission runs. A registration applied at once while
+// a submission is being recorded (made by another of the game's threads, or by the game ahead of a replay on its own
+// thread) therefore reaches that submission's draws, the ones recorded before it too: they sample the slot's next
+// texture, which the game registered for a later frame. Ocarina registers a round of 30 to 60 streamed textures every
+// four or five frames in a town. So a registration takes effect between submissions, in the order the game made it:
+//  - replay on the submitting thread: one made while a submission is being recorded (reg_replaying, set by vk_sync.c
+//    submit around the replay) waits in regq, and so does any made behind it; the submitting thread applies them once
+//    the submission has gone to the queue (vk_reg_replay_end). With no submission being recorded it is applied at once.
+//  - replay on its own thread: every one waits in regq and goes into the replay queue ahead of the next item queued
+//    (regs_queue_locked), so the replay applies it where the game made it among its submits, as it does a release.
+// A texture finalized while its own registration waits is released at once as before, and once more behind that
+// registration (RG_RELEASE): vk_texture_release marks the slots the texture object registered, which the first call
+// could not see. NX2NAVI_REG_ORDER=0: every registration at once, as before.
+enum { RG_TEX, RG_SAMP, RG_RELEASE };
+typedef struct {
+    int kind, id, has_view;
+    tex_t t;      // RG_TEX, RG_RELEASE: a copy (the game may reuse the object's memory)
+    view_t v;
+    samp_par s;   // RG_SAMP
+} RegItem;
+static PlatMutex regl = PLAT_MUTEX_INIT;   // regq and the order of what is applied at once; taken before the texture lock
+static RegItem *regq;
+static int regn, regcap;
+static int reg_replaying;   // atomic
+static uint64_t reg_waited, reg_rounds;   // under regl: registrations that waited for a submission, and the times some did
+
+static int reg_order_on(void) {
+    static int on = -1;
+    if (on < 0) {
+        on = rt_knob("REG_ORDER", "1") != 0;
+        rt_log("vk: texture and sampler registrations take effect %s (NX2NAVI_REG_ORDER=%d)",
+               on ? "between submissions, in the order the game made them" : "at once", on);
+    }
+    return on;
+}
+
+static void reg_apply(const RegItem *q, int n) {
+    for (int i = 0; i < n; i++) {
+        if (q[i].kind == RG_TEX) vk_texture_register(q[i].id, &q[i].t, q[i].has_view ? &q[i].v : NULL);
+        else if (q[i].kind == RG_SAMP) vk_sampler_register(q[i].id, &q[i].s);
+        else vk_texture_release(&q[i].t);
+    }
+}
+
+static void reg_push(const RegItem *r) {   // regl held
+    if (regn == regcap) {
+        int n = regcap ? regcap * 2 : 256;
+        RegItem *p = realloc(regq, sizeof *p * (size_t)n);
+        if (!p) rt_fatal("vk: out of memory (registrations)");
+        regq = p, regcap = n;
+    }
+    regq[regn++] = *r;
+}
+
+// A registration from the game: applied now, or kept for its place among the submissions.
+static void reg_add(const RegItem *r) {
+    plat_mutex_lock(&regl);
+    if (gfx_replay_async() || regn || __atomic_load_n(&reg_replaying, __ATOMIC_ACQUIRE)) reg_push(r);
+    else reg_apply(r, 1);   // under regl: in call order with the other threads' registrations
+    plat_mutex_unlock(&regl);
+}
+
+// vk_sync.c submit, around the replay of a submission of the game's commands, under its submit lock.
+void vk_reg_replay_begin(void) {
+    if (gfx_replay_async() || !reg_order_on()) return;
+    __atomic_store_n(&reg_replaying, 1, __ATOMIC_RELEASE);
+}
+
+// The submission has gone to the queue with its set: what waited is applied, before the next submission is recorded.
+void vk_reg_replay_end(uint64_t fence) {
+    if (gfx_replay_async() || !reg_order_on()) return;
+    plat_mutex_lock(&regl);
+    __atomic_store_n(&reg_replaying, 0, __ATOMIC_RELEASE);
+    int n = regn;
+    if (n) {
+        reg_apply(regq, n);   // under regl: a registration made meanwhile comes after these
+        regn = 0;
+        reg_waited += (uint64_t)n, reg_rounds++;
+        if (reg_rounds <= 8 || reg_rounds % 1024 == 0)
+            rt_log("vk: %d registrations made while submission %llu was being replayed took effect after it (frame %llu; "
+                   "%llu so far, %llu times)", n, (unsigned long long)fence,
+                   (unsigned long long)__atomic_load_n(&gfx_counters.frames, __ATOMIC_RELAXED),
+                   (unsigned long long)reg_waited, (unsigned long long)reg_rounds);
+    }
+    plat_mutex_unlock(&regl);
+}
+
 // ---- Asynchronous replay
-enum { RI_SUBMIT, RI_PRESENT, RI_RELEASE };
+enum { RI_SUBMIT, RI_PRESENT, RI_RELEASE, RI_REGS };
 typedef struct {
     int kind, n, has_crop, crop[4];
     uint64_t value, submit_no, at, queued;
     uint64_t *h;
     const tex_t *tex;   // RI_RELEASE: a copy the item owns (the game may reuse the object's memory)
+    RegItem *regs;      // RI_REGS: n registrations the item owns, in the order the game made them
 } ReplayItem;
 #define RQ_CAP 1024   // a frame queues 1 submit and 1 present, and up to a few hundred releases
 enum { DR_POOL_FINI, DR_MAP_VIRTUAL, DR_TEX_RELEASE, DR_PROG_RELEASE, DR_FINISH, DR_READ_TEXELS, DR_WRITE_TEXELS,
@@ -433,6 +524,9 @@ static void replay_main(void *arg) {
             vk_present_drain();
             vk_texture_release(it.tex);
             free((void *)it.tex);
+        } else if (it.kind == RI_REGS) {
+            reg_apply(it.regs, it.n);
+            free(it.regs);
         } else {
             uint64_t frame = __atomic_add_fetch(&gfx_counters.frames, 1, __ATOMIC_RELAXED);
             vk_dlssg_submitted(frame);   // the frame's submissions were replayed before its present
@@ -441,7 +535,7 @@ static void replay_main(void *arg) {
         }
         plat_mutex_lock(&rl);
         if (it.kind == RI_SUBMIT) r_submit_done = it.submit_no;
-        else r_presents_done++;
+        else if (it.kind != RI_REGS) r_presents_done++;
         r_busy = 0;
         plat_cond_broadcast(&rcv);
     }
@@ -471,22 +565,42 @@ static void core_back_unlocked(int *away) {
     plat_mutex_lock(&rl);
 }
 
-// Queues an item (waits while the queue is full) and returns its value, reserved under rl so values follow queue order.
-static uint64_t replay_enqueue(ReplayItem *it) {
-    plat_mutex_lock(&rl);
+// rl held: the replay thread runs, and the queue has room for `room` items (waits while it has not).
+static void rq_room(int room) {
     if (!r_started) {
         if (!plat_thread_spawn(replay_main, NULL, "vk replay", 0)) rt_fatal("vk: cannot start the replay thread");
         r_started = 1;
     }
     // rl is dropped while the core is taken back, and another thread may fill the queue again meanwhile: check again.
-    while (rq_n == RQ_CAP) {
+    while (rq_n > RQ_CAP - room) {
         uint64_t p0 = perf_begin();
         dr_calls[DR_FULL]++, dr_tcalls[DR_FULL]++;
         int away = core_away_unlocked();
-        while (rq_n == RQ_CAP) plat_cond_wait(&rcv, &rl);
+        while (rq_n > RQ_CAP - room) plat_cond_wait(&rcv, &rl);
         wait_counted(DR_FULL, p0);
         core_back_unlocked(&away);
     }
+}
+
+// rl held, with room for an item (rq_room): the registrations made since the last queued item go into the queue, to
+// be applied before whatever is queued next ("registrations in submit order"). Taken and queued without letting go of
+// rl, so two threads' lists keep their order.
+static void regs_queue_locked(void) {
+    plat_mutex_lock(&regl);
+    ReplayItem ri = {.kind = RI_REGS, .n = regn, .regs = regq};
+    regq = NULL, regn = regcap = 0;
+    plat_mutex_unlock(&regl);
+    if (!ri.n) return;
+    rq[(rq_head + rq_n) % RQ_CAP] = ri;
+    rq_n++;
+    plat_cond_broadcast(&rcv);
+}
+
+// Queues an item (waits while the queue is full) and returns its value, reserved under rl so values follow queue order.
+static uint64_t replay_enqueue(ReplayItem *it) {
+    plat_mutex_lock(&rl);
+    rq_room(2);   // the item, and the registrations made before it
+    regs_queue_locked();
     it->value = it->kind == RI_RELEASE ? 0 : vk_sync_reserve();   // a release signals nothing
     rq[(rq_head + rq_n) % RQ_CAP] = *it;
     rq_n++;
@@ -500,6 +614,10 @@ static void replay_drain(int why) {
     if (!gfx_replay_async()) return;
     plat_mutex_lock(&rl);
     dr_calls[why]++, dr_tcalls[why]++;
+    if (__atomic_load_n(&regn, __ATOMIC_RELAXED)) {   // this thread's own registrations are in: replayed too
+        rq_room(1);
+        regs_queue_locked();
+    }
     if (rq_n || r_busy) {
         uint64_t p0 = perf_begin();
         int away = core_away_unlocked();
@@ -597,8 +715,20 @@ static int vk_be_pool_map_virtual(pool_t *v, int n, const void *requests) {
     vk_present_drain();
     return vk_pool_map_virtual(v, n, requests);
 }
-static void vk_be_texture_register(int id, const tex_t *t, const view_t *v) { vk_sync_start(); vk_texture_register(id, t, v); }
-static void vk_be_sampler_register(int id, const samp_par *s) { vk_sync_start(); vk_sampler_register(id, s); }
+// Both take effect between submissions ("registrations in submit order").
+static void vk_be_texture_register(int id, const tex_t *t, const view_t *v) {
+    vk_sync_start();
+    if (!t || !reg_order_on()) { vk_texture_register(id, t, v); return; }
+    RegItem r = {.kind = RG_TEX, .id = id, .has_view = v != NULL, .t = *t};
+    if (v) r.v = *v;
+    reg_add(&r);
+}
+static void vk_be_sampler_register(int id, const samp_par *s) {
+    vk_sync_start();
+    if (!s || !reg_order_on()) { vk_sampler_register(id, s); return; }
+    RegItem r = {.kind = RG_SAMP, .id = id, .s = *s};
+    reg_add(&r);
+}
 // Asynchronous: the release is queued behind the submissions before it, so their replay still resolves the texture's
 // slots to its image (a released slot follows another live texture at the storage), and the ones after it see it
 // released, the order of the synchronous replay. vk_texture_release reads only the object's key fields: a copy.
@@ -607,6 +737,19 @@ static void vk_be_texture_release(const tex_t *t) {
     if (!gfx_replay_async()) {
         vk_present_drain();
         vk_texture_release(t);
+        // Its own registration still waits for the submission being replayed: released again behind it, which marks
+        // the slot that registration fills ("registrations in submit order").
+        if (t && t->mem && __atomic_load_n(&regn, __ATOMIC_RELAXED)) {
+            plat_mutex_lock(&regl);
+            int mine = 0;
+            for (int i = 0; i < regn && !mine; i++)
+                mine = regq[i].kind == RG_TEX && regq[i].t.serial == t->serial && regq[i].t.mem == t->mem;
+            if (mine) {
+                RegItem r = {.kind = RG_RELEASE, .t = *t};
+                reg_push(&r);
+            }
+            plat_mutex_unlock(&regl);
+        }
         return;
     }
     if (!t || !t->mem) return;

runtime/vk_sync.c

@@ -820,6 +820,9 @@ static void gpu_stats(PerfGpu *g) {
 }
 
 WEAK void vk_exec_mesh_flush(VkCommandBuffer cb) { (void)cb; }   // without vk_exec_draw.c: nothing is queued
+// without vk_backend.c: nothing registers through a back end, so no registration waits for a submission
+WEAK void vk_reg_replay_begin(void) {}
+WEAK void vk_reg_replay_end(uint64_t fence) { (void)fence; }
 
 // Weak stand-in for vk_exec.c's replay: only empty recordings ({OP_BEGIN, OP_END}) can be submitted.
 WEAK int vk_exec_record(VkCommandBuffer cb, const uint64_t *handles, int n, uint64_t fence) {
@@ -1266,8 +1269,10 @@ static uint64_t submit(const uint64_t *handles, int n, const OwnCommands *own) {
     vk_gpu_passes_fence(fence);   // the submission the parts and results that follow belong to
     vk_gpu_passes_part(s->cb, 0);
     int recorded = 1;
+    // The game's registrations made from here to the queue call below take effect after it (vk_backend.c
+    // "registrations in submit order"): the replay writes the submission's texture pool set as it goes.
     if (own) own->record(s->cb, own->arg);
-    else recorded = vk_exec_record(s->cb, handles, n, fence);
+    else vk_reg_replay_begin(), recorded = vk_exec_record(s->cb, handles, n, fence);
     s = &slots[cur_slot];   // the last part (vk_sync_split)
     vk_gpu_passes_part(s->cb, 1);
     if (recorded) ts_end(s->cb, fence);
@@ -1294,6 +1299,7 @@ static uint64_t submit(const uint64_t *handles, int n, const OwnCommands *own) {
                         .signalSemaphoreInfoCount = both ? 2u : 1u, .pSignalSemaphoreInfos = sig};
     ts_submitted(cbi, ncb);
     queue_submit(c, &si, fence);   // held or handed to the queue (submit batching)
+    if (!own) vk_reg_replay_end(fence);
     vk_mem_lazy_handed();
     s->fence = sent ? fence : 0;
     vk_mem_note_submit(fence);