Merge the two performance changes into the 0.7.0 release line: a signalled event wakes only its own waiters, and a thread of its own writes the log.

One conflict, the list of setting names, which keeps both sides' additions.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
chrg6
2026-11-10 07:32:34 -05:00
17 changed files with 778 additions and 28 deletions

View File

@@ -40,6 +40,10 @@ from other projects, with its license, is listed in [THIRD_PARTY_NOTICES.md](../
- Settings and menu: every key with its default is in one table (`cfg.c`), read from `nx2navi.ini` and `NX2NAVI_*` - Settings and menu: every key with its default is in one table (`cfg.c`), read from `nx2navi.ini` and `NX2NAVI_*`
variables and written back when the in-game menu changes it. The menu and the performance overlay are Dear ImGui, variables and written back when the in-game menu changes it. The menu and the performance overlay are Dear ImGui,
drawn into the swapchain image (`runtime/ui/`); their text is in `ui_text.c`. drawn into the swapchain image (`runtime/ui/`); their text is in `ui_text.c`.
- Log: in a game run the lines go through a queue to a thread that writes them (`runtime/logq.c`), so a stalled
write to the log file or the console does not hold the thread that logged; a fatal error, the crash report and the
end of the process write the queue out first (`rt_log_sync`). `NX2NAVI_LOG_ASYNC=0` writes each line where it is
logged.
- OS calls the host makes go through `runtime/platform/` (`platform.h`, `win32.c`). - OS calls the host makes go through `runtime/platform/` (`platform.h`, `win32.c`).
#### Threads and cores #### Threads and cores
@@ -57,6 +61,9 @@ keeps per-core data without locks and relies on that, so the host models it (`ru
and, if its host rip is in lifted code at host depth 0, moves it to `kern_ap_stub`, which saves the volatile and, if its host rip is in lifted code at host depth 0, moves it to `kern_ap_stub`, which saves the volatile
registers and gives the core up there. A holder of a guest mutex counts at the priority of its best waiter, and one registers and gives the core up there. A holder of a guest mutex counts at the priority of its best waiter, and one
inside an SDK spin lock is not preempted. inside an SDK spin lock is not preempted.
- Wakes. A signalled event or a thread that ends wakes only the `WaitSynchronization` waits that hold a handle of it
(`wake_sync_waiters`): a woken thread takes a core before it looks, so waking the others cost running threads
their cores for nothing (`NX2NAVI_SYNC_WAKE_ALL=1` wakes them all).
- Host work gives the core back. Queue submits, presents, GPU waits, shader translation, `nn::fs` calls and NVN memory - Host work gives the core back. Queue submits, presents, GPU waits, shader translation, `nn::fs` calls and NVN memory
pool setup run without a core (`NVN_BLOCKING`, `kern_core_away`). pool setup run without a core (`NVN_BLOCKING`, `kern_core_away`).
- Spin waits. Loops that only another thread can end send their back-edges through `rt_spin_gate`, which pauses the - Spin waits. Loops that only another thread can end send their back-edges through `rt_spin_gate`, which pauses the

View File

@@ -299,6 +299,18 @@ A change to the step Ocarina is given each frame (`runtime/frame_rate.c` `ocarin
display times: a held 60 fps limit gives exactly one frame every frame, a 240 fps limit never less than a quarter display times: a held 60 fps limit gives exactly one frame every frame, a 240 fps limit never less than a quarter
frame, uneven frame times one steady step, and game time follows the clock. No game files. frame, uneven frame times one steady step, and game time follows the clock. No game files.
A change to which waits a signalled event or an ended thread wakes (`runtime/kernel.c` `wake_sync_waiters`, the
`WaitSynchronization` service) passes `python tests\sync_wake_check.py`, which builds `kernel.c` with the platform
layer and the rest of the runtime stubbed and runs guest threads whose entries are host functions: a signal wakes the
waits on its event and leaves the others asleep, a wait on several handles reports the right one, a thread's end ends
the wait on it, timed waits keep their time, a cancel ends any wait, and 4,000 signals to eight looping waiters are
each seen. It runs a second time with `NX2NAVI_SYNC_WAKE_ALL=1`. No game files.
A change to the log queue (`runtime/logq.c`) passes `python tests\logq_check.py`, which builds it with the platform
layer and a destination of its own: 160,000 lines from eight threads arrive whole and in each thread's order, a
destination that takes 30 ms a write does not hold the threads that log, one that stops while four times the queue's
size is logged loses nothing, and `logq_sync` returns with everything written. No game files.
A change to the button prompts (`tools/make_romfs_patches.py --prompts`, `tools/button_icons.py`) passes A change to the button prompts (`tools/make_romfs_patches.py --prompts`, `tools/button_icons.py`) passes
`.venv-py314\Scripts\python tests\romfs_prompts_test.py`, which runs the tool on a RomFS it makes itself (no game `.venv-py314\Scripts\python tests\romfs_prompts_test.py`, which runs the tool on a RomFS it makes itself (no game
files); a change to the settings passes `nx2navi.exe --probe-cfg`. files); a change to the settings passes `nx2navi.exe --probe-cfg`.

View File

@@ -621,6 +621,17 @@
# Log kernel services that block longer than this many ms, with the service number (0: off). A stalled frame shows up as # Log kernel services that block longer than this many ms, with the service number (0: off). A stalled frame shows up as
# time in the replay; this says whether the guest was waiting in a service, and which one. # time in the replay; this says whether the guest was waiting in a service, and which one.
# svc_slow_ms = 0 # svc_slow_ms = 0
# 1: every signalled event and every thread that ends wakes all the game threads that wait for any event, as before;
# each takes one of the game's virtual cores to find that its own event is not the one, often from a thread that was
# working. 0 (default): only the threads that wait for that event or thread are woken. With trace = sched the
# "sched:" line counts both ("signals woke ..., left ... asleep").
# sync_wake_all = 0
# 1 (default): a thread of its own writes the log lines, to the log file and the console, so a write that stalls (a
# busy disk, a virus scanner, a console that scrolls) no longer holds the thread that logged, which is often the one
# that draws the frame. A crash, a fatal error and the end of the game write out what is still queued first. 0: each
# line is written by the thread that logs it, as before. Either way a "log: ... writes of the log took over 2 ms" line
# counts the slow writes and says whose time they took.
# log_async = 1
# 1: the frame-time summary also splits the replay by op kind (draw, bind, transfer), which says where a stalled frame # 1: the frame-time summary also splits the replay by op kind (draw, bind, transfer), which says where a stalled frame
# went. The clock reads cost a fraction of a millisecond a frame, so it is off by default. # went. The clock reads cost a fraction of a millisecond a frame, so it is off by default.
# replay_time = 0 # replay_time = 0

View File

@@ -69,7 +69,7 @@ static const char *const known[] = {
"BIND_REUSE", "DYN_DELTA", "STG_ARENA", "SUBMIT_BATCH", "ASTC_GPU_DESWIZZLE", "ASTC_SCRATCH_KEEP", "ASTC_ARENA", "BIND_REUSE", "DYN_DELTA", "STG_ARENA", "SUBMIT_BATCH", "ASTC_GPU_DESWIZZLE", "ASTC_SCRATCH_KEEP", "ASTC_ARENA",
"TEXUP_LOG_MS", "DYN_RES", "DYN_RES_TARGET", "DYN_RES_STEP", "TEXUP_LOG_MS", "DYN_RES", "DYN_RES_TARGET", "DYN_RES_STEP",
"RECORDER_CORE", "RECORDER_PRIO", "REPLAY_TIME", "LOCK_TIME", "POOL_CREATE_BUDGET", "POOL_CREATE_MS", "POOL_LOAD_BACKLOG", "POOL_LOAD_MS", "RECORDER_CORE", "RECORDER_PRIO", "REPLAY_TIME", "LOCK_TIME", "POOL_CREATE_BUDGET", "POOL_CREATE_MS", "POOL_LOAD_BACKLOG", "POOL_LOAD_MS",
"VMA_BLOCK_MB", "ASTC_BC_PROBE", "VMA_BLOCK_MB", "ASTC_BC_PROBE", "SYNC_WAKE_ALL", "LOG_ASYNC",
}; };
// The nx2navi.ini section a key cfg_set may write goes to when the file lacks it (nx2navi.ini.example's sections). // The nx2navi.ini section a key cfg_set may write goes to when the file lacks it (nx2navi.ini.example's sections).

View File

@@ -44,6 +44,7 @@ static uint64_t g_nhost;
void rt_fatal(const char *fmt, ...) { void rt_fatal(const char *fmt, ...) {
va_list ap; va_list ap;
va_start(ap, fmt); va_start(ap, fmt);
rt_log_sync(); // main.c: the lines queued for the log thread, then these, written here
rt_log_raw("FATAL: ", 7); rt_log_raw("FATAL: ", 7);
rt_vlogf(fmt, ap); rt_vlogf(fmt, ap);
rt_log_raw("\n", 1); rt_log_raw("\n", 1);

View File

@@ -23,6 +23,7 @@ void rt_logf(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
void rt_vlogf(const char *fmt, va_list ap); void rt_vlogf(const char *fmt, va_list ap);
void rt_log_raw(const char *p, size_t n); void rt_log_raw(const char *p, size_t n);
const char *rt_log_path(void); // the log file, NULL when this run writes none const char *rt_log_path(void); // the log file, NULL when this run writes none
void rt_log_sync(void); // writes the queued log lines now; later lines are written as they are made
enum { RT_TRACE_SVC = 1, RT_TRACE_HOSTCALL = 2, RT_TRACE_IPC = 4, RT_TRACE_FS = 8, RT_TRACE_GFX = 16, RT_TRACE_PERF = 32, enum { RT_TRACE_SVC = 1, RT_TRACE_HOSTCALL = 2, RT_TRACE_IPC = 4, RT_TRACE_FS = 8, RT_TRACE_GFX = 16, RT_TRACE_PERF = 32,
RT_TRACE_SCHED = 64, RT_TRACE_AUDIO = 128, RT_TRACE_PRESENT = 256 }; RT_TRACE_SCHED = 64, RT_TRACE_AUDIO = 128, RT_TRACE_PRESENT = 256 };
extern unsigned rt_trace; // from NX2NAVI_TRACE=svc,host,ipc,fs,gfx,perf,sched,audio,present (or all) extern unsigned rt_trace; // from NX2NAVI_TRACE=svc,host,ipc,fs,gfx,perf,sched,audio,present (or all)

View File

@@ -87,6 +87,8 @@ struct RtThread {
int st; // S_* state since st_since int st; // S_* state since st_since
uint64_t st_since, st_ns[S_N], nblk, npre, pre_since, cpu_last; uint64_t st_since, st_ns[S_N], nblk, npre, pre_since, cpu_last;
uint64_t wobj; // what the current wait is on: owner handle, key, address or first handle uint64_t wobj; // what the current wait is on: owner handle, key, address or first handle
const uint32_t *whs; // a WaitSynchronization that blocks: its handles, in the waiter's own memory (holding G)
int wn;
uint32_t waker; // the thread whose signal ended the last wait (0: a host thread or a timeout) uint32_t waker; // the thread whose signal ended the last wait (0: a host thread or a timeout)
uint64_t wake_at, grant_at; // when a wait was signalled and a core granted, until the thread runs again uint64_t wake_at, grant_at; // when a wait was signalled and a core granted, until the thread runs again
uint64_t lw_ns, lw_n, lw_max, lg_ns, lg_n, lg_max, sl_n, sl_req, sl_act; // those latencies; SleepThread asked/took uint64_t lw_ns, lw_n, lw_max, lg_ns, lg_n, lg_max, sl_n, sl_req, sl_act; // those latencies; SleepThread asked/took
@@ -153,6 +155,7 @@ typedef struct { int st; uint32_t waker; uint64_t obj, ns; } WaitRec;
static WaitRec g_rw[RW_N], g_mw[RW_N]; // the render and main thread's waits by object and waker static WaitRec g_rw[RW_N], g_mw[RW_N]; // the render and main thread's waits by object and waker
static uint64_t g_pre_req, g_pre_n, g_pre_ns, g_pre_max, g_pre_stale; // preempt requests; releases after one and their latency static uint64_t g_pre_req, g_pre_n, g_pre_ns, g_pre_max, g_pre_stale; // preempt requests; releases after one and their latency
static uint64_t g_sync_calls, g_sync_blocks; // WaitSynchronization calls and the blocks inside them static uint64_t g_sync_calls, g_sync_blocks; // WaitSynchronization calls and the blocks inside them
static uint64_t g_sync_woken, g_sync_spared; // such waits a signal woke, and those it left asleep
static void wait_add(WaitRec *w, int st, uint64_t obj, uint32_t waker, uint64_t ns) { static void wait_add(WaitRec *w, int st, uint64_t obj, uint32_t waker, uint64_t ns) {
for (int i = 0; i < RW_N; i++) for (int i = 0; i < RW_N; i++)
@@ -246,10 +249,31 @@ static void wake(RtThread *t, uint32_t res) {
plat_cond_signal(&t->cv); plat_cond_signal(&t->cv);
} }
static void wake_sync_waiters(void) { // 1 when one of the handles w waits on names obj, an event (either a readable handle of it) or a thread.
static int sync_watches(const RtThread *w, const void *obj) {
for (int i = 0; i < w->wn; i++) {
unsigned k = w->whs[i] - 0x1000;
if (k > 0 && k < MAX_H && g_h[k].obj == obj && (g_h[k].type == O_EVENT_R || g_h[k].type == O_THREAD)) return 1;
}
return 0;
}
// obj became signalled: an event (its Event) or a thread that exited. The WaitSynchronization waits that hold a handle
// of it are woken to look again. Every other such wait stays asleep: woken, it took a virtual core (block: core_take,
// which asks a running thread of lower priority for its core when every core is held), found nothing signalled and
// blocked again, and the audio renderer alone signals its event 200 times a second, each time waking every thread
// that waits for a message, a timer or the end of another thread. NX2NAVI_SYNC_WAKE_ALL=1 wakes them all, as before
// ("ab": in the B blocks of NX2NAVI_AB); the sched line counts the waits woken and left asleep.
static void wake_sync_waiters(const void *obj) {
static int mode = -1;
if (mode < 0) mode = rt_knob("SYNC_WAKE_ALL", "0");
int all = RT_KNOB_ON(mode);
for (RtThread *w = g_wait; w;) { for (RtThread *w = g_wait; w;) {
RtThread *n = w->next; RtThread *n = w->next;
if (w->wkind == W_SYNC) wake(w, (uint32_t)-1); if (w->wkind == W_SYNC) {
if (all || sync_watches(w, obj)) g_sync_woken++, wake(w, (uint32_t)-1);
else g_sync_spared++;
}
w = n; w = n;
} }
} }
@@ -1184,7 +1208,7 @@ void kern_sched_report(uint64_t frame) {
} }
} }
uint64_t pre_stale = g_pre_stale, pre_req = g_pre_req, pre_n = g_pre_n, pre_ns = g_pre_ns, pre_max = g_pre_max; uint64_t pre_stale = g_pre_stale, pre_req = g_pre_req, pre_n = g_pre_n, pre_ns = g_pre_ns, pre_max = g_pre_max;
uint64_t sc = g_sync_calls, sb = g_sync_blocks; uint64_t sc = g_sync_calls, sb = g_sync_blocks, sw = g_sync_woken, ss = g_sync_spared;
for (RtThread *t = g_all; t; t = t->all) { for (RtThread *t = g_all; t; t = t->all) {
memset(t->st_ns, 0, sizeof t->st_ns); memset(t->st_ns, 0, sizeof t->st_ns);
t->nblk = t->npre = 0; t->nblk = t->npre = 0;
@@ -1194,14 +1218,16 @@ void kern_sched_report(uint64_t frame) {
memset(g_rw, 0, sizeof g_rw); memset(g_rw, 0, sizeof g_rw);
memset(g_mw, 0, sizeof g_mw); memset(g_mw, 0, sizeof g_mw);
g_pre_req = g_pre_n = g_pre_ns = g_pre_max = g_pre_stale = 0; g_pre_req = g_pre_n = g_pre_ns = g_pre_max = g_pre_stale = 0;
g_sync_calls = g_sync_blocks = 0; g_sync_calls = g_sync_blocks = g_sync_woken = g_sync_spared = 0;
unlock(); unlock();
if (!span) return; if (!span) return;
rt_log("sched: frame %llu: %.2f s, cores busy %.2f of %d, core wait %.2f thread-s, preempt req %llu released %llu " rt_log("sched: frame %llu: %.2f s, cores busy %.2f of %d, core wait %.2f thread-s, preempt req %llu released %llu "
"avg %.2f ms max %.2f ms, WaitSynchronization %llu calls %llu blocks, stale preempt %llu%s", "avg %.2f ms max %.2f ms, WaitSynchronization %llu calls %llu blocks (signals woke %llu, left %llu asleep), "
"stale preempt %llu%s",
(unsigned long long)frame, span / 1e9, (double)all_run / span, NCORE, all_core / 1e9, (unsigned long long)frame, span / 1e9, (double)all_run / span, NCORE, all_core / 1e9,
(unsigned long long)pre_req, (unsigned long long)pre_n, pre_n ? pre_ns / 1e6 / pre_n : 0.0, pre_max / 1e6, (unsigned long long)pre_req, (unsigned long long)pre_n, pre_n ? pre_ns / 1e6 / pre_n : 0.0, pre_max / 1e6,
(unsigned long long)sc, (unsigned long long)sb, (unsigned long long)pre_stale, self ? "" : " (render thread is not a guest thread)"); (unsigned long long)sc, (unsigned long long)sb, (unsigned long long)sw, (unsigned long long)ss,
(unsigned long long)pre_stale, self ? "" : " (render thread is not a guest thread)");
for (int i = 0; i < nr; i++) { for (int i = 0; i < nr; i++) {
Row *r = &rows[i]; Row *r = &rows[i];
char d[128], s[256]; char d[128], s[256];
@@ -1336,7 +1362,7 @@ static void thread_finish(RtThread *t) {
if (t->hthr) plat_intr_free(t->hthr); // no waiter interrupts a thread without a core if (t->hthr) plat_intr_free(t->hthr); // no waiter interrupts a thread without a core
t->hthr = NULL; t->hthr = NULL;
t->exited = 1; t->exited = 1;
wake_sync_waiters(); wake_sync_waiters(t);
if (t->closed) thr_free(t); if (t->closed) thr_free(t);
unlock(); unlock();
} }
@@ -1858,7 +1884,7 @@ static int kern_svc_inner(Cpu *c, uint32_t imm) {
if (r == R_BAD_HANDLE) rt_log("kernel: event svc %#x: invalid handle %#x", imm, h); if (r == R_BAD_HANDLE) rt_log("kernel: event svc %#x: invalid handle %#x", imm, h);
} else if (imm == 0x11) { } else if (imm == 0x11) {
e->signaled = 1; e->signaled = 1;
wake_sync_waiters(); wake_sync_waiters(e);
} else if (imm == 0x17 && !e->signaled) { } else if (imm == 0x17 && !e->signaled) {
r = R_BAD_STATE; r = R_BAD_STATE;
} else { } else {
@@ -1939,7 +1965,9 @@ static int kern_svc_inner(Cpu *c, uint32_t imm) {
} }
t->wobj = n ? hs[0] | (uint64_t)n << 32 : 0; t->wobj = n ? hs[0] | (uint64_t)n << 32 : 0;
g_sync_blocks++; g_sync_blocks++;
t->whs = hs, t->wn = n;
woke = block(t, W_SYNC, left); woke = block(t, W_SYNC, left);
t->whs = NULL, t->wn = 0;
if (woke == R_TIMEOUT) { if (woke == R_TIMEOUT) {
r = R_TIMEOUT; r = R_TIMEOUT;
break; break;
@@ -2010,7 +2038,7 @@ void kern_event_signal(uint32_t hw) {
Event *e = h_peek(hw, O_EVENT_W); Event *e = h_peek(hw, O_EVENT_W);
if (!e) rt_fatal("kern_event_signal: %#x is not a writable event handle", hw); if (!e) rt_fatal("kern_event_signal: %#x is not a writable event handle", hw);
e->signaled = 1; e->signaled = 1;
wake_sync_waiters(); wake_sync_waiters(e);
unlock(); unlock();
} }

141
runtime/logq.c Normal file
View File

@@ -0,0 +1,141 @@
// Log writes on a thread of their own.
//
// A log line was written by the thread that logged it: to stderr and to the log file (main.c log_sink). A write can
// stall: a console that scrolls, a pipe whose reader is busy, a virus scanner, a disk that flushes; hundreds of
// milliseconds at worst. The render thread logs too, and a frame it spends in such a write is a hitch with nothing
// else to show for it.
//
// With the queue on (logq_async) a line is copied into a buffer under a lock that is held for the copy alone, and the
// "log" thread writes the buffer out, so lines keep the order they were queued in. logq_sync writes what is queued on
// the calling thread and turns the queue off for the rest of the run: a fatal error, the crash report and the end of
// the process call it (rt_log_sync), so no line is lost to an exit and their own lines are written as they are made.
//
// Every write is timed. One that took LOGQ_SLOW_NS or longer is counted, and a line says so at most every ten
// seconds: with the queue that time was the log thread's, without it (NX2NAVI_LOG_ASYNC=0) the logging thread's.
#include <stdlib.h>
#include <string.h>
#include "host.h"
#include "logq.h"
#include "platform/platform.h"
#define LOGQ_CAP (1u << 20)
#define LOGQ_SLOW_NS 2000000ll
#define LOGQ_SAY_NS 10000000000ull
static struct {
PlatMutex q, io; // q: buf and n; io: the queue's writes and out. io is taken first
PlatCond more;
char *buf, *out; // LOGQ_CAP bytes each: the queue, and what the log thread is writing
size_t n;
LogqSink sink;
volatile int32_t on, saying;
volatile int64_t slow, said, worst_ns, direct; // slow writes, those reported, the longest; lines written past a busy queue
volatile uint64_t said_at;
} Q = {.q = PLAT_MUTEX_INIT, .io = PLAT_MUTEX_INIT, .more = PLAT_COND_INIT};
void logq_init(LogqSink sink) { Q.sink = sink; }
// The write itself, timed.
static void put_out(const char *p, size_t n) {
uint64_t t0 = plat_now_ns();
Q.sink(p, n);
int64_t d = (int64_t)(plat_now_ns() - t0);
if (d < LOGQ_SLOW_NS) return;
__atomic_add_fetch(&Q.slow, 1, __ATOMIC_RELAXED);
int64_t w = __atomic_load_n(&Q.worst_ns, __ATOMIC_RELAXED);
while (d > w && !__atomic_compare_exchange_n(&Q.worst_ns, &w, d, 0, __ATOMIC_RELAXED, __ATOMIC_RELAXED)) {}
}
// The line about slow writes, when there are new ones and the last line is ten seconds old. Called with no lock held:
// the line goes through logq_write like any other.
static void say_slow(void) {
int64_t slow = __atomic_load_n(&Q.slow, __ATOMIC_RELAXED);
if (slow == __atomic_load_n(&Q.said, __ATOMIC_RELAXED)) return;
uint64_t now = plat_now_ns(), at = __atomic_load_n(&Q.said_at, __ATOMIC_RELAXED);
if ((at && now - at < LOGQ_SAY_NS) || plat_xchg32(&Q.saying, 1)) return;
__atomic_store_n(&Q.said, slow, __ATOMIC_RELAXED);
__atomic_store_n(&Q.said_at, now ? now : 1, __ATOMIC_RELAXED);
rt_log("log: %lld writes of the log took over %d ms so far, the longest %.1f ms, %s (NX2NAVI_LOG_ASYNC); %lld lines "
"were written past a busy queue", (long long)slow, (int)(LOGQ_SLOW_NS / 1000000),
__atomic_load_n(&Q.worst_ns, __ATOMIC_RELAXED) / 1e6,
__atomic_load_n(&Q.on, __ATOMIC_ACQUIRE) ? "on the log thread" : "on the threads that logged",
(long long)__atomic_load_n(&Q.direct, __ATOMIC_RELAXED));
plat_xchg32(&Q.saying, 0);
}
// Writes out what is queued, in order. Any thread.
static void drain(void) {
plat_mutex_lock(&Q.io);
plat_mutex_lock(&Q.q);
char *b = Q.buf;
size_t n = Q.n;
Q.buf = Q.out, Q.out = b, Q.n = 0;
plat_mutex_unlock(&Q.q);
if (n) put_out(b, n);
plat_mutex_unlock(&Q.io);
}
// 1 when the bytes were queued, 0 when the caller is to write them. The queue's lock is only tried: a thread that logs
// while the lock's holder cannot run (stopped for a profile sample or a core hand-over) writes its line itself.
static int put(const char *p, size_t n) {
if (!__atomic_load_n(&Q.on, __ATOMIC_ACQUIRE) || n > LOGQ_CAP / 4) return 0;
for (int tries = 0; !plat_mutex_trylock(&Q.q); tries++) {
if (tries == 4000) {
__atomic_add_fetch(&Q.direct, 1, __ATOMIC_RELAXED);
return 0;
}
plat_pause();
}
int on = __atomic_load_n(&Q.on, __ATOMIC_ACQUIRE);
if (!on || Q.n + n > LOGQ_CAP) { // full: the log thread is held up in a write. What is queued goes out first
plat_mutex_unlock(&Q.q);
if (on) drain();
return 0;
}
int first = !Q.n;
memcpy(Q.buf + Q.n, p, n);
Q.n += n;
plat_mutex_unlock(&Q.q);
if (first) plat_cond_signal(&Q.more);
return 1;
}
void logq_write(const char *p, size_t n) {
if (put(p, n)) return;
put_out(p, n);
say_slow();
}
static void log_main(void *arg) {
(void)arg;
for (;;) {
plat_mutex_lock(&Q.q);
while (!Q.n) plat_cond_wait(&Q.more, &Q.q);
plat_mutex_unlock(&Q.q);
drain();
say_slow();
}
}
int logq_async(void) {
if (!Q.sink || __atomic_load_n(&Q.on, __ATOMIC_ACQUIRE)) return 0;
if (!Q.buf) Q.buf = malloc(LOGQ_CAP), Q.out = malloc(LOGQ_CAP);
if (!Q.buf || !Q.out || !plat_thread_spawn(log_main, NULL, "log", 0)) return 0;
__atomic_store_n(&Q.on, 1, __ATOMIC_RELEASE);
return 1;
}
void logq_sync(void) {
if (!plat_xchg32(&Q.on, 0)) return;
// Each lock is tried for half a second: the thread that holds it may be the one that crashed, or be stuck in the
// write this call is there to get past. Without the lock the queue is written as it stands.
int io = 0, q = 0;
for (int i = 0; i < 500 && !(io = plat_mutex_trylock(&Q.io)); i++) plat_sleep_ms(1);
for (int i = 0; i < 500 && !(q = plat_mutex_trylock(&Q.q)); i++) plat_sleep_ms(1);
size_t n = Q.n;
Q.n = 0;
if (n && n <= LOGQ_CAP) put_out(Q.buf, n);
if (q) plat_mutex_unlock(&Q.q);
if (io) plat_mutex_unlock(&Q.io);
}

10
runtime/logq.h Normal file
View File

@@ -0,0 +1,10 @@
// Log writes on a thread of their own (logq.c). main.c owns the log's destinations and hands every line here.
#pragma once
#include <stddef.h>
typedef void (*LogqSink)(const char *p, size_t n); // writes the bytes to the log's destinations; may block
void logq_init(LogqSink sink); // before the first logq_write; lines are written by the thread that logs them
int logq_async(void); // from now on the log thread writes them; 0 when it could not be started
void logq_write(const char *p, size_t n); // queues the bytes, or writes them when the queue is off, full or busy
void logq_sync(void); // writes what is queued, on the calling thread, and turns the queue off

View File

@@ -42,6 +42,7 @@
#include "cfg.h" #include "cfg.h"
#include "host.h" #include "host.h"
#include "logq.h"
#include "prof.h" #include "prof.h"
#include "platform/platform.h" #include "platform/platform.h"
#include "sha256.h" #include "sha256.h"
@@ -66,21 +67,29 @@ static char g_log_early[32768];
static size_t g_log_early_n; static size_t g_log_early_n;
static uint64_t g_log_bytes; static uint64_t g_log_bytes;
void rt_log_raw(const char *p, size_t n) { // The log's destinations. logq.c calls it: from the log thread in a game run, else from the thread that logs.
static void log_sink(const char *p, size_t n) {
fwrite(p, 1, n, stderr); fwrite(p, 1, n, stderr);
int state = __atomic_load_n(&g_log_state, __ATOMIC_ACQUIRE); if (__atomic_load_n(&g_log_state, __ATOMIC_ACQUIRE) != LOG_FILE) return;
if (state == LOG_EARLY) { uint64_t at = __atomic_fetch_add(&g_log_bytes, n, __ATOMIC_RELAXED);
if (at + n <= LOG_CAP) {
plat_append_write(g_log_file, p, n);
} else if (at <= LOG_CAP) { // the first write past the cap says so, once
static const char cut[] = "[host] log: the file reached 64 MiB, the rest of this run goes to stderr only\n";
plat_append_write(g_log_file, cut, sizeof cut - 1);
}
}
void rt_log_sync(void) { logq_sync(); }
void rt_log_raw(const char *p, size_t n) {
if (__atomic_load_n(&g_log_state, __ATOMIC_ACQUIRE) == LOG_EARLY) {
fwrite(p, 1, n, stderr);
size_t at = __atomic_fetch_add(&g_log_early_n, n, __ATOMIC_RELAXED); size_t at = __atomic_fetch_add(&g_log_early_n, n, __ATOMIC_RELAXED);
if (at + n <= sizeof g_log_early) memcpy(g_log_early + at, p, n); if (at + n <= sizeof g_log_early) memcpy(g_log_early + at, p, n);
} else if (state == LOG_FILE) { return;
uint64_t at = __atomic_fetch_add(&g_log_bytes, n, __ATOMIC_RELAXED);
if (at + n <= LOG_CAP) {
plat_append_write(g_log_file, p, n);
} else if (at <= LOG_CAP) { // the first write past the cap says so, once
static const char cut[] = "[host] log: the file reached 64 MiB, the rest of this run goes to stderr only\n";
plat_append_write(g_log_file, cut, sizeof cut - 1);
}
} }
logq_write(p, n);
} }
void rt_vlogf(const char *fmt, va_list ap) { void rt_vlogf(const char *fmt, va_list ap) {
@@ -115,6 +124,7 @@ const char *rt_log_path(void) {
// Opens the log file for a game run (game = 1) and writes the buffered lines to it; anything else drops them. Runs on // Opens the log file for a game run (game = 1) and writes the buffered lines to it; anything else drops them. Runs on
// the main thread before the boot starts other threads. // the main thread before the boot starts other threads.
static void log_open(int game) { static void log_open(int game) {
logq_init(log_sink);
char dir[PLAT_PATH_MAX * 3], from[PLAT_PATH_MAX * 3 + 16], to[PLAT_PATH_MAX * 3 + 16]; char dir[PLAT_PATH_MAX * 3], from[PLAT_PATH_MAX * 3 + 16], to[PLAT_PATH_MAX * 3 + 16];
char *cut = NULL; char *cut = NULL;
if (game && !plat_stderr_path(dir, sizeof dir) && plat_exe_path(dir, sizeof dir)) { if (game && !plat_stderr_path(dir, sizeof dir) && plat_exe_path(dir, sizeof dir)) {
@@ -136,14 +146,20 @@ static void log_open(int game) {
} }
if (!g_log_file) { if (!g_log_file) {
__atomic_store_n(&g_log_state, LOG_OFF, __ATOMIC_RELEASE); __atomic_store_n(&g_log_state, LOG_OFF, __ATOMIC_RELEASE);
return; } else {
size_t n = __atomic_load_n(&g_log_early_n, __ATOMIC_RELAXED);
if (n > sizeof g_log_early) n = sizeof g_log_early;
plat_append_write(g_log_file, g_log_early, n);
g_log_bytes = n;
__atomic_store_n(&g_log_state, LOG_FILE, __ATOMIC_RELEASE);
rt_log("log: %s", g_log_path);
} }
size_t n = __atomic_load_n(&g_log_early_n, __ATOMIC_RELAXED); // A game run writes its log from the log thread (logq.c), whether the lines go to a log file or to stderr alone.
if (n > sizeof g_log_early) n = sizeof g_log_early; const char *e = cfg_get("NX2NAVI_LOG_ASYNC");
plat_append_write(g_log_file, g_log_early, n); if (!game || (e && !strcmp(e, "0"))) return;
g_log_bytes = n; plat_exit_hook = rt_log_sync;
__atomic_store_n(&g_log_state, LOG_FILE, __ATOMIC_RELEASE); if (logq_async()) rt_log("log: lines are written by a thread of their own (NX2NAVI_LOG_ASYNC)");
rt_log("log: %s", g_log_path); else rt_log("log: no log thread; every line is written by the thread that logs it");
} }
static void parse_trace(void) { static void parse_trace(void) {
@@ -243,6 +259,7 @@ static int readable(uint64_t a, uint64_t n) { return plat_mem_readable(a, n); }
static void crash_handler(const PlatCrash *x) { static void crash_handler(const PlatCrash *x) {
static volatile int32_t busy; static volatile int32_t busy;
if (plat_xchg32(&busy, 1)) plat_exit(71); if (plat_xchg32(&busy, 1)) plat_exit(71);
rt_log_sync(); // the lines before the crash, then the report, each written as it is made
char d[160]; char d[160];
uintptr_t rip = x->rip; uintptr_t rip = x->rip;
rt_logf("FATAL: %s (%#x) at host %p (", x->what, x->code, (void *)rip); rt_logf("FATAL: %s (%#x) at host %p (", x->what, x->code, (void *)rip);

View File

@@ -1672,7 +1672,10 @@ int plat_stderr_path(char *out, size_t cap) {
int __llvm_profile_dump(void); int __llvm_profile_dump(void);
#endif #endif
void (*plat_exit_hook)(void);
void plat_exit(int code) { void plat_exit(int code) {
if (plat_exit_hook) plat_exit_hook();
#ifdef NX2NAVI_PGO_GEN #ifdef NX2NAVI_PGO_GEN
__llvm_profile_dump(); __llvm_profile_dump();
#endif #endif

View File

@@ -303,6 +303,7 @@ void plat_aligned_free(void *p);
// ---- process // ---- process
_Noreturn void plat_exit(int code); // ends the process at once: other threads stop, no atexit handlers run _Noreturn void plat_exit(int code); // ends the process at once: other threads stop, no atexit handlers run
extern void (*plat_exit_hook)(void); // called first by plat_exit when set (main.c: the queued log lines)
int plat_setenv(const char *name, const char *value); // value NULL removes it; 1 on success int plat_setenv(const char *name, const char *value); // value NULL removes it; 1 on success
uint32_t plat_process_id(void); uint32_t plat_process_id(void);
// CPU time of the process since it started (user and kernel), and wall time since it started, in ns; 0 when unknown. // CPU time of the process since it started (user and kernel), and wall time since it started, in ns; 0 when unknown.

View File

@@ -1135,7 +1135,10 @@ __attribute__((constructor)) static void pgo_flush_start(void) {
} }
#endif #endif
void (*plat_exit_hook)(void);
void plat_exit(int code) { void plat_exit(int code) {
if (plat_exit_hook) plat_exit_hook();
#ifdef NX2NAVI_PGO_GEN #ifdef NX2NAVI_PGO_GEN
__llvm_profile_dump(); __llvm_profile_dump();
#endif #endif

189
tests/logq_check.c Normal file
View File

@@ -0,0 +1,189 @@
// runtime/logq.c: log lines queued for the log thread, with a sink of the test's own that can be slow or held.
// Built and run by tests/logq_check.py; prints each failed check and exits 1 on any.
#include "logq.c"
#include <stdarg.h>
#include <stdio.h>
static int fails;
#define CHECK(c, ...) do { if (!(c)) { fails++; printf("FAIL line %d: ", __LINE__); printf(__VA_ARGS__); printf("\n"); } } while (0)
// ---- the sink: everything written, in the order it arrived; each call can take sink_ms, or wait while held is set
#define OUT_CAP (64u << 20)
static char *out;
static volatile size_t out_n;
static volatile int32_t sink_ms, held, sink_calls, in_sink;
static void sink(const char *p, size_t n) {
plat_xchg32(&in_sink, 1);
while (__atomic_load_n(&held, __ATOMIC_ACQUIRE)) plat_sleep_ms(1);
int ms = __atomic_load_n(&sink_ms, __ATOMIC_ACQUIRE);
if (ms) plat_sleep_ms((uint32_t)ms);
size_t at = __atomic_fetch_add(&out_n, n, __ATOMIC_SEQ_CST);
if (at + n <= OUT_CAP) memcpy(out + at, p, n);
__atomic_add_fetch(&sink_calls, 1, __ATOMIC_SEQ_CST);
plat_xchg32(&in_sink, 0);
}
// ---- the runtime logq.c reaches: its line about slow writes comes through here
void rt_log(const char *fmt, ...) {
char buf[512];
va_list ap;
va_start(ap, fmt);
int n = vsnprintf(buf, sizeof buf - 1, fmt, ap);
va_end(ap);
if (n < 0) return;
if (n > (int)sizeof buf - 2) n = (int)sizeof buf - 2;
buf[n++] = '\n';
logq_write(buf, (size_t)n);
}
// ---- producers: thread t writes lines "t <t> <seq> <padding>\n"
enum { THREADS = 8 };
typedef struct { int id, lines, pad; uint64_t worst_ns; } Prod;
static void producer(void *arg) {
Prod *p = arg;
char line[512];
for (int i = 0; i < p->lines; i++) {
int n = snprintf(line, sizeof line, "t %d %d %.*s\n", p->id, i, p->pad,
"................................................................................................"
"................................................................................................"
"................................................................................................");
uint64_t t0 = plat_now_ns();
logq_write(line, (size_t)n);
uint64_t d = plat_now_ns() - t0;
if (d > p->worst_ns) p->worst_ns = d;
}
}
// Waits until the sink has taken everything written so far.
static void settle(size_t want) {
for (int i = 0; i < 20000 && __atomic_load_n(&out_n, __ATOMIC_SEQ_CST) < want; i++) plat_sleep_ms(1);
}
// Every line of the output from at on is whole, and each thread's lines come in their order. Returns the lines seen;
// the slow-write lines ("log: ...") are counted in *notes.
static int check_lines(size_t at, int *next, int *notes) {
int lines = 0;
size_t end = __atomic_load_n(&out_n, __ATOMIC_SEQ_CST);
while (at < end) {
const char *nlp = memchr(out + at, '\n', end - at);
if (!nlp) { CHECK(0, "the output ends inside a line"); break; }
int t, seq;
if (!strncmp(out + at, "log: ", 5)) {
++*notes;
} else if (sscanf(out + at, "t %d %d", &t, &seq) == 2 && t >= 0 && t < THREADS) {
if (seq != next[t]) CHECK(0, "thread %d: line %d came where %d was due", t, seq, next[t]);
next[t] = seq + 1;
lines++;
} else {
CHECK(0, "a torn line at byte %zu: %.40s", at, out + at);
}
at = (size_t)(nlp - out) + 1;
}
return lines;
}
static size_t run(Prod *pr, int n, int lines, int pad) {
PlatThread *th[THREADS];
size_t bytes = 0;
for (int i = 0; i < n; i++) {
pr[i] = (Prod){.id = i, .lines = lines, .pad = pad};
th[i] = plat_thread_new(producer, &pr[i], "producer");
if (!th[i]) { printf("FATAL: no thread\n"); exit(3); }
}
for (int i = 0; i < n; i++) plat_thread_join(th[i]);
for (int i = 0; i < n; i++)
for (int k = 0; k < lines; k++) bytes += (size_t)snprintf(NULL, 0, "t %d %d %.*s\n", i, k, pad, "");
return bytes + (size_t)n * (size_t)lines * (size_t)pad;
}
int main(void) {
setvbuf(stdout, NULL, _IONBF, 0);
out = malloc(OUT_CAP);
if (!out) return 3;
Prod pr[THREADS];
int next[THREADS] = {0}, notes = 0;
logq_init(sink);
// 1. Before the queue is on, a line is written by the thread that logs it, before logq_write returns.
logq_write("t 0 0 \n", 7);
CHECK(out_n == 7 && sink_calls == 1, "a line without the queue: %zu bytes in %d writes", (size_t)out_n, (int)sink_calls);
size_t at = 0;
CHECK(check_lines(at, next, &notes) == 1, "the first line");
at = out_n;
for (int i = 0; i < THREADS; i++) next[i] = 0;
// 2. With the queue: eight threads, 20,000 lines each; every line arrives whole and each thread's in order.
CHECK(logq_async(), "the log thread did not start");
size_t want = at + run(pr, THREADS, 20000, 40);
settle(want);
CHECK(out_n == want, "%zu bytes written, %zu queued", (size_t)out_n - at, want - at);
CHECK(check_lines(at, next, &notes) == THREADS * 20000, "lines lost or doubled");
CHECK(sink_calls < THREADS * 20000 / 2, "%d writes for %d lines: the queue did not batch", (int)sink_calls, THREADS * 20000);
at = out_n;
for (int i = 0; i < THREADS; i++) next[i] = 0;
// 3. A slow destination (30 ms a write) does not hold the threads that log: 100 lines, a millisecond apart.
__atomic_store_n(&sink_ms, 30, __ATOMIC_RELEASE);
uint64_t worst = 0;
for (int i = 0; i < 100; i++) {
char line[64];
int n = snprintf(line, sizeof line, "t 0 %d \n", i);
uint64_t t0 = plat_now_ns();
logq_write(line, (size_t)n);
uint64_t d = plat_now_ns() - t0;
if (d > worst) worst = d;
want += (size_t)n;
plat_sleep_ms(1);
}
CHECK(worst < 15000000, "a line waited %.1f ms for a 30 ms write", worst / 1e6);
settle(want);
__atomic_store_n(&sink_ms, 0, __ATOMIC_RELEASE);
settle(want);
plat_sleep_ms(50); // the log thread's line about the slow writes
CHECK(check_lines(at, next, &notes) == 100, "lines lost behind a slow write");
CHECK(notes == 1, "%d lines about slow writes within ten seconds, want 1", notes);
want = at = out_n;
for (int i = 0; i < THREADS; i++) next[i] = 0;
// 4. A destination that stops while more is logged than the queue holds: the threads wait, nothing is lost.
__atomic_store_n(&held, 1, __ATOMIC_RELEASE);
logq_write("t 0 0 \n", 7); // the log thread takes it and stops in the sink
for (int i = 0; i < 5000 && !__atomic_load_n(&in_sink, __ATOMIC_ACQUIRE); i++) plat_sleep_ms(1);
CHECK(in_sink, "the log thread did not reach the sink");
PlatThread *th[THREADS];
for (int i = 0; i < THREADS; i++) {
pr[i] = (Prod){.id = i, .lines = 2000, .pad = 250}; // 8 x 2000 x 260 bytes: four times the queue
th[i] = plat_thread_new(producer, &pr[i], "producer");
}
plat_sleep_ms(200);
size_t stuck = out_n;
__atomic_store_n(&held, 0, __ATOMIC_RELEASE);
for (int i = 0; i < THREADS; i++) plat_thread_join(th[i]);
plat_sleep_ms(200);
CHECK(stuck == at, "%zu bytes were written while the destination was stopped", stuck - at);
at += 7; // past the line that stopped the log thread
CHECK(check_lines(at, next, &notes) == THREADS * 2000, "lines lost at a full queue");
at = out_n;
for (int i = 0; i < THREADS; i++) next[i] = 0;
// 5. logq_sync: what is queued is written before it returns, and later lines are written by their own thread.
__atomic_store_n(&sink_ms, 20, __ATOMIC_RELEASE);
for (int i = 0; i < 50; i++) {
char line[64];
int n = snprintf(line, sizeof line, "t 1 %d \n", i);
logq_write(line, (size_t)n);
}
logq_sync();
__atomic_store_n(&sink_ms, 0, __ATOMIC_RELEASE);
CHECK(check_lines(at, next, &notes) == 50, "logq_sync returned before the queued lines were written");
at = out_n;
int calls = sink_calls;
logq_write("t 2 0 \n", 7);
CHECK(out_n == at + 7 && sink_calls == calls + 1, "a line after logq_sync was not written at once");
logq_sync(); // a second call does nothing
CHECK(check_lines(at, next, &notes) == 1, "the line after logq_sync");
printf("logq_check: %s (%zu bytes in %d writes)\n", fails ? "FAILED" : "ok", (size_t)out_n, (int)sink_calls);
return fails ? 1 : 0;
}

52
tests/logq_check.py Normal file
View File

@@ -0,0 +1,52 @@
"""Build and run tests/logq_check.c: the queue that hands log lines to the log thread (runtime/logq.c), with the
platform layer linked and a destination of the test's own. No game files: the module layout the runtime includes is a
stand-in written to the test's build folder.
Usage: logq_check.py
"""
import os
import shutil
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
MSYS_BIN = Path(r'C:\msys64\ucrt64\bin')
def compiler():
if os.environ.get('CC'):
return os.environ['CC']
if os.name == 'nt' and (MSYS_BIN / 'clang.exe').exists():
return str(MSYS_BIN / 'clang.exe')
return shutil.which('clang') or 'clang'
LAYOUT = '''// A stand-in for the generated layout.h (tests/logq_check.py).
#pragma once
#define MOD_MAIN_BASE 0x7000000000ull
#define MOD_MAIN_SIZE 0x4ea0000ull
'''
def main():
work = ROOT / 'build' / 'test' / 'logq'
work.mkdir(parents=True, exist_ok=True)
(work / 'layout.h').write_text(LAYOUT)
win = os.name == 'nt'
exe = work / ('logq_check.exe' if win else 'logq_check')
env = dict(os.environ)
if win and MSYS_BIN.exists():
env['PATH'] = str(MSYS_BIN) + os.pathsep + env.get('PATH', '')
rt = ROOT / 'runtime'
plat = [rt / 'platform' / ('win32.c' if win else 'linux.c'), rt / 'platform' / 'common.c']
libs = ['-lbcrypt', '-lpsapi', '-lwinmm', '-ldbghelp', '-lshell32', '-lole32', '-luser32', '-ladvapi32'] if win \
else ['-lpthread']
subprocess.check_call([compiler(), '-O1', '-g', '-Wall', '-Wno-unused-function', '-fno-strict-aliasing',
'-march=x86-64-v3', '-I', str(rt), '-I', str(work),
str(ROOT / 'tests' / 'logq_check.c'), *map(str, plat), '-o', str(exe), '-lm', *libs],
env=env)
return subprocess.run([str(exe)], env=env, timeout=300).returncode
if __name__ == '__main__':
sys.exit(main())

221
tests/sync_wake_check.c Normal file
View File

@@ -0,0 +1,221 @@
// runtime/kernel.c: which WaitSynchronization waits a signalled event or an ended thread wakes (wake_sync_waiters),
// on guest threads whose entries are host functions, with the rest of the runtime stubbed. Built and run by
// tests/sync_wake_check.py; prints each failed check and exits 1 on any. With the argument "all" it runs with
// NX2NAVI_SYNC_WAKE_ALL=1 and checks that every wait is woken and blocks again.
#include "kernel.c"
#include <stdarg.h>
static int fails, wake_all;
#define CHECK(c, ...) do { if (!(c)) { fails++; printf("FAIL line %d: ", __LINE__); printf(__VA_ARGS__); printf("\n"); } } while (0)
// ---- the runtime kernel.c reaches
unsigned rt_trace;
uint64_t rt_guest_start_ns;
volatile uint32_t prof_replay_tid;
const char *cfg_get(const char *name) { return !strcmp(name, "NX2NAVI_SYNC_WAKE_ALL") && wake_all ? "1" : NULL; }
const char *cfg_value(const char *name) { (void)name; return "0"; } // no asynchronous preemption, no slow-service log
int cfg_bool(const char *name) { (void)name; return 0; }
int rt_ab(void) { return 0; }
int rt_knob(const char *name, const char *def) {
char key[64];
snprintf(key, sizeof key, "NX2NAVI_%s", name);
const char *e = cfg_get(key);
return strcmp(e && *e ? e : def, "0") != 0;
}
int64_t frame_step_world_us(void) { return -1; }
void ipc_handle_closed(int kind, void *obj) { (void)kind, (void)obj; }
const char *ldr_describe(uint64_t a, char *buf, int n) { snprintf(buf, (size_t)n, "%#llx", (unsigned long long)a); return buf; }
uint64_t ldr_symbol(const char *name) { (void)name; return 0; }
void perf_main_thread(void) {}
int rt_linked_modules(RtModule *out, int max) { (void)out, (void)max; return 0; }
const RtGuestLayout *mem_layout(void) { return NULL; }
void *mem_kernel_alloc(uint64_t size, uint64_t align) {
void *p = plat_aligned_alloc(size, align < 64 ? 64 : align);
if (!p) rt_fatal("out of memory");
memset(p, 0, size);
return p;
}
void mem_kernel_free(void *p, uint64_t size) { (void)size; plat_aligned_free(p); }
void rt_log(const char *fmt, ...) { (void)fmt; }
void rt_logf(const char *fmt, ...) { (void)fmt; }
void rt_fatal(const char *fmt, ...) {
va_list ap;
va_start(ap, fmt);
printf("FATAL: ");
vprintf(fmt, ap);
printf("\n");
va_end(ap);
fflush(stdout);
_Exit(3);
}
void rt_dispatch(Cpu *c) { (void)c; rt_fatal("rt_dispatch in the test"); }
void rt_svc(Cpu *c, uint32_t imm, uint64_t pc) { (void)pc; kern_svc(c, imm); }
// A guest entry here is a host function taking the thread's Cpu, with its argument in x0.
uint64_t rt_call(Cpu *c, uint64_t fn, int n, ...) {
va_list ap;
va_start(ap, n);
for (int i = 0; i < n; i++) c->x[i] = va_arg(ap, uint64_t);
va_end(ap);
((void (*)(Cpu *))(uintptr_t)fn)(c);
return c->x[0];
}
// ---- waiters
typedef struct {
uint32_t hs[2];
int n, loop; // loop: clear the event and wait again after each wake, counting in `wakes`
int64_t ns;
volatile uint32_t res, thread;
volatile int idx, done, wakes;
} Wait;
static void waiter(Cpu *c) {
Wait *w = (Wait *)(uintptr_t)c->x[0];
do {
c->x[1] = (uint64_t)(uintptr_t)w->hs, c->x[2] = (uint64_t)w->n, c->x[3] = (uint64_t)w->ns;
kern_svc(c, 0x18);
w->res = (uint32_t)c->x[0], w->idx = (int)c->x[1];
if (w->loop && !w->res) {
c->x[0] = w->hs[0];
kern_svc(c, 0x12); // ClearEvent on the readable end
__atomic_add_fetch(&w->wakes, 1, __ATOMIC_SEQ_CST);
}
} while (w->loop > 0 && !w->res);
__atomic_store_n(&w->done, 1, __ATOMIC_SEQ_CST);
kern_svc(c, 0x0A);
rt_fatal("ExitThread returned");
}
static uint32_t spawn(Wait *w) {
Cpu c = {0};
c.x[1] = (uint64_t)(uintptr_t)waiter, c.x[2] = (uint64_t)(uintptr_t)w;
c.x[3] = (uint64_t)(uintptr_t)mem_kernel_alloc(0x4000, 16) + 0x4000;
c.x[4] = 44, c.x[5] = (uint64_t)(int64_t)-2;
kern_svc(&c, 0x08);
if (c.x[0]) rt_fatal("CreateThread failed %#llx", (unsigned long long)c.x[0]);
w->thread = (uint32_t)c.x[1];
c.x[0] = w->thread;
kern_svc(&c, 0x09);
return w->thread;
}
static int until(volatile int *flag, int want, int ms) {
for (int i = 0; i < ms * 10; i++) {
if (__atomic_load_n(flag, __ATOMIC_SEQ_CST) == want) return 1;
plat_wait_ns(100000);
}
return 0;
}
static int in_sync(uint32_t thread, int ms) {
for (int i = 0; i < ms * 10; i++) {
if (kern_thread_state(thread) == KERN_WAIT_SYNC) return 1;
plat_wait_ns(100000);
}
return 0;
}
static void counts(uint64_t *woken, uint64_t *spared, uint64_t *blocks) {
lock();
*woken = g_sync_woken, *spared = g_sync_spared, *blocks = g_sync_blocks;
unlock();
}
// Waits blocked now: those woken for nothing are back in their wait.
static void settle(Wait **ws, int n) {
plat_wait_ns(20000000);
for (int i = 0; i < n; i++) CHECK(in_sync(ws[i]->thread, 2000), "waiter %d is not back in its wait", i);
}
int main(int argc, char **argv) {
wake_all = argc > 1 && !strcmp(argv[1], "all");
setvbuf(stdout, NULL, _IONBF, 0);
uint32_t ea = kern_event_new(), eb = kern_event_new(), ec = kern_event_new(), ed = kern_event_new();
uint32_t ra = kern_event_readable(ea), rb = kern_event_readable(eb), rc = kern_event_readable(ec);
uint64_t wk, sp, bl;
// 1. A waits on a, B on b, C on b and c. A signal of a wakes A; B and C stay in their waits.
Wait A = {.hs = {ra}, .n = 1, .ns = -1}, B = {.hs = {rb}, .n = 1, .ns = -1}, C = {.hs = {rb, rc}, .n = 2, .ns = -1};
spawn(&A), spawn(&B), spawn(&C);
CHECK(in_sync(A.thread, 2000) && in_sync(B.thread, 2000) && in_sync(C.thread, 2000), "the waiters did not block");
kern_event_signal(ea);
CHECK(until(&A.done, 1, 2000) && A.res == 0 && A.idx == 0, "A: result %#x index %d", A.res, A.idx);
settle((Wait *[]){&B, &C}, 2);
CHECK(!B.done && !C.done, "a signal of another event ended a wait");
// Two wakes pass B and C by: the signal of a, and the end of A's thread.
counts(&wk, &sp, &bl);
if (wake_all) CHECK(wk == 5 && sp == 0 && bl == 7, "all: woken %llu left %llu blocks %llu, want 5 0 7",
(unsigned long long)wk, (unsigned long long)sp, (unsigned long long)bl);
else CHECK(wk == 1 && sp == 4 && bl == 3, "woken %llu left %llu blocks %llu, want 1 4 3",
(unsigned long long)wk, (unsigned long long)sp, (unsigned long long)bl);
// 2. A wait on several handles is woken by its second one and reports its index.
kern_event_signal(ec);
CHECK(until(&C.done, 1, 2000) && C.res == 0 && C.idx == 1, "C: result %#x index %d", C.res, C.idx);
CHECK(in_sync(B.thread, 2000) && !B.done, "B left its wait at a signal of c");
// 3. A thread that ends wakes the wait on its handle: E waits for B's thread, and b ends B.
Wait E = {.hs = {B.thread}, .n = 1, .ns = -1};
spawn(&E);
CHECK(in_sync(E.thread, 2000), "E did not block");
kern_event_signal(eb);
CHECK(until(&B.done, 1, 2000) && B.res == 0 && B.idx == 0, "B: result %#x index %d", B.res, B.idx);
CHECK(until(&E.done, 1, 2000) && E.res == 0 && E.idx == 0, "E: the end of B's thread did not end its wait (result %#x)", E.res);
// 4. A timed wait keeps its time across other events' signals; a wait without handles is a sleep.
kern_event_clear(ea);
Wait F = {.hs = {ra}, .n = 1, .ns = 150000000}, Z = {.n = 0, .ns = 150000000};
uint64_t t0 = kern_now_ns();
spawn(&F), spawn(&Z);
CHECK(in_sync(F.thread, 2000) && in_sync(Z.thread, 2000), "F and Z did not block");
for (int i = 0; i < 20; i++) kern_event_signal(ed), plat_wait_ns(1000000);
CHECK(until(&F.done, 1, 2000) && F.res == R_TIMEOUT, "F: result %#x, want a timeout", F.res);
CHECK(until(&Z.done, 1, 2000) && Z.res == R_TIMEOUT, "Z: result %#x, want a timeout", Z.res);
CHECK(kern_now_ns() - t0 >= 150000000, "a timed wait ended after %.1f ms, before its 150 ms", (kern_now_ns() - t0) / 1e6);
// 5. CancelSynchronization ends a wait whatever it waits on.
Wait G = {.hs = {ra}, .n = 1, .ns = -1};
spawn(&G);
CHECK(in_sync(G.thread, 2000), "G did not block");
Cpu c = {0};
c.x[0] = G.thread;
kern_svc(&c, 0x19);
CHECK(until(&G.done, 1, 2000) && G.res == R_CANCELLED, "G: result %#x, want cancelled", G.res);
// 6. No signal is lost: eight threads each wait on their own event in a loop; every signal is seen by its thread.
enum { N = 8, ROUNDS = 4000 };
static Wait L[N];
uint32_t ew[N];
for (int i = 0; i < N; i++) {
ew[i] = kern_event_new();
L[i] = (Wait){.hs = {kern_event_readable(ew[i])}, .n = 1, .ns = -1, .loop = 1};
spawn(&L[i]);
}
unsigned seed = 12345;
int lost = 0;
for (int r = 0; r < ROUNDS && !lost; r++) {
seed = seed * 1664525u + 1013904223u;
int i = (int)(seed >> 24) % N;
int before = __atomic_load_n(&L[i].wakes, __ATOMIC_SEQ_CST);
kern_event_signal(ew[i]);
uint64_t end = kern_now_ns() + 5000000000ull;
while (__atomic_load_n(&L[i].wakes, __ATOMIC_SEQ_CST) == before && kern_now_ns() < end) plat_yield();
if (__atomic_load_n(&L[i].wakes, __ATOMIC_SEQ_CST) == before) {
lost = 1;
CHECK(0, "round %d: thread %d never saw its signal", r, i);
}
}
int total = 0;
for (int i = 0; i < N; i++) total += L[i].wakes;
CHECK(lost || total == ROUNDS, "%d wakes for %d signals", total, ROUNDS);
for (int i = 0; i < N; i++) { // end them: the last wake is not followed by another wait
L[i].loop = -1;
kern_event_signal(ew[i]);
CHECK(until(&L[i].done, 1, 5000), "looping waiter %d did not end", i);
}
counts(&wk, &sp, &bl);
printf("sync_wake_check%s: %s (signals woke %llu waits and left %llu asleep; %llu blocks)\n", wake_all ? " all" : "",
fails ? "FAILED" : "ok", (unsigned long long)wk, (unsigned long long)sp, (unsigned long long)bl);
return fails ? 1 : 0;
}

53
tests/sync_wake_check.py Normal file
View File

@@ -0,0 +1,53 @@
"""Build and run tests/sync_wake_check.c: which WaitSynchronization waits a signalled event or an ended thread wakes
(runtime/kernel.c wake_sync_waiters), with the platform layer linked and the rest of the runtime stubbed. No game files:
the module layout the runtime includes is a stand-in written to the test's build folder.
Usage: sync_wake_check.py
"""
import os
import shutil
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
MSYS_BIN = Path(r'C:\msys64\ucrt64\bin')
def compiler():
if os.environ.get('CC'):
return os.environ['CC']
if os.name == 'nt' and (MSYS_BIN / 'clang.exe').exists():
return str(MSYS_BIN / 'clang.exe')
return shutil.which('clang') or 'clang'
LAYOUT = '''// A stand-in for the generated layout.h (tests/sync_wake_check.py).
#pragma once
#define MOD_MAIN_BASE 0x7000000000ull
#define MOD_MAIN_SIZE 0x4ea0000ull
'''
def main():
work = ROOT / 'build' / 'test' / 'sync_wake'
work.mkdir(parents=True, exist_ok=True)
(work / 'layout.h').write_text(LAYOUT)
win = os.name == 'nt'
exe = work / ('sync_wake_check.exe' if win else 'sync_wake_check')
env = dict(os.environ)
if win and MSYS_BIN.exists():
env['PATH'] = str(MSYS_BIN) + os.pathsep + env.get('PATH', '')
rt = ROOT / 'runtime'
plat = [rt / 'platform' / ('win32.c' if win else 'linux.c'), rt / 'platform' / 'common.c']
libs = ['-lbcrypt', '-lpsapi', '-lwinmm', '-ldbghelp', '-lshell32', '-lole32', '-luser32', '-ladvapi32'] if win \
else ['-lpthread']
subprocess.check_call([compiler(), '-O1', '-g', '-Wall', '-Wno-unused-function', '-fno-strict-aliasing',
'-march=x86-64-v3', '-I', str(rt), '-I', str(work),
str(ROOT / 'tests' / 'sync_wake_check.c'), *map(str, plat), '-o', str(exe), '-lm', *libs],
env=env)
rc = subprocess.run([str(exe)], env=env, timeout=300).returncode
return subprocess.run([str(exe), 'all'], env=env, timeout=300).returncode or rc
if __name__ == '__main__':
sys.exit(main())