Add a hang watch that logs a report when no frame is shown with the window in front, or the GPU does not finish, for 20 s.

A present, a submission or a completion that never came back (a deadlock, a driver wait that does not return, a
GPU hang without a reset) left a log that just stopped: kernel.c wait_watch covers guest lock and thread-end waits
only and is off by default, and nothing watched the present, submit or completion threads.

hang_watch.c runs a thread of its own in a game run (main.c starts it before the boot; never in the tool modes).
Once a second it reads, without a lock, gfx_counters.frames and the submissions issued and done (the new
vk_sync.c vk_sync_progress, no device call). It reports a stall of the frames while the guest runs, the window is
in front and not minimized (gfx_window_in_front) and no orderly end has begun (rt_exit_started): the game stops
presenting without the focus, so a focus change starts the count again, and the boot screen is not judged. It
reports submissions issued and not done, "done" unchanged for N s, whatever the focus. One report per stall, a
new stall needs progress, at most 3 a run. A report names the stall with the counts, the focus and the world
clock; who holds the present, queue and window-start locks (gfx_present.c gfx_lock_holders, read from
PlatRMutex.owner and depth); the guest threads' lock and thread-end waits (kernel.c kern_wait_report, wait_watch's
walk factored out as wait_pass, run with limit 0 under ten 10 ms trylocks of the kernel lock); and then writes the
log out with logq_flush, never rt_log_sync, which would put every later line on the thread that logs it in a run
that may recover. It never ends the process and adds no wait to a drawing thread. The per-thread wait sites and
lock owner wrappers are left for a later change.

NX2NAVI_HANG_WATCH_S sets the seconds (default 20, 0..600, read at start, 0 off): a metas row, a known[] name, a
diagnostics sections[] entry and an nx2navi.ini.example entry.

Not reproduced before the change (a diagnostic). The new tests/hang_watch_check.py drives hang_watch.c's decisions
on made-up readings: no report while things move, unfocused, minimized or ending; one per stall of the frames with
the window in front and one again after progress; one for submissions not done whatever the focus; none for an
idle GPU; at most 3; logq_flush called and logq_sync never. tests/cfg_rows_check.py (0 problems), the scratchpad
rows check ("missing []") and the compile checks of every runtime file pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
chrg6
2026-11-13 17:38:20 -05:00
parent 0fbd2b8f31
commit b4556957c2
11 changed files with 332 additions and 20 deletions

View File

@@ -25,6 +25,9 @@ void rt_log_raw(const char *p, size_t n);
const char *rt_log_path(void); // the log file, NULL when this run writes none
void rt_log_sync(void); // writes the queued log lines now; later lines are written as they are made
void rt_log_ending(void); // the process is ending: its lines reach the log file past its 64 MiB cap
void hang_watch_start(void); // hang_watch.c: NX2NAVI_HANG_WATCH_S's thread, in a game run
void kern_wait_report(void); // kernel.c: the guest threads' lock and thread-end waits, for a hang report
void vk_sync_progress(uint64_t *issued, uint64_t *done); // vk_sync.c: submissions issued and done, no lock or device call
enum { RT_TRACE_SVC = 1, RT_TRACE_HOSTCALL = 2, RT_TRACE_IPC = 4, RT_TRACE_FS = 8, RT_TRACE_GFX = 16, RT_TRACE_PERF = 32,
RT_TRACE_SCHED = 64, RT_TRACE_AUDIO = 128, RT_TRACE_PRESENT = 256 };
extern unsigned rt_trace; // from NX2NAVI_TRACE=svc,host,ipc,fs,gfx,perf,sched,audio,present (or all)
@@ -116,6 +119,7 @@ void gfx_window_closed(void); // gfx_present.c (gfx_wsys.h): t
// --seconds). rt_exit_begin: 1 for the first caller; another thread waits there for good; 0 for that caller's own
// thread again. rt_exit_steps: the saves and the put-backs, the slow pipeline save late and the final save flush last.
int rt_exit_begin(void);
int rt_exit_started(void); // an orderly end has begun (the hang watch stops judging presents)
enum { RT_EXIT_SETTINGS = 1 }; // rt_exit_steps also writes the menu's changes to nx2navi.ini
void rt_exit_steps(unsigned what);
void perf_gpu_wait(uint64_t t0); // a wait for a GPU fence ended (any thread; the render thread's also counts as PERF_FENCE)