Files
NX2navi/tests/lazy_fetch_check.py
chrg6 ef0673fba0 Mark a region for the lazy write-back before its pages become inaccessible, so that a read during an arm is not passed to the crash handler.
mem_lazy_arm made a range's pages inaccessible (lazy_protect_locked: the protect, then the back end's clean test) and set the region's lzr mark only after the segment was in the list. lazy_fault returns 0 at once for an unmarked region, without taking frz_lock, so a CPU read in that window found the page inaccessible in access_fault and went to the crash handler as an access violation. A write survived only because thaw_fault waits for frz_lock. Arms come from the completion thread (vk_memory.c vk_gpu_written_back) when a submission's fence completes, which is when the game's own fence wait returns and it reads the results; the window lasts as long as the protect and the clean test, which can take tens of milliseconds while the driver pins memory.

The mark is now set before the protect, after the cheap refusals, and put back when the protect or the clean test refuses the arm. A fault from the protect on finds the mark, waits for frz_lock and then finds the segment, or the pages given back. Every store to lzr is made under frz_lock, which the arm holds, so putting it back is exact.

Not reproduced in the game before the change. The new tests\lazy_fetch_check.py reproduces it with the real Windows fault handler: a thread reads a page while it is armed and given back 20,000 times (arm_race: an access violation in 5 of 6 runs before the change), and once with a clean test of 100 ms (arm_race_slow: an access violation in every run). Both pass in every run since. wb_lazy_check, heap_lazy_copy_check, freeze_hold_check and mem_growth_check passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 20:12:50 -05:00

91 lines
3.5 KiB
Python

"""Build and run tests/lazy_fetch_check.c: runtime/memory.c's lazy write-back (memory.c "Lazy write-back") with the
Windows platform layer and its real fault handler, the rest of the runtime stubbed and the back end's fetch a stand-in
that only waits. Each scenario runs as its own process, so an access violation passed to the crash handler shows as
its exit code; any exit code other than 0, or a scenario that does not end within 120 s, fails.
Windows only (the fault handler is): no game files, no GPU. The module layout the runtime includes is a stand-in
written to the test's build folder.
Usage: lazy_fetch_check.py [scenario ...]
"""
import os
import shutil
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
MSYS_BIN = Path(r'C:\msys64\ucrt64\bin')
# name: (scenario argument, extra environment)
SCENARIOS = {
'arm_race': ('arm_race', {}),
'arm_race_slow': ('arm_race_slow', {}),
}
def compiler():
if os.environ.get('CC'):
return os.environ['CC']
if os.name == 'nt' and (MSYS_BIN / 'clang.exe').exists():
return str(MSYS_BIN / 'clang.exe')
return shutil.which('clang') or 'clang'
LAYOUT = '''// A stand-in for the generated layout.h (tests/lazy_fetch_check.py).
#pragma once
#define MOD_MAIN_BASE 0x7000000000ull
#define MOD_MAIN_SIZE 0x4ea0000ull
#define MOD_COUNT 3
'''
def main():
if os.name != 'nt':
print('lazy_fetch_check: the fault handler it runs is Windows only; skipped')
return 0
names = sys.argv[1:] or list(SCENARIOS)
unknown = [n for n in names if n not in SCENARIOS]
if unknown:
print('lazy_fetch_check: unknown scenario %s; known: %s' % (', '.join(unknown), ', '.join(SCENARIOS)))
return 2
work = ROOT / 'build' / 'test' / 'lazy_fetch'
work.mkdir(parents=True, exist_ok=True)
(work / 'layout.h').write_text(LAYOUT)
exe = work / 'lazy_fetch_check.exe'
env = dict(os.environ)
if MSYS_BIN.exists():
env['PATH'] = str(MSYS_BIN) + os.pathsep + env.get('PATH', '')
rt = ROOT / 'runtime'
plat = [rt / 'platform' / 'win32.c', rt / 'platform' / 'common.c']
libs = ['-lbcrypt', '-lws2_32', '-lpsapi', '-lwinmm', '-ldbghelp', '-lshell32', '-lole32', '-luser32', '-ladvapi32']
subprocess.check_call([compiler(), '-O1', '-g', '-Wall', '-Wno-unused-function', '-fno-strict-aliasing',
'-march=x86-64-v3', '-I', str(rt), '-I', str(work),
str(ROOT / 'tests' / 'lazy_fetch_check.c'), *map(str, plat), '-o', str(exe), '-lm',
*libs], env=env)
bad = 0
for name in names:
arg, extra = SCENARIOS[name]
try:
p = subprocess.run([str(exe), arg], env={**env, **extra}, capture_output=True, text=True, timeout=120)
except subprocess.TimeoutExpired:
bad += 1
print('lazy_fetch_check: %s: FAIL: did not end within 120 s' % name)
continue
code = p.returncode & 0xFFFFFFFF
if code:
bad += 1
print('lazy_fetch_check: %s: FAIL: exit code %#x%s' % (
name, code, ' (an access violation)' if code == 0xC0000005 else ''))
sys.stdout.write(p.stdout)
else:
print('lazy_fetch_check: %s: passed' % name)
if bad:
print('lazy_fetch_check: %d of %d scenarios failed' % (bad, len(names)))
return 1
print('lazy_fetch_check: all %d scenarios passed' % len(names))
return 0
if __name__ == '__main__':
sys.exit(main())