mirror of
https://github.com/tangerineturtle1/NX2navi.git
synced 2026-11-15 17:09:57 -04:00
6b95c867c8
vk_memory.c handed out page write generations from a 32-bit counter (gen_now) and every consumer compared them as 32-bit values, with 0 meaning "never stamped". At 124 to 336 generations a frame (a 0.10.0 log with trace = perf) the counter wrapped after about 66 h of play at 60 fps, 27.6 h at 144 and 15 to 17 h at 240. Past the wrap a stamp made before it compared newer than every later check and one made after it older: textures and VRAM mirrors missed the CPU's writes (stale textures, and the lazy fetch copying older mirror bytes over a CPU write), pages nothing wrote read as written at every check (a hash or upload every time, and every such page of a mirrored pool copied up at each sync), and the generation handed out at the wrap was 0. At 64 bits nothing wraps: 2^64 lasts millions of years at 336 a frame and 240 fps. Below 2^32 every value and every compare is the same as before. What changed: gen_now, both tables (gen_chunk, cgen_chunk), reg_max and creg_max, PoolRec's synced_gen, mgen and sgen, dirty_gen and DIRTY_UNSYNCED, VSample.mgen, every local and parameter that holds a generation in vk_memory.c, vk_texture.c (Tex.wgen and gpu_gen, PEnt.since, storage_rewritten and its callers, sweep_hard's write-back check included) and tex_probe.c, and vk.h's vk_mem_changed, vk_mem_changed_snap and vk_mem_unchanged(_why)_locked. The verify mode's mismatch line prints the generations with %llu. The readers that take no gen_lock (lazy_clean, cpu_wrote_since, verify_cb, vk_mem_sizes) stay lock-free at 64 bits: the slots, mgen, gen_now and a new chunk pointer are now stored atomically (relaxed, or release where a reader acquires), the same instructions on x86-64, and a static assertion holds that 64-bit atomics are lock-free. No lock, wait or thread changes. gen_next, which hands out every generation, no longer warns at 0xC0000000 and up or at the wrap, which cannot come; it logs one line when the count reaches 2^32, where earlier versions ran out, with the frame, the hours and the rate. Cost: the tables take 8 bytes a page instead of 4 (512 KiB more per 256 MiB of guest address space stamped, both tables together, allocated as before when a chunk is first stamped), reg_max and creg_max 2 MiB each instead of 1, Tex and PoolRec 8 bytes more, PEnt 8 more. The page walks read 8 bytes a slot; a micro-benchmark of the walk measured no difference. Not reproduced in the game before the change: the wrong answers were reproduced by compiling vk_memory.c's generation code with the counter seeded below 2^32. tests/gen_wrap_check now runs that code, with the readers outside its section, across 2^32, 2^40 and 2^63 and checks that the compares answer as below the boundary; on the code before the change 19 of its checks fail at 2^32 and the other two seeds do not fit the counter. The new tests/gen_width_check compiles vk_memory.c, vk_texture.c and tex_probe.c with -Wshorten-64-to-32, -Wformat and -Wincompatible-pointer-types as errors, so a 32-bit local left behind is found (shown by reverting single locals, fields and formats). Checks that passed: gen_wrap_check, gen_width_check, pass_cc_check of every runtime file, no new -Wall -Wextra warnings in the three files, tex_release_check, tex_compact_check, wb_lazy_check, gpa_check, mem_growth_check, mem_budget_check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
72 lines
3.3 KiB
Python
72 lines
3.3 KiB
Python
"""No 64-to-32-bit narrowing in the files that hold page write generations: runtime/vk_memory.c, runtime/vk_texture.c
|
|
and runtime/tex_probe.c, compiled with the build's flags plus -Wshorten-64-to-32, -Wformat and
|
|
-Wincompatible-pointer-types as errors (clang -fsyntax-only). The generations are 64 bits (vk_memory.c "page write
|
|
generations"); one local, field or format left at 32 bits would bring the wrap of 2^32 back without a word, and the
|
|
normal build sees a narrowing only as a silent conversion. The baseline is zero such diagnostics in the whole of the
|
|
three files: a narrowing elsewhere in them takes an explicit cast, which also says it is meant.
|
|
|
|
Needs clang, the Vulkan headers, VMA and the shader headers of tools/build_shaders.py (runtime/*_spv.h). No GPU, no
|
|
game files, any Python.
|
|
|
|
Usage: gen_width_check.py [--cc <clang>]
|
|
"""
|
|
import argparse
|
|
import os
|
|
import re
|
|
import subprocess
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
ROOT = Path(__file__).resolve().parents[1]
|
|
sys.path.insert(0, str(ROOT / 'tools'))
|
|
import build_game # noqa: E402 (RT_FLAGS, the compiler and the include folders of the build)
|
|
|
|
FILES = ['vk_memory.c', 'vk_texture.c', 'tex_probe.c']
|
|
STRICT = ['-Werror=shorten-64-to-32', '-Werror=format', '-Werror=incompatible-pointer-types']
|
|
# the diagnostics this check is about, by the flag clang names at the end of the line
|
|
WATCHED = re.compile(r'^(?P<where>[^\n]*?:\d+:\d+): (?:error|warning): (?P<what>[^\n]*?)'
|
|
r'\[(?:-Werror,)?(?P<flag>-Wshorten-64-to-32|-Wformat[\w-]*|-Wincompatible[\w-]*pointer-types)\]',
|
|
re.M)
|
|
|
|
|
|
def main():
|
|
ap = argparse.ArgumentParser()
|
|
ap.add_argument('--cc', help='the C compiler (default: the build\'s clang)')
|
|
args = ap.parse_args()
|
|
if not (ROOT / 'runtime' / 'wb_merge_spv.h').is_file():
|
|
print('gen_width_check: FAIL: runtime/wb_merge_spv.h is missing: run tools/build_shaders.py first')
|
|
return 1
|
|
cc = args.cc or os.environ.get('CC') or str(build_game.CLANG)
|
|
build_game.set_cpu('v3') # adds -DRT_CPU_LEVEL=3, as the build does
|
|
flags = [f for f in build_game.RT_FLAGS if f != '-g']
|
|
inc = ['-I', str(ROOT / 'runtime'), '-I', str(build_game.vma_include())]
|
|
vk = build_game.vulkan_include(cc)
|
|
if vk:
|
|
inc = ['-I', str(vk)] + inc
|
|
env = dict(os.environ)
|
|
if os.name == 'nt':
|
|
env['PATH'] = str(Path(cc).parent) + os.pathsep + env.get('PATH', '')
|
|
bad = 0
|
|
for name in FILES:
|
|
src = ROOT / 'runtime' / name
|
|
p = subprocess.run([cc, '-fsyntax-only', '-std=gnu11'] + flags + STRICT + inc + [str(src)], capture_output=True,
|
|
text=True, errors='replace', env=env)
|
|
hits = [m for m in WATCHED.finditer(p.stderr)]
|
|
for m in hits:
|
|
print(f'gen_width_check: FAIL: {m.group("where")}: {m.group("what").strip()} [{m.group("flag")}]')
|
|
if p.returncode and not hits:
|
|
print(f'gen_width_check: FAIL: runtime/{name} does not compile:\n{p.stderr[-3000:]}')
|
|
bad += 1
|
|
bad += len(hits)
|
|
if not hits and not p.returncode:
|
|
print(f'gen_width_check: runtime/{name}: no narrowing, format or pointer-type diagnostic')
|
|
if bad:
|
|
print(f'gen_width_check: {bad} problems')
|
|
return 1
|
|
print('gen_width_check: all checks passed')
|
|
return 0
|
|
|
|
|
|
if __name__ == '__main__':
|
|
sys.exit(main())
|