Redraw the in-game menu with the game's own interface textures, and bring in the launcher's save import and faster builds.

In-game menu: the settings window, its tabs, rows, buttons and cursor, the performance overlay and the Preparing
shaders box are drawn with the game's interface textures where the RomFS holds them (runtime/ui/menu_skin.cpp,
perf_panel.cpp; plain shapes until they are decoded). Restart joins Continue and Quit, each behind a confirmation;
VSync is an on/off switch; DLSS is listed only on NVIDIA GPUs; the menu sounds are quieter; Picture shape is gone and
the setting descriptions are shorter; edge cases of input, layout and fonts are fixed. tools/menu_preview.py draws the
menu without the game.

Launcher: Import Saves takes a save from the game folder's saves/ folder or one the player picks; downloads fall back
to Windows' own certificate check when Python's refuses a server; updates are faster; the title bar is drawn in the
window and Navi is every icon. tools/make_game.py builds with CLANG64's clang where the launcher installed it, packs
the RomFS beside the lift and the compile, takes a prebuilt runtime, and compiles the game's resources without cmd.exe,
so a user folder with a space in its name builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
chrg6
2026-11-08 03:57:01 -05:00
parent 5b28da3a06
commit 8daedbd806
42 changed files with 4138 additions and 972 deletions

View File

@@ -10,7 +10,8 @@ Steps, each skipped when a run before finished it for the same inputs (state in
2. exefs the five ExeFS files, every module's segments unpacked (ZBIC through the player's own `zbic` package)
and stored plainly, as the runtime's loader reads them -> <out>/exefs
3. romfs every RomFS file, checked against the container's sha256, into one image -> <out>/romfs/romfs.bin and
index.json (16-byte aligned, in the byte order of the paths)
index.json (16-byte aligned, in the byte order of the paths). When the lift and the compile follow, it
runs on a thread of its own beside them: only the RomFS index and romfs.c wait for the image
4. lift recomp/gen.py with recomp/patches-v111.txt -> <out>/build-cache/lift, plus the two SDK bring-up fixes
5. compile the lifted C and runtime/ with MSYS2's clang, linked into <out>/ocarina.exe; the MSYS2 DLLs it loads,
the shader translator and romfs-index.bin beside it
@@ -20,8 +21,16 @@ Steps, each skipped when a run before finished it for the same inputs (state in
The exefs and romfs steps depend only on the game files; lift and compile depend on this kit's code too. A newer kit
therefore lifts and compiles again but keeps the unpacked game files, and --rebuild needs no game files at all. The
compile step keeps every object with the dependency list clang wrote for it (build-cache/obj/objects.json) and
compiles a file again only when it, a header it includes, the flags or the compiler changed. The new ocarina.exe
replaces the old one only once it links, so a failed build leaves the last good one in place.
compiles a file again only when it, a header it includes, the flags or the compiler changed; the kit's own folder
is not part of that key, so a kit in another folder (the launcher keeps each version in its own) reuses the objects.
A kit with lib/libnx2navirt.a built by --make-runtime-lib for the same runtime/ and the same compiler links the
runtime from it and compiles only the runtime files that include headers written per install. The new ocarina.exe
replaces the old one only once it links (with lld when the toolchain has it), so a failed build leaves the last good
one in place. Compiles run on every logical processor, the largest files first. The lifted code is built for this
PC's x86-64 level (cpu_level), with MSYS2 CLANG64's clang when it sits beside the UCRT64 folder and reports the same
version (fast_clang: the same objects, about a fifth faster).
python tools/make_game.py --make-runtime-lib <kit>/lib --msys <ucrt64> # the release's prebuilt runtime
Nothing is written outside --out. The game folder then holds only what this PC built from the player's files: never
share it. Needs Python 3.10+ with numpy, lz4, zstandard and zbic (pip), and MSYS2 UCRT64's clang and libraries
@@ -43,6 +52,7 @@ import shutil
import struct
import subprocess
import sys
import threading
import time
from pathlib import Path
@@ -76,11 +86,14 @@ LIFT_FIXES = [
LINK_LIBS = ['-lvulkan-1', '-lSDL3', '-lopus', '-lvpx', '-l:libdav1d.a', '-lzstd', '-l:libatomic.a', '-lbcrypt',
'-limm32', '-luuid', '-lole32', '-luser32', '-lshell32', '-lgdi32', '-ladvapi32', '-lwinmm']
SYSTEM_DLLS_SKIP = {'vulkan-1.dll'} # the graphics driver's loader, never a copy
# GNU windres's preprocessor, named explicitly: left to find it, windres puts the full path of the gcc beside it on a
# cmd.exe command line, and once that path and the .rc's both hold a space, cmd.exe drops the outer quotes and splits
# the command at the first space. gcc is then found on PATH (the toolchain's bin first); the arguments are windres's own.
WINDRES_PP = ['--preprocessor=gcc', '--preprocessor-arg=-E', '--preprocessor-arg=-xc', '--preprocessor-arg=-DRC_INVOKED']
EXE_NAME = 'ocarina.exe'
# Headers this build writes per install (gen.py, romfs_index.py): a runtime source that includes one, directly or
# through a runtime header, is compiled on the player's PC; every other runtime source can come prebuilt.
GENERATED_HEADERS = ('layout.h', 'patches.h', 'romfs_config.h')
RUNTIME_LIB = 'libnx2navirt.a'
RUNTIME_PATTERNS = ('*.c', '*.cpp', 'platform/*.c', 'ui/*.cpp', 'third_party/imgui/*.cpp',
'third_party/imgui/backends/*.cpp')
INCLUDE = re.compile(r'^[ \t]*#[ \t]*include[ \t]+"([^"]+)"', re.M)
# Runs a script the way `python script.py` does, with the script's folder first on sys.path. The launcher's
# embeddable Python leaves the script folder off sys.path (its ._pth file), so every script this build starts goes
@@ -89,24 +102,53 @@ RUN_SCRIPT = ('import os, runpy, sys; s = sys.argv[1]; sys.argv = sys.argv[1:];
'sys.path.insert(0, os.path.dirname(os.path.abspath(s))); runpy.run_path(s, run_name="__main__")')
EVENTS = False
OUT_LOCK = threading.Lock() # the RomFS step prints from its own thread (main)
def say(msg):
print(msg, flush=True)
with OUT_LOCK:
print(msg, flush=True)
def emit(ev, **fields):
"""One "@@{json}" line for the launcher (--events); nothing otherwise."""
if EVENTS:
print('@@' + json.dumps({'ev': ev, **fields}, ensure_ascii=False), flush=True)
line = '@@' + json.dumps({'ev': ev, **fields}, ensure_ascii=False)
with OUT_LOCK:
print(line, flush=True)
def fail(msg):
emit('error', message=msg)
print(f'\nerror: {msg}', file=sys.stderr, flush=True)
with OUT_LOCK:
print(f'\nerror: {msg}', file=sys.stderr, flush=True)
sys.exit(1)
class Background:
"""One step on a thread of its own. join() waits for it and raises its failure (fail's SystemExit included) in
the caller; failed() tells the caller early, so it can stop work that would be thrown away."""
def __init__(self, fn):
self.exc = None
self.thread = threading.Thread(target=self._run, args=(fn,), name='romfs', daemon=True)
self.thread.start()
def _run(self, fn):
try:
fn()
except BaseException as e: # noqa: B036 - handed to join()
self.exc = e
def failed(self):
return self.exc is not None
def join(self):
self.thread.join()
if self.exc is not None:
raise self.exc
def py_cmd(script, *args):
"""The command line that runs a Python script of this kit with this interpreter (see RUN_SCRIPT)."""
flags = ['-I'] if sys.flags.isolated else []
@@ -317,11 +359,11 @@ def step_romfs(src, out, jobs):
done += size
emit('progress', step='romfs', done=done, total=total, unit='bytes')
if start // batch % 40 == 0:
say(f' {done / 1e9:5.1f} of {total / 1e9:.1f} GB, {time.time() - t0:.0f} s')
say(f' RomFS {done / 1e9:5.1f} of {total / 1e9:.1f} GB, {time.time() - t0:.0f} s')
part.replace(dst / 'romfs.bin')
index = {'status': 'pass', 'romfs_size': pos, 'romfs_sha256': h.hexdigest(), 'files': entries}
(dst / 'index.json').write_text(json.dumps(index), encoding='utf-8')
say(f' image {pos / 1e9:.2f} GB in {time.time() - t0:.0f} s')
say(f' RomFS image {pos / 1e9:.2f} GB in {time.time() - t0:.0f} s')
# ---------------------------------------------------------------- lift
@@ -491,17 +533,33 @@ def parse_deps(text):
class ObjectCache:
"""Which objects are current: per object the key of its command and source, and the sha256 of every file its
dependency list names (build-cache/obj/objects.json)."""
"""Which objects are current: per object the key of its command and source, and the sha256 (with size and
modification time, to skip hashing an untouched file) of every file its dependency list names
(build-cache/obj/objects.json). Paths under a root are kept relative to it ({'<kit>': KIT}), so the same files
in a kit at another place still count as the same."""
def __init__(self, path):
def __init__(self, path, roots=None):
self.path = path
self.roots = {k: os.path.normcase(os.path.abspath(v)) for k, v in (roots or {}).items()}
try:
self.entries = json.loads(path.read_text(encoding='utf-8')) if path.is_file() else {}
except (OSError, ValueError):
self.entries = {}
self._sha = {}
def norm(self, p):
a = os.path.normcase(os.path.abspath(p))
for name, root in self.roots.items():
if a == root or a.startswith(root + os.sep):
return name + a[len(root):].replace(os.sep, '/')
return a
def real(self, p):
for name, root in self.roots.items():
if p.startswith(name):
return root + p[len(name):].replace('/', os.sep)
return p
def file_sha(self, p):
p = os.path.normcase(os.path.abspath(p))
if p not in self._sha:
@@ -511,18 +569,37 @@ class ObjectCache:
self._sha[p] = None
return self._sha[p]
def _same(self, p, rec):
path = self.real(p)
if isinstance(rec, str): # a cache without sizes and times
return self.file_sha(path) == rec
digest, size, mtime = rec
try:
st = os.stat(path)
except OSError:
return False
if st.st_size == size and st.st_mtime_ns == mtime:
return True
return st.st_size == size and self.file_sha(path) == digest
def current(self, obj, key):
e = self.entries.get(obj.name)
return (e is not None and e.get('key') == key and obj.is_file()
and all(self.file_sha(p) == h for p, h in e.get('deps', {}).items()))
and all(self._same(p, rec) for p, rec in e.get('deps', {}).items()))
def record(self, obj, key, dep_file, src):
try:
deps = parse_deps(Path(dep_file).read_text(encoding='utf-8', errors='replace'))
except OSError:
deps = []
deps = deps or [str(src)]
self.entries[obj.name] = {'key': key, 'deps': {p: self.file_sha(p) for p in deps}}
out = {}
for p in deps or [str(src)]:
try:
st = os.stat(p)
out[self.norm(p)] = [self.file_sha(p), st.st_size, st.st_mtime_ns]
except OSError:
out[self.norm(p)] = [None, -1, -1]
self.entries[obj.name] = {'key': key, 'deps': out}
def forget(self, obj):
self.entries.pop(obj.name, None)
@@ -539,22 +616,31 @@ def compiler_id(clang, env):
return sha256_bytes(f'{p.stdout}\n{st.st_size}\n{int(st.st_mtime)}'.encode())
def compile_all(srcs, cmd_for, cache, jobs, env, progress=None, salt=''):
def compile_all(srcs, cmd_for, cache, jobs, env, progress=None, salt='', stop=None):
"""Compile each (source, object) whose object is not current in cache. cmd_for(src, dst) -> argv without the
dependency-file options; salt identifies the compiler. Returns (compiled, reused, failures) with failures as
(src, stderr tail)."""
(src, stderr tail). The cache's roots stand for their paths in the flags too. The largest sources start first:
compile time grows with size, and a big file that started last would run on alone after the others finished.
stop() true makes the compiles not yet started end at once, unrecorded (a step beside this one failed)."""
todo, keys = [], {}
for src, dst in srcs:
cmd = cmd_for(src, dst)
flags = [c for c in cmd if c not in (str(src), str(dst))]
flags = [c for c in cmd[1:] if c not in (str(src), str(dst))]
for name, root in cache.roots.items():
flags = [name + os.path.normcase(f)[len(root):].replace(os.sep, '/') if os.path.normcase(f).startswith(root)
else f for f in flags]
flags = [os.path.basename(cmd[0])] + flags
keys[dst] = sha256_bytes(json.dumps([salt, flags, cache.file_sha(src)]).encode())
if not cache.current(dst, keys[dst]):
todo.append((src, dst))
reused = len(srcs) - len(todo)
todo.sort(key=lambda job: job[0].stat().st_size, reverse=True)
def cc(job):
src, dst = job
dep = dst.with_suffix('.d')
if stop and stop():
return src, dst, dep, None, ''
cmd = cmd_for(src, dst)
cmd[1:1] = ['-MMD', '-MF', str(dep)]
p = subprocess.run(cmd, capture_output=True, text=True, env=env, errors='replace')
@@ -566,6 +652,8 @@ def compile_all(srcs, cmd_for, cache, jobs, env, progress=None, salt=''):
with concurrent.futures.ThreadPoolExecutor(max(1, jobs)) as ex:
for src, dst, dep, rc, err in ex.map(cc, todo):
done += 1
if rc is None: # not started (stop)
continue
if rc:
bad.append((src, err))
cache.forget(dst)
@@ -577,33 +665,236 @@ def compile_all(srcs, cmd_for, cache, jobs, env, progress=None, salt=''):
return len(todo), reused, bad
def step_compile(out, cache_dir, msys, jobs):
def runtime_sources():
"""(source, object name) of every runtime file the Windows build compiles."""
rt = KIT / 'runtime'
out = []
for pat in RUNTIME_PATTERNS:
for p in sorted(rt.glob(pat)):
if p.stem.endswith('_sdl') or p.name == 'linux.c':
continue
out.append((p, 'rt_' + p.relative_to(rt).as_posix().replace('/', '_').rsplit('.', 1)[0] + '.o'))
return out
def per_install_names(generated=GENERATED_HEADERS):
"""The basenames of the generated headers and of every runtime header that includes one, at any depth."""
rt = KIT / 'runtime'
headers = {}
for p in list(rt.rglob('*.h')) + list(rt.rglob('*.inc')):
headers[p.name] = {os.path.basename(i) for i in INCLUDE.findall(p.read_text(encoding='utf-8', errors='replace'))}
tainted = set(generated)
grew = True
while grew:
grew = False
for name, incs in headers.items():
if name not in tainted and incs & tainted:
tainted.add(name)
grew = True
return tainted
def per_install(src, tainted):
incs = {os.path.basename(i) for i in INCLUDE.findall(Path(src).read_text(encoding='utf-8', errors='replace'))}
return bool(incs & tainted)
def clang_version(clang, env):
p = subprocess.run([str(clang), '--version'], capture_output=True, text=True, env=env)
return (p.stdout.splitlines() or [''])[0].strip()
def runtime_digest():
return tree_digest(KIT / 'runtime', ('**/*',))
# The x86-64 levels the lifted code is built and tested for (docs/building.md, "Checks"), as the macros clang defines
# for a CPU that has them.
CPU_V2 = {'__SSE3__', '__SSSE3__', '__SSE4_1__', '__SSE4_2__', '__POPCNT__', '__LAHF_SAHF__',
'__GCC_HAVE_SYNC_COMPARE_AND_SWAP_16'}
CPU_V3 = CPU_V2 | {'__AVX__', '__AVX2__', '__BMI__', '__BMI2__', '__F16C__', '__FMA__', '__LZCNT__', '__MOVBE__',
'__XSAVE__'}
def cpu_level(clang, env):
"""This PC's x86-64 level for the lifted code: 3 (AVX2, FMA, BMI2), 2 (SSE4.2, POPCNT) or 0 when clang cannot
tell. The game runs on the PC that builds it, so it takes this CPU's level, as tools/build_game.py --cpu does by
hand. Without one the lifted code would be generic x86-64: every fused multiply-add of the game a call to
runtime/fma.c's software fma, and a level the differential tests do not cover."""
p = subprocess.run([str(clang), '-march=native', '-E', '-dM', '-x', 'c', os.devnull], capture_output=True,
text=True, env=env, errors='replace')
have = set(re.findall(r'^#define (__\w+) ', p.stdout, re.M)) if p.returncode == 0 else set()
return 3 if CPU_V3 <= have else 2 if CPU_V2 <= have else 0
def clang_ident(clang, env):
"""(version, target) a clang reports, or None when it does not run."""
try:
p = subprocess.run([str(clang), '--version'], capture_output=True, text=True, env=env, errors='replace')
except OSError:
return None
v, t = re.search(r'clang version (\S+)', p.stdout), re.search(r'^Target: (\S+)', p.stdout, re.M)
return (v.group(1), t.group(1)) if p.returncode == 0 and v and t else None
def fast_clang(msys, clang, env):
"""MSYS2's CLANG64 clang beside the UCRT64 one (msys64/clang64, which the launcher installs): the same LLVM release
built with profile-guided optimisation, about a fifth faster on the lifted code. With UCRT64's headers
(--sysroot) it writes the same objects as UCRT64's clang, byte for byte but the COFF timestamp; it is used only
when it reports the same version and target. None otherwise."""
fast = Path(msys).parent / 'clang64' / 'bin' / 'clang.exe'
if not fast.is_file():
return None
mine = clang_ident(clang, env)
return fast if mine and clang_ident(fast, env) == mine else None
def runtime_cmd(clang, clangxx, inc, level=0, fast=None, msys=None):
"""fast: fast_clang's compiler for the lifted code, with msys as its sysroot."""
def cmd_for(src, dst):
cpp = src.suffix in ('.cpp', '.cxx')
# the lifted code at -O1 (its size makes -O2 slow to build), the runtime at -O2; neither may assume strict
# aliasing: the lifted code reaches guest memory through casts of one address to several types
# (tools/build_game.py's flags). The lifted code is built for this PC's level (cpu_level); the runtime
# stays generic, so the prebuilt one serves every PC (its vectors are 16 bytes, passed alike at every level).
guest = dst.name.startswith(('main_', 'sdk_', 'subsdk0_'))
march = [f'-march=x86-64-v{level}'] if guest and level else []
cc = [str(fast), f'--sysroot={msys}'] if guest and fast else [str(clangxx if cpp else clang)]
return [*cc, '-c', '-std=gnu++17' if cpp else '-std=gnu11', '-O1' if guest else '-O2',
*march, '-fno-strict-aliasing', '-w', *inc, '-o', str(dst), str(src)]
return cmd_for
def make_runtime_lib(dest, msys, jobs, obj=None):
"""--make-runtime-lib: compile every runtime source that does not include a per-install header into
<dest>/libnx2navirt.a, with runtime-lib.json naming the runtime/ tree and the compiler it was built from. The
objects stay in obj (default <dest>/obj), which a release keeps out of what it ships."""
dest = Path(dest).resolve()
dest.mkdir(parents=True, exist_ok=True)
obj = Path(obj).resolve() if obj else dest / 'obj'
obj.mkdir(parents=True, exist_ok=True)
clang, clangxx = msys / 'bin' / 'clang.exe', msys / 'bin' / 'clang++.exe'
env = dict(os.environ, PATH=str(msys / 'bin') + os.pathsep + os.environ.get('PATH', ''))
rt = KIT / 'runtime'
tainted = per_install_names()
srcs = [(p, obj / name) for p, name in runtime_sources() if not per_install(p, tainted)]
inc = ['-I', str(rt), '-I', str(rt / 'third_party' / 'imgui'), '-I', str(msys / 'include' / 'vma')]
cache = ObjectCache(obj / 'objects.json', {'<kit>': KIT})
t0 = time.time()
compiled, reused, bad = compile_all(srcs, runtime_cmd(clang, clangxx, inc), cache, jobs, env,
lambda d, t: (d % 20 == 0 or d == t) and say(f' {d} of {t}'),
salt=compiler_id(clang, env))
if bad:
for src, err in bad[:3]:
print(f'--- {src}\n{err}', file=sys.stderr)
fail(f'{len(bad)} runtime files did not compile')
lib = dest / RUNTIME_LIB
lib.unlink(missing_ok=True)
rsp = dest / 'ar.rsp'
rsp.write_text('\n'.join('"%s"' % o.as_posix() for _, o in srcs), encoding='utf-8')
ar = os.environ.get('NX2NAVI_AR') or str(msys / 'bin' / 'ar.exe')
if subprocess.run([ar, 'rcs', str(lib), '@' + str(rsp)], env=env).returncode:
fail('the runtime library could not be archived')
meta = {'runtime': runtime_digest(), 'compiler': clang_version(clang, env),
'objects': sorted(o.name for _, o in srcs)}
(dest / 'runtime-lib.json').write_text(json.dumps(meta, indent=1), encoding='utf-8')
rsp.unlink(missing_ok=True)
say(f'{lib}: {len(srcs)} runtime files ({compiled} compiled, {reused} kept), {lib.stat().st_size / 1e6:.1f} MB, '
f'{time.time() - t0:.0f} s')
def prebuilt_runtime(clang, env):
"""(library, object names it holds) when the kit's lib/ matches this runtime/ and this compiler, else None."""
lib, meta = KIT / 'lib' / RUNTIME_LIB, KIT / 'lib' / 'runtime-lib.json'
try:
m = json.loads(meta.read_text(encoding='utf-8'))
except (OSError, ValueError):
return None
if not lib.is_file() or m.get('runtime') != runtime_digest() or m.get('compiler') != clang_version(clang, env):
say(' the prebuilt runtime does not match this kit or compiler: compiling runtime/')
return None
return lib, set(m.get('objects', []))
def romfs_index(out, gen):
if subprocess.run(py_cmd(KIT / 'tools' / 'romfs_index.py', '--romfs', out / 'romfs', '--out', gen)).returncode:
fail('the RomFS index could not be written')
def windres_command(msys, rco):
"""MSYS2's windres on nx2navi.rc, run with runtime/ as the working directory (compile_resources says why)."""
pp = ('-E', '-xc', '-DRC_INVOKED') # windres's own preprocessor flags, which --preprocessor replaces
return [str(msys / 'bin' / 'windres.exe'), '--use-temp-file', f'--preprocessor={msys / "bin" / "gcc.exe"}',
*(f'--preprocessor-arg={a}' for a in pp), '-O', 'coff', '-i', 'nx2navi.rc', '-o', str(rco)]
def compile_resources(msys, rt, obj, env):
"""runtime/nx2navi.rc into obj/nx2navi_rc.o: the manifest of the game's exe (per-monitor DPI awareness, so the
picture keeps the monitor's real size rather than one Windows stretches) and its icon.
windres preprocesses the .rc with gcc through one command line it writes itself, quoting each path that holds a
space. By default it hands that line to cmd.exe /c, which drops the first and the last double quote of a line
holding more than two: gcc's quoted path loses its opening quote, and under a user folder with a space in its
name (C:\\Users\\Ali Sayed) cmd.exe stops at that space ("'C:\\Users\\Ali' is not recognized").
With --use-temp-file windres starts gcc itself instead, no shell involved: it splits the line into arguments and
passes each one to gcc intact. Its splitter reads a quoted first word right, but it cuts a quoted -I path at its
spaces and reads past the end of the line after a quoted last word, so the line holds no path but gcc's, its
first word: the .rc goes by its bare name from runtime/ as the working directory, where windres also finds the
manifest and the icon, and there is no -I. windres's temporary files go to a folder of the build cache, deleted
afterwards (it leaves two of them behind on every run).
The object is written under another name and replaces the old one only when its bytes differ, so an unchanged
resource keeps its time and does not make the link key change (a rebuild with nothing new does not link again)."""
rco, new = obj / 'nx2navi_rc.o', obj / 'nx2navi_rc.o.new'
tmp = obj / 'windres-tmp'
shutil.rmtree(tmp, ignore_errors=True)
tmp.mkdir()
try:
r = subprocess.run(windres_command(msys, new), cwd=rt, env=dict(env, TMP=str(tmp), TEMP=str(tmp)),
capture_output=True, text=True, errors='replace')
finally:
shutil.rmtree(tmp, ignore_errors=True)
if r.returncode:
new.unlink(missing_ok=True)
fail(f'windres failed on runtime/nx2navi.rc: {r.stderr.strip()[:400]}')
if rco.is_file() and rco.read_bytes() == new.read_bytes():
new.unlink()
else:
os.replace(new, rco)
return rco
def step_compile(out, cache_dir, msys, jobs, romfs_job=None):
"""romfs_job: the RomFS step still running beside this one (a first build). Everything that does not include
romfs_config.h compiles meanwhile; the RomFS index and the few files that include it follow once it is done."""
lift, gen, obj = cache_dir / 'lift', cache_dir / 'gen', cache_dir / 'obj'
gen.mkdir(parents=True, exist_ok=True)
obj.mkdir(parents=True, exist_ok=True)
rt = KIT / 'runtime'
clang, clangxx = msys / 'bin' / 'clang.exe', msys / 'bin' / 'clang++.exe'
env = dict(os.environ, PATH=str(msys / 'bin') + os.pathsep + os.environ.get('PATH', ''))
if subprocess.run(py_cmd(KIT / 'tools' / 'romfs_index.py', '--romfs', out / 'romfs', '--out', gen)).returncode:
fail('the RomFS index could not be written')
if not romfs_job:
romfs_index(out, gen)
srcs = [(p, obj / f'{mod}_{p.stem}.o') for mod in ('main', 'sdk', 'subsdk0') for p in sorted((lift / mod).glob('*.c'))]
for pat in ('*.c', '*.cpp', 'platform/*.c', 'ui/*.cpp', 'third_party/imgui/*.cpp', 'third_party/imgui/backends/*.cpp'):
for p in sorted(rt.glob(pat)):
if p.stem.endswith('_sdl') or p.name == 'linux.c':
continue
srcs.append((p, obj / ('rt_' + p.relative_to(rt).as_posix().replace('/', '_').rsplit('.', 1)[0] + '.o')))
prebuilt = prebuilt_runtime(clang, env)
runtime = [(p, obj / name) for p, name in runtime_sources()]
if prebuilt:
runtime = [(p, o) for p, o in runtime if o.name not in prebuilt[1]]
say(f' the runtime comes prebuilt; {len(runtime)} runtime files are compiled here')
srcs += runtime
srcs += [(modules_c(lift, out, gen), obj / 'modules.o'), (gen / 'shim.c', obj / 'shim.o')]
inc = ['-I', str(rt), '-I', str(lift), '-I', str(gen), '-I', str(rt / 'third_party' / 'imgui'),
'-I', str(msys / 'include' / 'vma')]
cid = compiler_id(clang, env)
def cmd_for(src, dst):
cpp = src.suffix in ('.cpp', '.cxx')
# the lifted code at -O1 (its size makes -O2 slow to build), the runtime at -O2; neither may assume strict aliasing:
# the lifted code reaches guest memory through casts of one address to several types (tools/build_game.py's flags)
guest = dst.name.startswith(('main_', 'sdk_', 'subsdk0_'))
return [str(clangxx if cpp else clang), '-c', '-std=gnu++17' if cpp else '-std=gnu11', '-O1' if guest else '-O2',
'-fno-strict-aliasing', '-w', *inc, '-o', str(dst), str(src)]
level = cpu_level(clang, env)
say(f' the lifted code is built for x86-64-v{level}, this processor\'s level' if level else
' the lifted code is built for generic x86-64: the processor\'s level could not be read')
fast = fast_clang(msys, clang, env)
if fast:
cid += compiler_id(fast, env)
say(f' the lifted code compiles with {fast}')
cmd_for = runtime_cmd(clang, clangxx, inc, level, fast, msys)
t0 = time.time()
last = [0.0]
@@ -617,29 +908,42 @@ def step_compile(out, cache_dir, msys, jobs):
say(f' compiled {done} of {total}, {now - t0:.0f} s')
say(f' {len(srcs)} files on {jobs} threads')
cache = ObjectCache(obj / 'objects.json')
compiled, reused, bad = compile_all(srcs, cmd_for, cache, jobs, env, progress, salt=cid)
cache = ObjectCache(obj / 'objects.json', {'<kit>': KIT})
late = []
if romfs_job: # the lifted chunks include only cpu.h and simd.h: only the runtime and generated files are read
tainted = per_install_names(('romfs_config.h',))
late = [(s, o) for s, o in runtime + srcs[-2:] if per_install(s, tainted)]
say(f' {len(late)} files wait for the RomFS index: {", ".join(s.name for s, _ in late)}')
early = [j for j in srcs if j not in late]
compiled, reused, bad = compile_all(early, cmd_for, cache, jobs, env, lambda d, t: progress(d, len(srcs)),
salt=cid, stop=romfs_job.failed if romfs_job else None)
if romfs_job:
if romfs_job.thread.is_alive():
say(' waiting for the RomFS image')
romfs_job.join() # raises the RomFS step's failure; a failed compile still keeps the finished image
if not bad:
romfs_index(out, gen)
c2, r2, bad = compile_all(late, cmd_for, cache, jobs, env,
lambda d, t: progress(len(early) + d, len(srcs)), salt=cid)
compiled, reused = compiled + c2, reused + r2
say(f' {compiled} compiled, {reused} unchanged since the last build')
if bad:
for src, err in bad[:3]:
print(f'--- {src}\n{err}', file=sys.stderr)
fail(f'{len(bad)} files did not compile')
# the manifest (per-monitor DPI awareness: the picture at the monitor's real size, not one Windows stretches) and the
# icon, runtime/nx2navi.rc through MSYS2's windres. windres runs its preprocessor through cmd.exe: run from the
# runtime folder with relative names and gcc by bare name (WINDRES_PP), that command line holds no path, so a space
# in the install folder (a user name) cannot split it.
rco = (obj / 'nx2navi_rc.o').resolve()
r = subprocess.run([str(msys / 'bin' / 'windres.exe'), *WINDRES_PP, '-I', '.', '-O', 'coff', '-i', 'nx2navi.rc',
'-o', str(rco)], capture_output=True, text=True, env=env, errors='replace', cwd=rt)
if r.returncode:
fail(f'windres failed on runtime/nx2navi.rc: {r.stderr.strip()[:400]}')
rco = compile_resources(msys, rt, obj, env)
objs = [o for _, o in srcs] + [rco]
exe = out / EXE_NAME
rsp = cache_dir / 'link.rsp'
rsp.write_text('\n'.join('"%s"' % o.as_posix() for o in objs), encoding='utf-8')
link_key = sha256_bytes(json.dumps([LINK_LIBS, cid] + [[o.name, o.stat().st_size, o.stat().st_mtime_ns]
for o in objs]).encode())
# lld links several times faster than GNU ld; the stack matches tools/build_game.py's build
ld = ['-fuse-ld=lld', '-Wl,--stack,16777216'] if (msys / 'bin' / 'ld.lld.exe').is_file() else []
lib = ['-Wl,--whole-archive', str(prebuilt[0]), '-Wl,--no-whole-archive'] if prebuilt else []
stamp = [[o.name, o.stat().st_size, o.stat().st_mtime_ns] for o in objs]
if prebuilt:
stamp.append([prebuilt[0].name, prebuilt[0].stat().st_size, sha256_file(prebuilt[0])])
link_key = sha256_bytes(json.dumps([LINK_LIBS, ld, cid] + stamp).encode())
state_link = cache_dir / 'link.key'
if exe.is_file() and state_link.is_file() and state_link.read_text(encoding='utf-8') == link_key:
say(' ocarina.exe is current, not linked again')
@@ -647,8 +951,8 @@ def step_compile(out, cache_dir, msys, jobs):
emit('phase', step='compile', phase='link')
say(' linking')
tmp = out / (EXE_NAME + '.new')
p = subprocess.run([str(clangxx), '-o', str(tmp), '@' + str(rsp), '-L', str(msys / 'lib'), *LINK_LIBS],
capture_output=True, text=True, env=env, errors='replace')
p = subprocess.run([str(clangxx), *ld, '-o', str(tmp), '@' + str(rsp), *lib, '-L', str(msys / 'lib'),
*LINK_LIBS], capture_output=True, text=True, env=env, errors='replace')
if p.returncode:
print(p.stderr[-4000:], file=sys.stderr)
tmp.unlink(missing_ok=True)
@@ -736,9 +1040,14 @@ def main():
ap.add_argument('--msys', default=r'C:\msys64\ucrt64', help='MSYS2 UCRT64 folder (default C:\\msys64\\ucrt64)')
ap.add_argument('--jobs', type=int, default=os.cpu_count() or 4, help='parallel compiles')
ap.add_argument('--events', action='store_true', help='machine-readable progress lines on stdout ("@@" + JSON)')
ap.add_argument('--make-runtime-lib', metavar='DIR', help='build the prebuilt runtime library of a release into DIR')
ap.add_argument('--obj', help='with --make-runtime-lib: the folder for its objects')
ap.add_argument('--until', choices=STEPS, help=argparse.SUPPRESS) # tests: stop after this step
a = ap.parse_args()
EVENTS = a.events
if a.make_runtime_lib:
make_runtime_lib(a.make_runtime_lib, Path(a.msys), a.jobs, a.obj)
return
given = bool(a.nup) + bool(a.exefs or a.romfs) + bool(a.rebuild)
if given != 1 or (not a.nup and not a.rebuild and not (a.exefs and a.romfs)):
ap.error('give --nup, both --exefs and --romfs, or --rebuild')
@@ -774,29 +1083,60 @@ def main():
state['kit'] = kit_version()
keep_awake()
t0 = time.time()
romfs_job = None
runs = {'exefs': lambda: step_exefs(src, out), 'romfs': lambda: step_romfs(src, out, min(a.jobs, 8)),
'lift': lambda: step_lift(out, cache), 'compile': lambda: step_compile(out, cache, msys, a.jobs),
'lift': lambda: step_lift(out, cache), 'compile': lambda: step_compile(out, cache, msys, a.jobs, romfs_job),
'shaders': lambda: step_shaders(out)}
for name in STEPS:
key = step_key(name, src.identity, out)
if state['steps'].get(name) == key:
say(f'{name}: done before')
emit('step', step=name, state='skip')
else:
# a step that runs makes every step after it run too: their keys hash its output, not its key
for later in STEPS[STEPS.index(name):]:
state['steps'].pop(later, None)
say(f'{name}:')
emit('step', step=name, state='start')
t = time.time()
runs[name]()
state['steps'][name] = step_key(name, src.identity, out)
state_lock = threading.Lock() # the RomFS step records itself from its own thread
def save():
with state_lock:
state_path.write_text(json.dumps(state, indent=1), encoding='utf-8')
say(f'{name}: {time.time() - t:.0f} s')
emit('step', step=name, state='done', seconds=round(time.time() - t, 1))
if name == a.until:
break
state_path.write_text(json.dumps(state, indent=1), encoding='utf-8')
def run(name):
t = time.time()
runs[name]()
with state_lock:
state['steps'][name] = step_key(name, src.identity, out)
save()
say(f'{name}: {time.time() - t:.0f} s')
emit('step', step=name, state='done', seconds=round(time.time() - t, 1))
try:
for name in STEPS:
key = step_key(name, src.identity, out)
if state['steps'].get(name) == key:
say(f'{name}: done before')
emit('step', step=name, state='skip')
else:
# a step that runs makes every step after it run too: their keys hash its output, not its key
with state_lock:
for later in STEPS[STEPS.index(name):]:
state['steps'].pop(later, None)
say(f'{name}:')
emit('step', step=name, state='start')
if name == 'shaders' and romfs_job: # the shaders come out of the RomFS image
romfs_job.join()
if name == 'romfs' and a.until != 'romfs':
# The RomFS image is mostly disk work, the lift and the compile are processor work: they run side
# by side, and only the RomFS index and the files that include it wait for the image
# (step_compile). A first build takes about as long as the longer of the two, not their sum.
romfs_job = Background(lambda: run('romfs'))
else:
run(name)
if romfs_job and romfs_job.failed():
romfs_job.join()
if name == a.until:
break
if romfs_job:
romfs_job.join()
except SystemExit:
# a failed lift or compile: the RomFS image still finishes and is kept, so the next run starts after it
if romfs_job and romfs_job.thread.is_alive():
say('waiting for the RomFS image before stopping')
romfs_job.thread.join()
raise
save()
say(f'\nfinished in {(time.time() - t0) / 60:.1f} min: {out / EXE_NAME}')
emit('end', exe=str(out / EXE_NAME), seconds=round(time.time() - t0, 1), kit=kit_version())

View File

@@ -12,12 +12,14 @@ checked against the sha256 in PINS.
The release folder (--out) holds:
NX2navi/ the launcher folder exactly as NX2navi-Setup.exe installs it:
NX2navi.exe launcher/native/nx2navi.c with the Triforce icon (launcher/ui/art.py)
NX2navi.exe launcher/native/nx2navi.c with the Navi icon (launcher/ui/art.py)
python/ python.org's embeddable Python 3.14.9, tkinter and Tcl/Tk 9.0 from its tcltk.msi,
the packages of tools/requirements-launcher.txt, runtime-id.txt
app/current.txt, app/<version>/ launcher/ (without native/), runtime/, recomp/, the tools make_game.py uses,
bin/nx2navi_xlate.exe and .id (and amd_fidelityfx_vk.dll with --ffx), toolchain.lock.json (the MSYS2 packages of today,
pinned), feed.json, VERSION, the licence files
bin/nx2navi_xlate.exe and .id (and amd_fidelityfx_vk.dll with --ffx),
lib/libnx2navirt.a (the runtime, prebuilt with the release's compiler),
toolchain.lock.json (the MSYS2 packages of today, pinned), feed.json, VERSION, the
licence files
NX2navi-Setup-<version>.exe launcher/native/setup.c with the folder above appended as a zip: the one file
players download
NX2navi-app-<version>.zip app/<version> alone: what the launcher downloads to update itself
@@ -196,10 +198,33 @@ def native_compiler(lock):
[windres, '--target=x86_64-w64-mingw32'])
def build_runtime_lib(folder, ver, out, wine):
"""app/<version>/lib/libnx2navirt.a: the runtime compiled once here with the release's MSYS2 clang, so players
compile only the few runtime files that include per-install headers (tools/make_game.py --make-runtime-lib). It
needs MSYS2's own clang.exe: native on Windows, through Wine with --wine elsewhere; without either players compile
runtime/ themselves."""
msys = Layout(root=MAIN_ROOT / 'toolchain' / 'launcher-msys').msys
app = folder / 'app' / ver
args = ['--make-runtime-lib', app / 'lib', '--obj', out / 'native' / 'runtime-obj', '--msys', msys]
if os.name == 'nt':
run([sys.executable, app / 'tools' / 'make_game.py', *args])
elif wine:
def win(p):
return 'Z:' + str(Path(p).resolve()).replace('/', '\\')
env = dict(os.environ, WINEDEBUG='-all', NX2NAVI_AR=win(msys / 'bin' / 'ar.exe'))
run(['wine', folder / 'python' / 'python.exe', win(app / 'tools' / 'make_game.py'),
*[win(a) if isinstance(a, Path) else a for a in args]], env=env)
else:
say('note: no prebuilt runtime (it is built on Windows, or with --wine): players compile runtime/ themselves')
return
if not (app / 'lib' / 'libnx2navirt.a').is_file():
sys.exit('the prebuilt runtime was not written')
def build_native(work, cc, windres):
from launcher.ui import art
work.mkdir(parents=True, exist_ok=True)
(work / 'nx2navi.ico').write_bytes(art.triforce_ico())
(work / 'nx2navi.ico').write_bytes(art.navi_ico())
shutil.copy2(ROOT / 'launcher' / 'native' / 'app.manifest', work / 'app.manifest')
shutil.copy2(ROOT / 'launcher' / 'native' / 'res.rc', work / 'res.rc')
res = (work / 'res.o').resolve()
@@ -261,6 +286,8 @@ def main():
ap.add_argument('--notes', type=Path, help='release notes (markdown) for the manifest and the GitHub release')
ap.add_argument('--lock', type=Path, help='a toolchain.lock.json to reuse instead of resolving today\'s packages')
ap.add_argument('--version', help='the release version instead of VERSION\'s (test releases)')
ap.add_argument('--wine', action='store_true', help='on Linux: build the prebuilt runtime with MSYS2\'s clang '
'through Wine')
ap.add_argument('--publish', action='store_true', help='upload as GitHub release v<version> with gh')
ap.add_argument('--draft', action='store_true', help='with --publish: as a draft')
a = ap.parse_args()
@@ -273,12 +300,15 @@ def main():
shutil.rmtree(out, ignore_errors=True)
folder = out / 'NX2navi'
t0 = time.time()
lock = json.loads(a.lock.read_text(encoding='utf-8')) if a.lock else toolchain.make_lock()
say(f'release {build}: toolchain lock with {len(lock["packages"])} packages')
lock = (json.loads(a.lock.read_text(encoding='utf-8')) if a.lock
else toolchain.make_lock(fast_repo=toolchain.FAST_REPO))
say(f'release {build}: toolchain lock with {len(lock["packages"])} packages'
+ (f' (CLANG64 clang left out: {lock["fast_skipped"]})' if lock.get('fast_skipped') else ''))
build_app(folder / 'app' / ver, ver, a.xlate, a.feed, lock, a.ffx)
(folder / 'app' / 'current.txt').write_text(ver + '\n', encoding='utf-8')
build_python(folder / 'python')
cc, windres = native_compiler(lock)
build_runtime_lib(folder, ver, out, a.wine)
stub, setup_stub = build_native(out / 'native', cc, windres)
shutil.copy2(stub, folder / 'NX2navi.exe')
check_folder(folder)

427
tools/menu_preview.cpp Normal file
View File

@@ -0,0 +1,427 @@
// Draws the in-game settings menu and the performance overlay without the game, for looking at their design: the
// menu's own row code (runtime/ui/option_rows.cpp), its shapes (ui/menu_skin.cpp), the overlay panel
// (ui/perf_panel.cpp) and the text catalog (ui_text.c), with the few runtime calls they make answered here, laid out as
// menu.cpp layout_menu lays out the window. Dear ImGui's draw lists are rasterised on the CPU into an RGBA image.
// Without an art folder the window has its plain border in place of the game's ornamented frame and every box is plain
// shapes; with one (tools/menu_preview.py --romfs) the boxes use the game's textures as in the game. The given font
// stands in for the game's either way.
//
// tools/menu_preview.py builds and runs it:
// menu_preview <font.ttf> <scene: menu | overlay | boot> <width> <height> <out.rgba> [art folder]
#include <math.h>
#include <stdarg.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <vector>
#include "imgui.h"
#include "imgui_internal.h"
#include "ui/menu_art.h"
#include "ui/menu_skin.h"
#include "ui/option_rows.h"
#include "ui/perf_panel.h"
extern "C" {
#include "cfg.h"
#include "menu_sound.h"
#include "ui_text.h"
}
// ---- what the rows and the catalog ask of the runtime: a few settings with the menu's values and defaults
static const CfgMeta kMeta[] = {
{"FULLSCREEN", "1", "0|1", 1}, {"WINDOW_SIZE", "", NULL, 1}, {"VK_PRESENT", "mailbox", "mailbox|fifo|immediate", 1},
{"GPU_INDEX", "", NULL, 0}, {"PERF_OVERLAY", "off", "off|fps|full", 1},
{"GAME_ASPECT", "16:9", "16:9|window|21:9|32:9", 0}, {"RENDER_SCALE", "", NULL, 0},
{"UPSCALER", "bilinear", "bilinear|fsr", 0}, {"SHARPNESS", "50", "0..100", 1},
{"ASPECT", "fit", "fit|stretch", 1}, {"MENU_LANG", "", NULL, 1},
};
static const struct { const char *key, *value; } kSet[] = {{"VK_PRESENT", "mailbox"}, {"PERF_OVERLAY", "full"}};
static const char *strip(const char *name) { return strncmp(name, "NX2NAVI_", 8) ? name : name + 8; }
extern "C" {
const CfgMeta *cfg_meta(const char *key) {
for (const CfgMeta &m : kMeta)
if (!strcasecmp(m.key, strip(key))) return &m;
return NULL;
}
const CfgMeta *cfg_meta_at(int i) { return i >= 0 && i < (int)(sizeof kMeta / sizeof *kMeta) ? &kMeta[i] : NULL; }
const char *cfg_get(const char *name) {
for (auto &s : kSet)
if (!strcasecmp(s.key, strip(name))) return s.value;
return NULL;
}
const char *cfg_value(const char *name) {
const char *v = cfg_get(name);
const CfgMeta *m = cfg_meta(name);
return v ? v : m ? m->def : "";
}
int cfg_int(const char *name) { return atoi(cfg_value(name)); }
double cfg_double(const char *name) { return atof(cfg_value(name)); }
int cfg_bool(const char *name) { return strcmp(cfg_value(name), "0") != 0; }
int cfg_from_env(const char *) { return 0; }
void cfg_set(const char *, const char *) {}
int cfg_save(void) { return 1; }
int cfg_reset(const char *const *, int) { return 0; }
unsigned cfg_version(void) { return 1; }
void menu_sound_play(MenuSound) {}
const char *ov_language(void) { return "en-US"; }
const char *plat_ui_language(void) { return "en"; }
uint64_t plat_now_ns(void) {
struct timespec t;
clock_gettime(CLOCK_MONOTONIC, &t);
return (uint64_t)t.tv_sec * 1000000000ull + (uint64_t)t.tv_nsec;
}
void rt_log(const char *, ...) {}
}
// ---- the game's art, from files menu_preview.py decoded: NN.art, a width and a height (32-bit) and the RGBA pixels of
// the MenuArt NN, with the smaller levels the game uploads
struct Level {
int w, h;
std::vector<uint8_t> px;
};
static std::vector<Level> g_art[MENU_ART_COUNT];
static void art_load(const char *dir) {
for (int i = 0; i < MENU_ART_COUNT; i++) {
char path[1024];
snprintf(path, sizeof path, "%s/%02d.art", dir, i);
FILE *f = fopen(path, "rb");
if (!f) continue;
uint32_t size[2];
if (fread(size, 4, 2, f) == 2) {
Level top{(int)size[0], (int)size[1], std::vector<uint8_t>((size_t)size[0] * size[1] * 4)};
if (fread(top.px.data(), 1, top.px.size(), f) == top.px.size()) {
skin_art_prepare(i, top.px.data(), size[0], size[1]);
g_art[i].push_back(std::move(top));
while (g_art[i].size() < 6 && (g_art[i].back().w > 1 || g_art[i].back().h > 1)) {
const Level &last = g_art[i].back();
Level next{last.w > 1 ? last.w / 2 : 1, last.h > 1 ? last.h / 2 : 1, {}};
next.px.resize((size_t)next.w * next.h * 4);
skin_art_halve(last.px.data(), last.w, last.h, next.px.data(), next.w, next.h);
g_art[i].push_back(std::move(next));
}
}
}
fclose(f);
}
}
// Texture ids in the draw lists: 1 is the font atlas, 2 and up the art.
extern "C" int menu_art(MenuArt id, ImTextureID *tex, float *w, float *h) {
if ((unsigned)id >= MENU_ART_COUNT || g_art[id].empty()) return 0;
*tex = (ImTextureID)(intptr_t)(2 + id);
if (w) *w = (float)g_art[id][0].w;
if (h) *h = (float)g_art[id][0].h;
return 1;
}
// ---- the scenes
static void backdrop(float w, float h) {
// a dim field and sky, as the game shows behind the menu; then the menu's own dimming (menu.cpp layout_menu)
ImDrawList *bg = ImGui::GetBackgroundDrawList();
bg->AddRectFilledMultiColor(ImVec2(0, 0), ImVec2(w, h * 0.55f), IM_COL32(62, 70, 74, 255), IM_COL32(70, 76, 78, 255),
IM_COL32(52, 64, 52, 255), IM_COL32(48, 60, 50, 255));
bg->AddRectFilledMultiColor(ImVec2(0, h * 0.55f), ImVec2(w, h), IM_COL32(52, 64, 52, 255), IM_COL32(48, 60, 50, 255),
IM_COL32(30, 38, 30, 255), IM_COL32(28, 34, 28, 255));
}
static void triforce(ImDrawList *dl, float x, float y, float h, ImU32 col) {
float w = h * 1.1547f;
ImVec2 t(x, y), l(x - w / 2, y + h), r(x + w / 2, y + h), ml(x - w / 4, y + h / 2), mr(x + w / 4, y + h / 2), mb(x, y + h);
dl->AddTriangleFilled(t, mr, ml, col);
dl->AddTriangleFilled(ml, mb, l, col);
dl->AddTriangleFilled(mr, r, mb, col);
}
static void scene_menu(float w, float h, float scale, ImFont *title, bool first) {
backdrop(w, h);
ImGui::GetBackgroundDrawList()->AddRectFilled(ImVec2(0, 0), ImVec2(w, h), IM_COL32(10, 6, 2, 150));
float ww = fminf(w * 0.88f, 1400 * scale), wh = fminf(h * 0.86f, 820 * scale);
ImGui::SetNextWindowPos(ImVec2(w * 0.5f, h * 0.5f), ImGuiCond_Always, ImVec2(0.5f, 0.5f));
ImGui::SetNextWindowSize(ImVec2(ww, wh), ImGuiCond_Always);
ImTextureID frame_tex;
const bool framed = menu_art(MENU_ART_FRAME_DECO, &frame_tex, nullptr, nullptr);
if (framed) ImGui::PushStyleVar(ImGuiStyleVar_WindowBorderSize, 0);
ImGui::Begin("##menu", nullptr, ImGuiWindowFlags_NoDecoration | ImGuiWindowFlags_NoMove | ImGuiWindowFlags_NoSavedSettings);
if (framed) ImGui::PopStyleVar();
const ImVec2 win_p = ImGui::GetWindowPos(), win_s = ImGui::GetWindowSize();
if (framed) { // as menu.cpp layout_menu draws it
ImDrawList *fdl = ImGui::GetWindowDrawList();
const float ox = 15 * scale, oy = 30 * scale;
fdl->PushClipRectFullScreen();
for (int turned = 0; turned < 2; turned++)
menu_art_draw9_turned(fdl, MENU_ART_FRAME_DECO, ImVec2(win_p.x - ox, win_p.y - oy),
ImVec2(win_p.x + win_s.x + ox, win_p.y + win_s.y + oy), 96 * scale, 0.2f,
IM_COL32(236, 196, 112, 235), 0, turned);
fdl->PopClipRect();
}
const ImVec4 gold(0.93f, 0.76f, 0.38f, 1);
ImGui::PushFont(title);
ImVec2 at = ImGui::GetCursorScreenPos();
float th = ImGui::GetTextLineHeight();
triforce(ImGui::GetWindowDrawList(), at.x + th * 0.6f, at.y + th * 0.02f, th * 0.96f, ImGui::GetColorU32(gold));
ImGui::SetCursorScreenPos(ImVec2(at.x + th * 1.5f, at.y));
ImGui::TextColored(gold, "Ocarina of Time");
ImGui::SameLine();
ImGui::TextDisabled("|");
ImGui::SameLine();
ImGui::TextUnformatted(menu_text(UI_MENU_TITLE));
const float title_end = ImGui::GetItemRectMax().x;
ImGui::PopFont();
const char *version = "NX2navi 0.5.0 - game 1.1.1";
float version_x = ImGui::GetWindowWidth() - ImGui::GetStyle().WindowPadding.x - ImGui::CalcTextSize(version).x;
if (ImGui::GetWindowPos().x + version_x >= title_end + ImGui::GetStyle().ItemSpacing.x * 2) {
ImGui::SameLine(version_x);
ImGui::TextDisabled("%s", version);
}
const ImGuiStyle &st = ImGui::GetStyle();
float bottom = ImGui::GetFrameHeight() + st.ItemSpacing.y * 2;
static const UiTextId tabs[] = {UI_MENU_TAB_SCREEN, UI_MENU_TAB_GAME, UI_MENU_TAB_SOUND, UI_MENU_TAB_CONTROLS};
if (ImGui::BeginTabBar("##tabs", ImGuiTabBarFlags_NoTabListScrollingButtons)) {
for (int k = 0; k < 4; k++) {
char label[96];
snprintf(label, sizeof label, " %s ###tab%d", menu_text(tabs[k]), k);
bool shown = skin_tab_item(label, menu_text(tabs[k]), 0, scale, title);
if (!shown) continue;
ImGui::Spacing();
opt_reserve_bottom(bottom);
if (first) opt_focus_first();
// the Screen tab's first rows, as menu.cpp tab_screen shows them
opt_begin("##screen", scale);
opt_toggle("NX2NAVI_FULLSCREEN", UI_MENU_FULLSCREEN, UI_MENU_FULLSCREEN_DESC);
static const char *const sizes[] = {"", "1920x1080"};
static const char *const size_names[] = {"1920 \xC3\x97 1080", "1920 \xC3\x97 1080"};
opt_choice_list("NX2NAVI_WINDOW_SIZE", UI_MENU_WINDOW_SIZE, sizes, size_names, 2, UI_MENU_WINDOW_SIZE_DESC, 0);
opt_toggle_value("NX2NAVI_VK_PRESENT", UI_MENU_PRESENT_VSYNC, UI_MENU_PRESENT_DESC, "mailbox", "immediate");
static const char *const gpus[] = {""};
static const char *const gpu_names[] = {"NVIDIA GeForce RTX 3050 Ti Laptop GPU"}; // a long name
opt_choice_list("NX2NAVI_GPU_INDEX", UI_MENU_GPU, gpus, gpu_names, 1, UI_MENU_GPU_DESC, 0);
static const UiTextId perf[] = {UI_MENU_PERF_OFF, UI_MENU_PERF_FPS, UI_MENU_PERF_FULL};
opt_choice("NX2NAVI_PERF_OVERLAY", UI_MENU_PERF_OVERLAY, perf, 3, UI_MENU_PERF_OVERLAY_DESC);
opt_group(UI_MENU_GRAPHICS);
static const UiTextId wide[] = {UI_MENU_GAME_ASPECT_16_9, UI_MENU_GAME_ASPECT_WINDOW, UI_MENU_GAME_ASPECT_21_9,
UI_MENU_GAME_ASPECT_32_9};
opt_choice("NX2NAVI_GAME_ASPECT", UI_MENU_GAME_ASPECT, wide, 4, UI_MENU_GAME_ASPECT_DESC);
opt_slider("NX2NAVI_SHARPNESS", UI_MENU_SHARPNESS, 0, 100, UI_MENU_SHARPNESS_DESC, 5, "%.0f %%");
opt_end();
ImGui::EndTabItem();
}
ImGui::EndTabBar();
}
ImGui::Spacing();
float y = ImGui::GetCursorPosY();
{
float sy = win_p.y + y, fh = ImGui::GetFrameHeight();
menu_art_draw(ImGui::GetWindowDrawList(), MENU_ART_GUIDE_BAR, ImVec2(win_p.x + st.WindowPadding.x, sy),
ImVec2(win_p.x + win_s.x - st.WindowPadding.x, sy + fh), IM_COL32(96, 70, 36, 170));
}
const char *cont = menu_text(UI_MENU_CONTINUE), *restart = menu_text(UI_MENU_RESTART_YES), *quit = menu_text(UI_MENU_QUIT);
float bw = fmaxf(fmaxf(ImGui::CalcTextSize(cont).x, ImGui::CalcTextSize(restart).x), ImGui::CalcTextSize(quit).x) + st.FramePadding.x * 4;
float gap = st.ItemSpacing.x + skin_cursor_reach(scale);
ImGui::SetCursorPos(ImVec2(fmaxf(st.WindowPadding.x, ImGui::GetWindowWidth() - st.WindowPadding.x - bw * 3 - gap * 2), y));
skin_widget_button(cont, ImVec2(bw, 0), scale);
ImGui::SameLine(0, gap);
skin_widget_button(restart, ImVec2(bw, 0), scale);
ImGui::SameLine(0, gap);
skin_widget_button(quit, ImVec2(bw, 0), scale);
ImGui::End();
}
static void scene_overlay(float w, float h, float scale) {
backdrop(w, h);
static PerfPanel p; // the numbers of the screenshot the overlay was redesigned from
PerfStats &s = p.s;
s.fps = 60, s.frame_ms_avg = 16.7f, s.frame_ms_max = 20.2f, s.low1_fps = 47, s.low01_fps = 8;
for (int i = 0; i < PERF_HISTORY; i++) s.frame_ms[i] = 16.7f + 1.4f * sinf(i * 0.7f) + (i % 17 == 0 ? 3.5f : 0);
snprintf(s.cpu_name, sizeof s.cpu_name, "AMD Ryzen 7 5800X3D 8-Core Processor");
s.cpu_total_pct = 50, s.cpu_game_pct = 44, s.cpu_temp_c = -1;
s.ram_used_mb = 23.0f * 1024, s.ram_total_mb = 31.9f * 1024, s.ram_game_mb = 8.6f * 1024;
snprintf(s.gpu_name, sizeof s.gpu_name, "AMD Radeon RX 7900 XT");
s.gpu_pct = 56, s.gpu_temp_c = 69, s.gpu_power_w = 107, s.gpu_clock_mhz = 1312, s.vram_used_mb = 5.3f * 1024,
s.vram_total_mb = 20.0f * 1024;
s.replay_ms = 0.60f, s.gpu_ms = 1.28f, s.pipe_wait_ms = 0, s.xlate_ms = 0, s.tex_uploads = 0;
p.version = "NX2navi 0.5.0, game 1.1.1";
snprintf(p.picture[0], 64, "1920x1080");
snprintf(p.picture[1], 64, "2560x1440");
snprintf(p.picture[3], 64, "bilinear");
snprintf(p.picture[4], 64, "60 FPS");
snprintf(p.picture[5], 64, "VSync");
p.scene_known = false, p.fg_fps = 60, p.under_menu = false;
perf_panel_draw(&p, w, scale);
}
static void scene_boot(float w, float h, float scale, ImFont *title) {
// the boot screen's box over the black the window shows then (menu.cpp menu_boot_frame), half way through the
// pipelines
ImGui::GetBackgroundDrawList()->AddRectFilled(ImVec2(0, 0), ImVec2(w, h), IM_COL32(0, 0, 0, 255));
char step[160];
snprintf(step, sizeof step, ui_text(UI_BOOT_PIPELINES), 1603LL, 3235LL);
skin_boot_panel(ImGui::GetForegroundDrawList(), title, title->FontSize, ImGui::GetFont(), ImGui::GetFontSize(),
ui_text(UI_BOOT_TITLE), step, ui_text(UI_BOOT_HINT), 1603.0f / 3235.0f, 0, w, h, scale);
}
// ---- the rasteriser: every triangle of the draw lists, its colour times the font atlas, blended over the image
struct Image {
int w, h;
std::vector<float> px; // straight RGBA, 0..1
};
// Twice the signed area of (e0, e1, p), computed with the edge's ends in one fixed order, so the two triangles that share
// an edge get exactly opposite values for a pixel and none is shaded twice or left out along it.
static float edge(ImVec2 e0, ImVec2 e1, float px, float py) {
bool swap = e0.x > e1.x || (e0.x == e1.x && e0.y > e1.y);
if (swap) {
ImVec2 t = e0;
e0 = e1, e1 = t;
}
float v = (e0.x - px) * (e1.y - py) - (e0.y - py) * (e1.x - px);
return swap ? -v : v;
}
// The atlas at (u, v), bilinear, clamped to its edges; 0..1.
static void sample(const uint8_t *atlas, int aw, int ah, float u, float v, float out[4]) {
float x = u * aw - 0.5f, y = v * ah - 0.5f;
int x0 = (int)floorf(x), y0 = (int)floorf(y);
float fx = x - x0, fy = y - y0;
for (int k = 0; k < 4; k++) out[k] = 0;
for (int j = 0; j < 2; j++)
for (int i = 0; i < 2; i++) {
int xx = x0 + i < 0 ? 0 : x0 + i >= aw ? aw - 1 : x0 + i, yy = y0 + j < 0 ? 0 : y0 + j >= ah ? ah - 1 : y0 + j;
float wgt = (i ? fx : 1 - fx) * (j ? fy : 1 - fy);
const uint8_t *t = atlas + 4 * ((size_t)yy * aw + xx);
for (int k = 0; k < 4; k++) out[k] += wgt * t[k] / 255.0f;
}
}
// The art at (u, v) at level of detail lod, trilinear between its levels, as the game's sampler reads it.
static void sample_art(const std::vector<Level> &levels, float u, float v, float lod, float out[4]) {
const float top = (float)levels.size() - 1;
lod = lod < 0 ? 0 : lod > top ? top : lod;
int l0 = (int)floorf(lod), l1 = l0 + 1 < (int)levels.size() ? l0 + 1 : l0;
float a[4], b[4], f = lod - l0;
sample(levels[l0].px.data(), levels[l0].w, levels[l0].h, u, v, a);
sample(levels[l1].px.data(), levels[l1].w, levels[l1].h, u, v, b);
for (int k = 0; k < 4; k++) out[k] = a[k] + (b[k] - a[k]) * f;
}
static void raster(const ImDrawData *dd, Image &img, const uint8_t *atlas, int aw, int ah) {
for (int l = 0; l < dd->CmdListsCount; l++) {
const ImDrawList *dl = dd->CmdLists[l];
const ImDrawVert *vb = dl->VtxBuffer.Data;
const ImDrawIdx *ib = dl->IdxBuffer.Data;
for (const ImDrawCmd &cmd : dl->CmdBuffer) {
if (cmd.UserCallback) continue;
int cx0 = (int)fmaxf(0, cmd.ClipRect.x), cy0 = (int)fmaxf(0, cmd.ClipRect.y);
int cx1 = (int)fminf((float)img.w, cmd.ClipRect.z), cy1 = (int)fminf((float)img.h, cmd.ClipRect.w);
const intptr_t tex_id = (intptr_t)cmd.GetTexID();
const std::vector<Level> *art = tex_id >= 2 && tex_id < 2 + MENU_ART_COUNT ? &g_art[tex_id - 2] : nullptr;
for (unsigned i = 0; i + 2 < cmd.ElemCount; i += 3) {
const ImDrawVert &a = vb[cmd.VtxOffset + ib[cmd.IdxOffset + i]], &b = vb[cmd.VtxOffset + ib[cmd.IdxOffset + i + 1]],
&c = vb[cmd.VtxOffset + ib[cmd.IdxOffset + i + 2]];
float area = (b.pos.x - a.pos.x) * (c.pos.y - a.pos.y) - (b.pos.y - a.pos.y) * (c.pos.x - a.pos.x);
if (fabsf(area) < 1e-6f) continue;
// the top-left rule: a pixel centre on an edge two triangles share is shaded by one of them only
// (edge k is the one opposite vertex k, run in the triangle's own winding)
const ImVec2 *vtx[3] = {&a.pos, &b.pos, &c.pos};
bool top_left[3];
for (int k = 0; k < 3; k++) {
ImVec2 e0 = *vtx[(k + 1) % 3], e1 = *vtx[(k + 2) % 3];
float dx = (e1.x - e0.x) * (area > 0 ? 1 : -1), dy = (e1.y - e0.y) * (area > 0 ? 1 : -1);
top_left[k] = (dy == 0 && dx < 0) || dy > 0;
}
int x0 = (int)fmaxf((float)cx0, floorf(fminf(a.pos.x, fminf(b.pos.x, c.pos.x))));
int x1 = (int)fminf((float)cx1, ceilf(fmaxf(a.pos.x, fmaxf(b.pos.x, c.pos.x))));
int y0 = (int)fmaxf((float)cy0, floorf(fminf(a.pos.y, fminf(b.pos.y, c.pos.y))));
int y1 = (int)fminf((float)cy1, ceilf(fmaxf(a.pos.y, fmaxf(b.pos.y, c.pos.y))));
float lod = 0;
if (art && !art->empty()) { // texels per screen pixel along the steeper axis
const float tw = (float)(*art)[0].w, th = (float)(*art)[0].h;
const float e1x = b.pos.x - a.pos.x, e1y = b.pos.y - a.pos.y, e2x = c.pos.x - a.pos.x, e2y = c.pos.y - a.pos.y;
const float du1 = (b.uv.x - a.uv.x) * tw, dv1 = (b.uv.y - a.uv.y) * th, du2 = (c.uv.x - a.uv.x) * tw, dv2 = (c.uv.y - a.uv.y) * th;
const float dudx = (du1 * e2y - du2 * e1y) / area, dvdx = (dv1 * e2y - dv2 * e1y) / area;
const float dudy = (du2 * e1x - du1 * e2x) / area, dvdy = (dv2 * e1x - dv1 * e2x) / area;
const float rho = fmaxf(sqrtf(dudx * dudx + dvdx * dvdx), sqrtf(dudy * dudy + dvdy * dvdy));
lod = rho > 0 ? log2f(rho) : 0;
}
for (int y = y0; y < y1; y++)
for (int x = x0; x < x1; x++) {
float px = x + 0.5f, py = y + 0.5f;
float w0 = edge(b.pos, c.pos, px, py) / area, w1 = edge(c.pos, a.pos, px, py) / area,
w2 = edge(a.pos, b.pos, px, py) / area;
if (w0 < 0 || w1 < 0 || w2 < 0) continue;
if ((w0 == 0 && !top_left[0]) || (w1 == 0 && !top_left[1]) || (w2 == 0 && !top_left[2])) continue;
float u = w0 * a.uv.x + w1 * b.uv.x + w2 * c.uv.x, v = w0 * a.uv.y + w1 * b.uv.y + w2 * c.uv.y;
float col[4];
for (int k = 0; k < 4; k++) {
float ca = (a.col >> (8 * k)) & 255, cb = (b.col >> (8 * k)) & 255, cc = (c.col >> (8 * k)) & 255;
col[k] = (w0 * ca + w1 * cb + w2 * cc) / 255.0f;
}
float t[4];
if (art && !art->empty()) sample_art(*art, u, v, lod, t);
else sample(atlas, aw, ah, u, v, t); // filtered, as the GPU samples the atlas's baked lines
for (int k = 0; k < 4; k++) col[k] *= t[k];
float *d = &img.px[4 * ((size_t)y * img.w + x)], sa = col[3];
for (int k = 0; k < 3; k++) d[k] = col[k] * sa + d[k] * (1 - sa);
d[3] = sa + d[3] * (1 - sa);
}
}
}
}
}
int main(int argc, char **argv) {
if (argc != 6 && argc != 7) {
fprintf(stderr, "usage: %s <font.ttf> <menu|overlay|boot> <width> <height> <out.rgba> [art folder]\n", argv[0]);
return 2;
}
const char *font = argv[1], *scene = argv[2], *out = argv[5];
if (argc == 7) art_load(argv[6]);
int w = atoi(argv[3]), h = atoi(argv[4]);
float scale = fmaxf(fminf((float)h, w * 0.68f), 720.0f) / 1080.0f; // as menu.cpp menu_scale at 100 % display scale
ImGui::CreateContext();
ImGuiIO &io = ImGui::GetIO();
io.IniFilename = nullptr;
io.DisplaySize = ImVec2((float)w, (float)h);
io.DeltaTime = 1.0f / 60;
io.ConfigFlags |= ImGuiConfigFlags_NavEnableKeyboard | ImGuiConfigFlags_NavEnableGamepad;
io.Fonts->AddFontFromFileTTF(font, 26.0f * scale);
ImFont *title = io.Fonts->AddFontFromFileTTF(font, 31.0f * scale);
skin_atlas_add(io.Fonts);
skin_atlas_fill(io.Fonts);
uint8_t *atlas;
int aw, ah;
io.Fonts->GetTexDataAsRGBA32(&atlas, &aw, &ah);
io.Fonts->SetTexID((ImTextureID)1);
ImGuiStyle &st = ImGui::GetStyle();
skin_style(st);
st.ScaleAllSizes(scale);
ImGui::SetCurrentFont(io.Fonts->Fonts[0]);
for (int frame = 0; frame < 4; frame++) { // a few frames: sizes settle and the first row takes the cursor
ImGui::NewFrame();
if (!strcmp(scene, "menu")) scene_menu((float)w, (float)h, scale, title, frame == 0);
else if (!strcmp(scene, "boot")) scene_boot((float)w, (float)h, scale, title);
else scene_overlay((float)w, (float)h, scale);
ImGui::Render();
}
Image img{w, h, std::vector<float>((size_t)w * h * 4, 0.0f)};
raster(ImGui::GetDrawData(), img, atlas, aw, ah);
FILE *f = fopen(out, "wb");
if (!f) return 1;
std::vector<uint8_t> bytes((size_t)w * h * 4);
for (size_t i = 0; i < bytes.size(); i++) bytes[i] = (uint8_t)(fminf(1, fmaxf(0, img.px[i])) * 255 + 0.5f);
fwrite(bytes.data(), 1, bytes.size(), f);
fclose(f);
ImGui::DestroyContext();
return 0;
}

101
tools/menu_preview.py Normal file
View File

@@ -0,0 +1,101 @@
"""Draw the in-game settings menu and the performance overlay without the game (tools/menu_preview.cpp).
python tools/menu_preview.py [--out build/menu-preview] [--font FILE] [--size 1920x1080] [--romfs FOLDER]
Builds tools/menu_preview.cpp with the menu's own row, shape, overlay and text code and Dear ImGui (host clang++, into
build/menu-preview), runs it for each scene and writes menu.png, overlay.png and boot.png. Without --romfs no game
files are read: the window has its plain border in place of the game's ornamented frame and the boxes are plain
shapes. With --romfs, an extracted RomFS folder, the textures listed in runtime/ui/menu_art.cpp are decoded into --out
and the boxes look as they do in the game (needs what tools/ui_textures.py needs). The font given (default: DejaVu
Sans, else the first TrueType font found) stands in for the game's either way.
"""
import argparse
import os
import re
import shutil
import struct
import subprocess
import sys
import zlib
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
RT = ROOT / 'runtime'
SOURCES = [ROOT / 'tools' / 'menu_preview.cpp', RT / 'ui' / 'option_rows.cpp', RT / 'ui' / 'menu_skin.cpp',
RT / 'ui' / 'perf_panel.cpp', RT / 'ui_text.c'] + sorted((RT / 'third_party' / 'imgui').glob('imgui*.cpp'))
FONTS = ['/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf', 'C:/Windows/Fonts/segoeui.ttf', 'C:/Windows/Fonts/arial.ttf']
def png(path, w, h, rgba):
"""An 8-bit RGBA PNG of the raw pixels."""
rows = b''.join(b'\0' + rgba[y * w * 4:(y + 1) * w * 4] for y in range(h))
def chunk(kind, data):
return struct.pack('>I', len(data)) + kind + data + struct.pack('>I', zlib.crc32(kind + data) & 0xffffffff)
Path(path).write_bytes(b'\x89PNG\r\n\x1a\n' + chunk(b'IHDR', struct.pack('>IIBBBBB', w, h, 8, 6, 0, 0, 0)) +
chunk(b'IDAT', zlib.compress(rows, 6)) + chunk(b'IEND', b''))
def build(out):
cxx = shutil.which('clang++') or shutil.which('g++')
if not cxx:
sys.exit('needs clang++ or g++ on PATH')
exe = out / ('menu_preview.exe' if os.name == 'nt' else 'menu_preview')
objs = []
for src in SOURCES:
obj = out / (src.stem + '.o')
if not obj.is_file() or obj.stat().st_mtime < max(src.stat().st_mtime, (RT / 'ui' / 'menu_skin.h').stat().st_mtime,
(RT / 'ui' / 'menu_art.h').stat().st_mtime):
lang = ['-x', 'c', '-std=gnu11'] if src.suffix == '.c' else ['-x', 'c++', '-std=gnu++17']
cmd = [cxx, *lang, '-c', '-O2', '-w', '-I', str(RT), '-I', str(RT / 'third_party' / 'imgui'), str(src), '-o',
str(obj)]
if subprocess.run(cmd).returncode:
sys.exit(f'{src.name} did not compile')
objs.append(str(obj))
if subprocess.run([cxx, *objs, '-o', str(exe), '-lm']).returncode:
sys.exit('the link failed')
return exe
def decode_art(romfs, out):
"""The textures of runtime/ui/menu_art.cpp kNames as out/art/NN.art for menu_preview.cpp; that folder."""
sys.path.insert(0, str(ROOT / 'tools'))
import ui_textures
source = (RT / 'ui' / 'menu_art.cpp').read_text(encoding='utf-8')
names = re.findall(r'"([^"]+)"', re.search(r'kNames\[MENU_ART_COUNT\] = \{(.*?)\};', source, re.S).group(1))
data = ui_textures.Romfs(romfs).read(f'{ui_textures.ARCHIVES}/Common.Product.100.Nin_Ounce_NVN2.bntx.zs')
textures = {t['name']: t for t in ui_textures.bntx_textures(data)}
art = out / 'art'
art.mkdir(parents=True, exist_ok=True)
lib = ui_textures.astc_library(out)
for i, name in enumerate(names):
rgba = ui_textures.decode(textures[name], data, lib)
(art / f'{i:02}.art').write_bytes(struct.pack('<II', rgba.shape[1], rgba.shape[0]) + rgba.tobytes())
return art
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('--out', type=Path, default=ROOT / 'build' / 'menu-preview')
ap.add_argument('--font', help='a TrueType font for the text')
ap.add_argument('--size', default='1920x1080')
ap.add_argument('--romfs', type=Path, help="an extracted RomFS folder: draw the game's textures")
a = ap.parse_args()
font = a.font or next((f for f in FONTS if Path(f).is_file()), None)
if not font:
sys.exit('no font found: give one with --font')
w, h = (int(x) for x in a.size.lower().split('x'))
a.out.mkdir(parents=True, exist_ok=True)
exe = build(a.out)
art = [str(decode_art(a.romfs, a.out))] if a.romfs else []
for scene in ('menu', 'overlay', 'boot'):
raw = a.out / f'{scene}.rgba'
if subprocess.run([str(exe), font, scene, str(w), str(h), str(raw), *art]).returncode:
sys.exit(f'the {scene} scene failed')
png(a.out / f'{scene}.png', w, h, raw.read_bytes())
raw.unlink()
print(a.out / f'{scene}.png')
if __name__ == '__main__':
main()

View File

@@ -1,6 +1,9 @@
# Packages of the launcher release's bundled Python 3.14.9 (tools/make_launcher.py): what the launcher and
# tools/make_game.py import. numpy: recomp/gen.py and the window's drawing (launcher/ui/art.py). lz4: NSO segments.
# zstandard: tools/nup.py. zbic is not bundled: the launcher downloads it from PyPI on the player's PC.
# zstandard: tools/nup.py. truststore: Windows' own certificate check for the launcher's downloads (launcher/net.py),
# pinned to its wheel's sha256 too (uv checks the hashes a line gives). zbic is not bundled: the launcher downloads
# it from PyPI on the player's PC.
numpy==2.5.4
lz4==4.4.5
zstandard==0.25.0
truststore==0.10.5 --hash=sha256:9aaaedaefaf06d8b206278cf8b5012bc897f485a874503501e12d776df78951c

245
tools/ui_textures.py Normal file
View File

@@ -0,0 +1,245 @@
"""List and dump the game's interface textures, for choosing the art the settings menu draws (runtime/ui/menu_art.cpp).
python tools/ui_textures.py ROMFS [--out build/ui-textures] [--archive Common] [--match REGEX] [--sheet]
ROMFS is an extracted RomFS folder. Without --match it only lists; with it the matching textures become PNGs under
--out, and --sheet also writes contact sheets of them (numbered; sheet.txt says which number is which). Reads files the
way runtime/menu_assets.c does, and decodes ASTC with the runtime's own decoder (runtime/tex_astc.c, built into --out).
Everything written is game data and stays under build/. Needs Python 3.14 (compression.zstd) and numpy.
"""
import argparse
import ctypes
import re
import shutil
import struct
import subprocess
import sys
import zlib
from compression import zstd
from pathlib import Path
import numpy as np
ROOT = Path(__file__).resolve().parents[1]
ARCHIVES = 'UI/Layout2Archive'
# the second byte of a BNTX format: name, block size, bytes per block
KINDS = {0x0b: ('RGBA8', 1, 4), 0x1a: ('BC1', 4, 8), 0x1b: ('BC2', 4, 16), 0x1c: ('BC3', 4, 16), 0x1d: ('BC4', 4, 8),
0x1e: ('BC5', 4, 16), 0x20: ('BC7', 4, 16), 0x2d: ('ASTC4x4', 4, 16)}
def u16(b, at): return struct.unpack_from('<H', b, at)[0]
def u32(b, at): return struct.unpack_from('<I', b, at)[0]
def u64(b, at): return struct.unpack_from('<Q', b, at)[0]
def sarc_members(p):
"""The members of a SARC archive as a name -> bytes dictionary."""
hl, data = u16(p, 4), u32(p, 0xc)
count, ents = u16(p, hl + 6), hl + 0xc
names = ents + 16 * count + 8
out = {}
for i in range(count):
e = ents + 16 * i
at = names + (u32(p, e + 4) & 0xffff) * 4
name = p[at:p.index(b'\0', at)].decode()
out[name] = p[data + u32(p, e + 8):data + u32(p, e + 12)]
return out
class Romfs:
def __init__(self, root):
self.root = Path(root)
self.dicts = None
def load_dicts(self):
self.dicts = {}
raw = zstd.decompress((self.root / 'Pack/ZsDic.pack.zs').read_bytes())
for name, data in sarc_members(raw).items():
d = zstd.ZstdDict(data)
self.dicts[d.dict_id] = d
def read(self, rel):
b = (self.root / rel).read_bytes()
if not rel.endswith('.zs'):
return b
id_ = zstd.get_frame_info(b).dictionary_id
if not id_:
return zstd.decompress(b)
if self.dicts is None:
self.load_dicts()
return zstd.decompress(b, zstd_dict=self.dicts[id_])
def bntx_textures(b):
"""Each texture of a BNTX file: name, format, size, block height, data offset and size."""
out = []
count, arr = u32(b, 0x24), u64(b, 0x28)
for i in range(count):
p = u64(b, arr + 8 * i)
if b[p:p + 4] != b'BRTI':
continue
no = u64(b, p + 0x60)
name = b[no + 2:no + 2 + u16(b, no)].decode()
out.append({'name': name, 'fmt': u32(b, p + 0x1c), 'w': u32(b, p + 0x24), 'h': u32(b, p + 0x28),
'th': u32(b, p + 0x34), 'size': u32(b, p + 0x50), 'data': u64(b, u64(b, p + 0x70))})
return out
def deswizzle(t, b, row, rows):
"""The first level as tightly packed rows, in the Switch 2 GOB layout (runtime/tex_deswizzle.c tex_bl_copy)."""
th = t['th']
gobs, unit, block = (row + 63) // 64, 8 << th, 512 << th
need = gobs * ((rows + unit - 1) // unit) * block
bl = np.zeros(max(need, t['size']) // 16 + 1, dtype='V16')
src = bytearray(bl.tobytes())
src[:t['size']] = b[t['data']:t['data'] + t['size']]
bl = np.frombuffer(bytes(src), dtype='V16')
cols = (row + 15) // 16
y = np.arange(rows)[:, None]
x = (np.arange(cols) * 16)[None, :]
at = (y // unit) * gobs * block + (x // 64) * block + ((y % unit) // 8) * 512 + \
(x % 64 // 32) * 256 + (y % 8 // 4) * 128 + (x % 32 // 16) * 64 + (y % 4) * 16
flat = np.frombuffer(bl[at // 16].tobytes(), dtype=np.uint8).reshape(rows, cols * 16)
return flat[:, :row]
def bc4(lin, wb, hb, w, h):
"""White with the mask as alpha (runtime/menu_assets.c bc4_decode)."""
blocks = lin.reshape(hb, wb, 8).astype(np.uint32)
a, b = blocks[..., 0], blocks[..., 1]
pal = np.zeros((hb, wb, 8), dtype=np.uint32)
pal[..., 0], pal[..., 1] = a, b
big = a > b
for i in range(6):
pal[..., 2 + i] = np.where(big, ((6 - i) * a + (i + 1) * b) // 7, 0)
for i in range(4):
pal[..., 2 + i] = np.where(big, pal[..., 2 + i], ((4 - i) * a + (i + 1) * b) // 5)
pal[..., 6] = np.where(big, pal[..., 6], 0)
pal[..., 7] = np.where(big, pal[..., 7], 255)
bits = np.zeros((hb, wb), dtype=np.uint64)
for i in range(6):
bits |= blocks[..., 2 + i].astype(np.uint64) << np.uint64(8 * i)
alpha = np.zeros((hb * 4, wb * 4), dtype=np.uint8)
for i in range(16):
sel = ((bits >> np.uint64(3 * i)) & np.uint64(7)).astype(np.intp)
alpha[i // 4::4, i % 4::4] = np.take_along_axis(pal, sel[..., None], 2)[..., 0]
rgba = np.full((h, w, 4), 255, dtype=np.uint8)
rgba[..., 3] = alpha[:h, :w]
return rgba
def astc_library(out):
"""The runtime's ASTC decoder as a library, built once into out."""
dll = out / 'astc.dll'
src = ROOT / 'runtime' / 'tex_astc.c'
if not dll.is_file() or dll.stat().st_mtime < src.stat().st_mtime:
cc = shutil.which('clang') or r'C:\msys64\ucrt64\bin\clang.exe'
vma = Path(cc).resolve().parents[1] / 'include' / 'vma' # tex.h reaches the Vulkan allocator's header
if subprocess.run([cc, '-shared', '-O2', '-static', '-w', '-I', str(ROOT / 'runtime'), '-I', str(vma), str(src), '-o',
str(dll)]).returncode:
sys.exit('tex_astc.c did not compile')
lib = ctypes.CDLL(str(dll))
lib.astc_decode_ldr.restype = None
lib.astc_decode_ldr.argtypes = [ctypes.c_char_p, ctypes.c_uint32, ctypes.c_uint32, ctypes.c_uint32, ctypes.c_uint32,
ctypes.c_bool, ctypes.c_void_p]
return lib
def decode(t, b, lib):
kind = t['fmt'] >> 8
if kind not in (0x0b, 0x1d, 0x2d):
return None
_, bs, bpb = KINDS[kind]
w, h = t['w'], t['h']
wb, hb = (w + bs - 1) // bs, (h + bs - 1) // bs
lin = deswizzle(t, b, wb * bpb, hb)
if kind == 0x1d:
return bc4(lin, wb, hb, w, h)
if kind == 0x0b:
return lin.reshape(h, w, 4).copy()
rgba = np.zeros((h, w, 4), dtype=np.uint8)
lib.astc_decode_ldr(lin.tobytes(), w, h, 4, 4, (t['fmt'] & 0xff) == 6, rgba.ctypes.data)
return rgba
def png(path, a):
h, w = a.shape[:2]
rows = b''.join(b'\0' + a[y].tobytes() for y in range(h))
def chunk(kind, data):
return struct.pack('>I', len(data)) + kind + data + struct.pack('>I', zlib.crc32(kind + data) & 0xffffffff)
Path(path).write_bytes(b'\x89PNG\r\n\x1a\n' + chunk(b'IHDR', struct.pack('>IIBBBBB', w, h, 8, 6, 0, 0, 0)) +
chunk(b'IDAT', zlib.compress(rows, 6)) + chunk(b'IEND', b''))
DIGITS = ['111101101101111', '010110010010111', '111001111100111', '111001111001111', '101101111001001',
'111100111001111', '111100111101111', '111001010010010', '111101111101111', '111101111001111']
def label(sheet, x, y, n):
for c in str(n):
for j in range(5):
for i in range(3):
if DIGITS[int(c)][j * 3 + i] == '1':
sheet[y + j * 2:y + j * 2 + 2, x + i * 2:x + i * 2 + 2] = (255, 255, 0, 255)
x += 8
def sheets(out, items, cell=220, cols=7, per=35):
"""Contact sheets over a mid brown, so white masks and dark art both show."""
lines = []
for s in range(0, len(items), per):
page = items[s:s + per]
rows = (len(page) + cols - 1) // cols
sheet = np.full((rows * cell, cols * cell, 4), (84, 70, 58, 255), dtype=np.uint8)
for k, (name, a) in enumerate(page):
n = s + k
lines.append(f'{n} {name} {a.shape[1]}x{a.shape[0]}')
step = max(1, max(a.shape[0], a.shape[1]) // (cell - 8) + (max(a.shape[:2]) % (cell - 8) > 0))
t = a[::step, ::step]
cy, cx = (k // cols) * cell + 4, (k % cols) * cell + 4
al = t[..., 3:4].astype(np.float32) / 255
bg = sheet[cy:cy + t.shape[0], cx:cx + t.shape[1], :3].astype(np.float32)
sheet[cy:cy + t.shape[0], cx:cx + t.shape[1], :3] = (t[..., :3] * al + bg * (1 - al)).astype(np.uint8)
label(sheet, cx, cy, n)
png(out / f'sheet{s // per:02}.png', sheet)
(out / 'sheet.txt').write_text('\n'.join(lines) + '\n')
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument('romfs', type=Path)
ap.add_argument('--out', type=Path, default=ROOT / 'build' / 'ui-textures')
ap.add_argument('--archive', default='', help='only archives whose name contains this')
ap.add_argument('--match', help='a regular expression: the textures to decode')
ap.add_argument('--sheet', action='store_true', help='write contact sheets of what was decoded')
a = ap.parse_args()
fs = Romfs(a.romfs)
a.out.mkdir(parents=True, exist_ok=True)
lib = astc_library(a.out) if a.match else None
decoded = []
for f in sorted((a.romfs / ARCHIVES).glob('*.bntx.zs')):
if a.archive not in f.name:
continue
b = fs.read(f'{ARCHIVES}/{f.name}')
ts = bntx_textures(b)
print(f'# {f.name}: {len(ts)} textures')
for t in sorted(ts, key=lambda t: t['name']):
kind = KINDS.get(t['fmt'] >> 8, (f"{t['fmt']:#x}",))[0]
if not a.match:
print(f"{t['name']} {kind} {t['w']}x{t['h']}")
elif re.search(a.match, t['name']):
rgba = decode(t, b, lib)
if rgba is None:
print(f"{t['name']}: format {kind} is not read by the runtime", file=sys.stderr)
continue
png(a.out / f"{t['name']}.png", rgba)
decoded.append((t['name'], rgba))
if a.match:
print(f'{len(decoded)} textures written to {a.out}')
if a.sheet and decoded:
sheets(a.out, decoded)
if __name__ == '__main__':
main()